Paired Samples CFA for Detecting Multivariate Change in Small Samples

Stemmler and Hammond (1997) proposed a version of Configuration Frequency Analysis (CFA) built for paired samples, and we treat it here as a practical way to pick up multivariate change when the number of participants is small. Our starting question concerned impulsive behavior in a Go/NoGo task. Such behavior shows up as a pair of signs together: responses get faster while accuracy drops. The usual assumption is that this pattern reflects a stable personality trait, impulsivity. We wanted to know whether it could also be pushed around by the reinforcement a person was working under.

Running the data through paired samples CFA, we found that impulsive responding was not fixed. Positive reinforcement, in the form of reward, brought it out. Indirect punishment, meaning the loss of an expected reward, held it back. So the same behavioral signature could be switched on or dampened depending on the contingencies in play.

To check that this was not an artifact of the method, we reanalyzed the same data with a repeated measures analysis of variance. The parametric result matched: there was a significant multivariate treatment effect, carried jointly by reaction latency and passive avoidance errors. Two independent routes, one case oriented and one based on distributional assumptions, pointed the same way.

Part of what makes paired samples CFA attractive for this kind of study is how few demands it places on the data. It can handle measurements of any dignity level, whether those are response latencies, ratings, or syndrome categories. It asks nothing about how the sample is distributed. And because it works from a cross-tabulation, the specific cases driving a significant effect can be read off directly rather than inferred.

For experimental and clinical work where samples are inevitably small and the design is within subjects, that combination (transparency, ease of interpretation, no distributional baggage) makes the method a reasonable choice for detecting change.

Marcus Ising conducted this work at the Max-Planck-Institut für Psychiatrie in Munich; Wilhelm Janke is based at the Institut für Psychologie in Würzburg.