Log-linear modeling and two-sample CFA in the search of discrimination types

When researchers want to compare data collected before and after some intervention, two-sample Configural Frequency Analysis (CFA) offers a practical way to do it. The measurements involved are often difference scores or improvement scores, and that is exactly where many standard methods run into trouble. Improvement scores drawn from two dependent samples tend not to follow a normal distribution, even when the underlying variable is measured on a metric scale. Because of that, parametric comparisons are usually a poor fit, and a procedure built for non-normal data makes more sense.

This paper by Mark Stemmler and C. Raymond Bingham sets two-sample CFA alongside log-linear modeling (LLM) and works through where the two approaches overlap and where they part ways. The two methods answer related questions, but they go about the work from opposite directions.

Log-linear modeling takes a global, model-fitting stance. It builds a null model that assumes no interaction between the grouping or sample variable and the outcome variables, then checks how well that model matches the observed data overall. The test is about the goodness-of-fit of the whole model.

CFA works locally instead. Rather than judging the fit of an entire model at once, it looks cell by cell, comparing what the hypothesized null model predicts against what the data actually show. The question shifts from "does the model hold in general" to "which specific patterns depart from it."

For the cell-level decisions, the authors bring in the Fisher-Yates test. It is the statistic for deciding whether a given pattern, or configuration, separates the two samples being studied. Patterns that stand out this way point to discrimination types: particular combinations of values that distinguish one sample from the other.

To show how this plays out, Stemmler and Bingham use real data from educational psychology, running both univariate and bivariate two-sample comparisons so the single-variable and paired-variable cases are each illustrated.

Key words: Two-sample comparison, Configural Frequency Analysis (CFA), two-sample CFA, log-linear modeling (LLM), nonparametric testing, contingency table analysis.