A Comparison of Measures Used for Configural Frequency Analysis of Large, Sparse Cross-Classifications

When researchers run Configural Frequency Analysis on contingency tables that are both large and sparse, the choice of which residual statistic to use is far from settled. This paper puts four candidate measures side by side: relative risk (RR), reworked versions of Log P and Pearson's statistic, and an approximation of Log P. The first three were already on the table thanks to DuMouchel (1999), and the last two surface in the companion piece by von Eye and Gutiérrez-Peña in this same issue.

The question driving the comparison is practical. Sparse tables cause trouble for the usual CFA machinery, so analysts need statistics that still behave sensibly when many cells carry tiny expected frequencies. Do these four measures agree about which cells deserve attention, or do they point in different directions?

A Monte Carlo study supplies the answer. Across the simulations, the four measures tend to rank the same cells as extreme whenever the gap between observed and expected frequencies is itself extreme. That agreement is the reassuring part: at the far end of the distribution, where types and antitypes actually live, the statistics mostly concur.

RR is the exception that complicates the picture. The convergence holds for the detection of both types and antitypes under the other three measures, but relative risk does not line up as cleanly across both. Users who rely on RR should keep that asymmetry in mind.

The approximation of Log P earns a qualified endorsement. It tracks the full version well, and it does so best precisely where it matters most, in cases of large discrepancies between what was observed and what the model expected. For small discrepancies the approximation is less dependable, but those cells are rarely the ones flagged as types or antitypes anyway.

The article closes on the computational side of things: numerical problems that can arise with these statistics, and what the simulation results imply for anyone applying CFA to sparse data in practice.