Configural Frequency Analysis for Large Sparse Cross-Classifications

When a cross-classification table grows large and most of its cells hold few or zero cases, the usual tools for Configural Frequency Analysis start to strain. Alexander von Eye and Eduardo Gutiérrez-Peña take up that problem here, sketching a way to run CFA on tables that are both big and sparse.

Their starting point is the approach DuMouchel described in 1999. The idea unfolds in three steps. First, you estimate how far each observed cell frequency sits from the frequency you would expect. Then you sort those discrepancy estimates by size. Finally, you take the ones that stand out most and treat them as candidates for types and antitypes, the patterns that occur far more or far less often than chance would predict.

Two routes through this are worked out. The first stays inside standard frequentist CFA and borrows measures that come out of the data mining literature. Several of those measures are laid side by side and checked against each other in a simulation, so the comparison rests on how they actually behave rather than on argument alone.

The second route is Bayesian. For that version the authors put forward three measures of their own, built to fit the Bayesian way of reading discrepancy and deciding which configurations deserve attention.

To show the methods on real numbers rather than invented ones, the paper draws on data from a study of how job applicants get selected. That example lets the frequentist and Bayesian procedures be seen at work on the kind of sparse table that motivated the whole exercise.