Artificial factors that appear when factor analysis is applied to dichotomous variables

A word of caution for anyone who runs factor analysis on items scored as yes/no, correct/incorrect, or any other two-category response. When the variables are dichotomous, the procedure has a habit of producing factors that nobody intended and that carry no real meaning. They are an artifact of the method rather than a property of what is being measured.

What tends to happen is that the extracted factors sort themselves according to item difficulty. Items of roughly similar difficulty cluster together and load on the same factor, so the solution ends up describing how hard the items are rather than any underlying trait or ability. You think you have found distinct dimensions of a construct, and instead you have found groups of easy items and groups of hard items.

A worked numerical example makes the point clear, since the pattern is easier to recognize once you watch it emerge from actual numbers rather than read about it in the abstract.

The source of the trouble is the correlation that gets factored. Pearson correlations, which are the usual input, behave poorly between two dichotomous variables because their maximum possible value shrinks as the two items differ in difficulty. That restriction is what drives the difficulty-based grouping. Using tetrachoric correlations instead sidesteps the problem and yields a factor structure that corresponds to the content of the items, which is what the analysis was supposed to reveal in the first place.

The recommendation, then, is straightforward: when the data are dichotomous, factor the tetrachoric correlations rather than the Pearson ones if you want results you can actually interpret in terms of content.