An Algorithm for Separating Skill from Working Style

When you accept that test-takers trade speed against accuracy, a familiar interpretive problem opens up: you can no longer say for certain what produced a given score on an ability test. Did the person know more, or did they simply slow down and answer more carefully? Two people can reach the same result by very different routes, and the number alone hides which route they took. This paper takes that problem as its starting point and asks how skill might be pulled apart from the manner in which someone works through a test.

The approach I set out combines a speed and power test model with ideas borrowed from two existing lines of research. One is the inspection time paradigm, which concerns how quickly a person can register and process a stimulus. The other is signal detection theory, which offers a way to describe decision quality independently of how cautious or hasty a responder happens to be. Fitted together, these let a multiple choice situation be scored for more than correctness, so that the pace and the reflexivity of someone's responses become measurable alongside what they actually got right.

Beyond describing the model in the abstract, I include concrete suggestions for building it into working psychological tests. The aim there is practical: a test author should be able to take the framework and operationalise working style without having to reconstruct the theory first.

To see whether any of this holds up, I applied the model to a variant of the vocabulary test LEWITE (Wagner-Menghin, 2002). The specific question was prediction of test pre-knowledge, meaning the kind of advantage a person gains when they have been coached and are, in effect, faking competence they do not have. Pre-knowledge did what you would expect: it pushed scores upward. The more telling finding was that the model could flag it. By watching how a person worked, not only what they answered, the pattern that betrays coached faking became visible.