Response time in computer-aided testing: a verbal memory test for routes and maps

When testing moves onto a computer, whether the version is simply assisted or fully adaptive, the machine records more than the final score. Every item also carries a response time. That extra layer of data raises a natural question: what can those timings tell us that the scores alone cannot?

The analysis here rests in part on results from 1622 young men who sat a computerized verbal memory test, looking both at the test as a whole and at individual items. Interest in response times is not new. Back in the early 1930s researchers already treated how long someone took to answer as a sign of personality, something they wanted to keep separate from a person's measured ability. Psychometric models developed much more recently have kept that same distinction alive.

Still, a basic uncertainty lingers. It remains open whether computer-adaptive testing, even with careful and programmed records of how each test was taken, actually yields diagnostic information worth having. The present work takes up that question rather than settling it.

One pattern from earlier research held up again. People spend noticeably more time on the items they get wrong than on the ones they get right. The data gathered here confirmed that gap.

The correlations between average item response time and test score turned out to be modest. For wrong answers the figure was .387; for correct answers it was .290. Neither points to a strong relationship. What does stand out is that the time spent on wrong answers tracks the score more closely than the time spent on correct ones, a difference sometimes called the F greater than C phenomenon.