by Deborah L. Schnipke

Executive Summary

When running out of time on a multiple-choice test such as the Law School Admission Test (LSAT), some test takers are likely to respond rapidly to the remaining unanswered items in an attempt to get some items right by chance. Because these responses will tend to be incorrect, the presence of "rapid-guessing behavior" could cause these items to appear more difficult statistically than they really are. Using data (correct/incorrect responses and response times) that were simulated to match real data, the present study found that this is indeed the case. Using the simulated response times, an attempt was made to remove responses that appeared to be the result of rapid-guessing behavior (because the responses had such short response times). A mathematical model was fit to the response time distribution of each item, and responses that were more likely, according to the model, to come from the rapid-guessing distribution (based on response times) were removed. After the fast responses (rapid guesses) were removed, estimated item difficulty was very close to the "true" item difficulty used to simulate the data. When test data are contaminated by speededness (rapid guesses), the present study shows that response times, if available, can be used to identify and remove rapid guesses and thereby recover the true item difficulty more accurately. If the LSAT is converted to a computer-delivered format, response times will be available, and we will be able to determine if rapid guesses are altering our estimates of item difficulty, and if so, to remove the rapid guesses and their influence.

The Influence of Speededness on Item-Parameter Estimation (CT-96-07)

Research Report Index