Model‐Based Treatment of Rapid Guessing

The increased availability of time‐related information as a result of computer‐based assessment has enabled new ways to measure test‐taking engagement. One of these ways is to distinguish between solution and rapid guessing behavior. Prior research has recommended response‐level filtering to deal wi...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of educational measurement 2021-06, Vol.58 (2), p.281-303
Hauptverfasser: Deribo, Tobias, Kroehne, Ulf, Goldhammer, Frank
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The increased availability of time‐related information as a result of computer‐based assessment has enabled new ways to measure test‐taking engagement. One of these ways is to distinguish between solution and rapid guessing behavior. Prior research has recommended response‐level filtering to deal with rapid guessing. Response‐level filtering can lead to parameter bias if rapid guessing depends on the measured trait or (un‐)observed covariates. Therefore, a model based on Mislevy and Wu (1996) was applied to investigate the assumption of ignorable missing data underlying response‐level filtering. The model allowed us to investigate different approaches to treating response‐level filtered responses in a single framework through model parameterization. The study found that lower‐ability test‐takers tend to rapidly guess more frequently and are more likely to be unable to solve an item they guessed on, indicating a violation of the assumption of ignorable missing data underlying response‐level filtering. Further ability estimation seemed sensitive to different approaches to treating response‐level filtered responses. Moreover, model‐based approaches exhibited better model fit and higher convergent validity evidence compared to more naïve treatments of rapid guessing. The results illustrate the need to thoroughly investigate the assumptions underlying specific treatments of rapid guessing as well as the need for robust methods.
ISSN:0022-0655
1745-3984
DOI:10.1111/jedm.12290