RETEST EFFECTS IN OPERATIONAL SELECTION SETTINGS: DEVELOPMENT AND TEST OF A FRAMEWORK

This study proposes a framework for examining the effects of retaking tests in operational selection settings. A central feature of this framework is the distinction between within‐person and between‐person retest effects. This framework is used to develop hypotheses about retest effects for exemplars of 3 types of tests (knowledge tests, cognitive ability tests, and situational judgment tests) and to test these hypotheses in a high stakes selection setting (admission to medical studies in Belgium). Analyses of within‐person retest effects showed that mean scores of repeat test takers were one‐third of a standard deviation higher for the knowledge test and situational judgment test and one‐half of a standard deviation higher for the cognitive ability test. The validity coefficients for the knowledge test differed significantly depending on whether examinees' test scores on the first versus second administration were used, with the latter being more valid. Analyses of between‐person retest effects on the prediction of academic performance showed that the same test score led to higher levels of performance for those passing on the first attempt than for those passing on the second attempt. The implications of these results are discussed in light of extant retesting practice.