Empirical Choice Weighting Under "Guess" and "Do Not Guess" Directions

This study investigated reliability and validity of scores resulting from empirical choice-weighting under two conditions: (1) examinees were told not to guess when unable to eliminate any choice and (2) they were told their scores would be the number of right responses. Unlike other choice-weighting studies which have reported disappointing results under one or the other of these two conditions, the present study used a predictor and criterion which were obtained independently yet had a substantial logical relationship. The results indicated that choice-weighting resulted in substantial increases in score reliability and predictive validity over number-right and corrected-for-guessing scores. Moreover, the validity coefficients associated with choice-weighted scores obtained under "Guess" directions were comparable to coefficients obtained under "Do Not Guess" directions. This latter result led to a recommendation against instruction to restrict guessing under choice-weighted scoring.