Subdivision of Grading Units Can Increase the Reliability of Performance Testing

This study examined the effect of subdividing grading units on performance test reliability. That is, instead of increasing test length by adding grading units comparable to existing grading units, this experimental approach attempted to increase test length, and hence reliability, by subdividing existing grading units into comparable subunits. The effect of subdividing grading units was assessed empirically using a performance test of the ultrasonic detection of cracks in pipe welds. Five-hour performance tests involving the examination of 10 pipe-weld specimens were completed by each of 52 experienced ultrasonic operators as part of their qualification for performing tasks of this type in nuclear power plants. Subdivision of grading units was found to increase the reliability of the test from 0.28 to 0.92, to decrease the standard error of measurement of the test from 13.81 to 1.35, and to decrease the 90% confidence band around test scores from ± 22.60 to ±2.20. Moreover, the increased reliability was predicted by the Spearman-Brown Prophecy Formula, the method commonly employed for predicting the effect of increased length on test reliability.