Assessing the Reliability of Criterion-Referenced Measures Used To Evaluate Health-Education Programs

The reliability of criterion-referenced tests (CRTs) used in health program evaluations can be conceptualized in different ways. Classical conceptualizations of reliability have limited usefulness when applied to health-related CRTs. When a cut-score is set, the test's reliability can be represented as the consistency of mastery/nonmastery classifications. When a cut-score is not set, the size of the standard error of measurement (SEM) of a domain score estimate is a reflection of the test's reliability. Formulas are presented for estimating appropriate SEMsfor CRTs. The SEM can be used in computing confidence intervals for domain score estimates andfor a cut-score.