Improving progress test score estimation using Bayesian statistics

Medical Education 2011: 45: 570–577 Objectives Progress tests give a continuous measure of a student’s growth in knowledge. However, the result at each test instance is subject to measurement error from a variety of sources. Previous tests contain useful information that might be used to reduce this error. A Bayesian statistical approach to using this prior information was investigated. Methods We first developed a Bayesian model that used the result from only one preceding test to update both the current estimated test score and its standard error of measurement (SEM). This was then extended to include results from all previous tests. Results The Bayesian model leads to an exponentially weighted combination of test scores. The results show smoothing of test scores when all previous tests are included in the model. The effective sample size is doubled, leading to a 30% reduction in measurement error. Conclusions A Bayesian approach can give improved score estimates and smaller SEMs. The method is simple to use with large cohorts of students and frequent tests. The smoothing of raw scores should give greater consistency in rank ordering of students and hence should better identify both high‐performing students and those in need of remediation.