Instructional Tools in Educational Measurement and Statistics (ITEMS) for School Personnel: Evaluation of Three Web-Based Training Modules

In the current No Child Left Behind era, K‐12 teachers and principals are expected to have a sophisticated understanding of standardized test results, use them to improve instruction, and communicate them to others. The goal of our project, funded by the National Science Foundation, was to develop and evaluate three Web‐based instructional modules in educational measurement and statistics to help school personnel acquire the “assessment literacy” required for these roles. Our first module, “What's the Score?” was administered in 2005 to 113 educators who also completed an assessment literacy quiz. Viewing the module had a small but statistically significant positive effect on quiz scores. Our second module, “What Test Scores Do and Don't Tell Us,” administered in 2006 to 104 educators, was even more effective, primarily among teacher education students. In evaluating our third module, “What's the Difference?” we were able to recruit only 33 participants. Although those who saw the module before taking the quiz outperformed those who did not, results were not statistically significant. Now that the research phase is complete, all ITEMS instructional materials are freely available on our Website.