Validity Research on Teacher Evaluation Systems Based on the Framework for Teaching

After decades of disinterest, evaluation of the performance of elementary and secondary teachers in the United States has become an important educational policy issue. As U.S. states and districts have tried to upgrade their evaluation processes, one of the models that has been increasingly used is the Framework for Teaching. This paper summarizes validity evidence pertaining to several different implementations of the Framework. It is based primarily on reviewing the published and unpublished studies that have looked at the relationship between teacher evaluation ratings made using systems based on the Framework and value-added measures of teacher effectiveness. The research results summarized suggest that teacher ratings made using evaluation systems based on or related to the Framework have a correlation with value-added estimates of teacher effectiveness, in the 0.2 to 0.3 range. Measurement error in both the value-added estimates and evaluation ratings likely bias these correlations downward. Though fragmentary, the evidence reviewed suggests that Framework-based evaluation ratings can show substantial inter-rater agreement and, if multiple observers and multiple occasions of observation are used, the ratings can be reliable. However, a score based on a single observation by one observer is likely to have low reliability. The paper also discusses ways to extend and improve validity research on teacher evaluation systems. "Inter-rater Agreement Details for Cincinnati" is appended to this document. The contributions to this research of current and former colleagues Steven Kimball, Herbert Heneman, Allan Odden, H. Alix Gallagher and Bradford White, are gratefully acknowledged. (Contains 6 tables, 2 figures, and 6 footnotes.)