Don't Blame the Teacher? The Need to Account for Classroom Characteristics in Evaluations of Teaching Quality

Previous research provides evidence that ratings from students and external observers can deliver reliable and valid measures of teaching quality. However, there is still a lack of research on how stable these measures are across time and classes, which is particularly important when they are used for evaluations. We addressed this issue in 3 different studies from Germany. Study 1 examined stability over time using a longitudinal sample of 174 teachers and 106 classes. The results show that student ratings of teaching quality are relatively stable when the same class rates the same teacher in 2 subsequent school years. However, we also found teacher-independent stabilities, which occurred when the same students rated different teachers. Study 2 examined stabilities in student ratings across classes. In this study, each teacher (N = 95) was evaluated by 2 different classes (N = 190). The results of a 3-level model reveal rather low stabilities across classes within teachers. Study 3 drew on a unique dataset in which 1 single teacher taught the same content to 30 different classes of the same grade level. Teaching quality as rated by students and external observers varied considerably between these classes. Differences in teaching quality were related to student learning, and the motivation of students in a class was a significant predictor of differences in teaching quality. In summary, the results of these studies show that both researchers and policymakers should be cautious when interpreting measures of teaching quality as indicators of teacher quality. Educational Impact and Implications Statement Ratings of teaching quality by students and external observers are increasingly used in research, in everyday school practice, and in the context of teacher evaluations. It is obviously a good idea to use direct measures of teaching quality when evaluating teachers because they can provide information on what actually happens in the classroom. However, it is an open question if these ratings of teaching quality can also be seen as indicators of teacher quality. To address this issue, we examined the stability of teaching quality measures across points in time and across classes: Do great teachers deliver great teaching every time and in every classroom? The results of our studies show that the stabilities over time (same teacher, same class) were relatively high. However, our findings also show that the teaching quality of a certain teacher can vary to a larg