Student and Teacher Ratings of Instructional Quality: Consistency of Ratings Over Time, Agreement, and Predictive Power

Prior research has shown that the agreement between teacher and student ratings of instructional quality is, at best, moderate, and the associations between measures of instructional quality and outcomes such as standardized achievement are typically small and somewhat mixed across both perspectives. One explanation for these low-to-moderate associations is the assumption that teacher and student ratings are not perfectly stable over time. By using a manifest-latent state-trait model, the present study investigated the following topics in a sample of 5th-grade students (74 classes) from vocational track schools: (a) the time consistency of teacher and student instructional quality ratings in math lessons over multiple measurement time points; (b) the agreement among teachers and students for time consistent rating components and single time point ratings; and (c) the predictive power of consistent rating components and single time point ratings with regard to a standardized math achievement test, and math self-concept. Results of multilevel factor analyses with 3 measurement time points over a period of 3 months showed a moderate-to-high time consistency for both teacher and student ratings. Furthermore, the agreement among teacher and student ratings regarding classroom management and goal clarity was higher for the consistent rating components than for ratings at single measurement time points, whereas this pattern was not found for support of autonomy. Finally, student consistent rating components predicted students' pretest adjusted math achievement and self-concept. The effect sizes for ratings at single measurement time points varied within quality dimensions.