An Evaluation of Observational Methods for Measuring Response to Classwide Intervention

Although there is much research to support the effectiveness of classwide interventions aimed at improving student engagement, there is also a great deal of variability in terms of how response to group-level intervention has been measured. The unfortunate consequence of this procedural variability is that it is difficult to determine whether differences in obtained results across studies are attributable to the way in which behavior was measured or actual intervention effectiveness. The purpose of this study was to comparatively evaluate the most commonly used observational methods for monitoring the effects of classwide interventions in terms of the degree to which obtained data represented actual behavior. The 5 most common sampling methods were identified and evaluated against a criterion generated by averaging across observations conducted on 14 students in one seventh-grade classroom. Results suggested that the best approximation of mean student engagement was obtained by observing a different student during each consecutive 15-s interval whereas observing an entire group of students during each interval underestimated the mean level of behavior within a phase and the degree of behavior change across phases. In contrast, when observations were restricted to the 3 students with the lowest levels of engagement, data revealed greater variability in engagement across baseline sessions and suggested a more notable change in student behavior subsequent to intervention implementation.