General Framework for Effect Sizes in Cluster Randomized Experiments

Cluster randomized experiments are ubiquitous in modern education research. Although a variety of modeling approaches are used to analyze these data, perhaps the most common methodology is a normal mixed effects model where some effects, such as the treatment effect, are regarded as fixed, and others, such as the effect of group random assignment or the neighborhoods where the students live, are regarded as random. For these models, the standard reference used by education researchers is Raudenbush and Bryk (2002). Although mixed effects models enjoy wide use in estimating parameters from and testing hypotheses about these experiments, the development of standardized mean difference effect size indices for them is relatively recent. This paper unifies the currently published effect sizes in a general mixed effects modeling framework. Researchers then apply this framework to suggest an effect size for a model that currently lacks a published effect size, a random slope model with heterogeneous treatment effects. The general framework proposed by researchers generates interpretable effect sizes for a wide class of mixed effects. Further, they illustrate the value of this framework by deriving a new effect size for a cluster randomized controlled trial with heterogeneous treatment effects. One figure is appended.