New Measures of Effect Size in Moderation Analysis

Measures of explained variance, ΔR2 and f,2 are routinely used to evaluate the size of moderation effects. However, they suffer from several drawbacks: (a) Not all the variance components of the outcome variable Y are related to the effect of moderation, and so an effect size with the total variance of Y as the denominator cannot accurately characterize the moderation effect; (b) moderation and interaction are conflated; and (c) the assumption of homoscedasticity might be violated when moderation exists. By arguing that measures for the size of moderation effect should be based on the variance of the outcome Y via the predictor variable X (i.e., X→Y), this article develops a new conceptualization of moderation effects that leads to 2 ways of defining new measures of moderation effects size. One is by using regression models that include the moderator, the predictor, and the product term sequentially. The other is based on a variance decomposition of the outcome variable Y. These new effect size measures effectively differentiate the role of the predictor variable from that of the moderator variable. Two empirical examples are provided to contrast the new measures against the traditional ΔR2 and f2, and to illustrate the applications of the new ones. R code is also provided for researchers to compute the new effect size measures. Translational Abstract Effect size plays an important role in quantifying findings in empirical research. Existing measures of effect sizes in moderation analysis changed the concept of moderation to interaction, and the homoscedasticity assumption with the moderated multiple regression model is also difficult to meet when an interaction effect truly exists. Based on the conceptual model for moderation analysis, this article proposed that a more appropriate baseline variance in evaluating a moderation effect should focus on the "varying relationship" between the predictor and the response variable. The proposal leads to four new measures of moderation effect. These new effect-size measures effectively differentiate the role of the predictor variable from that of the moderator variable and closely match the concept of moderation effect. Therefore, the new measures quantify the size of moderation effect more accurately and allow researchers to appreciate moderation effects from different perspectives.