Summary-Statistics-Based Power Analysis: A New and Practical Method to Determine Sample Size for Mixed-Effects Modeling

This article proposes a summary-statistics-based power analysis-a practical method for conducting power analysis for mixed-effects modeling with two-level nested data (for both binary and continuous predictors), complementing the existing formula-based and simulation-based methods. The proposed method bases its logic on conditional equivalence of the summary-statistics approach and mixed-effects modeling, paring back the power analysis for mixed-effects modeling to that for a simpler statistical analysis (e.g., one-sample t test). Accordingly, the proposed method allows us to conduct power analysis for mixed-effects modeling using popular software such as G*Power or the pwr package in R and, with minimum input from relevant prior work (e.g., t value). We provide analytic proof and a series of statistical simulations to show the validity and robustness of the summary-statistics-based power analysis and show illustrative examples with real published work. We also developed a web app (https://koumurayama.shinyapps.io/summary_statistics_based_power/) to facilitate the utility of the proposed method. While the proposed method has limited flexibilities compared with the existing methods in terms of the models and designs that can be appropriately handled, it provides a convenient alternative for applied researchers when there is limited information to conduct power analysis. Translational AbstractFor applied researchers, statistical power analysis with mixed-effects modeling (or multilevel modeling) poses a big challenge, because it requires substantive expertise on modeling, use of special software, and a number of input parameters which are usually not available in published work. In fact, despite the number of research articles on this topic, we found that applied researchers rarely use appropriate statistical power analysis recommended by experts. To improve the current state of practice, this article proposes an easy and practical method to conduct statistical power analysis for mixed-effects modeling, called summary-statistics-based power analysis. While the proposed method has limited flexibilities (e.g., it can be applied only to two-level nested data), it has greater advantages over traditional power-analysis methods (formula- and simulation-based power analyses) in terms of usability and practicality. In fact, the proposed method can determine appropriate Level-2 sample size of a new study by using only a t value and Level-2 sample size from a previous