Statistical significance versus fit: estimating the importance of individual factors in ecological analysis of variance

Although analysis of variance (ANOVA) is widely used by ecologists, the full potential of ANOVA as a descriptive tool has not been realized in most ecological studies. As questions addressed by ecologists become more complex, and experimental and sampling designs more complicated, it is necessary for ecologists to estimate both statistical significance and fit when comparing the relative importance of individual factors in an explanatory model, especially when models are multi-factorial. Yet, with few exceptions, ecologists are only presenting significance values with ANOVA results. Here we review methods for estimating statistical fit (magnitude of effect) for individual ANOVA factors based on variance components and provide examples of their application to field data. Furthermore, we detail the potential occurrence of negative variance components when determining magnitude of effects in ANOVA and describe simple remediation procedures. The techniques we advocate are efficient and will greatly enhance analyses of a wide variety of ANOVA models used in ecological studies. Estimation of magnitude of effects will particularly benefit the analysis of complex multi-factorial ANOVAs where emphasis is on interpreting the relative importance of many individual factors.