Assumption Violations and Rates of Type I Error for the Tukey Multiple Comparison Test: A Review and Empirical Investigation via a Coefficient of Variance Variation

While numerous investigations have examined the effects of assumption violations on the empirical probability of a Type I error for Tukey's multiple comparison test, no study to date has numerically quantified and systematically varied the degree of total variation resulting from combining unequal variances with unequal sample sizes. The present investigation employed a coefficient of variance variation to index the degree of heterogeneity and compared the effects of varying degrees of heterogeneity on the harmonic mean, Kramer (11) and Miller (14) unequal group forms of the Tukey test. The discrepancies between the empirical and nominal significance rates of Type I error were related to a bias ratio provided by Box (1) and were found to markedly vary as a function of the magnitude of this ratio. The Kramer unequal group form of the Tukey test is recommended, as it consistently resulted in empirical Type I rates of error deviating less from the nominal significance level than either of the other two unequal$\text{n}_{\text{k}}$forms. In addition, even when sample sizes were equal, the rate of Type I error was seriously inflated, when the degree of variance heterogeneity was large.