The Pairwise Multiple Comparison Multiplicity Problem: An Alternative Approach to Familywise and Comparisonwise Type I Error Control

When undertaking many tests of significance, researchers are faced with the problem of how best to control the probability of committing a Type I error. The familywise approach deals directly with multiplicity problems by setting a level of significance for an entire set of related hypotheses; the comparison approach ignores the issue by setting the rate of error on each individual hypothesis. A new formulation of control, the false discovery rate, does not provide control as stringent as that of the familywise rate, but concomitant with this relaxation in stringency is an increase in sensitivity to detect effects relative to the sensitivity of familywise control. Type I error and power rates for 4 relatively powerful and easily computed pairwise multiple comparison procedures were compared with the false discovery rate procedure for various 1-way layouts by use of test statistics that do not assume variance homogeneity.