Tests of Significance for Differences in Counts of Rare Events in Two Treatment Groups

Counts of rare events tend to have Poisson or J-shaped distributions that render parametric assumptions questionable. Six different methods for testing the significance of the difference between location parameters for two such distributions are evaluated in this article. A binomial test for the difference between means is known to be the most powerful unbiased test when population distributions are truly Poisson; however, it appears extremely non-robust against departures from the distributional assumption. Robust enough to provide appropriate protection against Type I error, Student's t test was among the most powerful of the tests when applied to counts of rare events in two treatment groups. The Mann-Whitney sum of ranks test also provided superior alpha protection and power even where a large number of tied ranks occurred in the zero-count category of a J-shaped distribution. Finally, a 1 df chi-square test for linear shift in proportional representation across frequency categories of one group as opposed to the other was the most powerful of three chi-square tests that were evaluated.