Post Hoc Quantile Test for One-Way Analysis of Variance Using a Double Bootstrap Method

Comparing samples from two or more populations is among the most common statistical tasks that engineers and scientists perform. Typically a t-test or one-way or two-way analysis of variance is used to compare the means of different populations. Although these tests are useful for describing differences in means for various populations, they are of limited use for comparison of other population parameters, such as quantiles (i.e., percentiles). Quantile comparisons are useful for examining the changes that occur to portions of the population away from the median. Comparison of speed percentiles is especially important to the traffic engineering profession to determine the effect of various treatments on speeds of faster drivers (e.g., 85th percentile speeds). Simply testing the mean speed values does not indicate whether the differences occur because of slower traffic driving more slowly, faster traffic driving more slowly, or both. Furthermore, are positive effects on faster drivers being masked by contrasting effects on slower drivers? The main obstacle to providing confidence intervals and tests for quantiles is calculating reasonable estimates of variances for the sample quantiles that are far from the median. This paper describes a nonparametric double bootstrapping procedure for direct comparison of quantiles of two or more sample populations. The first bootstrap simulation is used to produce estimates of standard errors for the desired quantiles and thereby overcome the inability to make reasonable variance estimations. The second layer of bootstrap simulations is used to determine the threshold cutoff values based on a desired level of confidence for the test of hypothesis. The cutoff values also may be used to form confidence intervals. The steps of the procedure along with an example of its use are provided.