Accuracy and Precision of Typical Quality Measures

Computer simulation studies were performed to evaluate the accuracy and precision of the estimators for three quality measures: percent within limits (PWL), average absolute deviation (AAD), and conformal index (CI). Sample sizes of 3, 5, and 10 were simulated for various levels of actual population PWL, AAD, or CI. The estimators for all three quality measures exhibited similar trends for the variabilities of their respective individual estimates. For each estimator, the variability decreased as the sample size increased. For PWL, the variabilities increased as the actual population PWL departed from either 0 or 100 PWL, peaking at 50 PWL. For AAD and CI, the variabilities increased as the actual population value, AAD or CI, departed from 0. Both PWL and AAD were unbiased estimators of their respective population parameters. However, the CI appeared to be a biased estimator, consistently underestimating the true population CI. Since the CI offers no benefits compared to AAD, and since it appears to be a biased estimator, AAD is preferred to the use of CI. In this study, PWL and AAD exhibited similar trends, so it is difficult to conclude that either method is preferred on the basis of bias and precision. Although there are many other differences between PWL and AAD, since they exhibited similar trends this study shows that the decision to choose PWL or AAD will need to be based on factors other than the bias and precision of their respective estimators.