Improving Variance Estimation in Biometric Systems

Measuring system performance seems conceptually straightforward. However, the interpretation of the results and predicting future performance remain as exceptional challenges in system evaluation. Robust experimental design is critical in evaluation, but there have been very few techniques to check designs for either overlooked associations or weak assumptions. For biometric & vision system evaluation, the complexity of the systems make a thorough exploration of the problem space impossible - this lack of verifiability in experimental design is a serious issue. In this paper, we present a new evaluation methodology that improves the accuracy of variance estimator via the discovery of false assumptions about the homogeneity of cofactors - i.e., when the data is not ''well mixed". The new methodology is then applied in the context of a biometric system evaluation with highly influential cofactors.