Performance Evaluation Efforts and Standardization of Biometric Systems

Biometrics is translated into Japanese as "Seitai-Ninshou" or "Shintai-Ninshou". This article gives an overview of the history of developments in the performance evaluation methodology for biometrics. It starts with a description of standardization activities for interoperable data formats and performance evaluation testing technologies developed by NIST and ANSI in USA, followed by international standardization activities related to biometric performance evaluation described in ISO/IEC JTC1 SC37. Although the term "performance evaluation" may remind readers of benchmark tests, the contents of the developed standard provide a framework for biometric system performance evaluation rather than test specifications for benchmarks. Published international standards and technical reports and those under development in SC37 are briefly referred to. Then several topics such as repeatability and reproducibility, a data sampling method without biasing, and the stochastic estimation of performance metrics using a sample data set are outlined as important issues. Confusion and misunderstanding regarding translated Japanese technical terms caused by various domain roots are mentioned. Finally, unsolved problems expected to be researched in future will be discussed.