Comparison of weighted kappa coefficients of multiple binary diagnostic tests done on the same subjects

Sensitivity and specificity are classic parameters to assess and compare the performance of binary diagnostic tests versus a gold standard in a population. Another useful parameter to assess and compare the performance of binary tests is the weighted kappa coefficient, which is defined as a measure of the beyond‐chance agreement between the diagnostic test and the gold standard. In this study, we deduce the maximum likelihood estimators of the weighted kappa coefficients of multiple binary tests and we propose an asymptotic method to compare the weighted kappa coefficients of multiple binary tests with regard to the same gold standard when all of the diagnostic tests are applied to the same sample of patients. We have carried out simulation experiments to study the type I error and the power of the method that we propose when we compared three binary tests. We have applied the results obtained to the diagnosis of coronary disease. Copyright © 2010 John Wiley & Sons, Ltd.