External validation of Acute Physiology and Chronic Health Evaluation IV in Dutch intensive care units and comparison with Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II

Abstract Purpose The aim of this study was to validate and compare the performance of the Acute Physiology and Chronic Health Evaluation (APACHE) IV in the Dutch intensive care unit (ICU) population to the APACHE II and Simplified Acute Physiology Score (SAPS) II. Materials and Methods This is a prospective study based on data from a national quality registry between 2006 and 2009 from 59 Dutch ICUs. The validation set consisted of 62 737 patients; the 3 models were compared using 44 112 patients. Measures of discrimination, accuracy, and calibration (area under the receiver operating characteristic curve (AUC), Brier score, R2 , and Ĉ-statistic) were calculated using bootstrapping. In addition, the standardized mortality ratios were calculated. Results The original APACHE IV showed good discrimination and accuracy (AUC = 0.87, Brier score = 0.10, R2 = 0.29) but poor calibration (Ĉ-statistic = 822.67). Customization significantly improved the performance of the APACHE IV. The overall discrimination and accuracy of the customized APACHE IV were statistically better, and the overall Ĉ-statistic was inferior to those of the customized APACHE II and SAPS II, but these differences were small in perspective of clinical use. Conclusions The 3 models have comparable capabilities for benchmarking purposes after customization. Main advantage of APACHE IV is the large number of diagnoses that enable subgroup analysis. The APACHE IV coronary artery bypass grafting (CABG) model has a good performance in the Dutch ICU population and can be used to complement the 3 models.