Evidence-Based Determination of Cut-Off Points for Increased Cardiac-Surgery Mortality Risk With EuroSCORE II and STS: The Best-Performing Risk Scoring Models in a Single-Centre Australian Population

Risk scoring models (RSMs) are commonly used for estimation of postoperative-mortality risk in patients undergoing cardiac surgery, but their prediction accuracy may vary in different populations and clinical situations. The prognostic accuracies of some RSMs have not yet been fully evaluated in the Australian population. In this retrospective observational study, our aims were to assess the performance of four contemporary RSMs, to identify the best RSMs for prediction of postoperative-mortality in the single-centre cohort, and to determine a statistical threshold for classification of patients with increased or “higher” mortality risk. The study population included patients who underwent cardiac surgery at Liverpool Hospital between January 2013 and December 2014. Demographic information was collected, and mortality risks were estimated with the ES2 (EuroSCORE II), STS (Society of Thoracic Surgeons Score), AS (AusSCORE total) and ASMR (AusSCORE multi-risk) RSMs. (Additive EuroSCORE) (AES) and LES (logistic EuroSCORE) were included for historical interest. Discrimination, the ability to stratify patients between mortality and no mortality outcomes, and calibration, the comparison of risk score estimated and observed outcome in the population, were evaluated for each RSM, to determine their predictive accuracy in the study population. Discrimination was assessed by the AUC (area under the receiver operating characteristic curve), and acceptable calibration by the p-value greater than 0.05 for the Hosmer–Lemeshow (H-L) test. The best AUCs in contempory models were compared using the DeLong test. For ES2 and STS risk scores, cut-off points, or thresholds, for patients at increased risk of mortality were derived using Youden’s J-statistics, calculated from sensitivity and specificity of models in predicting mortality. From a total study population of 898 patients, 738 had scores for all six RSMs. The three EuroSCORE risk models and Youden’s J-statistics analysis included the total population. Of the models in contemporary use, ES2 had higher discrimination (AUC=0.850) in this population than ASMR (AUC=0.767, p=0.024) and AS (AUC=0.739) and non-significantly higher discrimination than STS (AUC=0.806, p=0.19). All contemporary models had acceptable calibration but the older LES (H-L p=0.024) did not. Estimated mortality was closest to observed mortality with the ES2 model. Both AES and LES over predicted mortality. The RSM with the highest discrimination in iso