Use of Machine Learning to Develop a Risk-Stratification Tool for Emergency Department Patients With Acute Heart Failure

We use variables from a recently derived acute heart failure risk-stratification rule (STRATIFY) as a basis to develop and optimize risk prediction using additional patient clinical data from electronic health records and machine-learning models. Using a retrospective cohort design, we identified all emergency department (ED) visits for acute heart failure between January 1, 2017, and December 31, 2018, among adult health plan members of a large system with 21 EDs. The primary outcome was any 30-day serious adverse event, including death, cardiopulmonary resuscitation, balloon-pump insertion, intubation, new dialysis, myocardial infarction, or coronary revascularization. Starting with the 13 variables from the STRATIFY rule (base model), we tested whether predictive accuracy in a different population could be enhanced with additional electronic health record–based variables or machine-learning approaches (compared with logistic regression). We calculated our derived model area under the curve (AUC), calculated test characteristics, and assessed admission rates across risk categories. Among 26,189 total ED encounters, mean patient age was 74 years, 51.7% were women, and 60.7% were white. The overall 30-day serious adverse event rate was 18.8%. The base model had an AUC of 0.76 (95% confidence interval 0.74 to 0.77). Incorporating additional variables led to improved accuracy with logistic regression (AUC 0.80; 95% confidence interval 0.79 to 0.82) and machine learning (AUC 0.85; 95% confidence interval 0.83 to 0.86). We found that 11.1%, 25.7%, and 48.9% of the study population had predicted serious adverse event risk of less than or equal to 3%, less than or equal to 5%, and less than or equal to 10%, respectively, and 28% of those with less than or equal to 3% risk were admitted. Use of a machine-learning model with additional variables improved 30-day risk prediction compared with conventional approaches.