An Artificial Intelligence Algorithm to Identify Documented Symptoms in Patients with Heart Failure who Received Cardiac Resynchronization Therapy (GP757)

Objectives Describe the importance of symptom identification for patients with congestive heart failure (CHF) undergoing cardiac resynchronization therapy (CRT). Describe the performance of GraphIE, a novel artificial intelligence (AI) algorithm, in identifying documented symptoms in a cohort of patients with CHF undergoing cardiac resynchronization therapy (CRT). Importance. Clinicians lack reliable methods to predict which patients with congestive heart failure (CHF) will benefit from cardiac resynchronization therapy (CRT). Symptom burden possibly predicts response, but this information is buried in free-text clinical notes. Artificial intelligence (AI) tools to identify symptoms in the electronic health record (EHR) may allow for the structured inclusion of these data into pre-procedural decision-making. Objective(s). To develop, train, and test an AI algorithm that identifies free-text documented symptoms in a cohort of patients with CHF receiving CRT. Method(s). We identified a random sample of clinical notes from a cohort of patients with CHF who received CRT. Investigators labeled documented symptoms as present, absent, and context-dependent (pathologic depending on the clinical situation). The algorithm trained on 80% and fine-tuned parameters on 10% of the notes. We tested the model on the remaining 10%. We compared the model's performance to investigators' annotations using accuracy, precision (positive predictive value), recall (sensitivity), and F1 score (a combined measure of precision and recall). Results. Investigators annotated 154 clinical notes (352,157 words) and identified 1340 present, 1300 absent, and 221 context-dependent symptoms. In the test set of 15 notes (35,467 words), the model's accuracy was 99.4% and recall was 66.8%. Precision was 77.6%. Consistent with established standards, overall F1 score was 71.8. F1 scores for present (70.8) and absent (74.7) symptoms were higher than that for context-dependent symptoms (48.3). Conclusion(s). An AI algorithm can be trained to capture symptoms in CHF patients who received CRT with satisfactory precision and recall. Impact. AI-based methods hold promise to efficiently extract symptoms from clinical notes. Future research will evaluate whether this methodology can predict which CHF patients will experience improvements in ejection fraction and quality of life after CRT.