Improving Valvular Pathologies and Ventricular Dysfunction Diagnostic Efficiency Using Combined Auscultation and Electrocardiography Data: A Multimodal AI Approach

Simple sensor-based procedures, including auscultation and electrocardiography (ECG), can facilitate early diagnosis of valvular diseases, resulting in timely treatment. This study assessed the impact of combining these sensor-based procedures with machine learning on diagnosing valvular abnormalities and ventricular dysfunction. Data from auscultation at three distinct locations and 12-lead ECGs were collected from 1052 patients undergoing echocardiography. An independent cohort of 103 patients was used for clinical validation. These patients were screened for severe aortic stenosis (AS), severe mitral regurgitation (MR), and left ventricular dysfunction (LVD) with ejection fractions ≤ 40%. Optimal neural networks were identified by a fourfold cross-validation training process using heart sounds and various ECG leads, and their outputs were combined using a stacking technique. This composite sensor model had high diagnostic efficiency (area under the receiver operating characteristic curve (AUC) values: AS, 0.93; MR, 0.80; LVD, 0.75). Notably, the contribution of individual sensors to disease detection was found to be disease-specific, underscoring the synergistic potential of the sensor fusion approach. Thus, machine learning models that integrate auscultation and ECG can efficiently detect conditions typically diagnosed via imaging. Moreover, this study highlights the potential of multimodal artificial intelligence applications.