Development and internal validation of prediction models for future hospital care utilization by patients with multimorbidity using electronic health record data

ObjectiveTo develop and internally validate prediction models for future hospital care utilization in patients with multiple chronic conditions.DesignRetrospective cohort study.SettingA teaching hospital in the Netherlands (542 beds).ParticipantsAll adult patients (n = 18.180) who received care at the outpatient clinic in 2017 for two chronic diagnoses or more (including oncological diagnoses) and who returned for hospital care or outpatient clinical care in 2018. Development and validation using a stratified random split-sample (n = 12.120 for development, n = 6.060 for internal validation).Outcomes≥2 emergency department visits in 2018, ≥1 hospitalization in 2018 and ≥12 outpatient visits in 2018.Statistical analysisMultivariable logistic regression with forward selection.ResultsEvaluation of the models' performance showed c-statistics of 0.70 (95% CI 0.69-0.72) for the hospitalization model, 0.72 (95% CI 0.70-0.74) for the ED visits model and 0.76 (95% 0.74-0.77) for the outpatient visits model. With regard to calibration, there was agreement between lower predicted and observed probability for all models, but the models overestimated the probability for patients with higher predicted probabilities.ConclusionsThese models showed promising results for further development of prediction models for future healthcare utilization using data from local electronic health records. This could be the first step in developing automated alert systems in electronic health records for identifying patients with multimorbidity with higher risk for high healthcare utilization, who might benefit from a more integrated care approach.