Predicting suicide attempts and suicide deaths among adolescents following outpatient visits

•This study examined whether a model trained and validated using data from all age groups works as well for adolescents or whether it could be improved.•Routinely collected information from electronic medical records and administrative claims can be used to predict adolescent suicide•Prediction models already in operational use by health systems can be reliably employed for identifying adolescents in need of further evaluation. Few studies report on machine learning models for suicide risk prediction in adolescents and their utility in identifying those in need of further evaluation. This study examined whether a model trained and validated using data from all age groups works as well for adolescents or whether it could be improved. We used healthcare data for 1.4 million specialty mental health and primary care outpatient visits among 256,823 adolescents across 7 health systems. The prediction target was 90-day risk of suicide attempt following a visit. We used logistic regression with least absolute shrinkage and selection operator (LASSO) and generalized estimating equations (GEE) to predict risk. We compared performance of three models: an existing model, a recalibrated version of that model, and a newly-learned model. Models were compared using area under the receiver operating curve (AUC), sensitivity, specificity, positive predictive value and negative predictive value. The AUC produced by the existing model for specialty mental health visits estimated in adolescents alone (0.796; [0.789, 0.802]) was not significantly different than the AUC of the recalibrated existing model (0.794; [0.787, 0.80]) or the newly-learned model (0.795; [0.789, 0.801]). Predicted risk following primary care visits was also similar: existing (0.855; [0.844, 0.866]), recalibrated (0.85 [0.839, 0.862]), newly-learned (0.842, [0.829, 0.854]). The models did not incorporate non-healthcare risk factors. The models relied on ICD9-CM codes for diagnoses and outcome measurement. Prediction models already in operational use by health systems can be reliably employed for identifying adolescents in need of further evaluation.