Triaging ophthalmology outpatient referrals with machine learning: A pilot study

Importance Triaging of outpatient referrals to ophthalmology services is required for the maintenance of patient care and appropriate resource allocation. Machine learning (ML), in particular natural language processing, may be able to assist with the triaging process. Background To determine whether ML can accurately predict triage category based on ophthalmology outpatient referrals. Design Retrospective cohort study. Participants The data of 208 participants was included in the project. Methods The synopses of consecutive ophthalmology outpatient referrals at a tertiary hospital were extracted along with their triage categorizations. Following pre‐processing, ML models were applied to determine how accurately they could predict the likely triage categorization allocated. Data was split into training and testing sets (75%/25% split). ML models were tested on an unseen test set, after development on the training dataset. Main Outcome Measure Area under the receiver operator curve (AUC) for category one vs non‐category one classification. Results For the main outcome measure, convolutional neural network (CNN) provided the best AUC (0.83) and accuracy on the test set (0.81), with the artificial neural network (AUC 0.81 and accuracy 0.77) being the next best performing model. When the CNN was applied to the classification task of identifying which referrals should be allocated a category one vs category two vs category three priority, a lower accuracy was achieved (0.65). Conclusions and Relevance ML may be able to accurately assist with the triaging of ophthalmology referrals. Future studies with data from multiple centres and larger sample sizes may be beneficial.