Evaluation of three machine learning models for self-referral decision support on low back pain in primary care

•Exploring the possibilities of using supervised machine learning in the design of a CDSS to support patients with LBP in their self-referral to primary care.•One way to prevent acute LBP from transiting into chronic LBP is to ensure that patients receive the right interventions at the right moment starting with helping patients with a new episode of LBP on what to do.•We show promising possibilities of using machine learning for this application and provide information on how these results can be extended in future research. Most people experience low back pain (LBP) at least once in their life and for some patients this evolves into a chronic condition. One way to prevent acute LBP from transiting into chronic LBP, is to ensure that patients receive the right interventions at the right moment. We started research in the design of a clinical decision support system (CDSS) to support patients with LBP in their self-referral to primary care. For this, we explored the possibilities of using supervised machine learning. We compared the performances of the three classification models − i.e. 1. decision tree, 2. random forest, and 3. boosted tree − to get insight in which model performs best and whether it is already acceptable to use this model in real practice. The three models were generated by means of supervised machine learning with 70% of a training dataset (1288 cases with 65% GP, 33% physio, 2% self-care cases). The cases in the training dataset were fictive cases on low back pain collected during a vignette study with primary healthcare professionals. We also wanted to know the performance of the models on real-life low back pain cases that were not used to train the models. Therefore we also collected real-life cases on low back pain as test dataset. These cases were collected with the help of patients and healthcare professionals in primary care. For each model, the performance was measured during model validation − with 30% of the training dataset −as well as during model testing − with the test dataset containing real-life cases. The total observed accuracy as well as the kappa, and the sensitivity, specificity, and precision were used as performance measures to compare the models. For the training dataset, the total observed accuracies of the decision tree, the random forest and boosted tree model were 70%, 69%, and 72% respectively. For the test dataset, the total observed accuracies were 71%, 53%, and 71% respectively. The boosted tree appeared