Utilizing a learning health system to capture real‐world patient data: Application of the reliable change index to evaluate and improve the outcome of a pain rehabilitation program

Background and Objectives The learning healthcare system (LHS) has been developed to integrate patients' clinical data into clinical decisions and improve treatment outcomes. Having little guidance on this integration process, we aim to explain (a) an applicable analytic tool for clinicians to evaluate the clinical outcomes at a group and an individual level and (b) our quality improvement (QI) project, analyzing the outcomes of a new outpatient pain rehabilitation program (“Back‐in‐Action”: BIA) and applying the analysis results to modify our clinical practice. Methods Through our LHS (CHOIR; https://choir.stanford.edu), we administered the Pain Catastrophizing Scale (PCS), Chronic Pain Acceptance Questionnaire (CPAQ), and Patient‐Reported Outcomes Measures (PROMIS)® before and after BIA. After searching for appropriate analytic tools, we decided to use the Reliable Change Index (RCI) to determine if an observed change in the direction of better (improvement) or worse (deterioration) would be beyond or within the measurement error (no change). Results Our RCI calculations revealed that at least a 9‐point decrease in the PCS scores and 10‐point increase in the CPAQ scores would indicate reliable improvement. RCIs for the PROMIS measures ranged from 5 to 8 T‐score points (i.e., 0.5–0.8 SD). When evaluating change scores of the PCS, CPAQ, and PROMIS measures, we found that 94% of patients showed improvement in at least one domain after BIA and 6% showed no reliable improvement. Conclusions Our QI project revealed RCI as a useful tool to evaluate treatment outcomes at a group and an individual level, and RCI could be incorporated into the LHS to generate a progress report automatically for clinicians. We further explained how clinicians could use RCI results to modify a clinical practice, to improve the outcomes of a pain program, and to develop individualized care plans. Lastly, we suggested future research areas to improve the LHS application in pain practice.