Evaluation of an Opioid Overdose Composite Risk Score Cutoff in Active Duty Military Service Members

Abstract Objective To evaluate the current cutoff score and a recalibrated adaptation of the Veterans Health Administration (VHA) Risk Index for Serious Prescription Opioid-Induced Respiratory Depression or Overdose (RIOSORD) in active duty service members. Design Retrospective case-control. Setting Military Health System. Subjects Active duty service members dispensed ≥ 1 opioid prescription between January 1, 2018, and December 31, 2019. Methods Service members with a documented opioid overdose were matched 1:10 to controls. An active duty-specific (AD) RIOSORD was constructed using the VHA RIOSORD components. Analyses examined the risk stratification and predictive characteristics of two RIOSORD versions (VHA and AD). Results Cases (n = 95) were matched with 950 controls. Only 6 of the original 17 elements were retained in the AD RIOSORD. Long-acting or extended-release opioid prescriptions, antidepressant prescriptions, hospitalization, and emergency department visits were associated with overdose events. The VHA RIOSORD had fair performance (C-statistic 0.77, 95% CI 0.75, 0.79), while the AD RIOSORD did not demonstrate statistically significant performance improvement (C-statistic 0.78, 95% CI, 0.77, 0.80). The DoD selected cut point (VHA RIOSORD > 32) only identified 22 of 95 ORD outcomes (Sensitivity 0.23), while an AD-specific cut point (AD RIOSORD > 16) correctly identified 53 of 95 adverse events (Sensitivity 0.56). Conclusions Results highlight the need to continually recalibrate predictive models and to consider multiple measures of performance. Although both models had similar overall performance with respect to the C-statistic, an AD-specific index threshold improves sensitivity. The calibrated AD RIOSORD does not represent an end-state, but a bridge to a future model developed on a wider range of patient variables, taking into consideration features that capture both care received, and care that was not received.