Tracking the randomized rollout of a Veterans Affairs opioid risk management tool: A multi-method implementation evaluation using the Consolidated Framework for Implementation Research (CFIR)

Background The Veterans Health Administration (VHA) developed the Stratification Tool for Opioid Risk Mitigation (STORM) dashboard to assist in identifying Veterans at risk for adverse opioid overdose or suicide-related events. In 2018, a policy was implemented requiring VHA facilities to complete case reviews of Veterans identified by STORM as very high risk for adverse events. Nationally, facilities were randomized in STORM implementation to four arms based on required oversight and by the timing of an increase in the number of required case reviews. To help evaluate this policy intervention, we aimed to (1) identify barriers and facilitators to implementing case reviews; (2) assess variation across the four arms; and (3) evaluate associations between facility characteristics and implementation barriers and facilitators. Method Using the Consolidated Framework for Implementation Research (CFIR), we developed a semi-structured interview guide to examine barriers to and facilitators of implementing the STORM policy. A total of 78 staff from 39 purposefully selected facilities were invited to participate in telephone interviews. Interview transcripts were coded and then organized into memos, which were rated using the −2 to + 2 CFIR rating system. Descriptive statistics were used to evaluate the mean ratings on each CFIR construct, the associations between ratings and study arm, and three facility characteristics (size, rurality, and academic detailing) associated with CFIR ratings. We used the mean CFIR rating for each site to determine which constructs differed between the sites with highest and lowest overall CFIR scores, and these constructs were described in detail. Results Two important CFIR constructs emerged as barriers to implementation: Access to knowledge and information and Evaluating and reflecting. Little time to complete the CASE reviews was a pervasive barrier. Sites with higher overall CFIR scores showed three important facilitators: Leadership engagement, Engaging, and Implementation climate. CFIR ratings were not significantly different between the four study arms, nor associated with facility characteristics. Plain Language Summary: The Veterans Health Administration (VHA) created a tool called the Stratification Tool for Opioid Risk Mitigation dashboard. This dashboard shows Veterans at risk for opioid overdose or suicide-related events. In 2018, a national policy required all VHA facilities to complete case reviews for Veterans who wer