Evaluating Measures of Fidelity for Substance Abuse Group Treatment with Incarcerated Adolescents

Abstract The evaluation of treatment fidelity has become increasingly important as the demand for evidence-based practice grows. The purpose of the present study is to describe the psychometric properties of two measures of treatment fidelity that can be used by therapists and supervisors - one for group-based Cognitive-Behavioral Therapy (CBT) and one for combined Substance Education and Twelve-Step Introduction (SET) for adolescent substance use. At the end of group sessions (CBT n = 307; SET n = 279), therapists and supervisors completed an evaluation measure assessing adherence to certain core components of the intervention. The supervisor version of the fidelity measure also included items for rating the level of competency the therapist demonstrated when providing each component of the intervention. Results from split-half cross-validation analyses provide strong support for an 11-item, three-factor CBT fidelity measure. Somewhat less consistent but adequate support for a nine-item, two-factor SET fidelity measure was found. Internal consistencies ranged from acceptable to good for both the CBT and SET adherence scales and from acceptable to good for the CBT and SET competency scales, with the exception of the CBT Practices competency scale. Preliminary validation of the measures suggests that both measures have adequate to strong factor structure, reliability, and concurrent and discriminant validity. The results of this study have implications for research and clinical settings, including the supervision process.