Establishing and measuring treatment fidelity of a complex cognitive rehabilitation intervention: The multicontext approach

Introduction This article describes the process of establishing and measuring treatment fidelity of a complex cognitive rehabilitation intervention that uses a metacognitive strategy framework (the multicontext approach). We adapted treatment guidelines for use within an inpatient rehabilitation setting for people with acquired brain injury; explored training methods and therapist perspectives; developed and examined inter-rater reliability of a fidelity measure; and examined adherence to and proficiency of treatment methods. Method Therapist perspectives of the intervention and training were obtained from written questionnaires and reflections. Inter-rater reliability of 21 treatment components across 22 video-recorded treatment sessions was assessed by examining absolute agreement between two raters using Cohen’s kappa. The proportion of treatment components implemented in individual sessions as well as average proficiency ratings was calculated. Results The fidelity measure demonstrated good inter-rater agreement, ranging from 91–100% for treatment adherence and kappa of .77 to .94 for therapist proficiency. Adherence and proficiency varied between therapists but increased to nearly proficient levels with adjustments to training and procedures. Therapists highlighted the critical importance of video review as a training tool. Conclusion The fidelity measure provides a clinical tool for therapist self-reflection, supervision and training, as well as a tool for future research.