A Standardized Patient Methodology to Assess Cognitive-Behavioral Therapy (CBT) Skills Performance: Development and Testing in a Randomized Controlled Trial of Web-Based Training

Competency-based training is increasingly important in mental health as practitioners are trained in the use of evidence-based treatments for posttraumatic stress disorder (PTSD) and other disorders. Clinical skills according to this model are assessed in various ways, including the use of standardized patient (SP) interviews. This study investigated the reliability and feasibility of using a telephone-based SP interview to evaluate cognitive-behavioral therapy (CBT) skills within a large, randomized trial of online training. This paper describes the development of a standardized SP methodology for the larger study, including case illustration, SP training, development of trainee skills assessment metrics, and rater training. A post hoc substudy evaluated SP performance across time. For the larger study, 2 out of 3 skills assessment metrics showed satisfactory performance and demonstrated consistently good interitem and interrater reliability. The substudy found that SPs were adherent and faithful to their intended character, with little evidence of performance drift. Based on these findings, telephone SP methodologies are a promising technology for large-scale clinical skills assessment in psychology.