Can Unannounced Standardized Patients Assess Professionalism and Communication Skills in the Emergency Department?

Objectives: The authors piloted unannounced standardized patients (USPs) in an emergency medicine (EM) residency to test feasibility, acceptability, and performance assessment of professionalism and communication skills. Methods: Fifteen postgraduate year (PGY)‐2 EM residents were scheduled to be visited by two USPs while working in the emergency department (ED). Multidisciplinary support was utilized to ensure successful USP introduction. Scores (% well done) were calculated for communication and professionalism skills using a 26‐item, behaviorally anchored checklist. Residents’ attitudes toward USPs and USP detection were also surveyed. Results: Of 27 USP encounters attempted, 17 (62%) were successfully completed. The detection rate was 44%. Eighty‐three percent of residents who encountered a USP felt that the encounter did not hinder daily practice and did not make them uncomfortable (86%) or suspicious of patients (71%). Overall, residents received a mean score of 60% for communication items rated “well done” (SD ± 28%, range = 23%–100%) and 53% of professionalism items “well done” (SD ± 20%, range = 23%‐85%). Residents’ communication skills were weakest for patient education and counseling (mean = 43%, SD ± 31%), compared with information gathering (68%, SD ± 36% and relationship development (62%, SD ± 32%). Scores of residents who detected USPs did not differ from those who had not. Conclusions: Implementing USPs in the ED is feasible and acceptable to staff. The unpredictability of the ED, specifically resident schedules, accounted for most incomplete encounters. USPs may represent a new way to assess real‐time resident physician performance without the need for faculty resources or the bias introduced by direct observation.