Standardized Direct Observation Assessment Tool: Using a Training Video

Abstract Background We developed a DVD training tool to educate physicians evaluating emergency residents on accurate Standardized Direct Observation Assessment Tool (SDOT) application. Objective Our goal was to assess whether this training video improved attendings' and senior residents' SDOT use. Methods Participants voluntarily completed SDOT evaluations based on a scripted “test” video. A DVD with “positive” and “negative” scenarios of proper SDOT use was viewed. It included education on appropriate recording of 26 behaviors. The test scenario was viewed again and follow-up SDOTs submitted. Performances by attendings and residents on the pre- and post-test SDOTs were compared. Results Twenty-six attendings and 26 senior residents participated. Prior SDOT experience was noted for 8 attendings and 11 residents. For 20 anchors, participants recorded observed behaviors with statistically significant difference on one each of the pretest (no. 20; p = 0.034) and post-test (no. 14; p = 0.041) SDOTs. On global competency assessments, pretest medical knowledge ( p = 0.016) differed significantly between groups. The training intervention changed one anchor (no. 5; p = 0.035) and one global assessment (systems-based practice; p = 0.031) more negatively for residents. Recording SDOTs with exact agreement occurred 48.73% for attendings pretest and 54.41% post-test; resident scores were 45.86% and 49.55%, respectively. DVD exposure slightly raised attending scores ( p = 0.289) and significantly lowered resident scores ( p = 0.046). Conclusions Exposure to an independently developed SDOT training video tended to raise attending scores, though without significance, while at the same time lowered senior resident scores statistically significantly. Emergency attendings' and senior residents' SDOT scoring rarely differed with significance; about half of anchor behaviors were recorded with exact agreement. This suggests senior residents, with appropriate education, may participate in SDOT assessment.