Does weekly direct observation and formal feedback improve intern patient care skills development? A randomized controlled trial

Direct observation (DO) is considered to be an effective way to evaluate patient care. This study assesses if weekly direct observation with formal feedback (DO-FF) increases (1) clinical skills and (2) comfort with patient care skills (CWPCS) during the first 12 weeks of internship. A single-blinded, stratified, randomized controlled trial (RCT) with allocation concealment incorporating interns at a community hospital family medicine residency program was performed. Interns (n=14) were stratified by the predicted number of calls in a 2:1 ratio of intervention: control group. The intervention group received DO-FF four times/month on inpatient history and physicals (H&Ps) by a family medicine senior resident or fellow. To assess skills, all interns were videotaped doing H&Ps at the beginning, middle, and end of the study. These were scored by two independent, blinded physicians using a validated tool. For self-assessment, all interns took a patient care comfort survey (PCCS) at baseline, 6 weeks, and 12 weeks. Analysis was done via intention to treat. At 6 weeks, interns who received weekly DO-FF had greater CWPCS on 32 of 35 questions. Eight were significantly different: (1) discussing end of life issues, (2) chest X ray interpretation, management of (3) congestive heart failure, (4) chronic obstructive pulmonary disease, (5) diabetes mellitus, (6) diabetic ketoacidosis, (7) stroke, and (8) venous thromboembolism/pulmonary embolism. In all but three questions, differences resolved by 12 weeks. Video data were not significantly different between groups. In certain aspects of patient care, DO-FF improved intern CWPCS at 6 weeks. There was no difference in the video assessments of clinical skills between the treatment groups. These results need to be further explored. However, this study showed that a well-designed RCT is feasible for educational research questions.