A simplified observation tool for residents in the outpatient clinic

The Accreditation Council for Graduate Medical Education promotes direct observation of residents as a key assessment tool for competency in patient care, professionalism, and communication skills. Although tools exist, validity and reliability have not been demonstrated for most, and many tools may have limited feasibility because of time constraints and other reasons. We conducted a study to measure feasibility of a simplified observation tool to evaluate these competencies and provide timely feedback. In the pediatric resident continuity clinic of a large children's hospital, we used a direct observation form with a 3-point scale for 16 items in the domains of patient care, professionalism, and communication skills. The form was divided by portion of visit, with specific items mapped to 1 or more of the competencies, and was used to provide direct oral feedback to the resident. Faculty and residents completed surveys rating the process (ease of use, satisfaction, and self-assessed usefulness) on a 5-point Likert scale. The study encompassed 89 surveys completed by attending physicians; 98% (87 of 89) of the time the form was easy to use, 99% (88) of the time its use did not interfere with patient flow, and 93% (83) of the observations provided useful information for resident feedback. Residents completed 70 surveys, with the majority (69%, 48) reporting they were comfortable about being observed by an attending physician; 87% (61) thought that direct observation did not significantly affect their efficiency. Ninety-seven percent of the time (68) residents reported that direct observation provided useful feedback. The data suggest the form was well-received by both faculty and residents, and enabled attending physicians to provide useful feedback.