Improving the National Board of Medical Examiners Internal Medicine Subject Exam for Use in Clerkship Evaluation

OBJECTIVE: To provide a consensus opinion on modifying the National Board of Medical Examiners (NBME) Medicine Subject Exam (Shelf) to: 1) reflect the internal medicine clerkship curriculum, developed by the Society of General Internal Medicine (SGIM) and the Clerkship Directors in Internal Medicine (CDIM); 2) emphasize knowledge important for a clerkship student; and 3) obtain feedback about students' performances on the Shelf. DESIGN: Two‐round Delphi technique. PARTICIPANTS: The CDIM Research and Evaluation Committee and CDIM members on NBME Step 2 Committees. MEASUREMENTS: Using 1–5 Likert scales (5 = highest ratings), the group rated test question content for relevance to the SGIM–CDIM Curriculum Guide and importance for clerkship students' knowledge. The Shelf content is organized into 4 physician tasks and into 11 sections that are generally organ system based. Each iteration of the Shelf has 100 questions. Participants indicated a desired distribution of questions by physician task and section, topics critical for inclusion on each exam, and new topics to include. They specified the types of feedback clerkship directors desired on students' performances. Following the first round, participants viewed pooled results prior to submitting their second‐round responses. RESULTS: Of 15 individuals contacted, 12 (80%) participated in each round. The desired distribution by physician task was: diagnosis (43), treatment (23), mechanism of disease (20), and health maintenance (15). The sections with the most questions requested were the cardiovascular (17), respiratory (15), and gastroenterology (12) sections. The fewest were requested in aging/ethics (4) and neurology, dermatology, and immunology (5 each). Examples of low‐rated content were Wilson's Disease, chancroid and tracheal rupture (all