Recommendations to address and research systemic bias in assessment: perspectives from directors of research in medical education

Addressing systemic bias in medical school assessment is an urgent task for medical education. This paper outlines recommendations on topic areas for further research on systemic bias, developed from a workshop discussion at the 2023 annual meeting of the Society of Directors of Research in Medical Education. During the workshop, directors engaged in small-group discussions on guidelines to address bias in assessment practices following a proposed categorization of 'Do's,' 'Don'ts,' and 'Don't knows' and listed their insights using anonymous sticky notes, which were shared and discussed with the larger group of participants. The authors performed a content analysis of the notes through deductive and inductive coding. We reviewed and discussed our analysis to reach consensus. The workshop included 31 participants from 28 institutions across the US and Canada, generating 51 unique notes. Participants identified 23 research areas in need of further study. The inductive analysis of proposed research areas revealed four main topics: 1) The role of interventions, including pre-medical academic interventions, medical-education interventions, assessment approaches, and wellness interventions; 2) Professional development, including the definition and assessment of professionalism and professional identity formation; 3) Context, including patient care and systemic influences; and 4) Research approaches. While limited to data from a single workshop, the results offered perspectives about areas for further research shared by a group of directors of medical education research units from diverse backgrounds. The workshop produced valuable insights into the need for more evidence-based interventions that promote more equitable assessment practices grounded in real-world situations and that attenuate the effects of bias.