Automated Metrics in a Virtual-Reality Myringotomy Simulator: Development and Construct Validity

OBJECTIVES:The objectives of this study were1) to develop and implement a set of automated performance metrics into the Western myringotomy simulator, and 2) to establish construct validity. STUDY DESIGN:Prospective simulator-based assessment study. SETTING:The Auditory Biophysics Laboratory at Western University, London, Ontario, Canada. PARTICIPANTS:Eleven participants were recruited from the Department of Otolaryngology–Head & Neck Surgery at Western Universityfour senior otolaryngology consultants and seven junior otolaryngology residents. INTERVENTIONS:Educational simulation. MAIN OUTCOME MEASURE:Discrimination between expert and novice participants on five primary automated performance metrics1) time to completion, 2) surgical errors, 3) incision angle, 4) incision length, and 5) the magnification of the microscope. METHODS:Automated performance metrics were developed, programmed, and implemented into the simulator. Participants were given a standardized simulator orientation and instructions on myringotomy and tube placement. Each participant then performed 10 procedures and automated metrics were collected. The metrics were analyzed using the Mann–Whitney U test with Bonferroni correction. RESULTS:All metrics discriminated senior otolaryngologists from junior residents with a significance of p