Development of a Tool for Evaluating Multimedia for Surgical Education

Background Educational multimedia has been designed to provide surgical trainees with expert operative information outside of the operating theater. The effectiveness of multimedia (e.g., CD-ROMs) for learning has been a common research topic since the 1990s. To date, however, little discussion has taken place on the mechanisms to evaluate the quality of multimedia-driven teaching. This may be because of a lack of research into the development of appropriate tools for evaluating multimedia, especially for surgical education. Methods This paper reports on a small-scale pilot and exploratory study ( n = 12) that developed a tool for surgical multimedia evaluation. The validity of the developed tool was established through adaptation of an existing tool, which was reviewed using experts in surgery, usability, and education. The reliability of the developed tool was tested with surgical trainees who used it to assess a multimedia CD-ROM created for teaching basic surgical skills. Results The findings contribute to an understanding of surgical trainees' experience of using educational multimedia, in terms of characteristics of the learning material for interface design and content and the process of developing evaluation tools, in terms of inclusion of appropriate assessment criteria. Conclusions The increasing use of multimedia in medical education necessitates the development of standardized tools for determining the quality of teaching and learning. Little research exists into the development of such tools and so the present work stimulates discussion on how to evaluate surgical training.