A call for the utilization of consensus standards in the surgical education literature

Abstract Background Assessment methods and theory continue to evolve in the general education literature. Nowhere is this more evident than in the framework of validity methods and concepts. The consensus standards of the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education have changed from “types of validity” (criterion, construct, and content) and “valid instruments,” last used in 1974, to a concept of identifying evidence for the validity of results and the use of those results. The purpose of this study was to evaluate the surgical education literature for the adoption of the current consensus standards. Methods As a representative sample of the surgical educational literature, the validation effort in laparoscopic simulator education was chosen. A MEDLINE search using the terms validity.tw and laparoscop$.tw between 1996 and 2008 (September week 1) yielded 192 citations. All titles and abstracts were reviewed, resulting in 47 studies appropriate for in-depth analysis. Results Validation studies have evaluated 21 different simulators. Twenty-three percent of the studies adhere, in part, to the new consensus standards for validity. One hundred percent use the old framework of types of validity including 75% using construct validity, 38% using face validity, and 11% using content. Conclusion The widespread use of the currently (after 1999) accepted framework for validity is lacking in the surgical education literature. Surgical educators must remain current and begin to investigate our assessments within the contemporary framework of validity to avoid improper judgments of performance.