Evaluation of a Medical Faculty Development Program: A Comparison of Traditional Pre/Post and Retrospective Pre/Post Self-Assessment Ratings

Pre- and postintervention self-assessments are commonly used to evaluate educational interventions. However, when training influences participants'criteria for their self-ratings (response shift), the validity of the traditional prefpost comparisons is suspect. We assessed the influence of this phenomenon in a national faculty development program for clinical teachers. We compared changes in traditional pre/post self-assessment ratings with changes in retrospective pre/post self-assessment ratings. Data included prelpost intervention faculty self-assessments and evaluations of faculty by housestaff and students. On dimensions addressed in the training program, retrospective pre/post comparisons revealed more significant changes in teaching performance and attitudes than traditional pre/post comparisons. Housestaff and student evaluations were more consistent with thefaculty 's retrospective pre/post than with traditional prelpost comparisons. We conclude that, compared to traditional prelpost self-assessment; retrospective pre/post ratings may provide a more sensitive and more valid measure of the effects offaculty development.