Facilitating Peer Evaluation in Team Contexts: The Impact of Frame-of-Reference Rater Training

This study extends research on peer evaluations by examining the impact of Frame-of-Reference rater training on team members' understanding of a specific model of teamwork, as well as on the quality of peer evaluations in a team-performance context. We examine whether participants who have completed Frame-of-Reference training can better understand and apply a model of team-member performance and provide higher quality peer ratings in an interactive team exercise than those who do not receive training. In studies at two universities, we found that trained participants are better able to classify team-member behaviors into the appropriate categories of a science-based model of teamwork and more accurately identify the performance level of those behaviors. Using round-robin data from students working on a highly interdependent task, we also utilized the social relations model (SRM) to test whether trained participants provide higher quality peer ratings. The SRM analysis estimates the variance in team members' ratings of teammates' performance attributable to various effects (rater, group, error, etc.). We found that trained groups had nearly twice the proportion of rating variance attributable to target effects (the people being rated) as untrained groups. We highlight implications of these results for practice and research.