Reliability of exercise ratings in the leaderless group discussion

The demonstrated low correlations of dimension ratings across assessment centre exercises has led to reservations about their use. A proposed alternative is the use of overall exercise ratings that may represent total performance on work simulations. Three forms of reliability of such exercise ratings were studied in this laboratory experiment. The study design incorporated four groups of assessors and 31 participants in multiple Leadership Group Discussion (LGD) problems. (LGD was chosen because it is extensively used in assessment centres.) Inter‐rater reliability, as measured by intra‐class correlations within an assessor group, ranged from .69 to .99. Intergroup reliability, as measured by correlations of consensus ratings between assessor groups that had observed identical LGD groups, was .66 to .84. Alternate form reliability, as measured by correlations of overall ratings by the same assessor of participants in two different LGD problems, was .35 to .62. The implications of these findings for the use of overall exercise ratings are discussed.