The Development of Performance Evaluation Scales for First-Level Army Civilian Supervisors

The Civilian Leadership Research Program was established in 1987 to improve the leadership of Army civilian personnel. A major component of the research was the development of a set of supervisory selection measures, including a scoreable in-basket exercise, a biographical data instrument, and an instrument for measuring temperament. This report describes the development of a set of supervisory rating scales that will be used to provide criteria against which the selection instruments will be validated. The critical incident approach was used to determine both the performance categories for which scales would be developed and the behavioral anchors that would be used in the scales. A total of 93 first- and second-level supervisors who attended seven critical incident workshops produced 841 incidents. The incidents were categorized by the researchers and then used in a series of retranslation workshops with different groups of supervisors. A total of 13 trial scales emerged from this process and later reviews of the scales. The scales were administered to first-and second- level supervisors of 53 Army civilian supervisors. Analyses of the obtained rating data suggested that a composite score based on the average of the 13 scale ratings would be highly reliable and could be used to validate selection instruments. Rater type comparisons, Critical incidents, Selection instrument validation, Rating scales, Supervisor evaluation.