Practical Considerations for Conducting Job Analysis Linkage Exercises

To follow best practices in creating selection tools, an important phase in job analysis is gathering linkage ratings between knowledge, skills, abilities, and other characteristics (KSAOs) and job tasks. However, the literature provides little guidance on best practices for collecting linkage ratings. Two studies were conducted to contribute to the limited research. Study 1 examined the interrater agreement of different types of raters: job incumbents, managers, or job analysts. Results revealed that job analysts have the highest interrater agreement. Study 2 examined the impact of a frame-of-reference (FOR) training for raters. Results suggested that a brief consensus training session vastly improved agreement among raters. These studies provide more guidance on best practices for obtaining high-quality linkage ratings.