Estimating Trait and Situational Variance in a Situational Judgment Test

In organizational research, situational judgment tests (SJTs) consistently demonstrate incremental validity, yet our theoretical understanding of SJTs is limited. Our knowledge could be advanced by decomposing the variance of SJT items into trait variance and situation variance; we do that by applying statistical methods used to analyze multitrait-multimethod matrices. A college-student sample (N = 2,747) was administered an SJT of goal orientation traits (i.e., mastery, performance-approach, and performance-avoid). Structural equation modeling was used to estimate the proportions of item variance to attributable to situational differences (across students) and to trait-based differences in students (across situations). Situation factors accounted for over three times the amount of variance as did individual difference factors. We conclude with general implications for the design of SJTs in organizational research.