Same as it ever was: A clarification on the sources of predictable variance in job performance ratings

The heart of their argument is: (a) there are multiple sources of variance in the ratings that are widely used as criteria in estimating validity, (b) generalizability theory gives us a tool for partitioning these sources of variance, (c) person (e.g., ratee) main effects are the only source of reliable between-ratee variance in performance ratings that is predictable, (d) existing work on partitioning variance gives estimates of person main effect variance as accounting for about 25% of rating variance, (e) we should rescale our validity estimates as a percentage of this 25% possible explainable variance, (f) and by doing so we find that our predictors explain a larger proportion of explainable variance in job performance ratings. [...]a subset of these components reflects between-person variance that is consistent across raters (i.e., reliable variance from a traditional perspective), namely the person main effect and interaction effects that involve persons but that do not include rater. [...]not all effects may be uniquely estimated depending on one’s measurement design. [...]as stated above, if one were to decompose the variance of two supervisors’ ratings (the traditional design of selection systems), then the person x source effect would actually be estimated as part of the person effect.