A Method for Analyzing Sparse Data Matrices in the Generalizability Theory Framework

In generalizability analyses, unstable, and potentially invalid, variance component estimates may result from using only a limited portion of available data. However, missing observations are common in operational performance assessment settings because of the nature of the assessment design. This article describes a procedure for overcoming the computational and technological limitations in analyzing data with missing observations by extracting data from a sparsely .lled data matrix into analyzable smaller subsets of data. This subdividing method is accomplished by creating data sets that exhibit structural designs that are common in generalizability analyses, namely, the crossed, MBIB, and nested designs. The validity of this subdividing method is examined using a Monte Carlo simulation. The method is demonstrated on an operational data set.