An Investigation of the Adequacy of Several Goodness of Fit Statistics

A study was conducted to evaluate six goodness-of-fit procedures using data simulation techniques. The procedures evaluated included the weighted average absolute deviation (WAAD), the mean absolute deviation (MAD), Bock's chi-square (BCHI), Yen's chi-square (YCHI), Wright and Mead's chi-square (WCHI), and the likelihood ratio chi-square (LCHI) statistics. These procedures were evaluated using data generated according to three different item response theory models and a factor analytic model. Three different distributions of ability and sample sizes were used. The results of this study yielded the following conclusions: (1) sample sizes of 500 to 1000 seemed to yield the best results; (2) the largest sample size (N=2000) seemed to make the fit procedures too sensitive; (3) shifts in the mean of the ability distribution caused minor fluctuations, but did not appear to be a major concern; (4) the chi-square statistics performed better than did the two non-chi-square statistics; and (5) the likelihood ratio chi-square procedure appeared to yield the best results. (Author/DWH)