Developing evidence for a validity argument for an English placement exam from multi-year test performance data

Background This study investigated the factor structure and factorial invariance of an English Placement Exam (EPE) from 1998 to 2011 to provide evidence for both the appropriateness of the test scores interpretations and for a validity argument. Methods Test performance data collected from 38,632 freshmen non-English majors from a university in central Taiwan from 13 years (1998–2001, and 2003–2011) was examined using both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). EFA was performed on 6 years of data (2006–2011) to establish a baseline structure, which was then further tested using CFA on data from all the years. Results Results from EFA supported a three-factor oblique (correlated) solution and CFA determined that a three-factor model as the best fit for the data. This model reflected the test structure posited by the test designers (grammar, reading, and listening sections) and remained factorially invariant and factorially distinct across all years, with insignificant variation in factor loadings and model fit indices. Conclusion The study concluded that the factor structure of the EPE provided evidence for a construct validity argument for the placement exam based on multi-year performance data, thus supporting the inferences regarding test scores interpretations and the soundness of score comparisons made across years.