Construct Validation of the Multiple-Choice Items of the English Subtest of the Advanced Subjects Test in Taiwan

Administered in each July in Taiwan, the Advanced Subjects Test (AST) is a high-stakes college entrance test, of which an English subtest (AST-E) is an integral part. The AST-E score affects considerably which universities test takers are qualified to be admitted to. To them, the major concern is: what English ability does the AST-E really measure? This study was intended to investigate the construct validity of the multiple-choice (MC) items of the AST-E through confirmatory factor analysis (CFA) using data on test takers' responses to the MC items from 2015 to 2016. In the first part of this study, the MC items were classified into several language components by five experienced raters based on the two classification frameworks of Purpura (1999, 2004). The chosen components represented an incipient model for describing the relationship between the unobserved components and the data. In the second part, a series of CFAs were performed, first on the incipient model and then on some alternative models, to look for a model that offers a relatively good fit to the data. The CFA results showed that, although the incipient model involved multiple components/factors, a one-factor model was found to best portray the characteristics of test takers' responses to the MC items for 2015 and 2016, suggesting that the MC items appeared to tap simply their general English reading ability rather than a set of their divisible English reading skills. Finally, this study concluded with some pedagogical and practical implications for Taiwan high school English teachers and AST-E test constructors.