Examining the Validity of a Scale Used in a Performance Assessment from Many Angles Using the Many-Faceted Rasch Model

An attempt is made is this paper to examine the validity of the American Council on the Teaching of Foreign Languages (ACTFL) scale through a comparison of the scaling of speaking tasks and speech performances by the scale and by a Rasch analysis of judgments made by "naive" persons. The results of the multi-faceted Rasch analysis seem to support the use of the scale in assessing developing second language proficiency. The unifying element was the underlying ACTFL scale. The results indicate a tendency towards convergence of the judgments made by "naive" judges across three different groups, made during separate phases of the test development project, made on different aspects of the project, and made using different methods of indicating decisions with the ACTFL scale. It is concluded that the use of the ACTFL Proficiency Guidelines is justified for developing performance-based assessments of speaking ability. Documentation is presented in 10 tables, and appendices provide: (1) the Structure of the Texas Oral Proficiency Test (TOPT)--Spanish; and (2) the TOPT Bilingual Education Teachers Job-Relatedness Survey. Contains 16 references. (LB)