Searching for the Holy Grail of Learning Outcomes. SERU Project and Consortium Research Paper. Research & Occasional Paper Series: CSHE.3.12

The search for the Holy Grail to measure learning gains started in the US, but the Organisation for Economic Co-operation and Development (OECD) wants to take it global. Here we tell a bit of this story and raise serious questions regarding the validity of the Collegiate Learning Assessment test and suggest there are alternatives. The merit of the CLA as a true assessment of learning outcomes is, we dare say, debatable. In part, the arrival and success of the CLA is a story of markets. In essence, it is a successfully marketed product that is fulfilling a growing demand with few recognized competitors. As a result, the CLA is winning the "learning outcomes race," essentially becoming the "gold standard" in the US. We worry that the CLA's early market success is potentially thwarting the development of other valuable and more nuanced alternatives--whether it be other types of standardized tests that attest to measuring the learning curve of students, or other approaches such as student portfolios, contextually designed surveys on the student experience, and alumni feedback. In a new study published in the journal "Higher Education", we examine the relative merits of student experience surveys in gauging learning outcomes by analyzing results from the data from the Student Experience in the Research University (SERU) Survey. This essay discusses some of the main points from that article. There are real problems with student self-assessments. But as we argue here, universities can probably learn more about learning outcomes in a wide range of disciplines via properly designed census surveys than by standardized tests like the CLA. At present, we suggest there is tension between the accountability desires of governments and the needs of individual universities who must focus on institutional self-improvement. One might hope that they would be synonymous. But how to make ministries and other policymakers more fully understand the perils of a silver bullet test tool?