Machine Learning Guided Evaluation of a College Program for Under-Prepared Students

Under-prepared college students are at risk of dropping out (Bettinger & Long, 2005) and having lower academic achievement to their better-prepared peers. Yet they would likely benefit most from higher education (Jaegar & Page, 1996; Kane & Rouse, 1995), highlighting the need for institutional support. While developmental courses in two-year community colleges are well studied (Martorell & McFarlin, 2011; Scott-Clayton & Rodriguez, 2015), four-year colleges have other tools at their disposal. The authors study one selective four-year state institution's (anonymized as the University) program, anonymized as the No-MisMatch Program (NMP). NMP identifies under-prepared students and requires them to take a variety of summer prep courses before enrollment, gives them extensive tutoring resources throughout the year, offers specialized course sections, and requires multiple advisor meetings each semester. Using machine learning with a regression discontinuity design, the authors estimate the effect of NMP assignment on academic outcomes. The key preliminary finding is that assignment to NMP increases the likelihood students subsequently enroll at the University around 20 percentage points. This paper suggests that NMP can act as a potential enrollment management tool for policy makers. Students' admission letters positively frame being assigned to NMP, offering students resources for success, rather than indicating under-preparation. Reaching out to these under-prepared students importantly promotes the institution's and higher education's mission to close potential achievement gaps. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]