The medium is the message: Comparing paper-based and web-based course evaluation modalities

•Online student evaluations are almost always lower than in class evaluations.•Differences between online and in class evaluations do not vary with class level.•Differences between online and in class evaluations do not vary with response rate.•Differences between course evaluation methods are larger for poorly received classes. An increasing number of universities have moved student evaluation of faculty and courses out of the classroom, where it had resided for many years, and onto the web. The increased efficiency of the web-based administrative modality of these instruments seems self-apparent. However, whether the measures obtained using the new modality are the same as the old modality is unclear. This paper compares the results of questionnaires administered on the web with those collected from the same students while they were in class. Data from 181 course offerings over seven terms at one university were utilized. Significantly lower evaluation scores for both the instructor and the course are produced when a web-based modality is used. In general, these results did not vary for courses at different levels of matriculation or at different levels of student participation. However, the magnitude of modality differences varied between highly rated and poorly rated courses. Implications for faculty evaluation are offered.