Using AutoTutor to Track Performance and Engagement in a Reading Comprehension Intervention for Adult Literacy Students

A large percentage of adults throughout the world have low reading skills. Computer technologies can potentially help these adults improve their literacy in addition to instructors at literacy centers. AutoTutor was designed to teach comprehension strategies by implementing conversational 'trialogues' in which two computer agents (tutor and peer) hold spoken interactions with the adult about words, sentences, and text in digital lessons. The agents model comprehension strategies, ask questions, and give feedback on adult answers. AutoTutor records in log files the adults' performance, namely the time and accuracy of answering questions in the conversation. We assessed the value of AutoTutor in a study with 52 adult literacy students in the United States and Canada who interacted with AutoTutor as part of a 4-month intervention with human instructors. Performance in AutoTutor was tracked at four theoretical discourse levels (words, explicit textbase, conceptual situation model, rhetorical structure) and also engagement, with an objective psychometric measure of comprehension skill both before and after the intervention. The results showed that AutoTutor provides nuanced performance and engagement measures that predicted comprehension improvements and can be used to guide formative assessment for instructors.