Automated Writing Evaluation: The Accuracy of Grammarly’s Feedback on Form

Automated Writing Evaluation (AWE) is gaining more and more presence, with Grammarly being a prominent example of this kind of software, which provides automated feedback on essay writing. While the Grammarly advertisement is full of praise regarding its ability to give meaningful feedback at various levels, including grammar or form, plagiarism, vocabulary and style, there has been little third party research to support these claims. Since Grammarly’s vocabulary feedback accuracy has already been examined to some extent (Dodigovic, Mlynarski & Wei, 2016), this study was designed to evaluate the accuracy of Grammarly’s grammar checker feedback. It did so by investigating the reports generated by Grammarly as a response to a small corpus of EFL writing, compiled from essays written by Armenian undergraduate students. The results were compared with those of human raters. In addition to Grammarly’s error detection power, the study also investigated the kind of remediation that Grammarly suggested. The comparisons revealed that on detected errors of form, Grammarly mostly provides accurate feedback, with occasional inconsistencies. It was also found to provide mainly adequate remediation for the correctly detected errors of form. However, it also left a number of errors undetected, which led to the omission of remediation. The study concluded that, despite its considerable usefulness, when using Grammarly, English learners should avoid relying solely on its feedback.