Framework for automation of short answer grading based on domain-specific pre-training

In the education sector, there is a growing need for efficient Automatic Short Answer Grading (ASAG) systems that require less training time, have faster inference, and provide higher accuracy. Over the past five years, researchers have made progress using deep learning for ASAG, but acquiring domain-specific labeled data remains a significant challenge. This study introduces an ASAG framework designed to learn from unlabeled domain-specific contexts while minimizing training and inference times. Our framework involves three main phases: 1) Building an unlabeled domain-specific corpus. 2) Training a pre-trained language model with this corpus. 3) Fine-tuning the pre-trained model with limited labeled domain-specific data. We explored various domain adaptations using two types of vocabulary during the pretraining stage. Our approach achieved an accuracy of 88.77% and an F1-score of 91.59% on the SPRAG dataset, improving state-of-the-art methods by 4%–5.7%. The models, including one trained from scratch with domain-specific data and vocabulary, showed robust performance, enhancing accuracy by 5.7% and F1-score by 4.84%. However, the framework's effectiveness in handling different languages or multilingual datasets remains to be explored. This framework's adaptability and efficiency enable educators to use AI for automated grading in various educational domains.