Towards Automated Assessment Generation in e-Learning Systems Using Combinatorial Testing and Formal Concept Analysis

In this paper, we research the use of software combinatorial testing techniques and the Formal Concept Analysis method for preparing sets of questions for student assessment in e-learning systems. Utilizing these techniques and methods, we ensure that the selected questions optimally cover the course material and that each question combines multiple topics. Therefore, in this paper we introduce our method for preparing student assessments that performs automated combinatorial testing and selection of questions, as well as automated generation of appropriate sequences of questions. The input for our method is a set of questions labelled with attributes or features. This set of questions is pre-processed using the Formal Concept Analysis method, and then the combinatorial testing of question features is performed, which generates a concise list of test-cases covering all pairs or triples of question features. Correspondingly, our method helps in identifying and selecting a subset of questions that covers all generated test-cases. Afterwards, the Formal Concept Analysis method automatically generates suitable sequences of selected questions for formative student assessments in e-learning systems. In this paper we implemented the proposed combinatorial testing method, and also demonstrated the feasibility of the proposed method on a use-case from an actual e-learning system.