A computer vision-based perceived attention monitoring technique for smart teaching

This paper aims to improve the lecture delivery mechanism in real-time in a classroom and remote sessions over web-based applications. In the traditional system, a lecturer observes their students’ attention levels from his/her experience. To date, no system automatically tracks the students’ attention level in a class in real-time (or while the lecturer is delivering his/her lectures remotely over web-based applications). On the other hand, our proposed system periodically will monitor the learning behaviour of the whole class and track the attentiveness of each student. The proposed system is not meant to identify the non-attentive students and punish them. Contrary to the punishment-based mechanism, it introduces a counseling-based mechanism. This deep learning-based real-time face monitoring system will allow lecturers to improvise/her delivery either through bringing diversity in the class contents or personal care to those non-attentive students. The concept of the deep learning technique in an ensemble configuration has been used to predict the likelihood of eyes’ openness. Separately, a student’s facial expressions are also recognized using our Convolutional Neural Network (CNN) model. Finally, the net learning behaviour of a student has been computed by a weighted average of these two features (that is, eyes’ openness and facial expressions). The student learning behaviour is validated twice with Pearson correlation coefficient and Spearman correlation coefficient measures between the openness of eye and facial expressions. Again, the Cosine similarity has been used to further examine the periodical similarity of the student’s learning patterns. The proposed pipeline has performed even better than the state-of-the-art models such as ResNet50, MobileNetV2, and EfficientNet-B0 in terms of accuracy and f1-score.