Real-Time System of Detection and Identification of Raised Hands and Persons by Deep Learning Algorithms for Teaching Support Systems

The authors have been developing a teaching support system by using neural network/deep learning technologies. This paper deals with detecting persons with or without a raised hand, tracking, and identifying individuals in a large classroom by using a camera. The algorithm consists of object detection by a deep learning algorithm YOLO and individual identification by cascading several neural networks. An experiment of automatically calling on a person among those with raised hands is also conducted in a real setting to avoid concentration to a very limited number of students. PTZ camera is shown to be essential in this system. In the real-time experiment, Jetson AGX Xavier, a portable GPU machine, is used to meet the requirement of computationally expensive task, and our system is shown to be useful.