Word-level arabic sign language recognition using millimeter wave radar and convolutional neural networks

This paper proposes and implements a real-time word-level Arabic Sign Language (ARSL) recognition system. The work aims to facilitate the interaction between people with hearing loss and the general public. The system uses a frequency-modulated continuous wave (FM-CW) radar operating at 77 GHz millimeter frequency to transmit and capture signals from a person performing hand gestures in front of the radar. The signals are then used to train a convolutional neural network (CNN) to translate the gestures into words. The CNN is built using the Keras platform on Python and has ten layers, including two 2D convolutional layers. In the preliminary work, the system is designed to identify ten important common words. Preliminary results show that the system can classify gestures correctly in real time with 81% accuracy.