STTI – CHGSR: Continuous hand gesture segmentation and recognition based on spatial-temporal and trajectory information

Verbal language is broken up into various categories; there are also different types of nonverbal communication. Human Computer Interaction (HCI) provides more intuitive way of interacting with computer and has long been an important and popular research field. The variations in gesture duration and unknown start and end points are the key issues in identifying uninterrupted gestures thus lowering the performing of conventional identification algorithms. In this paper, we first present a survey on issues and limitations of hand gesture recognition. Later, we propose a framework for ‘Continuous Hand Gesture Segmentation and Recognition (CHGSR) based on Spatial-Temporal and Trajectory Information (STTI). The framework utilizes cognitive Deep Learning Networks and outperforms in recognizing continuous hand gestures. We utilized TensorFlow and Keras Deep Learning Libraries for implementing our Deep Learning Network. Three different persons in different session for evaluation of framework videos recorded by using Microsoft Kinect V2 sensor. In presence of continuous gestures and different gesticulation behaviors, the investigational results signify that the intended framework has a high acknowledgment accuracy and F-score of up to 0.98.