A Visual Analytic in Deep Learning Approach to Eye Movement for Human-Machine Interaction Based on Inertia Measurement

This paper proposes a hand free human-machine interaction (HMI) system to establish a novel way for communication between humans and computers. A regular interaction system based on the computer mouse puts the user's hand for too long in a pronation posture that increases inflammation in the wrist and hand. Additionally, the need for hand obstructs the use of computers for handicap people. In this paper, we develop a new pointing device for differently able people based on open and closed human eyes with inertia measurement that restrict to deal with carpal tunnel syndrome (CTS) for regular people and enables a novel way to interact with computers for the handicap people. The proposed system carries the human head gesture and eyes to perform the movement and clicking event of the mouse cursor. A combined three-axis accelerometer and gyroscope is used to detect the head gesture and translate it into the position of the mouse cursor on the computer monitor. To perform the left and right-clicking event, the user needs to shut down the left and right eye for a moment while opening another eye. This paper is also carried out the design of a deep learning approach to classify the individual openness and closeness of both human eyes with quite a high accuracy of 95.36% that ensures the comprehensive control over the clicking performance. The use of complementary filter removes the noise and drift from the obtained performance and confirms the smooth and accurate operation of the proposed device. An experimental validation is added to show the effectiveness of the proposed HMI system. The experimental details along with the performance evaluation prove that the proposed HMI system has extensive control over its performance for differently able people.