Eye behavior recognition of eye–computer interaction

Eye behavior recognition has found widespread applications in augmented reality devices and simplified traditional human–computer interaction methods. However, several eye-machine interaction methods currently studied, such as pupil detection and gaze tracking, may not be well-suited for practical applications. Wearable devices or portable outdoor equipment require eye movement recognition in close proximity to the eyes, demanding rich eye-machine interaction commands and strong robustness. In this context, it is crucial to achieve eye behavior recognition that is suitable for practical application scenarios. To achieve this objective, the current study employs three enhancement strategies to modify YOLOv5 into YOLOv5-Eye State Recognition (YOLOv5-ESR) and trains it on a dataset of human eye images captured by intelligent eyeglasses to obtain an eye state recognition model. The study categorizes the eye states detected by the model into three types: "open," "closed," and "squint." Additionally, the user's eye behaviors are classified into four categories: "gaze," "long blink," "long squint," and "double blink." Through experimental testing, the average recognition accuracy of eye behavior reaches 96.25%, with a detection speed of 78.66 frames per second (fps). Moreover, this eye behavior recognition technology demonstrates its applicability in various application environments. The proposed technology is characterized by its suitability for wearable devices, high detection accuracy, real-time performance, strong robustness, and excellent human–computer interaction capabilities.