An integrated neural network model for pupil detection and tracking

The accurate detection and tracking of pupil is important to many applications such as human–computer interaction, driver’s fatigue detection and diagnosis of brain diseases. Existing approaches however face challenges in handing low quality of pupil images. In this paper, we propose an integrated pupil tracking framework, namely LVCF, based on deep learning. LVCF consists of the pupil detection model VCF which is an end-to-end network, and the LSTM pupil motion prediction model which applies LSTM to track pupil’s position. The proposed network was trained and evaluated on 10600 images and 75 videos taken from 3 realistic datasets. Within an error threshold of 5 pixels, VCF achieves an accuracy of more than 81%, and LVCF outperforms the state of arts by 9% in terms of percentage of pupils tracked. The project of LCVF is available at https://github.com/UnderTheMangoTree/LVCF .