EyeCoD: Eye Tracking System Acceleration via FlatCam-Based Algorithm and Hardware Co-Design

Eye tracking has become an essential human–machine interaction modality in virtual reality (VR) and augmented reality (AR) applications requiring high throughput (e.g., more than 240 frames per second), small form factor, and enhanced visual privacy. Existing eye tracking systems have adopted bulky, lens-based cameras, and thus suffer from both a large form factor and high communication cost between the camera and back-end processor. This work presents a camera, algorithm, and accelerator co-designed lensless eye tracking system dubbed EyeCoD, which, to the best of our knowledge, is the first to provide a general, front-end eye tracking solution for AR/VR while satisfying the requirements for both high throughput and smaller form factor. Specifically, EyeCoD integrates system-, algorithm-, and accelerator-level techniques to boost system efficiency without sacrificing eye tracking accuracy. We believe that our EyeCoD system will pave the way for next-generation eye tracking solutions in VR/AR and shed light on future innovations for intelligent imaging systems.