EyeCoD: Eye Tracking System Acceleration via FlatCam-Based Algorithm and Hardware Co-Design
Eye tracking has become an essential human–machine interaction modality in virtual reality (VR) and augmented reality (AR) applications requiring high throughput (e.g., more than 240 frames per second), small form factor, and enhanced visual privacy. Existing eye tracking systems have adopted bulky,...
Gespeichert in:
| Veröffentlicht in: | IEEE MICRO 2023-07, Vol.43 (4), p.88-97 |
|---|---|
| Hauptverfasser: | , , , , , , , , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 97 |
|---|---|
| container_issue | 4 |
| container_start_page | 88 |
| container_title | IEEE MICRO |
| container_volume | 43 |
| creator | You, Haoran Zhao, Yang Wan, Cheng Yu, Zhongzhi Fu, Yonggan Yuan, Jiayi Wu, Shang Zhang, Shunyao Zhang, Yongan Li, Chaojian Boominathan, Vivek Veeraraghavan, Ashok Li, Ziyun Lin, Yingyan Celine |
| description | Eye tracking has become an essential human–machine interaction modality in virtual reality (VR) and augmented reality (AR) applications requiring high throughput (e.g., more than 240 frames per second), small form factor, and enhanced visual privacy. Existing eye tracking systems have adopted bulky, lens-based cameras, and thus suffer from both a large form factor and high communication cost between the camera and back-end processor. This work presents a camera, algorithm, and accelerator co-designed lensless eye tracking system dubbed EyeCoD, which, to the best of our knowledge, is the first to provide a general, front-end eye tracking solution for AR/VR while satisfying the requirements for both high throughput and smaller form factor. Specifically, EyeCoD integrates system-, algorithm-, and accelerator-level techniques to boost system efficiency without sacrificing eye tracking accuracy. We believe that our EyeCoD system will pave the way for next-generation eye tracking solutions in VR/AR and shed light on future innovations for intelligent imaging systems. |
| doi_str_mv | 10.1109/MM.2023.3274736 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2831521748</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2831521748</sourcerecordid><originalsourceid>FETCH-LOGICAL-c223t-ad4628924493180185305a07a4457ca76e7ddd4708476b91bfd07a24101623463</originalsourceid><addsrcrecordid>eNotkM9PwjAYhhujiYievTbxPGi_dmvnDSeICYsH8eSh-VgLDvcD26Hhv3cETs_hffO-yUPIPWcjzlk6zvMRMBAjAUoqkVyQAU-FiiSX4pIMGCiIuBJwTW5C2DLGYmB6QD6nB5e1z4-0J116LL7LZkPfD6FzNZ0Uhaucx65sG_pbIp1V2GVYR08YnKWTatP6svuqKTaWztHbP_SOZm307EK5aW7J1Rqr4O7OHJKP2XSZzaPF28trNllEBYDoIrQyAZ2ClKngmnEdCxYjUyhlrApUiVPWWqmYlipZpXy1tn0IkjOegJCJGJKH0-7Otz97Fzqzbfe-6S8NaMFj4ErqvjU-tQrfhuDd2ux8WaM_GM7M0aDJc3M0aM4GxT9WbmBE</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2831521748</pqid></control><display><type>article</type><title>EyeCoD: Eye Tracking System Acceleration via FlatCam-Based Algorithm and Hardware Co-Design</title><source>IEEE Electronic Library (IEL)</source><creator>You, Haoran ; Zhao, Yang ; Wan, Cheng ; Yu, Zhongzhi ; Fu, Yonggan ; Yuan, Jiayi ; Wu, Shang ; Zhang, Shunyao ; Zhang, Yongan ; Li, Chaojian ; Boominathan, Vivek ; Veeraraghavan, Ashok ; Li, Ziyun ; Lin, Yingyan Celine</creator><creatorcontrib>You, Haoran ; Zhao, Yang ; Wan, Cheng ; Yu, Zhongzhi ; Fu, Yonggan ; Yuan, Jiayi ; Wu, Shang ; Zhang, Shunyao ; Zhang, Yongan ; Li, Chaojian ; Boominathan, Vivek ; Veeraraghavan, Ashok ; Li, Ziyun ; Lin, Yingyan Celine</creatorcontrib><description>Eye tracking has become an essential human–machine interaction modality in virtual reality (VR) and augmented reality (AR) applications requiring high throughput (e.g., more than 240 frames per second), small form factor, and enhanced visual privacy. Existing eye tracking systems have adopted bulky, lens-based cameras, and thus suffer from both a large form factor and high communication cost between the camera and back-end processor. This work presents a camera, algorithm, and accelerator co-designed lensless eye tracking system dubbed EyeCoD, which, to the best of our knowledge, is the first to provide a general, front-end eye tracking solution for AR/VR while satisfying the requirements for both high throughput and smaller form factor. Specifically, EyeCoD integrates system-, algorithm-, and accelerator-level techniques to boost system efficiency without sacrificing eye tracking accuracy. We believe that our EyeCoD system will pave the way for next-generation eye tracking solutions in VR/AR and shed light on future innovations for intelligent imaging systems.</description><identifier>ISSN: 0272-1732</identifier><identifier>EISSN: 1937-4143</identifier><identifier>DOI: 10.1109/MM.2023.3274736</identifier><language>eng</language><publisher>Los Alamitos: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</publisher><subject>Algorithms ; Augmented reality ; Cameras ; Co-design ; Disclaimers ; Eye movements ; Form factors ; Frames per second ; Microprocessors ; Tracking systems ; Virtual reality ; Visual aspects</subject><ispartof>IEEE MICRO, 2023-07, Vol.43 (4), p.88-97</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c223t-ad4628924493180185305a07a4457ca76e7ddd4708476b91bfd07a24101623463</cites><orcidid>0000-0001-7919-049X ; 0000-0001-5043-7460 ; 0000-0001-6070-6310 ; 0000-0001-6467-7166 ; 0000-0001-5946-203X ; 0000-0002-6227-6905 ; 0000-0003-4875-3135 ; 0000-0002-7483-2921 ; 0000-0002-2873-2153 ; 0000-0002-2295-3481 ; 0000-0001-8023-1551 ; 0000-0003-4030-9777 ; 0000-0002-9981-4981 ; 0009-0005-9208-6677</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,27903,27904</link.rule.ids></links><search><creatorcontrib>You, Haoran</creatorcontrib><creatorcontrib>Zhao, Yang</creatorcontrib><creatorcontrib>Wan, Cheng</creatorcontrib><creatorcontrib>Yu, Zhongzhi</creatorcontrib><creatorcontrib>Fu, Yonggan</creatorcontrib><creatorcontrib>Yuan, Jiayi</creatorcontrib><creatorcontrib>Wu, Shang</creatorcontrib><creatorcontrib>Zhang, Shunyao</creatorcontrib><creatorcontrib>Zhang, Yongan</creatorcontrib><creatorcontrib>Li, Chaojian</creatorcontrib><creatorcontrib>Boominathan, Vivek</creatorcontrib><creatorcontrib>Veeraraghavan, Ashok</creatorcontrib><creatorcontrib>Li, Ziyun</creatorcontrib><creatorcontrib>Lin, Yingyan Celine</creatorcontrib><title>EyeCoD: Eye Tracking System Acceleration via FlatCam-Based Algorithm and Hardware Co-Design</title><title>IEEE MICRO</title><description>Eye tracking has become an essential human–machine interaction modality in virtual reality (VR) and augmented reality (AR) applications requiring high throughput (e.g., more than 240 frames per second), small form factor, and enhanced visual privacy. Existing eye tracking systems have adopted bulky, lens-based cameras, and thus suffer from both a large form factor and high communication cost between the camera and back-end processor. This work presents a camera, algorithm, and accelerator co-designed lensless eye tracking system dubbed EyeCoD, which, to the best of our knowledge, is the first to provide a general, front-end eye tracking solution for AR/VR while satisfying the requirements for both high throughput and smaller form factor. Specifically, EyeCoD integrates system-, algorithm-, and accelerator-level techniques to boost system efficiency without sacrificing eye tracking accuracy. We believe that our EyeCoD system will pave the way for next-generation eye tracking solutions in VR/AR and shed light on future innovations for intelligent imaging systems.</description><subject>Algorithms</subject><subject>Augmented reality</subject><subject>Cameras</subject><subject>Co-design</subject><subject>Disclaimers</subject><subject>Eye movements</subject><subject>Form factors</subject><subject>Frames per second</subject><subject>Microprocessors</subject><subject>Tracking systems</subject><subject>Virtual reality</subject><subject>Visual aspects</subject><issn>0272-1732</issn><issn>1937-4143</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><recordid>eNotkM9PwjAYhhujiYievTbxPGi_dmvnDSeICYsH8eSh-VgLDvcD26Hhv3cETs_hffO-yUPIPWcjzlk6zvMRMBAjAUoqkVyQAU-FiiSX4pIMGCiIuBJwTW5C2DLGYmB6QD6nB5e1z4-0J116LL7LZkPfD6FzNZ0Uhaucx65sG_pbIp1V2GVYR08YnKWTatP6svuqKTaWztHbP_SOZm307EK5aW7J1Rqr4O7OHJKP2XSZzaPF28trNllEBYDoIrQyAZ2ClKngmnEdCxYjUyhlrApUiVPWWqmYlipZpXy1tn0IkjOegJCJGJKH0-7Otz97Fzqzbfe-6S8NaMFj4ErqvjU-tQrfhuDd2ux8WaM_GM7M0aDJc3M0aM4GxT9WbmBE</recordid><startdate>20230701</startdate><enddate>20230701</enddate><creator>You, Haoran</creator><creator>Zhao, Yang</creator><creator>Wan, Cheng</creator><creator>Yu, Zhongzhi</creator><creator>Fu, Yonggan</creator><creator>Yuan, Jiayi</creator><creator>Wu, Shang</creator><creator>Zhang, Shunyao</creator><creator>Zhang, Yongan</creator><creator>Li, Chaojian</creator><creator>Boominathan, Vivek</creator><creator>Veeraraghavan, Ashok</creator><creator>Li, Ziyun</creator><creator>Lin, Yingyan Celine</creator><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0001-7919-049X</orcidid><orcidid>https://orcid.org/0000-0001-5043-7460</orcidid><orcidid>https://orcid.org/0000-0001-6070-6310</orcidid><orcidid>https://orcid.org/0000-0001-6467-7166</orcidid><orcidid>https://orcid.org/0000-0001-5946-203X</orcidid><orcidid>https://orcid.org/0000-0002-6227-6905</orcidid><orcidid>https://orcid.org/0000-0003-4875-3135</orcidid><orcidid>https://orcid.org/0000-0002-7483-2921</orcidid><orcidid>https://orcid.org/0000-0002-2873-2153</orcidid><orcidid>https://orcid.org/0000-0002-2295-3481</orcidid><orcidid>https://orcid.org/0000-0001-8023-1551</orcidid><orcidid>https://orcid.org/0000-0003-4030-9777</orcidid><orcidid>https://orcid.org/0000-0002-9981-4981</orcidid><orcidid>https://orcid.org/0009-0005-9208-6677</orcidid></search><sort><creationdate>20230701</creationdate><title>EyeCoD: Eye Tracking System Acceleration via FlatCam-Based Algorithm and Hardware Co-Design</title><author>You, Haoran ; Zhao, Yang ; Wan, Cheng ; Yu, Zhongzhi ; Fu, Yonggan ; Yuan, Jiayi ; Wu, Shang ; Zhang, Shunyao ; Zhang, Yongan ; Li, Chaojian ; Boominathan, Vivek ; Veeraraghavan, Ashok ; Li, Ziyun ; Lin, Yingyan Celine</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c223t-ad4628924493180185305a07a4457ca76e7ddd4708476b91bfd07a24101623463</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Algorithms</topic><topic>Augmented reality</topic><topic>Cameras</topic><topic>Co-design</topic><topic>Disclaimers</topic><topic>Eye movements</topic><topic>Form factors</topic><topic>Frames per second</topic><topic>Microprocessors</topic><topic>Tracking systems</topic><topic>Virtual reality</topic><topic>Visual aspects</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>You, Haoran</creatorcontrib><creatorcontrib>Zhao, Yang</creatorcontrib><creatorcontrib>Wan, Cheng</creatorcontrib><creatorcontrib>Yu, Zhongzhi</creatorcontrib><creatorcontrib>Fu, Yonggan</creatorcontrib><creatorcontrib>Yuan, Jiayi</creatorcontrib><creatorcontrib>Wu, Shang</creatorcontrib><creatorcontrib>Zhang, Shunyao</creatorcontrib><creatorcontrib>Zhang, Yongan</creatorcontrib><creatorcontrib>Li, Chaojian</creatorcontrib><creatorcontrib>Boominathan, Vivek</creatorcontrib><creatorcontrib>Veeraraghavan, Ashok</creatorcontrib><creatorcontrib>Li, Ziyun</creatorcontrib><creatorcontrib>Lin, Yingyan Celine</creatorcontrib><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE MICRO</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>You, Haoran</au><au>Zhao, Yang</au><au>Wan, Cheng</au><au>Yu, Zhongzhi</au><au>Fu, Yonggan</au><au>Yuan, Jiayi</au><au>Wu, Shang</au><au>Zhang, Shunyao</au><au>Zhang, Yongan</au><au>Li, Chaojian</au><au>Boominathan, Vivek</au><au>Veeraraghavan, Ashok</au><au>Li, Ziyun</au><au>Lin, Yingyan Celine</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>EyeCoD: Eye Tracking System Acceleration via FlatCam-Based Algorithm and Hardware Co-Design</atitle><jtitle>IEEE MICRO</jtitle><date>2023-07-01</date><risdate>2023</risdate><volume>43</volume><issue>4</issue><spage>88</spage><epage>97</epage><pages>88-97</pages><issn>0272-1732</issn><eissn>1937-4143</eissn><abstract>Eye tracking has become an essential human–machine interaction modality in virtual reality (VR) and augmented reality (AR) applications requiring high throughput (e.g., more than 240 frames per second), small form factor, and enhanced visual privacy. Existing eye tracking systems have adopted bulky, lens-based cameras, and thus suffer from both a large form factor and high communication cost between the camera and back-end processor. This work presents a camera, algorithm, and accelerator co-designed lensless eye tracking system dubbed EyeCoD, which, to the best of our knowledge, is the first to provide a general, front-end eye tracking solution for AR/VR while satisfying the requirements for both high throughput and smaller form factor. Specifically, EyeCoD integrates system-, algorithm-, and accelerator-level techniques to boost system efficiency without sacrificing eye tracking accuracy. We believe that our EyeCoD system will pave the way for next-generation eye tracking solutions in VR/AR and shed light on future innovations for intelligent imaging systems.</abstract><cop>Los Alamitos</cop><pub>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</pub><doi>10.1109/MM.2023.3274736</doi><tpages>10</tpages><orcidid>https://orcid.org/0000-0001-7919-049X</orcidid><orcidid>https://orcid.org/0000-0001-5043-7460</orcidid><orcidid>https://orcid.org/0000-0001-6070-6310</orcidid><orcidid>https://orcid.org/0000-0001-6467-7166</orcidid><orcidid>https://orcid.org/0000-0001-5946-203X</orcidid><orcidid>https://orcid.org/0000-0002-6227-6905</orcidid><orcidid>https://orcid.org/0000-0003-4875-3135</orcidid><orcidid>https://orcid.org/0000-0002-7483-2921</orcidid><orcidid>https://orcid.org/0000-0002-2873-2153</orcidid><orcidid>https://orcid.org/0000-0002-2295-3481</orcidid><orcidid>https://orcid.org/0000-0001-8023-1551</orcidid><orcidid>https://orcid.org/0000-0003-4030-9777</orcidid><orcidid>https://orcid.org/0000-0002-9981-4981</orcidid><orcidid>https://orcid.org/0009-0005-9208-6677</orcidid></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0272-1732 |
| ispartof | IEEE MICRO, 2023-07, Vol.43 (4), p.88-97 |
| issn | 0272-1732 1937-4143 |
| language | eng |
| recordid | cdi_proquest_journals_2831521748 |
| source | IEEE Electronic Library (IEL) |
| subjects | Algorithms Augmented reality Cameras Co-design Disclaimers Eye movements Form factors Frames per second Microprocessors Tracking systems Virtual reality Visual aspects |
| title | EyeCoD: Eye Tracking System Acceleration via FlatCam-Based Algorithm and Hardware Co-Design |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-21T19%3A53%3A13IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=EyeCoD:%20Eye%20Tracking%20System%20Acceleration%20via%20FlatCam-Based%20Algorithm%20and%20Hardware%20Co-Design&rft.jtitle=IEEE%20MICRO&rft.au=You,%20Haoran&rft.date=2023-07-01&rft.volume=43&rft.issue=4&rft.spage=88&rft.epage=97&rft.pages=88-97&rft.issn=0272-1732&rft.eissn=1937-4143&rft_id=info:doi/10.1109/MM.2023.3274736&rft_dat=%3Cproquest_cross%3E2831521748%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2831521748&rft_id=info:pmid/&rfr_iscdi=true |