Watch & Do: A Smart IoT Interaction System with Object Detection and Gaze Estimation
The Internet of Things (IoT) attempts to help people access Internet-connected devices, applications, and services anytime and anywhere. However, how providing an efficient and intuitive method of interaction between people and IoT devices is still an open challenge. In this paper, we propose a nove...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on consumer electronics 2019-05, Vol.65 (2), p.195-204 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 204 |
|---|---|
| container_issue | 2 |
| container_start_page | 195 |
| container_title | IEEE transactions on consumer electronics |
| container_volume | 65 |
| creator | Kim, Jung-Hwa Choi, Seung-June Jeong, Jin-Woo |
| description | The Internet of Things (IoT) attempts to help people access Internet-connected devices, applications, and services anytime and anywhere. However, how providing an efficient and intuitive method of interaction between people and IoT devices is still an open challenge. In this paper, we propose a novel interaction system called Watch & Do , where users can control an IoT device by gazing at it and doing simple gestures. The proposed system mainly consists of: 1) object detection module; 2) gaze estimation module; 3) hand gesture recognition module; and 4) IoT controller module. The target device is identified by various deep learning-based gaze estimation and object detection techniques. Afterwards, hand gesture recognition is applied to generate an IoT device control command which is transmitted to the IoT platform. The experimental results and case studies demonstrate the feasibility of the proposed system and imply the future research directions. |
| doi_str_mv | 10.1109/TCE.2019.2897758 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_crossref_primary_10_1109_TCE_2019_2897758</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>8634883</ieee_id><sourcerecordid>2215060760</sourcerecordid><originalsourceid>FETCH-LOGICAL-c338t-e1b1176f2266f44ab355b969f255a883a59f892ed78a0ffec155256006f82c683</originalsourceid><addsrcrecordid>eNo9kEFLAzEQhYMoWKt3wUtA8LZ1kmyyibfS1loo9NAVjyG7TegWu1uTiNRfb8oWTwMz7828-RC6JzAiBNRzOZmNKBA1olIVBZcXaEA4l1lOaHGJBgBKZgwEu0Y3IewASM6pHKDyw8R6i5_wtHvBY7zeGx_xoivxoo3Wmzo2XYvXxxDtHv80cYtX1c7WEU9ttP3QtBs8N78Wz0Js9ubUu0VXznwGe3euQ_T-Oisnb9lyNV9MxsusZkzGzJKKkEI4SoVweW4qxnmlhHKUcyMlM1w5qajdFNKAc7ZOD1EuAISTtBaSDdFjv_fgu69vG6Ledd--TSc1pYSDgEJAUkGvqn0XgrdOH3wK6o-agD6x04mdPrHTZ3bJ8tBbGmvtv1wKlqdY7A_evWfx</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2215060760</pqid></control><display><type>article</type><title>Watch & Do: A Smart IoT Interaction System with Object Detection and Gaze Estimation</title><source>IEEE Electronic Library (IEL)</source><creator>Kim, Jung-Hwa ; Choi, Seung-June ; Jeong, Jin-Woo</creator><creatorcontrib>Kim, Jung-Hwa ; Choi, Seung-June ; Jeong, Jin-Woo</creatorcontrib><description>The Internet of Things (IoT) attempts to help people access Internet-connected devices, applications, and services anytime and anywhere. However, how providing an efficient and intuitive method of interaction between people and IoT devices is still an open challenge. In this paper, we propose a novel interaction system called Watch & Do , where users can control an IoT device by gazing at it and doing simple gestures. The proposed system mainly consists of: 1) object detection module; 2) gaze estimation module; 3) hand gesture recognition module; and 4) IoT controller module. The target device is identified by various deep learning-based gaze estimation and object detection techniques. Afterwards, hand gesture recognition is applied to generate an IoT device control command which is transmitted to the IoT platform. The experimental results and case studies demonstrate the feasibility of the proposed system and imply the future research directions.</description><identifier>ISSN: 0098-3063</identifier><identifier>EISSN: 1558-4127</identifier><identifier>DOI: 10.1109/TCE.2019.2897758</identifier><identifier>CODEN: ITCEDA</identifier><language>eng</language><publisher>New York: IEEE</publisher><subject>Deep learning ; Estimation ; Feasibility studies ; gaze estimation ; Gaze tracking ; Gesture recognition ; Head ; Internet of Things ; Machine learning ; Modules ; Object detection ; Object recognition ; smart interaction ; Target recognition</subject><ispartof>IEEE transactions on consumer electronics, 2019-05, Vol.65 (2), p.195-204</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2019</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c338t-e1b1176f2266f44ab355b969f255a883a59f892ed78a0ffec155256006f82c683</citedby><cites>FETCH-LOGICAL-c338t-e1b1176f2266f44ab355b969f255a883a59f892ed78a0ffec155256006f82c683</cites><orcidid>0000-0001-9313-6860</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/8634883$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,796,27924,27925,54758</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/8634883$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Kim, Jung-Hwa</creatorcontrib><creatorcontrib>Choi, Seung-June</creatorcontrib><creatorcontrib>Jeong, Jin-Woo</creatorcontrib><title>Watch & Do: A Smart IoT Interaction System with Object Detection and Gaze Estimation</title><title>IEEE transactions on consumer electronics</title><addtitle>T-CE</addtitle><description>The Internet of Things (IoT) attempts to help people access Internet-connected devices, applications, and services anytime and anywhere. However, how providing an efficient and intuitive method of interaction between people and IoT devices is still an open challenge. In this paper, we propose a novel interaction system called Watch & Do , where users can control an IoT device by gazing at it and doing simple gestures. The proposed system mainly consists of: 1) object detection module; 2) gaze estimation module; 3) hand gesture recognition module; and 4) IoT controller module. The target device is identified by various deep learning-based gaze estimation and object detection techniques. Afterwards, hand gesture recognition is applied to generate an IoT device control command which is transmitted to the IoT platform. The experimental results and case studies demonstrate the feasibility of the proposed system and imply the future research directions.</description><subject>Deep learning</subject><subject>Estimation</subject><subject>Feasibility studies</subject><subject>gaze estimation</subject><subject>Gaze tracking</subject><subject>Gesture recognition</subject><subject>Head</subject><subject>Internet of Things</subject><subject>Machine learning</subject><subject>Modules</subject><subject>Object detection</subject><subject>Object recognition</subject><subject>smart interaction</subject><subject>Target recognition</subject><issn>0098-3063</issn><issn>1558-4127</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNo9kEFLAzEQhYMoWKt3wUtA8LZ1kmyyibfS1loo9NAVjyG7TegWu1uTiNRfb8oWTwMz7828-RC6JzAiBNRzOZmNKBA1olIVBZcXaEA4l1lOaHGJBgBKZgwEu0Y3IewASM6pHKDyw8R6i5_wtHvBY7zeGx_xoivxoo3Wmzo2XYvXxxDtHv80cYtX1c7WEU9ttP3QtBs8N78Wz0Js9ubUu0VXznwGe3euQ_T-Oisnb9lyNV9MxsusZkzGzJKKkEI4SoVweW4qxnmlhHKUcyMlM1w5qajdFNKAc7ZOD1EuAISTtBaSDdFjv_fgu69vG6Ledd--TSc1pYSDgEJAUkGvqn0XgrdOH3wK6o-agD6x04mdPrHTZ3bJ8tBbGmvtv1wKlqdY7A_evWfx</recordid><startdate>20190501</startdate><enddate>20190501</enddate><creator>Kim, Jung-Hwa</creator><creator>Choi, Seung-June</creator><creator>Jeong, Jin-Woo</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SP</scope><scope>8FD</scope><scope>F28</scope><scope>FR3</scope><scope>L7M</scope><orcidid>https://orcid.org/0000-0001-9313-6860</orcidid></search><sort><creationdate>20190501</creationdate><title>Watch & Do: A Smart IoT Interaction System with Object Detection and Gaze Estimation</title><author>Kim, Jung-Hwa ; Choi, Seung-June ; Jeong, Jin-Woo</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c338t-e1b1176f2266f44ab355b969f255a883a59f892ed78a0ffec155256006f82c683</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Deep learning</topic><topic>Estimation</topic><topic>Feasibility studies</topic><topic>gaze estimation</topic><topic>Gaze tracking</topic><topic>Gesture recognition</topic><topic>Head</topic><topic>Internet of Things</topic><topic>Machine learning</topic><topic>Modules</topic><topic>Object detection</topic><topic>Object recognition</topic><topic>smart interaction</topic><topic>Target recognition</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Kim, Jung-Hwa</creatorcontrib><creatorcontrib>Choi, Seung-June</creatorcontrib><creatorcontrib>Jeong, Jin-Woo</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ANTE: Abstracts in New Technology & Engineering</collection><collection>Engineering Research Database</collection><collection>Advanced Technologies Database with Aerospace</collection><jtitle>IEEE transactions on consumer electronics</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Kim, Jung-Hwa</au><au>Choi, Seung-June</au><au>Jeong, Jin-Woo</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Watch & Do: A Smart IoT Interaction System with Object Detection and Gaze Estimation</atitle><jtitle>IEEE transactions on consumer electronics</jtitle><stitle>T-CE</stitle><date>2019-05-01</date><risdate>2019</risdate><volume>65</volume><issue>2</issue><spage>195</spage><epage>204</epage><pages>195-204</pages><issn>0098-3063</issn><eissn>1558-4127</eissn><coden>ITCEDA</coden><abstract>The Internet of Things (IoT) attempts to help people access Internet-connected devices, applications, and services anytime and anywhere. However, how providing an efficient and intuitive method of interaction between people and IoT devices is still an open challenge. In this paper, we propose a novel interaction system called Watch & Do , where users can control an IoT device by gazing at it and doing simple gestures. The proposed system mainly consists of: 1) object detection module; 2) gaze estimation module; 3) hand gesture recognition module; and 4) IoT controller module. The target device is identified by various deep learning-based gaze estimation and object detection techniques. Afterwards, hand gesture recognition is applied to generate an IoT device control command which is transmitted to the IoT platform. The experimental results and case studies demonstrate the feasibility of the proposed system and imply the future research directions.</abstract><cop>New York</cop><pub>IEEE</pub><doi>10.1109/TCE.2019.2897758</doi><tpages>10</tpages><orcidid>https://orcid.org/0000-0001-9313-6860</orcidid></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | ISSN: 0098-3063 |
| ispartof | IEEE transactions on consumer electronics, 2019-05, Vol.65 (2), p.195-204 |
| issn | 0098-3063 1558-4127 |
| language | eng |
| recordid | cdi_crossref_primary_10_1109_TCE_2019_2897758 |
| source | IEEE Electronic Library (IEL) |
| subjects | Deep learning Estimation Feasibility studies gaze estimation Gaze tracking Gesture recognition Head Internet of Things Machine learning Modules Object detection Object recognition smart interaction Target recognition |
| title | Watch & Do: A Smart IoT Interaction System with Object Detection and Gaze Estimation |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-04T15%3A34%3A24IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Watch%20&%20Do:%20A%20Smart%20IoT%20Interaction%20System%20with%20Object%20Detection%20and%20Gaze%20Estimation&rft.jtitle=IEEE%20transactions%20on%20consumer%20electronics&rft.au=Kim,%20Jung-Hwa&rft.date=2019-05-01&rft.volume=65&rft.issue=2&rft.spage=195&rft.epage=204&rft.pages=195-204&rft.issn=0098-3063&rft.eissn=1558-4127&rft.coden=ITCEDA&rft_id=info:doi/10.1109/TCE.2019.2897758&rft_dat=%3Cproquest_RIE%3E2215060760%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2215060760&rft_id=info:pmid/&rft_ieee_id=8634883&rfr_iscdi=true |