Robust and Efficient RGB-D SLAM in Dynamic Environments

Simultaneous localization and mapping (SLAM) using an RGB-D camera is a key enabling technique for many augmented reality (AR) applications. However, most existing RGB-D SLAM methods could fail in dynamic scenarios due to non-trivial pose estimation errors arising from moving objects. In this study,...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on multimedia 2021, Vol.23, p.4208-4219
Hauptverfasser: Yang, Xin, Yuan, Zikang, Zhu, Dongfu, Chi, Cheng, Li, Kun, Liao, Chunyuan
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 4219
container_issue
container_start_page 4208
container_title IEEE transactions on multimedia
container_volume 23
creator Yang, Xin
Yuan, Zikang
Zhu, Dongfu
Chi, Cheng
Li, Kun
Liao, Chunyuan
description Simultaneous localization and mapping (SLAM) using an RGB-D camera is a key enabling technique for many augmented reality (AR) applications. However, most existing RGB-D SLAM methods could fail in dynamic scenarios due to non-trivial pose estimation errors arising from moving objects. In this study, we present an accurate and robust RGB-D SLAM system for dynamic scenarios which can run real-time on a single dual-core CPU. The core of our system is a robust and efficient dynamic keypoint exclusion method which consists of three steps: 1) grouping spatially and appearance related pixels of a keyframe into regions; 2) identifying dynamic regions by checking motion consistency of keypoints in every region; 3) excluding keypoints in the identified dynamic regions as well as the matching points in the 3D local map. The dynamic keypoint exclusion method can be easily integrated into any keypoint based RGB-D SLAM system for improving the accuracy and robustness in dynamic scenes with trivial time increase (16.6ms per frame). Experimental results on the TUM dataset demonstrates that our method which runs on an Intel i7-4900 CPU is even 2.3X faster than the state-of-the-art method DS-SLAM [1] which runs parallel on a P4000 GPU and a comparable CPU. In addition, our system outperforms the state-of-the-art methods [1]-[4] in terms of smaller absolute trajectory errors (ATE). We also apply our system to a real AR application and live experiments with a hand-held RGB-D camera demonstrate the robustness and generalizability of our method in practical scenarios. 1 1 A demo video is provided on https://github.com/cc-qy/Dynamic-RGB-D-SLAM
doi_str_mv 10.1109/TMM.2020.3038323
format Article
fullrecord <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_ieee_primary_9261135</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9261135</ieee_id><sourcerecordid>2599215231</sourcerecordid><originalsourceid>FETCH-LOGICAL-c291t-af34ffba2ee3c7ec87ef628426c8e643b42196e2d96eb820a6103b3cecd9ccc93</originalsourceid><addsrcrecordid>eNo9kE1LAzEQhoMoWKt3wUvA89bJZL9yrLVWoUWo9Ryy6QRSbLZutkL_vSktXmbm8LwzzMPYvYCREKCeVovFCAFhJEHWEuUFGwiViwygqi7TXCBkCgVcs5sYNwAiL6AasGrZNvvYcxPWfOqct55Cz5ez5-yFf87HC-4DfzkEs_WWT8Ov79qwTUS8ZVfOfEe6O_ch-3qdriZv2fxj9j4ZzzOLSvSZcTJ3rjFIJG1Ftq7IlVjnWNqaylw2OQpVEq5TaWoEUwqQjbRk18paq-SQPZ727rr2Z0-x15t234V0UmOh0kMFSpEoOFG2a2PsyOld57emO2gB-qhHJz36qEef9aTIwyniiegfV1gKIQv5B5rNXvI</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2599215231</pqid></control><display><type>article</type><title>Robust and Efficient RGB-D SLAM in Dynamic Environments</title><source>IEEE Electronic Library (IEL)</source><creator>Yang, Xin ; Yuan, Zikang ; Zhu, Dongfu ; Chi, Cheng ; Li, Kun ; Liao, Chunyuan</creator><creatorcontrib>Yang, Xin ; Yuan, Zikang ; Zhu, Dongfu ; Chi, Cheng ; Li, Kun ; Liao, Chunyuan</creatorcontrib><description>Simultaneous localization and mapping (SLAM) using an RGB-D camera is a key enabling technique for many augmented reality (AR) applications. However, most existing RGB-D SLAM methods could fail in dynamic scenarios due to non-trivial pose estimation errors arising from moving objects. In this study, we present an accurate and robust RGB-D SLAM system for dynamic scenarios which can run real-time on a single dual-core CPU. The core of our system is a robust and efficient dynamic keypoint exclusion method which consists of three steps: 1) grouping spatially and appearance related pixels of a keyframe into regions; 2) identifying dynamic regions by checking motion consistency of keypoints in every region; 3) excluding keypoints in the identified dynamic regions as well as the matching points in the 3D local map. The dynamic keypoint exclusion method can be easily integrated into any keypoint based RGB-D SLAM system for improving the accuracy and robustness in dynamic scenes with trivial time increase (16.6ms per frame). Experimental results on the TUM dataset demonstrates that our method which runs on an Intel i7-4900 CPU is even 2.3X faster than the state-of-the-art method DS-SLAM [1] which runs parallel on a P4000 GPU and a comparable CPU. In addition, our system outperforms the state-of-the-art methods [1]-[4] in terms of smaller absolute trajectory errors (ATE). We also apply our system to a real AR application and live experiments with a hand-held RGB-D camera demonstrate the robustness and generalizability of our method in practical scenarios.&lt;xref ref-type="fn" rid="fn1"&gt; 1 1 A demo video is provided on https://github.com/cc-qy/Dynamic-RGB-D-SLAM</description><identifier>ISSN: 1520-9210</identifier><identifier>EISSN: 1941-0077</identifier><identifier>DOI: 10.1109/TMM.2020.3038323</identifier><identifier>CODEN: ITMUF8</identifier><language>eng</language><publisher>Piscataway: IEEE</publisher><subject>Augmented reality ; Cameras ; Central processing units ; CPUs ; Dynamics ; Errors ; Motion segmentation ; Pose estimation ; robot sensing systems ; Robotics and automation ; robots ; Robustness ; Simultaneous localization and mapping ; Three-dimensional displays</subject><ispartof>IEEE transactions on multimedia, 2021, Vol.23, p.4208-4219</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c291t-af34ffba2ee3c7ec87ef628426c8e643b42196e2d96eb820a6103b3cecd9ccc93</citedby><cites>FETCH-LOGICAL-c291t-af34ffba2ee3c7ec87ef628426c8e643b42196e2d96eb820a6103b3cecd9ccc93</cites><orcidid>0000-0001-6252-1061</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9261135$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,796,4024,27923,27924,27925,54758</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/9261135$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Yang, Xin</creatorcontrib><creatorcontrib>Yuan, Zikang</creatorcontrib><creatorcontrib>Zhu, Dongfu</creatorcontrib><creatorcontrib>Chi, Cheng</creatorcontrib><creatorcontrib>Li, Kun</creatorcontrib><creatorcontrib>Liao, Chunyuan</creatorcontrib><title>Robust and Efficient RGB-D SLAM in Dynamic Environments</title><title>IEEE transactions on multimedia</title><addtitle>TMM</addtitle><description>Simultaneous localization and mapping (SLAM) using an RGB-D camera is a key enabling technique for many augmented reality (AR) applications. However, most existing RGB-D SLAM methods could fail in dynamic scenarios due to non-trivial pose estimation errors arising from moving objects. In this study, we present an accurate and robust RGB-D SLAM system for dynamic scenarios which can run real-time on a single dual-core CPU. The core of our system is a robust and efficient dynamic keypoint exclusion method which consists of three steps: 1) grouping spatially and appearance related pixels of a keyframe into regions; 2) identifying dynamic regions by checking motion consistency of keypoints in every region; 3) excluding keypoints in the identified dynamic regions as well as the matching points in the 3D local map. The dynamic keypoint exclusion method can be easily integrated into any keypoint based RGB-D SLAM system for improving the accuracy and robustness in dynamic scenes with trivial time increase (16.6ms per frame). Experimental results on the TUM dataset demonstrates that our method which runs on an Intel i7-4900 CPU is even 2.3X faster than the state-of-the-art method DS-SLAM [1] which runs parallel on a P4000 GPU and a comparable CPU. In addition, our system outperforms the state-of-the-art methods [1]-[4] in terms of smaller absolute trajectory errors (ATE). We also apply our system to a real AR application and live experiments with a hand-held RGB-D camera demonstrate the robustness and generalizability of our method in practical scenarios.&lt;xref ref-type="fn" rid="fn1"&gt; 1 1 A demo video is provided on https://github.com/cc-qy/Dynamic-RGB-D-SLAM</description><subject>Augmented reality</subject><subject>Cameras</subject><subject>Central processing units</subject><subject>CPUs</subject><subject>Dynamics</subject><subject>Errors</subject><subject>Motion segmentation</subject><subject>Pose estimation</subject><subject>robot sensing systems</subject><subject>Robotics and automation</subject><subject>robots</subject><subject>Robustness</subject><subject>Simultaneous localization and mapping</subject><subject>Three-dimensional displays</subject><issn>1520-9210</issn><issn>1941-0077</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNo9kE1LAzEQhoMoWKt3wUvA89bJZL9yrLVWoUWo9Ryy6QRSbLZutkL_vSktXmbm8LwzzMPYvYCREKCeVovFCAFhJEHWEuUFGwiViwygqi7TXCBkCgVcs5sYNwAiL6AasGrZNvvYcxPWfOqct55Cz5ez5-yFf87HC-4DfzkEs_WWT8Ov79qwTUS8ZVfOfEe6O_ch-3qdriZv2fxj9j4ZzzOLSvSZcTJ3rjFIJG1Ftq7IlVjnWNqaylw2OQpVEq5TaWoEUwqQjbRk18paq-SQPZ727rr2Z0-x15t234V0UmOh0kMFSpEoOFG2a2PsyOld57emO2gB-qhHJz36qEef9aTIwyniiegfV1gKIQv5B5rNXvI</recordid><startdate>2021</startdate><enddate>2021</enddate><creator>Yang, Xin</creator><creator>Yuan, Zikang</creator><creator>Zhu, Dongfu</creator><creator>Chi, Cheng</creator><creator>Li, Kun</creator><creator>Liao, Chunyuan</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0001-6252-1061</orcidid></search><sort><creationdate>2021</creationdate><title>Robust and Efficient RGB-D SLAM in Dynamic Environments</title><author>Yang, Xin ; Yuan, Zikang ; Zhu, Dongfu ; Chi, Cheng ; Li, Kun ; Liao, Chunyuan</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c291t-af34ffba2ee3c7ec87ef628426c8e643b42196e2d96eb820a6103b3cecd9ccc93</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Augmented reality</topic><topic>Cameras</topic><topic>Central processing units</topic><topic>CPUs</topic><topic>Dynamics</topic><topic>Errors</topic><topic>Motion segmentation</topic><topic>Pose estimation</topic><topic>robot sensing systems</topic><topic>Robotics and automation</topic><topic>robots</topic><topic>Robustness</topic><topic>Simultaneous localization and mapping</topic><topic>Three-dimensional displays</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Yang, Xin</creatorcontrib><creatorcontrib>Yuan, Zikang</creatorcontrib><creatorcontrib>Zhu, Dongfu</creatorcontrib><creatorcontrib>Chi, Cheng</creatorcontrib><creatorcontrib>Li, Kun</creatorcontrib><creatorcontrib>Liao, Chunyuan</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE transactions on multimedia</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Yang, Xin</au><au>Yuan, Zikang</au><au>Zhu, Dongfu</au><au>Chi, Cheng</au><au>Li, Kun</au><au>Liao, Chunyuan</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Robust and Efficient RGB-D SLAM in Dynamic Environments</atitle><jtitle>IEEE transactions on multimedia</jtitle><stitle>TMM</stitle><date>2021</date><risdate>2021</risdate><volume>23</volume><spage>4208</spage><epage>4219</epage><pages>4208-4219</pages><issn>1520-9210</issn><eissn>1941-0077</eissn><coden>ITMUF8</coden><abstract>Simultaneous localization and mapping (SLAM) using an RGB-D camera is a key enabling technique for many augmented reality (AR) applications. However, most existing RGB-D SLAM methods could fail in dynamic scenarios due to non-trivial pose estimation errors arising from moving objects. In this study, we present an accurate and robust RGB-D SLAM system for dynamic scenarios which can run real-time on a single dual-core CPU. The core of our system is a robust and efficient dynamic keypoint exclusion method which consists of three steps: 1) grouping spatially and appearance related pixels of a keyframe into regions; 2) identifying dynamic regions by checking motion consistency of keypoints in every region; 3) excluding keypoints in the identified dynamic regions as well as the matching points in the 3D local map. The dynamic keypoint exclusion method can be easily integrated into any keypoint based RGB-D SLAM system for improving the accuracy and robustness in dynamic scenes with trivial time increase (16.6ms per frame). Experimental results on the TUM dataset demonstrates that our method which runs on an Intel i7-4900 CPU is even 2.3X faster than the state-of-the-art method DS-SLAM [1] which runs parallel on a P4000 GPU and a comparable CPU. In addition, our system outperforms the state-of-the-art methods [1]-[4] in terms of smaller absolute trajectory errors (ATE). We also apply our system to a real AR application and live experiments with a hand-held RGB-D camera demonstrate the robustness and generalizability of our method in practical scenarios.&lt;xref ref-type="fn" rid="fn1"&gt; 1 1 A demo video is provided on https://github.com/cc-qy/Dynamic-RGB-D-SLAM</abstract><cop>Piscataway</cop><pub>IEEE</pub><doi>10.1109/TMM.2020.3038323</doi><tpages>12</tpages><orcidid>https://orcid.org/0000-0001-6252-1061</orcidid></addata></record>
fulltext fulltext_linktorsrc
identifier ISSN: 1520-9210
ispartof IEEE transactions on multimedia, 2021, Vol.23, p.4208-4219
issn 1520-9210
1941-0077
language eng
recordid cdi_ieee_primary_9261135
source IEEE Electronic Library (IEL)
subjects Augmented reality
Cameras
Central processing units
CPUs
Dynamics
Errors
Motion segmentation
Pose estimation
robot sensing systems
Robotics and automation
robots
Robustness
Simultaneous localization and mapping
Three-dimensional displays
title Robust and Efficient RGB-D SLAM in Dynamic Environments
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-30T01%3A57%3A19IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Robust%20and%20Efficient%20RGB-D%20SLAM%20in%20Dynamic%20Environments&rft.jtitle=IEEE%20transactions%20on%20multimedia&rft.au=Yang,%20Xin&rft.date=2021&rft.volume=23&rft.spage=4208&rft.epage=4219&rft.pages=4208-4219&rft.issn=1520-9210&rft.eissn=1941-0077&rft.coden=ITMUF8&rft_id=info:doi/10.1109/TMM.2020.3038323&rft_dat=%3Cproquest_RIE%3E2599215231%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2599215231&rft_id=info:pmid/&rft_ieee_id=9261135&rfr_iscdi=true