GPU-Accelerated Real-Time Stereo Estimation With Binary Neural Network
Depth estimation from stereo images is essential to many applications such as robotics and autonomous vehicles, most of which ask for the real-time response, high energy and storage efficiency. Recent work has shown deep neural networks (DNN) perform extremely well for stereo estimation. However, th...
Gespeichert in:
Veröffentlicht in: | IEEE transactions on parallel and distributed systems 2020-12, Vol.31 (12), p.2896-2907 |
---|---|
Hauptverfasser: | , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 2907 |
---|---|
container_issue | 12 |
container_start_page | 2896 |
container_title | IEEE transactions on parallel and distributed systems |
container_volume | 31 |
creator | Chen, Gang Meng, Haitao Liang, Yucheng Huang, Kai |
description | Depth estimation from stereo images is essential to many applications such as robotics and autonomous vehicles, most of which ask for the real-time response, high energy and storage efficiency. Recent work has shown deep neural networks (DNN) perform extremely well for stereo estimation. However, these state-of-the-art DNN based algorithms are challenging to be deployed into real-world applications due to the high computational complexities of DNNs. Most of them are too slow for real-time inference and require several seconds of GPU computation to process image frames. In this article, we address the problem of fast stereo estimation and propose an efficient and light-weighted stereo matching system, called StereoBit, to produce a disparity map in a real-time manner while achieving close to state-of-the-art accuracy. To achieve this goal, we propose a binary neural network to generate weighted Hamming distance for an efficient similarity join in stereo estimation. In addition, we propose a novel approximation approach to derive StereoBit network directly from the well-trained network with the cosine similarity. Our approximation strategies enable a significant speedup while maintaining almost the same accuracy compared to the network with the cosine similarity. Furthermore, we present an optimization framework for fully exploiting the computing power of StereoBit. The framework provides a significant speedup of stereo estimation routines, and at the same time, reduces the memory usage for storing parameters. The effectiveness of StereoBit is evaluated by comprehensive experiments. StereoBit can achieve 60 frames per second on an NVIDIA TITAN Xp GPU on KITTI 2012 benchmark while achieving 3-pixel non-occluded stereo error 3.56 percent. |
doi_str_mv | 10.1109/TPDS.2020.3006238 |
format | Article |
fullrecord | <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_proquest_journals_2424188219</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9130887</ieee_id><sourcerecordid>2424188219</sourcerecordid><originalsourceid>FETCH-LOGICAL-c293t-104a2ea1c2ed771bfaf6cdffedc8cd4f7ec4ceeceee6d4335731c7972c8f48603</originalsourceid><addsrcrecordid>eNo9UE1LAzEUDKJgrf4A8bLgOTVfu8kea22rULTYFo8hZl9w67ZbkyzSf29Ki_Bg3mHmzZtB6JaSAaWkfFjOnxYDRhgZcEIKxtUZ6tE8V5hRxc_TTkSOS0bLS3QVwpoQKnIiemgyna_w0FpowJsIVfYOpsHLegPZIoKHNhuHWG9MrNtt9lHHr-yx3hq_z16h86ZJEH9b_32NLpxpAtycsI9Wk_Fy9Ixnb9OX0XCGLSt5xOkLw8BQy6CSkn464wpbOQeVVbYSToIVFiANFJXgPJecWllKZpUTqiC8j-6Pd3e-_ekgRL1uO79NlpoJJqhSKWJi0SPL-jYED07vfMrg95oSfahLH-rSh7r0qa6kuTtq6mT-zy8pJ0pJ_geFvWah</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2424188219</pqid></control><display><type>article</type><title>GPU-Accelerated Real-Time Stereo Estimation With Binary Neural Network</title><source>IEEE Electronic Library (IEL)</source><creator>Chen, Gang ; Meng, Haitao ; Liang, Yucheng ; Huang, Kai</creator><creatorcontrib>Chen, Gang ; Meng, Haitao ; Liang, Yucheng ; Huang, Kai</creatorcontrib><description>Depth estimation from stereo images is essential to many applications such as robotics and autonomous vehicles, most of which ask for the real-time response, high energy and storage efficiency. Recent work has shown deep neural networks (DNN) perform extremely well for stereo estimation. However, these state-of-the-art DNN based algorithms are challenging to be deployed into real-world applications due to the high computational complexities of DNNs. Most of them are too slow for real-time inference and require several seconds of GPU computation to process image frames. In this article, we address the problem of fast stereo estimation and propose an efficient and light-weighted stereo matching system, called StereoBit, to produce a disparity map in a real-time manner while achieving close to state-of-the-art accuracy. To achieve this goal, we propose a binary neural network to generate weighted Hamming distance for an efficient similarity join in stereo estimation. In addition, we propose a novel approximation approach to derive StereoBit network directly from the well-trained network with the cosine similarity. Our approximation strategies enable a significant speedup while maintaining almost the same accuracy compared to the network with the cosine similarity. Furthermore, we present an optimization framework for fully exploiting the computing power of StereoBit. The framework provides a significant speedup of stereo estimation routines, and at the same time, reduces the memory usage for storing parameters. The effectiveness of StereoBit is evaluated by comprehensive experiments. StereoBit can achieve 60 frames per second on an NVIDIA TITAN Xp GPU on KITTI 2012 benchmark while achieving 3-pixel non-occluded stereo error 3.56 percent.</description><identifier>ISSN: 1045-9219</identifier><identifier>EISSN: 1558-2183</identifier><identifier>DOI: 10.1109/TPDS.2020.3006238</identifier><identifier>CODEN: ITDSEO</identifier><language>eng</language><publisher>New York: IEEE</publisher><subject>Algorithms ; Approximation ; Artificial neural networks ; binary neural network ; Computational modeling ; Convolution ; Energy storage ; Estimation ; Frames (data processing) ; Frames per second ; GPU acceleration ; Graphics processing units ; Mathematical analysis ; Neural networks ; Optimization ; Real time ; Real-time systems ; Robotics ; Similarity ; stereo estimation ; Time response</subject><ispartof>IEEE transactions on parallel and distributed systems, 2020-12, Vol.31 (12), p.2896-2907</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c293t-104a2ea1c2ed771bfaf6cdffedc8cd4f7ec4ceeceee6d4335731c7972c8f48603</citedby><cites>FETCH-LOGICAL-c293t-104a2ea1c2ed771bfaf6cdffedc8cd4f7ec4ceeceee6d4335731c7972c8f48603</cites><orcidid>0000-0001-9917-5625 ; 0000-0003-0359-7810 ; 0000-0003-4234-1359</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9130887$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,796,27924,27925,54758</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/9130887$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Chen, Gang</creatorcontrib><creatorcontrib>Meng, Haitao</creatorcontrib><creatorcontrib>Liang, Yucheng</creatorcontrib><creatorcontrib>Huang, Kai</creatorcontrib><title>GPU-Accelerated Real-Time Stereo Estimation With Binary Neural Network</title><title>IEEE transactions on parallel and distributed systems</title><addtitle>TPDS</addtitle><description>Depth estimation from stereo images is essential to many applications such as robotics and autonomous vehicles, most of which ask for the real-time response, high energy and storage efficiency. Recent work has shown deep neural networks (DNN) perform extremely well for stereo estimation. However, these state-of-the-art DNN based algorithms are challenging to be deployed into real-world applications due to the high computational complexities of DNNs. Most of them are too slow for real-time inference and require several seconds of GPU computation to process image frames. In this article, we address the problem of fast stereo estimation and propose an efficient and light-weighted stereo matching system, called StereoBit, to produce a disparity map in a real-time manner while achieving close to state-of-the-art accuracy. To achieve this goal, we propose a binary neural network to generate weighted Hamming distance for an efficient similarity join in stereo estimation. In addition, we propose a novel approximation approach to derive StereoBit network directly from the well-trained network with the cosine similarity. Our approximation strategies enable a significant speedup while maintaining almost the same accuracy compared to the network with the cosine similarity. Furthermore, we present an optimization framework for fully exploiting the computing power of StereoBit. The framework provides a significant speedup of stereo estimation routines, and at the same time, reduces the memory usage for storing parameters. The effectiveness of StereoBit is evaluated by comprehensive experiments. StereoBit can achieve 60 frames per second on an NVIDIA TITAN Xp GPU on KITTI 2012 benchmark while achieving 3-pixel non-occluded stereo error 3.56 percent.</description><subject>Algorithms</subject><subject>Approximation</subject><subject>Artificial neural networks</subject><subject>binary neural network</subject><subject>Computational modeling</subject><subject>Convolution</subject><subject>Energy storage</subject><subject>Estimation</subject><subject>Frames (data processing)</subject><subject>Frames per second</subject><subject>GPU acceleration</subject><subject>Graphics processing units</subject><subject>Mathematical analysis</subject><subject>Neural networks</subject><subject>Optimization</subject><subject>Real time</subject><subject>Real-time systems</subject><subject>Robotics</subject><subject>Similarity</subject><subject>stereo estimation</subject><subject>Time response</subject><issn>1045-9219</issn><issn>1558-2183</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNo9UE1LAzEUDKJgrf4A8bLgOTVfu8kea22rULTYFo8hZl9w67ZbkyzSf29Ki_Bg3mHmzZtB6JaSAaWkfFjOnxYDRhgZcEIKxtUZ6tE8V5hRxc_TTkSOS0bLS3QVwpoQKnIiemgyna_w0FpowJsIVfYOpsHLegPZIoKHNhuHWG9MrNtt9lHHr-yx3hq_z16h86ZJEH9b_32NLpxpAtycsI9Wk_Fy9Ixnb9OX0XCGLSt5xOkLw8BQy6CSkn464wpbOQeVVbYSToIVFiANFJXgPJecWllKZpUTqiC8j-6Pd3e-_ekgRL1uO79NlpoJJqhSKWJi0SPL-jYED07vfMrg95oSfahLH-rSh7r0qa6kuTtq6mT-zy8pJ0pJ_geFvWah</recordid><startdate>20201201</startdate><enddate>20201201</enddate><creator>Chen, Gang</creator><creator>Meng, Haitao</creator><creator>Liang, Yucheng</creator><creator>Huang, Kai</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0001-9917-5625</orcidid><orcidid>https://orcid.org/0000-0003-0359-7810</orcidid><orcidid>https://orcid.org/0000-0003-4234-1359</orcidid></search><sort><creationdate>20201201</creationdate><title>GPU-Accelerated Real-Time Stereo Estimation With Binary Neural Network</title><author>Chen, Gang ; Meng, Haitao ; Liang, Yucheng ; Huang, Kai</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c293t-104a2ea1c2ed771bfaf6cdffedc8cd4f7ec4ceeceee6d4335731c7972c8f48603</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Algorithms</topic><topic>Approximation</topic><topic>Artificial neural networks</topic><topic>binary neural network</topic><topic>Computational modeling</topic><topic>Convolution</topic><topic>Energy storage</topic><topic>Estimation</topic><topic>Frames (data processing)</topic><topic>Frames per second</topic><topic>GPU acceleration</topic><topic>Graphics processing units</topic><topic>Mathematical analysis</topic><topic>Neural networks</topic><topic>Optimization</topic><topic>Real time</topic><topic>Real-time systems</topic><topic>Robotics</topic><topic>Similarity</topic><topic>stereo estimation</topic><topic>Time response</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Chen, Gang</creatorcontrib><creatorcontrib>Meng, Haitao</creatorcontrib><creatorcontrib>Liang, Yucheng</creatorcontrib><creatorcontrib>Huang, Kai</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE transactions on parallel and distributed systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Chen, Gang</au><au>Meng, Haitao</au><au>Liang, Yucheng</au><au>Huang, Kai</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>GPU-Accelerated Real-Time Stereo Estimation With Binary Neural Network</atitle><jtitle>IEEE transactions on parallel and distributed systems</jtitle><stitle>TPDS</stitle><date>2020-12-01</date><risdate>2020</risdate><volume>31</volume><issue>12</issue><spage>2896</spage><epage>2907</epage><pages>2896-2907</pages><issn>1045-9219</issn><eissn>1558-2183</eissn><coden>ITDSEO</coden><abstract>Depth estimation from stereo images is essential to many applications such as robotics and autonomous vehicles, most of which ask for the real-time response, high energy and storage efficiency. Recent work has shown deep neural networks (DNN) perform extremely well for stereo estimation. However, these state-of-the-art DNN based algorithms are challenging to be deployed into real-world applications due to the high computational complexities of DNNs. Most of them are too slow for real-time inference and require several seconds of GPU computation to process image frames. In this article, we address the problem of fast stereo estimation and propose an efficient and light-weighted stereo matching system, called StereoBit, to produce a disparity map in a real-time manner while achieving close to state-of-the-art accuracy. To achieve this goal, we propose a binary neural network to generate weighted Hamming distance for an efficient similarity join in stereo estimation. In addition, we propose a novel approximation approach to derive StereoBit network directly from the well-trained network with the cosine similarity. Our approximation strategies enable a significant speedup while maintaining almost the same accuracy compared to the network with the cosine similarity. Furthermore, we present an optimization framework for fully exploiting the computing power of StereoBit. The framework provides a significant speedup of stereo estimation routines, and at the same time, reduces the memory usage for storing parameters. The effectiveness of StereoBit is evaluated by comprehensive experiments. StereoBit can achieve 60 frames per second on an NVIDIA TITAN Xp GPU on KITTI 2012 benchmark while achieving 3-pixel non-occluded stereo error 3.56 percent.</abstract><cop>New York</cop><pub>IEEE</pub><doi>10.1109/TPDS.2020.3006238</doi><tpages>12</tpages><orcidid>https://orcid.org/0000-0001-9917-5625</orcidid><orcidid>https://orcid.org/0000-0003-0359-7810</orcidid><orcidid>https://orcid.org/0000-0003-4234-1359</orcidid></addata></record> |
fulltext | fulltext_linktorsrc |
identifier | ISSN: 1045-9219 |
ispartof | IEEE transactions on parallel and distributed systems, 2020-12, Vol.31 (12), p.2896-2907 |
issn | 1045-9219 1558-2183 |
language | eng |
recordid | cdi_proquest_journals_2424188219 |
source | IEEE Electronic Library (IEL) |
subjects | Algorithms Approximation Artificial neural networks binary neural network Computational modeling Convolution Energy storage Estimation Frames (data processing) Frames per second GPU acceleration Graphics processing units Mathematical analysis Neural networks Optimization Real time Real-time systems Robotics Similarity stereo estimation Time response |
title | GPU-Accelerated Real-Time Stereo Estimation With Binary Neural Network |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T21%3A19%3A57IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=GPU-Accelerated%20Real-Time%20Stereo%20Estimation%20With%20Binary%20Neural%20Network&rft.jtitle=IEEE%20transactions%20on%20parallel%20and%20distributed%20systems&rft.au=Chen,%20Gang&rft.date=2020-12-01&rft.volume=31&rft.issue=12&rft.spage=2896&rft.epage=2907&rft.pages=2896-2907&rft.issn=1045-9219&rft.eissn=1558-2183&rft.coden=ITDSEO&rft_id=info:doi/10.1109/TPDS.2020.3006238&rft_dat=%3Cproquest_RIE%3E2424188219%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2424188219&rft_id=info:pmid/&rft_ieee_id=9130887&rfr_iscdi=true |