Adv-Depth: Self-Supervised Monocular Depth Estimation With an Adversarial Loss

Loss function plays a key role in self-supervised monocular depth estimation methods. Current reprojection loss functions are hand-designed and mainly focus on local patch similarity but overlook the global distribution differences between a synthetic image and a target image. In this paper, we leve...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE signal processing letters 2021, Vol.28, p.638-642
Hauptverfasser: Li, Kunhong, Fu, Zhiheng, Wang, Hanyun, Chen, Zonghao, Guo, Yulan
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 642
container_issue
container_start_page 638
container_title IEEE signal processing letters
container_volume 28
creator Li, Kunhong
Fu, Zhiheng
Wang, Hanyun
Chen, Zonghao
Guo, Yulan
description Loss function plays a key role in self-supervised monocular depth estimation methods. Current reprojection loss functions are hand-designed and mainly focus on local patch similarity but overlook the global distribution differences between a synthetic image and a target image. In this paper, we leverage global distribution differences by introducing an adversarial loss into the training stage of self-supervised depth estimation. Specifically, we formulate this task as a novel view synthesis problem. We use a depth estimation module and a pose estimation module to form a generator, and then design a discriminator to learn the global distribution differences between real and synthetic images. With the learned global distribution differences, the adversarial loss can be back-propagated to the depth estimation module to improve its performance. Experiments on the KITTI dataset have demonstrated the effectiveness of the adversarial loss. The adversarial loss is further combined with the reprojection loss to achieve the state-of-the-art performance on the KITTI dataset.
doi_str_mv 10.1109/LSP.2021.3065203
format Article
fullrecord <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_crossref_primary_10_1109_LSP_2021_3065203</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9376591</ieee_id><sourcerecordid>2515863572</sourcerecordid><originalsourceid>FETCH-LOGICAL-c291t-1f39a31c3044d7dda930e4eff9a6a31692ce8572558cf5ba9ed7163ce8da1f253</originalsourceid><addsrcrecordid>eNo9UE1Lw0AQXUTBWr0LXgKeE2d2u0nWW6n1A-IHVPG4rMkspsQk7iYF_71bWzzNzOO9mXmPsXOEBBHUVbF6SThwTASkkoM4YBOUMo-5SPEw9JBBrBTkx-zE-zUA5JjLCXuaV5v4hvrh8zpaUWPj1diT29Sequixa7tybIyL_gjR0g_1lxnqro3e6zCbNgpqct642jRR0Xl_yo6saTyd7euUvd0uXxf3cfF897CYF3HJFQ4xWqGMwFLAbFZlVWWUAJqRtcqkAU8VLymXGQ8GSis_jKIqw1QEsDJouRRTdrnb27vueyQ_6HU3ujac1FyizFMR1IEFO1bpwm-OrO5dcOB-NILepqZDanqbmt6nFiQXO0lNRP90JbJUKhS_7DJn9Q</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2515863572</pqid></control><display><type>article</type><title>Adv-Depth: Self-Supervised Monocular Depth Estimation With an Adversarial Loss</title><source>IEEE Electronic Library (IEL)</source><creator>Li, Kunhong ; Fu, Zhiheng ; Wang, Hanyun ; Chen, Zonghao ; Guo, Yulan</creator><creatorcontrib>Li, Kunhong ; Fu, Zhiheng ; Wang, Hanyun ; Chen, Zonghao ; Guo, Yulan</creatorcontrib><description>Loss function plays a key role in self-supervised monocular depth estimation methods. Current reprojection loss functions are hand-designed and mainly focus on local patch similarity but overlook the global distribution differences between a synthetic image and a target image. In this paper, we leverage global distribution differences by introducing an adversarial loss into the training stage of self-supervised depth estimation. Specifically, we formulate this task as a novel view synthesis problem. We use a depth estimation module and a pose estimation module to form a generator, and then design a discriminator to learn the global distribution differences between real and synthetic images. With the learned global distribution differences, the adversarial loss can be back-propagated to the depth estimation module to improve its performance. Experiments on the KITTI dataset have demonstrated the effectiveness of the adversarial loss. The adversarial loss is further combined with the reprojection loss to achieve the state-of-the-art performance on the KITTI dataset.</description><identifier>ISSN: 1070-9908</identifier><identifier>EISSN: 1558-2361</identifier><identifier>DOI: 10.1109/LSP.2021.3065203</identifier><identifier>CODEN: ISPLEM</identifier><language>eng</language><publisher>New York: IEEE</publisher><subject>Back propagation ; Datasets ; Estimation ; Feature extraction ; Gallium nitride ; Generative adversarial networks ; Generators ; Modules ; Monocular depth estimation ; self-supervised learning ; single-image depth prediction ; Target recognition ; Task analysis ; Training</subject><ispartof>IEEE signal processing letters, 2021, Vol.28, p.638-642</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c291t-1f39a31c3044d7dda930e4eff9a6a31692ce8572558cf5ba9ed7163ce8da1f253</citedby><cites>FETCH-LOGICAL-c291t-1f39a31c3044d7dda930e4eff9a6a31692ce8572558cf5ba9ed7163ce8da1f253</cites><orcidid>0000-0001-7051-841X ; 0000-0002-8320-4230</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9376591$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,796,4024,27923,27924,27925,54758</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/9376591$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Li, Kunhong</creatorcontrib><creatorcontrib>Fu, Zhiheng</creatorcontrib><creatorcontrib>Wang, Hanyun</creatorcontrib><creatorcontrib>Chen, Zonghao</creatorcontrib><creatorcontrib>Guo, Yulan</creatorcontrib><title>Adv-Depth: Self-Supervised Monocular Depth Estimation With an Adversarial Loss</title><title>IEEE signal processing letters</title><addtitle>LSP</addtitle><description>Loss function plays a key role in self-supervised monocular depth estimation methods. Current reprojection loss functions are hand-designed and mainly focus on local patch similarity but overlook the global distribution differences between a synthetic image and a target image. In this paper, we leverage global distribution differences by introducing an adversarial loss into the training stage of self-supervised depth estimation. Specifically, we formulate this task as a novel view synthesis problem. We use a depth estimation module and a pose estimation module to form a generator, and then design a discriminator to learn the global distribution differences between real and synthetic images. With the learned global distribution differences, the adversarial loss can be back-propagated to the depth estimation module to improve its performance. Experiments on the KITTI dataset have demonstrated the effectiveness of the adversarial loss. The adversarial loss is further combined with the reprojection loss to achieve the state-of-the-art performance on the KITTI dataset.</description><subject>Back propagation</subject><subject>Datasets</subject><subject>Estimation</subject><subject>Feature extraction</subject><subject>Gallium nitride</subject><subject>Generative adversarial networks</subject><subject>Generators</subject><subject>Modules</subject><subject>Monocular depth estimation</subject><subject>self-supervised learning</subject><subject>single-image depth prediction</subject><subject>Target recognition</subject><subject>Task analysis</subject><subject>Training</subject><issn>1070-9908</issn><issn>1558-2361</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNo9UE1Lw0AQXUTBWr0LXgKeE2d2u0nWW6n1A-IHVPG4rMkspsQk7iYF_71bWzzNzOO9mXmPsXOEBBHUVbF6SThwTASkkoM4YBOUMo-5SPEw9JBBrBTkx-zE-zUA5JjLCXuaV5v4hvrh8zpaUWPj1diT29Sequixa7tybIyL_gjR0g_1lxnqro3e6zCbNgpqct642jRR0Xl_yo6saTyd7euUvd0uXxf3cfF897CYF3HJFQ4xWqGMwFLAbFZlVWWUAJqRtcqkAU8VLymXGQ8GSis_jKIqw1QEsDJouRRTdrnb27vueyQ_6HU3ujac1FyizFMR1IEFO1bpwm-OrO5dcOB-NILepqZDanqbmt6nFiQXO0lNRP90JbJUKhS_7DJn9Q</recordid><startdate>2021</startdate><enddate>2021</enddate><creator>Li, Kunhong</creator><creator>Fu, Zhiheng</creator><creator>Wang, Hanyun</creator><creator>Chen, Zonghao</creator><creator>Guo, Yulan</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0001-7051-841X</orcidid><orcidid>https://orcid.org/0000-0002-8320-4230</orcidid></search><sort><creationdate>2021</creationdate><title>Adv-Depth: Self-Supervised Monocular Depth Estimation With an Adversarial Loss</title><author>Li, Kunhong ; Fu, Zhiheng ; Wang, Hanyun ; Chen, Zonghao ; Guo, Yulan</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c291t-1f39a31c3044d7dda930e4eff9a6a31692ce8572558cf5ba9ed7163ce8da1f253</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Back propagation</topic><topic>Datasets</topic><topic>Estimation</topic><topic>Feature extraction</topic><topic>Gallium nitride</topic><topic>Generative adversarial networks</topic><topic>Generators</topic><topic>Modules</topic><topic>Monocular depth estimation</topic><topic>self-supervised learning</topic><topic>single-image depth prediction</topic><topic>Target recognition</topic><topic>Task analysis</topic><topic>Training</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Li, Kunhong</creatorcontrib><creatorcontrib>Fu, Zhiheng</creatorcontrib><creatorcontrib>Wang, Hanyun</creatorcontrib><creatorcontrib>Chen, Zonghao</creatorcontrib><creatorcontrib>Guo, Yulan</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE signal processing letters</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Li, Kunhong</au><au>Fu, Zhiheng</au><au>Wang, Hanyun</au><au>Chen, Zonghao</au><au>Guo, Yulan</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Adv-Depth: Self-Supervised Monocular Depth Estimation With an Adversarial Loss</atitle><jtitle>IEEE signal processing letters</jtitle><stitle>LSP</stitle><date>2021</date><risdate>2021</risdate><volume>28</volume><spage>638</spage><epage>642</epage><pages>638-642</pages><issn>1070-9908</issn><eissn>1558-2361</eissn><coden>ISPLEM</coden><abstract>Loss function plays a key role in self-supervised monocular depth estimation methods. Current reprojection loss functions are hand-designed and mainly focus on local patch similarity but overlook the global distribution differences between a synthetic image and a target image. In this paper, we leverage global distribution differences by introducing an adversarial loss into the training stage of self-supervised depth estimation. Specifically, we formulate this task as a novel view synthesis problem. We use a depth estimation module and a pose estimation module to form a generator, and then design a discriminator to learn the global distribution differences between real and synthetic images. With the learned global distribution differences, the adversarial loss can be back-propagated to the depth estimation module to improve its performance. Experiments on the KITTI dataset have demonstrated the effectiveness of the adversarial loss. The adversarial loss is further combined with the reprojection loss to achieve the state-of-the-art performance on the KITTI dataset.</abstract><cop>New York</cop><pub>IEEE</pub><doi>10.1109/LSP.2021.3065203</doi><tpages>5</tpages><orcidid>https://orcid.org/0000-0001-7051-841X</orcidid><orcidid>https://orcid.org/0000-0002-8320-4230</orcidid></addata></record>
fulltext fulltext_linktorsrc
identifier ISSN: 1070-9908
ispartof IEEE signal processing letters, 2021, Vol.28, p.638-642
issn 1070-9908
1558-2361
language eng
recordid cdi_crossref_primary_10_1109_LSP_2021_3065203
source IEEE Electronic Library (IEL)
subjects Back propagation
Datasets
Estimation
Feature extraction
Gallium nitride
Generative adversarial networks
Generators
Modules
Monocular depth estimation
self-supervised learning
single-image depth prediction
Target recognition
Task analysis
Training
title Adv-Depth: Self-Supervised Monocular Depth Estimation With an Adversarial Loss
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-06T01%3A12%3A46IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Adv-Depth:%20Self-Supervised%20Monocular%20Depth%20Estimation%20With%20an%20Adversarial%20Loss&rft.jtitle=IEEE%20signal%20processing%20letters&rft.au=Li,%20Kunhong&rft.date=2021&rft.volume=28&rft.spage=638&rft.epage=642&rft.pages=638-642&rft.issn=1070-9908&rft.eissn=1558-2361&rft.coden=ISPLEM&rft_id=info:doi/10.1109/LSP.2021.3065203&rft_dat=%3Cproquest_RIE%3E2515863572%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2515863572&rft_id=info:pmid/&rft_ieee_id=9376591&rfr_iscdi=true