Shape Estimation from Shading, Defocus, and Correspondence Using Light-Field Angular Coherence
Light-field cameras are quickly becoming commodity items, with consumer and industrial applications. They capture many nearby views simultaneously using a single image with a micro-lens array, thereby providing a wealth of cues for depth recovery: defocus, correspondence, and shading. In particular,...
Gespeichert in:
Veröffentlicht in: | IEEE transactions on pattern analysis and machine intelligence 2017-03, Vol.39 (3), p.546-560 |
---|---|
Hauptverfasser: | , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 560 |
---|---|
container_issue | 3 |
container_start_page | 546 |
container_title | IEEE transactions on pattern analysis and machine intelligence |
container_volume | 39 |
creator | Tao, Michael W. Srinivasan, Pratul P. Hadap, Sunil Malik, Jitendra Ramamoorthi, Ravi |
description | Light-field cameras are quickly becoming commodity items, with consumer and industrial applications. They capture many nearby views simultaneously using a single image with a micro-lens array, thereby providing a wealth of cues for depth recovery: defocus, correspondence, and shading. In particular, apart from conventional image shading, one can refocus images after acquisition, and shift one's viewpoint within the sub-apertures of the main lens, effectively obtaining multiple views. We present a principled algorithm for dense depth estimation that combines defocus and correspondence metrics. We then extend our analysis to the additional cue of shading, using it to refine fine details in the shape. By exploiting an all-in-focus image, in which pixels are expected to exhibit angular coherence, we define an optimization framework that integrates photo consistency, depth consistency, and shading consistency. We show that combining all three sources of information: defocus, correspondence, and shading, outperforms state-of-the-art light-field depth estimation algorithms in multiple scenarios. |
doi_str_mv | 10.1109/TPAMI.2016.2554121 |
format | Article |
fullrecord | <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_proquest_miscellaneous_1826672206</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>7452621</ieee_id><sourcerecordid>1866377457</sourcerecordid><originalsourceid>FETCH-LOGICAL-c444t-2e6da8753434f23169e9b5fff65c70ecf056401e0e4a0df3712e8d52dae6043a3</originalsourceid><addsrcrecordid>eNpdkE1Lw0AQhhdRbK3-AQVZ8OKhqfud5FjqV6GiYL0a1uxsm5Jm625y8N-7tdWDp4GZ550ZHoTOKRlRSvKb-cv4aTpihKoRk1JQRg9Qn-Y8T7jk-SHqxwlLsoxlPXQSwooQKiThx6jHUkqozLM-en9d6g3gu9BWa91WrsHWuzWOXVM1iyG-BevKLgyxbgyeOO8hbFxjoCkBv4WI4Fm1WLbJfQW1weNm0dXaR3AJfsucoiOr6wBn-zpAb_d388ljMnt-mE7Gs6QUQrQJA2V0lkouuLCMU5VD_iGttUqWKYHSEqkEoUBAaGIsTymDzEhmNCgiuOYDdL3bu_Hus4PQFusqlFDXugHXhYJmTKmUMaIievUPXbnON_G7SCnF01TINFJsR5XeheDBFhsfDfmvgpJia7_4sV9s7Rd7-zF0uV_dfazB_EV-dUfgYgdUAPA3jgeZivFvJtKHmQ</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1866377457</pqid></control><display><type>article</type><title>Shape Estimation from Shading, Defocus, and Correspondence Using Light-Field Angular Coherence</title><source>IEEE Electronic Library (IEL)</source><creator>Tao, Michael W. ; Srinivasan, Pratul P. ; Hadap, Sunil ; Malik, Jitendra ; Ramamoorthi, Ravi</creator><creatorcontrib>Tao, Michael W. ; Srinivasan, Pratul P. ; Hadap, Sunil ; Malik, Jitendra ; Ramamoorthi, Ravi</creatorcontrib><description>Light-field cameras are quickly becoming commodity items, with consumer and industrial applications. They capture many nearby views simultaneously using a single image with a micro-lens array, thereby providing a wealth of cues for depth recovery: defocus, correspondence, and shading. In particular, apart from conventional image shading, one can refocus images after acquisition, and shift one's viewpoint within the sub-apertures of the main lens, effectively obtaining multiple views. We present a principled algorithm for dense depth estimation that combines defocus and correspondence metrics. We then extend our analysis to the additional cue of shading, using it to refine fine details in the shape. By exploiting an all-in-focus image, in which pixels are expected to exhibit angular coherence, we define an optimization framework that integrates photo consistency, depth consistency, and shading consistency. We show that combining all three sources of information: defocus, correspondence, and shading, outperforms state-of-the-art light-field depth estimation algorithms in multiple scenarios.</description><identifier>ISSN: 0162-8828</identifier><identifier>EISSN: 1939-3539</identifier><identifier>EISSN: 2160-9292</identifier><identifier>DOI: 10.1109/TPAMI.2016.2554121</identifier><identifier>PMID: 27101598</identifier><identifier>CODEN: ITPIDJ</identifier><language>eng</language><publisher>United States: IEEE</publisher><subject>3D reconstruction ; Apertures ; Cameras ; Coherence ; Consistency ; Cues ; depth cues ; Estimation ; Field cameras ; Geometry ; Image acquisition ; Industrial applications ; Lenses ; Light fields ; Lighting ; reflection components separation ; Shading ; Shape ; shape from shading ; specular-free image</subject><ispartof>IEEE transactions on pattern analysis and machine intelligence, 2017-03, Vol.39 (3), p.546-560</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2017</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c444t-2e6da8753434f23169e9b5fff65c70ecf056401e0e4a0df3712e8d52dae6043a3</citedby><cites>FETCH-LOGICAL-c444t-2e6da8753434f23169e9b5fff65c70ecf056401e0e4a0df3712e8d52dae6043a3</cites><orcidid>0000-0002-6312-4989</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/7452621$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,796,27924,27925,54758</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/7452621$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/27101598$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Tao, Michael W.</creatorcontrib><creatorcontrib>Srinivasan, Pratul P.</creatorcontrib><creatorcontrib>Hadap, Sunil</creatorcontrib><creatorcontrib>Malik, Jitendra</creatorcontrib><creatorcontrib>Ramamoorthi, Ravi</creatorcontrib><title>Shape Estimation from Shading, Defocus, and Correspondence Using Light-Field Angular Coherence</title><title>IEEE transactions on pattern analysis and machine intelligence</title><addtitle>TPAMI</addtitle><addtitle>IEEE Trans Pattern Anal Mach Intell</addtitle><description>Light-field cameras are quickly becoming commodity items, with consumer and industrial applications. They capture many nearby views simultaneously using a single image with a micro-lens array, thereby providing a wealth of cues for depth recovery: defocus, correspondence, and shading. In particular, apart from conventional image shading, one can refocus images after acquisition, and shift one's viewpoint within the sub-apertures of the main lens, effectively obtaining multiple views. We present a principled algorithm for dense depth estimation that combines defocus and correspondence metrics. We then extend our analysis to the additional cue of shading, using it to refine fine details in the shape. By exploiting an all-in-focus image, in which pixels are expected to exhibit angular coherence, we define an optimization framework that integrates photo consistency, depth consistency, and shading consistency. We show that combining all three sources of information: defocus, correspondence, and shading, outperforms state-of-the-art light-field depth estimation algorithms in multiple scenarios.</description><subject>3D reconstruction</subject><subject>Apertures</subject><subject>Cameras</subject><subject>Coherence</subject><subject>Consistency</subject><subject>Cues</subject><subject>depth cues</subject><subject>Estimation</subject><subject>Field cameras</subject><subject>Geometry</subject><subject>Image acquisition</subject><subject>Industrial applications</subject><subject>Lenses</subject><subject>Light fields</subject><subject>Lighting</subject><subject>reflection components separation</subject><subject>Shading</subject><subject>Shape</subject><subject>shape from shading</subject><subject>specular-free image</subject><issn>0162-8828</issn><issn>1939-3539</issn><issn>2160-9292</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2017</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNpdkE1Lw0AQhhdRbK3-AQVZ8OKhqfud5FjqV6GiYL0a1uxsm5Jm625y8N-7tdWDp4GZ550ZHoTOKRlRSvKb-cv4aTpihKoRk1JQRg9Qn-Y8T7jk-SHqxwlLsoxlPXQSwooQKiThx6jHUkqozLM-en9d6g3gu9BWa91WrsHWuzWOXVM1iyG-BevKLgyxbgyeOO8hbFxjoCkBv4WI4Fm1WLbJfQW1weNm0dXaR3AJfsucoiOr6wBn-zpAb_d388ljMnt-mE7Gs6QUQrQJA2V0lkouuLCMU5VD_iGttUqWKYHSEqkEoUBAaGIsTymDzEhmNCgiuOYDdL3bu_Hus4PQFusqlFDXugHXhYJmTKmUMaIievUPXbnON_G7SCnF01TINFJsR5XeheDBFhsfDfmvgpJia7_4sV9s7Rd7-zF0uV_dfazB_EV-dUfgYgdUAPA3jgeZivFvJtKHmQ</recordid><startdate>20170301</startdate><enddate>20170301</enddate><creator>Tao, Michael W.</creator><creator>Srinivasan, Pratul P.</creator><creator>Hadap, Sunil</creator><creator>Malik, Jitendra</creator><creator>Ramamoorthi, Ravi</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0002-6312-4989</orcidid></search><sort><creationdate>20170301</creationdate><title>Shape Estimation from Shading, Defocus, and Correspondence Using Light-Field Angular Coherence</title><author>Tao, Michael W. ; Srinivasan, Pratul P. ; Hadap, Sunil ; Malik, Jitendra ; Ramamoorthi, Ravi</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c444t-2e6da8753434f23169e9b5fff65c70ecf056401e0e4a0df3712e8d52dae6043a3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2017</creationdate><topic>3D reconstruction</topic><topic>Apertures</topic><topic>Cameras</topic><topic>Coherence</topic><topic>Consistency</topic><topic>Cues</topic><topic>depth cues</topic><topic>Estimation</topic><topic>Field cameras</topic><topic>Geometry</topic><topic>Image acquisition</topic><topic>Industrial applications</topic><topic>Lenses</topic><topic>Light fields</topic><topic>Lighting</topic><topic>reflection components separation</topic><topic>Shading</topic><topic>Shape</topic><topic>shape from shading</topic><topic>specular-free image</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Tao, Michael W.</creatorcontrib><creatorcontrib>Srinivasan, Pratul P.</creatorcontrib><creatorcontrib>Hadap, Sunil</creatorcontrib><creatorcontrib>Malik, Jitendra</creatorcontrib><creatorcontrib>Ramamoorthi, Ravi</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>MEDLINE - Academic</collection><jtitle>IEEE transactions on pattern analysis and machine intelligence</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Tao, Michael W.</au><au>Srinivasan, Pratul P.</au><au>Hadap, Sunil</au><au>Malik, Jitendra</au><au>Ramamoorthi, Ravi</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Shape Estimation from Shading, Defocus, and Correspondence Using Light-Field Angular Coherence</atitle><jtitle>IEEE transactions on pattern analysis and machine intelligence</jtitle><stitle>TPAMI</stitle><addtitle>IEEE Trans Pattern Anal Mach Intell</addtitle><date>2017-03-01</date><risdate>2017</risdate><volume>39</volume><issue>3</issue><spage>546</spage><epage>560</epage><pages>546-560</pages><issn>0162-8828</issn><eissn>1939-3539</eissn><eissn>2160-9292</eissn><coden>ITPIDJ</coden><abstract>Light-field cameras are quickly becoming commodity items, with consumer and industrial applications. They capture many nearby views simultaneously using a single image with a micro-lens array, thereby providing a wealth of cues for depth recovery: defocus, correspondence, and shading. In particular, apart from conventional image shading, one can refocus images after acquisition, and shift one's viewpoint within the sub-apertures of the main lens, effectively obtaining multiple views. We present a principled algorithm for dense depth estimation that combines defocus and correspondence metrics. We then extend our analysis to the additional cue of shading, using it to refine fine details in the shape. By exploiting an all-in-focus image, in which pixels are expected to exhibit angular coherence, we define an optimization framework that integrates photo consistency, depth consistency, and shading consistency. We show that combining all three sources of information: defocus, correspondence, and shading, outperforms state-of-the-art light-field depth estimation algorithms in multiple scenarios.</abstract><cop>United States</cop><pub>IEEE</pub><pmid>27101598</pmid><doi>10.1109/TPAMI.2016.2554121</doi><tpages>15</tpages><orcidid>https://orcid.org/0000-0002-6312-4989</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext_linktorsrc |
identifier | ISSN: 0162-8828 |
ispartof | IEEE transactions on pattern analysis and machine intelligence, 2017-03, Vol.39 (3), p.546-560 |
issn | 0162-8828 1939-3539 2160-9292 |
language | eng |
recordid | cdi_proquest_miscellaneous_1826672206 |
source | IEEE Electronic Library (IEL) |
subjects | 3D reconstruction Apertures Cameras Coherence Consistency Cues depth cues Estimation Field cameras Geometry Image acquisition Industrial applications Lenses Light fields Lighting reflection components separation Shading Shape shape from shading specular-free image |
title | Shape Estimation from Shading, Defocus, and Correspondence Using Light-Field Angular Coherence |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T03%3A20%3A01IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Shape%20Estimation%20from%20Shading,%20Defocus,%20and%20Correspondence%20Using%20Light-Field%20Angular%20Coherence&rft.jtitle=IEEE%20transactions%20on%20pattern%20analysis%20and%20machine%20intelligence&rft.au=Tao,%20Michael%20W.&rft.date=2017-03-01&rft.volume=39&rft.issue=3&rft.spage=546&rft.epage=560&rft.pages=546-560&rft.issn=0162-8828&rft.eissn=1939-3539&rft.coden=ITPIDJ&rft_id=info:doi/10.1109/TPAMI.2016.2554121&rft_dat=%3Cproquest_RIE%3E1866377457%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1866377457&rft_id=info:pmid/27101598&rft_ieee_id=7452621&rfr_iscdi=true |