Missing data reconstruction in VHR images based on progressive structure prediction and texture generation

Very high resolution (VHR) satellite and aerial images often suffer from scene occlusion caused by redundant objects. The task of removing these redundant objects can be solved by missing data reconstruction technology. However, when dealing with VHR images with large-scale missing regions, existing...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:ISPRS journal of photogrammetry and remote sensing 2021-01, Vol.171, p.266-277
Hauptverfasser: Xu, Hanwen, Tang, Xinming, Ai, Bo, Gao, Xiaoming, Yang, Fanlin, Wen, Zhen
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 277
container_issue
container_start_page 266
container_title ISPRS journal of photogrammetry and remote sensing
container_volume 171
creator Xu, Hanwen
Tang, Xinming
Ai, Bo
Gao, Xiaoming
Yang, Fanlin
Wen, Zhen
description Very high resolution (VHR) satellite and aerial images often suffer from scene occlusion caused by redundant objects. The task of removing these redundant objects can be solved by missing data reconstruction technology. However, when dealing with VHR images with large-scale missing regions, existing spatial-based methods often destroy the structural information of ground objects. To alleviate this problem, this paper proposes a novel missing data reconstruction method based on deep learning. The reconstruction process is divided into two parts: structure prediction and texture generation. First, a progressive edge generation network (PEGN) is designed to predict the edges of objects in missing regions in a progressive manner. Then, the edge map predicted by PEGN is input to a texture generation network (TGN) as structural information to produce the reconstruction results. This is a spatial-based method that can produce realistic and reasonable results without any need for auxiliary spectral or temporal data. Experiments demonstrate that our model can better restore the structure of ground objects in VHR images than other spatial-based methods and outperform them in SSIM and PSNR indices. In addition, our model also has a strong generalization capability by introducing Poisson blending and histogram matching.
doi_str_mv 10.1016/j.isprsjprs.2020.11.020
format Article
fullrecord <record><control><sourceid>elsevier_webof</sourceid><recordid>TN_cdi_webofscience_primary_000604406500018</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S0924271620303270</els_id><sourcerecordid>S0924271620303270</sourcerecordid><originalsourceid>FETCH-LOGICAL-a338t-5c559b2a6e0d2d5e72f730fc32f344d902aca6007664313efdf262ca37a1f9ca3</originalsourceid><addsrcrecordid>eNqNkV1LwzAUhoMoOKe_wdxLaz7atL0cRZ0wEUS9DVlyUlK0HUk39d-brmO3ehFeeDlPyHmC0DUlKSVU3LapCxsf2nhSRlhsaRrjBM1oWbCkZDw_RTNSsSxhBRXn6CKElhBCc1HOUPvkQnBdg40aFPag-y4MfqsH13fYdfh9-YLdp2og4LUKYHCsN75vPERsB3ga3nqILRg3caozeIDvfd1AB16N9SU6s-ojwNUh5-jt_u61Xiar54fHerFKFOflkOQ6z6s1UwKIYSaHgtmCE6s5szzLTEWY0koQUgiRccrBGssE04oXitoq5hwV073a9yF4sHLj4wr-R1IiR2WylUdlclQmKZUxIllO5Besexu0g07DkY7OBMkyIvLRXlm7Yb9W3W-7IaI3_0fj9GKahuhh58DLA2Fc_IJBmt79-dhfQcideQ</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Missing data reconstruction in VHR images based on progressive structure prediction and texture generation</title><source>Access via ScienceDirect (Elsevier)</source><creator>Xu, Hanwen ; Tang, Xinming ; Ai, Bo ; Gao, Xiaoming ; Yang, Fanlin ; Wen, Zhen</creator><creatorcontrib>Xu, Hanwen ; Tang, Xinming ; Ai, Bo ; Gao, Xiaoming ; Yang, Fanlin ; Wen, Zhen</creatorcontrib><description>Very high resolution (VHR) satellite and aerial images often suffer from scene occlusion caused by redundant objects. The task of removing these redundant objects can be solved by missing data reconstruction technology. However, when dealing with VHR images with large-scale missing regions, existing spatial-based methods often destroy the structural information of ground objects. To alleviate this problem, this paper proposes a novel missing data reconstruction method based on deep learning. The reconstruction process is divided into two parts: structure prediction and texture generation. First, a progressive edge generation network (PEGN) is designed to predict the edges of objects in missing regions in a progressive manner. Then, the edge map predicted by PEGN is input to a texture generation network (TGN) as structural information to produce the reconstruction results. This is a spatial-based method that can produce realistic and reasonable results without any need for auxiliary spectral or temporal data. Experiments demonstrate that our model can better restore the structure of ground objects in VHR images than other spatial-based methods and outperform them in SSIM and PSNR indices. In addition, our model also has a strong generalization capability by introducing Poisson blending and histogram matching.</description><identifier>ISSN: 0924-2716</identifier><identifier>EISSN: 1872-8235</identifier><identifier>DOI: 10.1016/j.isprsjprs.2020.11.020</identifier><language>eng</language><publisher>AMSTERDAM: Elsevier B.V</publisher><subject>Deep learning ; Geography, Physical ; Geology ; Geosciences, Multidisciplinary ; Imaging Science &amp; Photographic Technology ; Missing data reconstruction ; Physical Geography ; Physical Sciences ; Progressive structure prediction ; Remote Sensing ; Science &amp; Technology ; Technology ; Texture generation ; VHR images</subject><ispartof>ISPRS journal of photogrammetry and remote sensing, 2021-01, Vol.171, p.266-277</ispartof><rights>2020 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS)</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>true</woscitedreferencessubscribed><woscitedreferencescount>11</woscitedreferencescount><woscitedreferencesoriginalsourcerecordid>wos000604406500018</woscitedreferencesoriginalsourcerecordid><citedby>FETCH-LOGICAL-a338t-5c559b2a6e0d2d5e72f730fc32f344d902aca6007664313efdf262ca37a1f9ca3</citedby><cites>FETCH-LOGICAL-a338t-5c559b2a6e0d2d5e72f730fc32f344d902aca6007664313efdf262ca37a1f9ca3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://dx.doi.org/10.1016/j.isprsjprs.2020.11.020$$EHTML$$P50$$Gelsevier$$H</linktohtml><link.rule.ids>315,781,785,3551,27928,27929,45999</link.rule.ids></links><search><creatorcontrib>Xu, Hanwen</creatorcontrib><creatorcontrib>Tang, Xinming</creatorcontrib><creatorcontrib>Ai, Bo</creatorcontrib><creatorcontrib>Gao, Xiaoming</creatorcontrib><creatorcontrib>Yang, Fanlin</creatorcontrib><creatorcontrib>Wen, Zhen</creatorcontrib><title>Missing data reconstruction in VHR images based on progressive structure prediction and texture generation</title><title>ISPRS journal of photogrammetry and remote sensing</title><addtitle>ISPRS J PHOTOGRAMM</addtitle><description>Very high resolution (VHR) satellite and aerial images often suffer from scene occlusion caused by redundant objects. The task of removing these redundant objects can be solved by missing data reconstruction technology. However, when dealing with VHR images with large-scale missing regions, existing spatial-based methods often destroy the structural information of ground objects. To alleviate this problem, this paper proposes a novel missing data reconstruction method based on deep learning. The reconstruction process is divided into two parts: structure prediction and texture generation. First, a progressive edge generation network (PEGN) is designed to predict the edges of objects in missing regions in a progressive manner. Then, the edge map predicted by PEGN is input to a texture generation network (TGN) as structural information to produce the reconstruction results. This is a spatial-based method that can produce realistic and reasonable results without any need for auxiliary spectral or temporal data. Experiments demonstrate that our model can better restore the structure of ground objects in VHR images than other spatial-based methods and outperform them in SSIM and PSNR indices. In addition, our model also has a strong generalization capability by introducing Poisson blending and histogram matching.</description><subject>Deep learning</subject><subject>Geography, Physical</subject><subject>Geology</subject><subject>Geosciences, Multidisciplinary</subject><subject>Imaging Science &amp; Photographic Technology</subject><subject>Missing data reconstruction</subject><subject>Physical Geography</subject><subject>Physical Sciences</subject><subject>Progressive structure prediction</subject><subject>Remote Sensing</subject><subject>Science &amp; Technology</subject><subject>Technology</subject><subject>Texture generation</subject><subject>VHR images</subject><issn>0924-2716</issn><issn>1872-8235</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>HGBXW</sourceid><recordid>eNqNkV1LwzAUhoMoOKe_wdxLaz7atL0cRZ0wEUS9DVlyUlK0HUk39d-brmO3ehFeeDlPyHmC0DUlKSVU3LapCxsf2nhSRlhsaRrjBM1oWbCkZDw_RTNSsSxhBRXn6CKElhBCc1HOUPvkQnBdg40aFPag-y4MfqsH13fYdfh9-YLdp2og4LUKYHCsN75vPERsB3ga3nqILRg3caozeIDvfd1AB16N9SU6s-ojwNUh5-jt_u61Xiar54fHerFKFOflkOQ6z6s1UwKIYSaHgtmCE6s5szzLTEWY0koQUgiRccrBGssE04oXitoq5hwV073a9yF4sHLj4wr-R1IiR2WylUdlclQmKZUxIllO5Besexu0g07DkY7OBMkyIvLRXlm7Yb9W3W-7IaI3_0fj9GKahuhh58DLA2Fc_IJBmt79-dhfQcideQ</recordid><startdate>202101</startdate><enddate>202101</enddate><creator>Xu, Hanwen</creator><creator>Tang, Xinming</creator><creator>Ai, Bo</creator><creator>Gao, Xiaoming</creator><creator>Yang, Fanlin</creator><creator>Wen, Zhen</creator><general>Elsevier B.V</general><general>Elsevier</general><scope>BLEPL</scope><scope>DTL</scope><scope>HGBXW</scope><scope>AAYXX</scope><scope>CITATION</scope></search><sort><creationdate>202101</creationdate><title>Missing data reconstruction in VHR images based on progressive structure prediction and texture generation</title><author>Xu, Hanwen ; Tang, Xinming ; Ai, Bo ; Gao, Xiaoming ; Yang, Fanlin ; Wen, Zhen</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a338t-5c559b2a6e0d2d5e72f730fc32f344d902aca6007664313efdf262ca37a1f9ca3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Deep learning</topic><topic>Geography, Physical</topic><topic>Geology</topic><topic>Geosciences, Multidisciplinary</topic><topic>Imaging Science &amp; Photographic Technology</topic><topic>Missing data reconstruction</topic><topic>Physical Geography</topic><topic>Physical Sciences</topic><topic>Progressive structure prediction</topic><topic>Remote Sensing</topic><topic>Science &amp; Technology</topic><topic>Technology</topic><topic>Texture generation</topic><topic>VHR images</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Xu, Hanwen</creatorcontrib><creatorcontrib>Tang, Xinming</creatorcontrib><creatorcontrib>Ai, Bo</creatorcontrib><creatorcontrib>Gao, Xiaoming</creatorcontrib><creatorcontrib>Yang, Fanlin</creatorcontrib><creatorcontrib>Wen, Zhen</creatorcontrib><collection>Web of Science Core Collection</collection><collection>Science Citation Index Expanded</collection><collection>Web of Science - Science Citation Index Expanded - 2021</collection><collection>CrossRef</collection><jtitle>ISPRS journal of photogrammetry and remote sensing</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Xu, Hanwen</au><au>Tang, Xinming</au><au>Ai, Bo</au><au>Gao, Xiaoming</au><au>Yang, Fanlin</au><au>Wen, Zhen</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Missing data reconstruction in VHR images based on progressive structure prediction and texture generation</atitle><jtitle>ISPRS journal of photogrammetry and remote sensing</jtitle><stitle>ISPRS J PHOTOGRAMM</stitle><date>2021-01</date><risdate>2021</risdate><volume>171</volume><spage>266</spage><epage>277</epage><pages>266-277</pages><issn>0924-2716</issn><eissn>1872-8235</eissn><abstract>Very high resolution (VHR) satellite and aerial images often suffer from scene occlusion caused by redundant objects. The task of removing these redundant objects can be solved by missing data reconstruction technology. However, when dealing with VHR images with large-scale missing regions, existing spatial-based methods often destroy the structural information of ground objects. To alleviate this problem, this paper proposes a novel missing data reconstruction method based on deep learning. The reconstruction process is divided into two parts: structure prediction and texture generation. First, a progressive edge generation network (PEGN) is designed to predict the edges of objects in missing regions in a progressive manner. Then, the edge map predicted by PEGN is input to a texture generation network (TGN) as structural information to produce the reconstruction results. This is a spatial-based method that can produce realistic and reasonable results without any need for auxiliary spectral or temporal data. Experiments demonstrate that our model can better restore the structure of ground objects in VHR images than other spatial-based methods and outperform them in SSIM and PSNR indices. In addition, our model also has a strong generalization capability by introducing Poisson blending and histogram matching.</abstract><cop>AMSTERDAM</cop><pub>Elsevier B.V</pub><doi>10.1016/j.isprsjprs.2020.11.020</doi><tpages>12</tpages></addata></record>
fulltext fulltext
identifier ISSN: 0924-2716
ispartof ISPRS journal of photogrammetry and remote sensing, 2021-01, Vol.171, p.266-277
issn 0924-2716
1872-8235
language eng
recordid cdi_webofscience_primary_000604406500018
source Access via ScienceDirect (Elsevier)
subjects Deep learning
Geography, Physical
Geology
Geosciences, Multidisciplinary
Imaging Science & Photographic Technology
Missing data reconstruction
Physical Geography
Physical Sciences
Progressive structure prediction
Remote Sensing
Science & Technology
Technology
Texture generation
VHR images
title Missing data reconstruction in VHR images based on progressive structure prediction and texture generation
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-17T10%3A28%3A56IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-elsevier_webof&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Missing%20data%20reconstruction%20in%20VHR%20images%20based%20on%20progressive%20structure%20prediction%20and%20texture%20generation&rft.jtitle=ISPRS%20journal%20of%20photogrammetry%20and%20remote%20sensing&rft.au=Xu,%20Hanwen&rft.date=2021-01&rft.volume=171&rft.spage=266&rft.epage=277&rft.pages=266-277&rft.issn=0924-2716&rft.eissn=1872-8235&rft_id=info:doi/10.1016/j.isprsjprs.2020.11.020&rft_dat=%3Celsevier_webof%3ES0924271620303270%3C/elsevier_webof%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rft_els_id=S0924271620303270&rfr_iscdi=true