Missing data reconstruction in VHR images based on progressive structure prediction and texture generation
Very high resolution (VHR) satellite and aerial images often suffer from scene occlusion caused by redundant objects. The task of removing these redundant objects can be solved by missing data reconstruction technology. However, when dealing with VHR images with large-scale missing regions, existing...
Gespeichert in:
Veröffentlicht in: | ISPRS journal of photogrammetry and remote sensing 2021-01, Vol.171, p.266-277 |
---|---|
Hauptverfasser: | , , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 277 |
---|---|
container_issue | |
container_start_page | 266 |
container_title | ISPRS journal of photogrammetry and remote sensing |
container_volume | 171 |
creator | Xu, Hanwen Tang, Xinming Ai, Bo Gao, Xiaoming Yang, Fanlin Wen, Zhen |
description | Very high resolution (VHR) satellite and aerial images often suffer from scene occlusion caused by redundant objects. The task of removing these redundant objects can be solved by missing data reconstruction technology. However, when dealing with VHR images with large-scale missing regions, existing spatial-based methods often destroy the structural information of ground objects. To alleviate this problem, this paper proposes a novel missing data reconstruction method based on deep learning. The reconstruction process is divided into two parts: structure prediction and texture generation. First, a progressive edge generation network (PEGN) is designed to predict the edges of objects in missing regions in a progressive manner. Then, the edge map predicted by PEGN is input to a texture generation network (TGN) as structural information to produce the reconstruction results. This is a spatial-based method that can produce realistic and reasonable results without any need for auxiliary spectral or temporal data. Experiments demonstrate that our model can better restore the structure of ground objects in VHR images than other spatial-based methods and outperform them in SSIM and PSNR indices. In addition, our model also has a strong generalization capability by introducing Poisson blending and histogram matching. |
doi_str_mv | 10.1016/j.isprsjprs.2020.11.020 |
format | Article |
fullrecord | <record><control><sourceid>elsevier_webof</sourceid><recordid>TN_cdi_webofscience_primary_000604406500018</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S0924271620303270</els_id><sourcerecordid>S0924271620303270</sourcerecordid><originalsourceid>FETCH-LOGICAL-a338t-5c559b2a6e0d2d5e72f730fc32f344d902aca6007664313efdf262ca37a1f9ca3</originalsourceid><addsrcrecordid>eNqNkV1LwzAUhoMoOKe_wdxLaz7atL0cRZ0wEUS9DVlyUlK0HUk39d-brmO3ehFeeDlPyHmC0DUlKSVU3LapCxsf2nhSRlhsaRrjBM1oWbCkZDw_RTNSsSxhBRXn6CKElhBCc1HOUPvkQnBdg40aFPag-y4MfqsH13fYdfh9-YLdp2og4LUKYHCsN75vPERsB3ga3nqILRg3caozeIDvfd1AB16N9SU6s-ojwNUh5-jt_u61Xiar54fHerFKFOflkOQ6z6s1UwKIYSaHgtmCE6s5szzLTEWY0koQUgiRccrBGssE04oXitoq5hwV073a9yF4sHLj4wr-R1IiR2WylUdlclQmKZUxIllO5Besexu0g07DkY7OBMkyIvLRXlm7Yb9W3W-7IaI3_0fj9GKahuhh58DLA2Fc_IJBmt79-dhfQcideQ</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Missing data reconstruction in VHR images based on progressive structure prediction and texture generation</title><source>Access via ScienceDirect (Elsevier)</source><creator>Xu, Hanwen ; Tang, Xinming ; Ai, Bo ; Gao, Xiaoming ; Yang, Fanlin ; Wen, Zhen</creator><creatorcontrib>Xu, Hanwen ; Tang, Xinming ; Ai, Bo ; Gao, Xiaoming ; Yang, Fanlin ; Wen, Zhen</creatorcontrib><description>Very high resolution (VHR) satellite and aerial images often suffer from scene occlusion caused by redundant objects. The task of removing these redundant objects can be solved by missing data reconstruction technology. However, when dealing with VHR images with large-scale missing regions, existing spatial-based methods often destroy the structural information of ground objects. To alleviate this problem, this paper proposes a novel missing data reconstruction method based on deep learning. The reconstruction process is divided into two parts: structure prediction and texture generation. First, a progressive edge generation network (PEGN) is designed to predict the edges of objects in missing regions in a progressive manner. Then, the edge map predicted by PEGN is input to a texture generation network (TGN) as structural information to produce the reconstruction results. This is a spatial-based method that can produce realistic and reasonable results without any need for auxiliary spectral or temporal data. Experiments demonstrate that our model can better restore the structure of ground objects in VHR images than other spatial-based methods and outperform them in SSIM and PSNR indices. In addition, our model also has a strong generalization capability by introducing Poisson blending and histogram matching.</description><identifier>ISSN: 0924-2716</identifier><identifier>EISSN: 1872-8235</identifier><identifier>DOI: 10.1016/j.isprsjprs.2020.11.020</identifier><language>eng</language><publisher>AMSTERDAM: Elsevier B.V</publisher><subject>Deep learning ; Geography, Physical ; Geology ; Geosciences, Multidisciplinary ; Imaging Science & Photographic Technology ; Missing data reconstruction ; Physical Geography ; Physical Sciences ; Progressive structure prediction ; Remote Sensing ; Science & Technology ; Technology ; Texture generation ; VHR images</subject><ispartof>ISPRS journal of photogrammetry and remote sensing, 2021-01, Vol.171, p.266-277</ispartof><rights>2020 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS)</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>true</woscitedreferencessubscribed><woscitedreferencescount>11</woscitedreferencescount><woscitedreferencesoriginalsourcerecordid>wos000604406500018</woscitedreferencesoriginalsourcerecordid><citedby>FETCH-LOGICAL-a338t-5c559b2a6e0d2d5e72f730fc32f344d902aca6007664313efdf262ca37a1f9ca3</citedby><cites>FETCH-LOGICAL-a338t-5c559b2a6e0d2d5e72f730fc32f344d902aca6007664313efdf262ca37a1f9ca3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://dx.doi.org/10.1016/j.isprsjprs.2020.11.020$$EHTML$$P50$$Gelsevier$$H</linktohtml><link.rule.ids>315,781,785,3551,27928,27929,45999</link.rule.ids></links><search><creatorcontrib>Xu, Hanwen</creatorcontrib><creatorcontrib>Tang, Xinming</creatorcontrib><creatorcontrib>Ai, Bo</creatorcontrib><creatorcontrib>Gao, Xiaoming</creatorcontrib><creatorcontrib>Yang, Fanlin</creatorcontrib><creatorcontrib>Wen, Zhen</creatorcontrib><title>Missing data reconstruction in VHR images based on progressive structure prediction and texture generation</title><title>ISPRS journal of photogrammetry and remote sensing</title><addtitle>ISPRS J PHOTOGRAMM</addtitle><description>Very high resolution (VHR) satellite and aerial images often suffer from scene occlusion caused by redundant objects. The task of removing these redundant objects can be solved by missing data reconstruction technology. However, when dealing with VHR images with large-scale missing regions, existing spatial-based methods often destroy the structural information of ground objects. To alleviate this problem, this paper proposes a novel missing data reconstruction method based on deep learning. The reconstruction process is divided into two parts: structure prediction and texture generation. First, a progressive edge generation network (PEGN) is designed to predict the edges of objects in missing regions in a progressive manner. Then, the edge map predicted by PEGN is input to a texture generation network (TGN) as structural information to produce the reconstruction results. This is a spatial-based method that can produce realistic and reasonable results without any need for auxiliary spectral or temporal data. Experiments demonstrate that our model can better restore the structure of ground objects in VHR images than other spatial-based methods and outperform them in SSIM and PSNR indices. In addition, our model also has a strong generalization capability by introducing Poisson blending and histogram matching.</description><subject>Deep learning</subject><subject>Geography, Physical</subject><subject>Geology</subject><subject>Geosciences, Multidisciplinary</subject><subject>Imaging Science & Photographic Technology</subject><subject>Missing data reconstruction</subject><subject>Physical Geography</subject><subject>Physical Sciences</subject><subject>Progressive structure prediction</subject><subject>Remote Sensing</subject><subject>Science & Technology</subject><subject>Technology</subject><subject>Texture generation</subject><subject>VHR images</subject><issn>0924-2716</issn><issn>1872-8235</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>HGBXW</sourceid><recordid>eNqNkV1LwzAUhoMoOKe_wdxLaz7atL0cRZ0wEUS9DVlyUlK0HUk39d-brmO3ehFeeDlPyHmC0DUlKSVU3LapCxsf2nhSRlhsaRrjBM1oWbCkZDw_RTNSsSxhBRXn6CKElhBCc1HOUPvkQnBdg40aFPag-y4MfqsH13fYdfh9-YLdp2og4LUKYHCsN75vPERsB3ga3nqILRg3caozeIDvfd1AB16N9SU6s-ojwNUh5-jt_u61Xiar54fHerFKFOflkOQ6z6s1UwKIYSaHgtmCE6s5szzLTEWY0koQUgiRccrBGssE04oXitoq5hwV073a9yF4sHLj4wr-R1IiR2WylUdlclQmKZUxIllO5Besexu0g07DkY7OBMkyIvLRXlm7Yb9W3W-7IaI3_0fj9GKahuhh58DLA2Fc_IJBmt79-dhfQcideQ</recordid><startdate>202101</startdate><enddate>202101</enddate><creator>Xu, Hanwen</creator><creator>Tang, Xinming</creator><creator>Ai, Bo</creator><creator>Gao, Xiaoming</creator><creator>Yang, Fanlin</creator><creator>Wen, Zhen</creator><general>Elsevier B.V</general><general>Elsevier</general><scope>BLEPL</scope><scope>DTL</scope><scope>HGBXW</scope><scope>AAYXX</scope><scope>CITATION</scope></search><sort><creationdate>202101</creationdate><title>Missing data reconstruction in VHR images based on progressive structure prediction and texture generation</title><author>Xu, Hanwen ; Tang, Xinming ; Ai, Bo ; Gao, Xiaoming ; Yang, Fanlin ; Wen, Zhen</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a338t-5c559b2a6e0d2d5e72f730fc32f344d902aca6007664313efdf262ca37a1f9ca3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Deep learning</topic><topic>Geography, Physical</topic><topic>Geology</topic><topic>Geosciences, Multidisciplinary</topic><topic>Imaging Science & Photographic Technology</topic><topic>Missing data reconstruction</topic><topic>Physical Geography</topic><topic>Physical Sciences</topic><topic>Progressive structure prediction</topic><topic>Remote Sensing</topic><topic>Science & Technology</topic><topic>Technology</topic><topic>Texture generation</topic><topic>VHR images</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Xu, Hanwen</creatorcontrib><creatorcontrib>Tang, Xinming</creatorcontrib><creatorcontrib>Ai, Bo</creatorcontrib><creatorcontrib>Gao, Xiaoming</creatorcontrib><creatorcontrib>Yang, Fanlin</creatorcontrib><creatorcontrib>Wen, Zhen</creatorcontrib><collection>Web of Science Core Collection</collection><collection>Science Citation Index Expanded</collection><collection>Web of Science - Science Citation Index Expanded - 2021</collection><collection>CrossRef</collection><jtitle>ISPRS journal of photogrammetry and remote sensing</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Xu, Hanwen</au><au>Tang, Xinming</au><au>Ai, Bo</au><au>Gao, Xiaoming</au><au>Yang, Fanlin</au><au>Wen, Zhen</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Missing data reconstruction in VHR images based on progressive structure prediction and texture generation</atitle><jtitle>ISPRS journal of photogrammetry and remote sensing</jtitle><stitle>ISPRS J PHOTOGRAMM</stitle><date>2021-01</date><risdate>2021</risdate><volume>171</volume><spage>266</spage><epage>277</epage><pages>266-277</pages><issn>0924-2716</issn><eissn>1872-8235</eissn><abstract>Very high resolution (VHR) satellite and aerial images often suffer from scene occlusion caused by redundant objects. The task of removing these redundant objects can be solved by missing data reconstruction technology. However, when dealing with VHR images with large-scale missing regions, existing spatial-based methods often destroy the structural information of ground objects. To alleviate this problem, this paper proposes a novel missing data reconstruction method based on deep learning. The reconstruction process is divided into two parts: structure prediction and texture generation. First, a progressive edge generation network (PEGN) is designed to predict the edges of objects in missing regions in a progressive manner. Then, the edge map predicted by PEGN is input to a texture generation network (TGN) as structural information to produce the reconstruction results. This is a spatial-based method that can produce realistic and reasonable results without any need for auxiliary spectral or temporal data. Experiments demonstrate that our model can better restore the structure of ground objects in VHR images than other spatial-based methods and outperform them in SSIM and PSNR indices. In addition, our model also has a strong generalization capability by introducing Poisson blending and histogram matching.</abstract><cop>AMSTERDAM</cop><pub>Elsevier B.V</pub><doi>10.1016/j.isprsjprs.2020.11.020</doi><tpages>12</tpages></addata></record> |
fulltext | fulltext |
identifier | ISSN: 0924-2716 |
ispartof | ISPRS journal of photogrammetry and remote sensing, 2021-01, Vol.171, p.266-277 |
issn | 0924-2716 1872-8235 |
language | eng |
recordid | cdi_webofscience_primary_000604406500018 |
source | Access via ScienceDirect (Elsevier) |
subjects | Deep learning Geography, Physical Geology Geosciences, Multidisciplinary Imaging Science & Photographic Technology Missing data reconstruction Physical Geography Physical Sciences Progressive structure prediction Remote Sensing Science & Technology Technology Texture generation VHR images |
title | Missing data reconstruction in VHR images based on progressive structure prediction and texture generation |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-17T10%3A28%3A56IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-elsevier_webof&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Missing%20data%20reconstruction%20in%20VHR%20images%20based%20on%20progressive%20structure%20prediction%20and%20texture%20generation&rft.jtitle=ISPRS%20journal%20of%20photogrammetry%20and%20remote%20sensing&rft.au=Xu,%20Hanwen&rft.date=2021-01&rft.volume=171&rft.spage=266&rft.epage=277&rft.pages=266-277&rft.issn=0924-2716&rft.eissn=1872-8235&rft_id=info:doi/10.1016/j.isprsjprs.2020.11.020&rft_dat=%3Celsevier_webof%3ES0924271620303270%3C/elsevier_webof%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rft_els_id=S0924271620303270&rfr_iscdi=true |