Fast and Robust Cascade Model for Multiple Degradation Single Image Super-Resolution

Single Image Super-Resolution (SISR) is one of the low-level computer vision problems that has received increased attention in the last few years. Current approaches are primarily based on harnessing the power of deep learning models and optimization techniques to reverse the degradation model. Owin...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on image processing 2021, Vol.30, p.4747-4759
Hauptverfasser: Lopez-Tapia, Santiago, de la Blanca, Nicolas Perez
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 4759
container_issue
container_start_page 4747
container_title IEEE transactions on image processing
container_volume 30
creator Lopez-Tapia, Santiago
de la Blanca, Nicolas Perez
description Single Image Super-Resolution (SISR) is one of the low-level computer vision problems that has received increased attention in the last few years. Current approaches are primarily based on harnessing the power of deep learning models and optimization techniques to reverse the degradation model. Owing to its hardness, isotropic blurring or Gaussians with small anisotropic deformations have been mainly considered. Here, we widen this scenario by including large non-Gaussian blurs that arise in real camera movements. Our approach leverages the degradation model and proposes a new formulation of the Convolutional Neural Network (CNN) cascade model, where each network sub-module is constrained to solve a specific degradation: deblurring or upsampling. A new densely connected CNN-architecture is proposed where the output of each sub-module is restricted using some external knowledge to focus it on its specific task. As far we know, this use of domain-knowledge to module-level is a novelty in SISR. To fit the finest model, a final sub-module takes care of the residual errors propagated by the previous sub-modules. We check our model with three state-of-the-art (SOTA) datasets in SISR and compare the results with the SOTA models. The results show that our model is the only one able to manage our wider set of deformations. Furthermore, our model overcomes all current SOTA methods for a standard set of deformations. In terms of computational load, our model also improves on the two closest competitors in terms of efficiency. Although the approach is non-blind and requires an estimation of the blur kernel, it shows robustness to blur kernel estimation errors, making it a good alternative to blind models.
doi_str_mv 10.1109/TIP.2021.3074821
format Article
fullrecord <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_ieee_primary_9416820</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9416820</ieee_id><sourcerecordid>2519321053</sourcerecordid><originalsourceid>FETCH-LOGICAL-c347t-1e3da50fc8161961aa6a4ad2abe4088b90435ecc1e0dc6e26edf3ecc3fd67ce33</originalsourceid><addsrcrecordid>eNpdkU1LAzEQQIMotlbvgiALXrxsnUmyX0epVguKYut5SZPZsmW7qcnuwX9vSqsHT5lk3gyZN4xdIowRobhbzN7HHDiOBWQy53jEhlhIjAEkPw4xJFmcoSwG7Mz7NQDKBNNTNhCigEQIHLLFVPkuUq2JPuyyD-FEea0MRa_WUBNV1kWvfdPV24aiB1o5ZVRX2zaa1-0qPM02akXRvN-Siz_I26bfZc_ZSaUaTxeHc8Q-p4-LyXP88vY0m9y_xFrIrIuRhFEJVDrHFIsUlUqVVIarJUnI82UBUiSkNRIYnRJPyVQi3EVl0kyTECN2u--7dfarJ9-Vm9prahrVku19yRMsBMfdqCN28w9d29614XeB4pwHM0UeKNhT2lnvHVXl1tUb5b5LhHJnvAzGy53x8mA8lFwfGvfLDZm_gl_FAbjaAzUR_aXDltKcg_gB_YCEDg</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2522214598</pqid></control><display><type>article</type><title>Fast and Robust Cascade Model for Multiple Degradation Single Image Super-Resolution</title><source>IEEE Electronic Library (IEL)</source><creator>Lopez-Tapia, Santiago ; de la Blanca, Nicolas Perez</creator><creatorcontrib>Lopez-Tapia, Santiago ; de la Blanca, Nicolas Perez</creatorcontrib><description>Single Image Super-Resolution (SISR) is one of the low-level computer vision problems that has received increased attention in the last few years. Current approaches are primarily based on harnessing the power of deep learning models and optimization techniques to reverse the degradation model. Owing to its hardness, isotropic blurring or Gaussians with small anisotropic deformations have been mainly considered. Here, we widen this scenario by including large non-Gaussian blurs that arise in real camera movements. Our approach leverages the degradation model and proposes a new formulation of the Convolutional Neural Network (CNN) cascade model, where each network sub-module is constrained to solve a specific degradation: deblurring or upsampling. A new densely connected CNN-architecture is proposed where the output of each sub-module is restricted using some external knowledge to focus it on its specific task. As far we know, this use of domain-knowledge to module-level is a novelty in SISR. To fit the finest model, a final sub-module takes care of the residual errors propagated by the previous sub-modules. We check our model with three state-of-the-art (SOTA) datasets in SISR and compare the results with the SOTA models. The results show that our model is the only one able to manage our wider set of deformations. Furthermore, our model overcomes all current SOTA methods for a standard set of deformations. In terms of computational load, our model also improves on the two closest competitors in terms of efficiency. Although the approach is non-blind and requires an estimation of the blur kernel, it shows robustness to blur kernel estimation errors, making it a good alternative to blind models.</description><identifier>ISSN: 1057-7149</identifier><identifier>EISSN: 1941-0042</identifier><identifier>DOI: 10.1109/TIP.2021.3074821</identifier><identifier>PMID: 33905331</identifier><identifier>CODEN: IIPRE4</identifier><language>eng</language><publisher>United States: IEEE</publisher><subject>Adaptation models ; Artificial neural networks ; Blurring ; cascade model ; Computational modeling ; Computer vision ; convolutional neural networks ; Deformation ; Degradation ; Feature extraction ; Image degradation ; Image resolution ; Kernel ; Kernels ; Modules ; multiple degradation deconvolution ; Optimization ; Optimization techniques ; Single image super-resolution ; super-resolution ; Superresolution ; Task analysis</subject><ispartof>IEEE transactions on image processing, 2021, Vol.30, p.4747-4759</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c347t-1e3da50fc8161961aa6a4ad2abe4088b90435ecc1e0dc6e26edf3ecc3fd67ce33</citedby><cites>FETCH-LOGICAL-c347t-1e3da50fc8161961aa6a4ad2abe4088b90435ecc1e0dc6e26edf3ecc3fd67ce33</cites><orcidid>0000-0003-2090-7446</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9416820$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,796,4024,27923,27924,27925,54758</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/9416820$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/33905331$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Lopez-Tapia, Santiago</creatorcontrib><creatorcontrib>de la Blanca, Nicolas Perez</creatorcontrib><title>Fast and Robust Cascade Model for Multiple Degradation Single Image Super-Resolution</title><title>IEEE transactions on image processing</title><addtitle>TIP</addtitle><addtitle>IEEE Trans Image Process</addtitle><description>Single Image Super-Resolution (SISR) is one of the low-level computer vision problems that has received increased attention in the last few years. Current approaches are primarily based on harnessing the power of deep learning models and optimization techniques to reverse the degradation model. Owing to its hardness, isotropic blurring or Gaussians with small anisotropic deformations have been mainly considered. Here, we widen this scenario by including large non-Gaussian blurs that arise in real camera movements. Our approach leverages the degradation model and proposes a new formulation of the Convolutional Neural Network (CNN) cascade model, where each network sub-module is constrained to solve a specific degradation: deblurring or upsampling. A new densely connected CNN-architecture is proposed where the output of each sub-module is restricted using some external knowledge to focus it on its specific task. As far we know, this use of domain-knowledge to module-level is a novelty in SISR. To fit the finest model, a final sub-module takes care of the residual errors propagated by the previous sub-modules. We check our model with three state-of-the-art (SOTA) datasets in SISR and compare the results with the SOTA models. The results show that our model is the only one able to manage our wider set of deformations. Furthermore, our model overcomes all current SOTA methods for a standard set of deformations. In terms of computational load, our model also improves on the two closest competitors in terms of efficiency. Although the approach is non-blind and requires an estimation of the blur kernel, it shows robustness to blur kernel estimation errors, making it a good alternative to blind models.</description><subject>Adaptation models</subject><subject>Artificial neural networks</subject><subject>Blurring</subject><subject>cascade model</subject><subject>Computational modeling</subject><subject>Computer vision</subject><subject>convolutional neural networks</subject><subject>Deformation</subject><subject>Degradation</subject><subject>Feature extraction</subject><subject>Image degradation</subject><subject>Image resolution</subject><subject>Kernel</subject><subject>Kernels</subject><subject>Modules</subject><subject>multiple degradation deconvolution</subject><subject>Optimization</subject><subject>Optimization techniques</subject><subject>Single image super-resolution</subject><subject>super-resolution</subject><subject>Superresolution</subject><subject>Task analysis</subject><issn>1057-7149</issn><issn>1941-0042</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNpdkU1LAzEQQIMotlbvgiALXrxsnUmyX0epVguKYut5SZPZsmW7qcnuwX9vSqsHT5lk3gyZN4xdIowRobhbzN7HHDiOBWQy53jEhlhIjAEkPw4xJFmcoSwG7Mz7NQDKBNNTNhCigEQIHLLFVPkuUq2JPuyyD-FEea0MRa_WUBNV1kWvfdPV24aiB1o5ZVRX2zaa1-0qPM02akXRvN-Siz_I26bfZc_ZSaUaTxeHc8Q-p4-LyXP88vY0m9y_xFrIrIuRhFEJVDrHFIsUlUqVVIarJUnI82UBUiSkNRIYnRJPyVQi3EVl0kyTECN2u--7dfarJ9-Vm9prahrVku19yRMsBMfdqCN28w9d29614XeB4pwHM0UeKNhT2lnvHVXl1tUb5b5LhHJnvAzGy53x8mA8lFwfGvfLDZm_gl_FAbjaAzUR_aXDltKcg_gB_YCEDg</recordid><startdate>2021</startdate><enddate>2021</enddate><creator>Lopez-Tapia, Santiago</creator><creator>de la Blanca, Nicolas Perez</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0003-2090-7446</orcidid></search><sort><creationdate>2021</creationdate><title>Fast and Robust Cascade Model for Multiple Degradation Single Image Super-Resolution</title><author>Lopez-Tapia, Santiago ; de la Blanca, Nicolas Perez</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c347t-1e3da50fc8161961aa6a4ad2abe4088b90435ecc1e0dc6e26edf3ecc3fd67ce33</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Adaptation models</topic><topic>Artificial neural networks</topic><topic>Blurring</topic><topic>cascade model</topic><topic>Computational modeling</topic><topic>Computer vision</topic><topic>convolutional neural networks</topic><topic>Deformation</topic><topic>Degradation</topic><topic>Feature extraction</topic><topic>Image degradation</topic><topic>Image resolution</topic><topic>Kernel</topic><topic>Kernels</topic><topic>Modules</topic><topic>multiple degradation deconvolution</topic><topic>Optimization</topic><topic>Optimization techniques</topic><topic>Single image super-resolution</topic><topic>super-resolution</topic><topic>Superresolution</topic><topic>Task analysis</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Lopez-Tapia, Santiago</creatorcontrib><creatorcontrib>de la Blanca, Nicolas Perez</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>MEDLINE - Academic</collection><jtitle>IEEE transactions on image processing</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Lopez-Tapia, Santiago</au><au>de la Blanca, Nicolas Perez</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Fast and Robust Cascade Model for Multiple Degradation Single Image Super-Resolution</atitle><jtitle>IEEE transactions on image processing</jtitle><stitle>TIP</stitle><addtitle>IEEE Trans Image Process</addtitle><date>2021</date><risdate>2021</risdate><volume>30</volume><spage>4747</spage><epage>4759</epage><pages>4747-4759</pages><issn>1057-7149</issn><eissn>1941-0042</eissn><coden>IIPRE4</coden><abstract>Single Image Super-Resolution (SISR) is one of the low-level computer vision problems that has received increased attention in the last few years. Current approaches are primarily based on harnessing the power of deep learning models and optimization techniques to reverse the degradation model. Owing to its hardness, isotropic blurring or Gaussians with small anisotropic deformations have been mainly considered. Here, we widen this scenario by including large non-Gaussian blurs that arise in real camera movements. Our approach leverages the degradation model and proposes a new formulation of the Convolutional Neural Network (CNN) cascade model, where each network sub-module is constrained to solve a specific degradation: deblurring or upsampling. A new densely connected CNN-architecture is proposed where the output of each sub-module is restricted using some external knowledge to focus it on its specific task. As far we know, this use of domain-knowledge to module-level is a novelty in SISR. To fit the finest model, a final sub-module takes care of the residual errors propagated by the previous sub-modules. We check our model with three state-of-the-art (SOTA) datasets in SISR and compare the results with the SOTA models. The results show that our model is the only one able to manage our wider set of deformations. Furthermore, our model overcomes all current SOTA methods for a standard set of deformations. In terms of computational load, our model also improves on the two closest competitors in terms of efficiency. Although the approach is non-blind and requires an estimation of the blur kernel, it shows robustness to blur kernel estimation errors, making it a good alternative to blind models.</abstract><cop>United States</cop><pub>IEEE</pub><pmid>33905331</pmid><doi>10.1109/TIP.2021.3074821</doi><tpages>13</tpages><orcidid>https://orcid.org/0000-0003-2090-7446</orcidid></addata></record>
fulltext fulltext_linktorsrc
identifier ISSN: 1057-7149
ispartof IEEE transactions on image processing, 2021, Vol.30, p.4747-4759
issn 1057-7149
1941-0042
language eng
recordid cdi_ieee_primary_9416820
source IEEE Electronic Library (IEL)
subjects Adaptation models
Artificial neural networks
Blurring
cascade model
Computational modeling
Computer vision
convolutional neural networks
Deformation
Degradation
Feature extraction
Image degradation
Image resolution
Kernel
Kernels
Modules
multiple degradation deconvolution
Optimization
Optimization techniques
Single image super-resolution
super-resolution
Superresolution
Task analysis
title Fast and Robust Cascade Model for Multiple Degradation Single Image Super-Resolution
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-20T04%3A44%3A46IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Fast%20and%20Robust%20Cascade%20Model%20for%20Multiple%20Degradation%20Single%20Image%20Super-Resolution&rft.jtitle=IEEE%20transactions%20on%20image%20processing&rft.au=Lopez-Tapia,%20Santiago&rft.date=2021&rft.volume=30&rft.spage=4747&rft.epage=4759&rft.pages=4747-4759&rft.issn=1057-7149&rft.eissn=1941-0042&rft.coden=IIPRE4&rft_id=info:doi/10.1109/TIP.2021.3074821&rft_dat=%3Cproquest_RIE%3E2519321053%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2522214598&rft_id=info:pmid/33905331&rft_ieee_id=9416820&rfr_iscdi=true