Broad Spectrum Image Deblurring via an Adaptive Super-Network

In blurry images, the degree of image blurs may vary drastically due to different factors, such as varying speeds of shaking cameras and moving objects, as well as defects of the camera lens. However, current end-to-end models failed to explicitly take into account such diversity of blurs. This unaw...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on image processing 2023, Vol.32, p.5270-5282
Hauptverfasser: Wu, Qiucheng, Jiang, Yifan, Wu, Junru, Kulikov, Victor, Goel, Vidit, Orlov, Nikita, Shi, Humphrey, Wang, Zhangyang, Chang, Shiyu
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 5282
container_issue
container_start_page 5270
container_title IEEE transactions on image processing
container_volume 32
creator Wu, Qiucheng
Jiang, Yifan
Wu, Junru
Kulikov, Victor
Goel, Vidit
Orlov, Nikita
Shi, Humphrey
Wang, Zhangyang
Chang, Shiyu
description In blurry images, the degree of image blurs may vary drastically due to different factors, such as varying speeds of shaking cameras and moving objects, as well as defects of the camera lens. However, current end-to-end models failed to explicitly take into account such diversity of blurs. This unawareness compromises the specialization at each blur level, yielding sub-optimal deblurred images as well as redundant post-processing. Therefore, how to specialize one model simultaneously at different blur levels, while still ensuring coverage and generalization, becomes an emerging challenge. In this work, we propose Ada-Deblur, a super-network that can be applied to a "broad spectrum" of blur levels with no re-training on novel blurs. To balance between individual blur level specialization and wide-range blur levels coverage, the key idea is to dynamically adapt the network architectures from a single well-trained super-network structure, targeting flexible image processing with different deblurring capacities at test time. Extensive experiments demonstrate that our work outperforms strong baselines by demonstrating better reconstruction accuracy while incurring minimal computational overhead. Besides, we show that our method is effective for both synthetic and realistic blurs compared to these baselines. The performance gap between our model and the state-of-the-art becomes more prominent when testing with unseen and strong blur levels. Specifically, our model demonstrates surprising deblurring performance on these images with PSNR improvements of around 1 dB. Our code is publicly available at https://github.com/wuqiuche/Ada-Deblur .
doi_str_mv 10.1109/TIP.2023.3312912
format Article
fullrecord <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_proquest_journals_2867396276</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10254493</ieee_id><sourcerecordid>2867396276</sourcerecordid><originalsourceid>FETCH-LOGICAL-c278t-81397019db7ebe96d8d92bf25ee8039b9bd6d9fe72534f81edb5f1ea925d6fe93</originalsourceid><addsrcrecordid>eNpdkD1PwzAQhi0EoqWwMzBEYmFJ8dmJHQ8MpXxVqgCpZbac-FKlJE2wkyL-PanaATHdDc_76u4h5BLoGICq2-Xsfcwo42POgSlgR2QIKoKQ0ogd9zuNZSghUgNy5v2aUohiEKdkwKVkkEg2JHf3rjY2WDSYta6rglllVhg8YFp2zhWbVbAtTGA2wcSapi22GCy6Bl34iu137T7PyUluSo8XhzkiH0-Py-lLOH97nk0n8zBjMmnDBLiSFJRNJaaohE2sYmnOYsSEcpWq1AqrcpQs5lGeANo0zgGNYrEVOSo-Ijf73sbVXx36VleFz7AszQbrzmuWCNH_RCX06PU_dF13btNft6MkV4JJ0VN0T2Wu9t5hrhtXVMb9aKB6p1b3avVOrT6o7SNX-0iBiH9wFkeR4vwXdZFxtg</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2867396276</pqid></control><display><type>article</type><title>Broad Spectrum Image Deblurring via an Adaptive Super-Network</title><source>IEEE Electronic Library (IEL)</source><creator>Wu, Qiucheng ; Jiang, Yifan ; Wu, Junru ; Kulikov, Victor ; Goel, Vidit ; Orlov, Nikita ; Shi, Humphrey ; Wang, Zhangyang ; Chang, Shiyu</creator><creatorcontrib>Wu, Qiucheng ; Jiang, Yifan ; Wu, Junru ; Kulikov, Victor ; Goel, Vidit ; Orlov, Nikita ; Shi, Humphrey ; Wang, Zhangyang ; Chang, Shiyu</creatorcontrib><description>In blurry images, the degree of image blurs may vary drastically due to different factors, such as varying speeds of shaking cameras and moving objects, as well as defects of the camera lens. However, current end-to-end models failed to explicitly take into account such diversity of blurs. This unawareness compromises the specialization at each blur level, yielding sub-optimal deblurred images as well as redundant post-processing. Therefore, how to specialize one model simultaneously at different blur levels, while still ensuring coverage and generalization, becomes an emerging challenge. In this work, we propose Ada-Deblur, a super-network that can be applied to a "broad spectrum" of blur levels with no re-training on novel blurs. To balance between individual blur level specialization and wide-range blur levels coverage, the key idea is to dynamically adapt the network architectures from a single well-trained super-network structure, targeting flexible image processing with different deblurring capacities at test time. Extensive experiments demonstrate that our work outperforms strong baselines by demonstrating better reconstruction accuracy while incurring minimal computational overhead. Besides, we show that our method is effective for both synthetic and realistic blurs compared to these baselines. The performance gap between our model and the state-of-the-art becomes more prominent when testing with unseen and strong blur levels. Specifically, our model demonstrates surprising deblurring performance on these images with PSNR improvements of around 1 dB. Our code is publicly available at https://github.com/wuqiuche/Ada-Deblur .</description><identifier>ISSN: 1057-7149</identifier><identifier>EISSN: 1941-0042</identifier><identifier>DOI: 10.1109/TIP.2023.3312912</identifier><identifier>PMID: 37721872</identifier><identifier>CODEN: IIPRE4</identifier><language>eng</language><publisher>New York: IEEE</publisher><subject>Adaptation models ; adaptive network ; Cameras ; Computer architecture ; Image processing ; Image reconstruction ; Image restoration ; Kernel ; Network architecture ; Shaking ; Task analysis ; Training</subject><ispartof>IEEE transactions on image processing, 2023, Vol.32, p.5270-5282</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c278t-81397019db7ebe96d8d92bf25ee8039b9bd6d9fe72534f81edb5f1ea925d6fe93</cites><orcidid>0000-0003-1026-8783 ; 0000-0002-2050-5693 ; 0000-0002-7113-305X ; 0000-0001-8483-6363 ; 0000-0003-4443-0873 ; 0000-0002-2922-5663</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10254493$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,796,4022,27922,27923,27924,54757</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/10254493$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Wu, Qiucheng</creatorcontrib><creatorcontrib>Jiang, Yifan</creatorcontrib><creatorcontrib>Wu, Junru</creatorcontrib><creatorcontrib>Kulikov, Victor</creatorcontrib><creatorcontrib>Goel, Vidit</creatorcontrib><creatorcontrib>Orlov, Nikita</creatorcontrib><creatorcontrib>Shi, Humphrey</creatorcontrib><creatorcontrib>Wang, Zhangyang</creatorcontrib><creatorcontrib>Chang, Shiyu</creatorcontrib><title>Broad Spectrum Image Deblurring via an Adaptive Super-Network</title><title>IEEE transactions on image processing</title><addtitle>TIP</addtitle><description>In blurry images, the degree of image blurs may vary drastically due to different factors, such as varying speeds of shaking cameras and moving objects, as well as defects of the camera lens. However, current end-to-end models failed to explicitly take into account such diversity of blurs. This unawareness compromises the specialization at each blur level, yielding sub-optimal deblurred images as well as redundant post-processing. Therefore, how to specialize one model simultaneously at different blur levels, while still ensuring coverage and generalization, becomes an emerging challenge. In this work, we propose Ada-Deblur, a super-network that can be applied to a "broad spectrum" of blur levels with no re-training on novel blurs. To balance between individual blur level specialization and wide-range blur levels coverage, the key idea is to dynamically adapt the network architectures from a single well-trained super-network structure, targeting flexible image processing with different deblurring capacities at test time. Extensive experiments demonstrate that our work outperforms strong baselines by demonstrating better reconstruction accuracy while incurring minimal computational overhead. Besides, we show that our method is effective for both synthetic and realistic blurs compared to these baselines. The performance gap between our model and the state-of-the-art becomes more prominent when testing with unseen and strong blur levels. Specifically, our model demonstrates surprising deblurring performance on these images with PSNR improvements of around 1 dB. Our code is publicly available at https://github.com/wuqiuche/Ada-Deblur .</description><subject>Adaptation models</subject><subject>adaptive network</subject><subject>Cameras</subject><subject>Computer architecture</subject><subject>Image processing</subject><subject>Image reconstruction</subject><subject>Image restoration</subject><subject>Kernel</subject><subject>Network architecture</subject><subject>Shaking</subject><subject>Task analysis</subject><subject>Training</subject><issn>1057-7149</issn><issn>1941-0042</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNpdkD1PwzAQhi0EoqWwMzBEYmFJ8dmJHQ8MpXxVqgCpZbac-FKlJE2wkyL-PanaATHdDc_76u4h5BLoGICq2-Xsfcwo42POgSlgR2QIKoKQ0ogd9zuNZSghUgNy5v2aUohiEKdkwKVkkEg2JHf3rjY2WDSYta6rglllVhg8YFp2zhWbVbAtTGA2wcSapi22GCy6Bl34iu137T7PyUluSo8XhzkiH0-Py-lLOH97nk0n8zBjMmnDBLiSFJRNJaaohE2sYmnOYsSEcpWq1AqrcpQs5lGeANo0zgGNYrEVOSo-Ijf73sbVXx36VleFz7AszQbrzmuWCNH_RCX06PU_dF13btNft6MkV4JJ0VN0T2Wu9t5hrhtXVMb9aKB6p1b3avVOrT6o7SNX-0iBiH9wFkeR4vwXdZFxtg</recordid><startdate>2023</startdate><enddate>2023</enddate><creator>Wu, Qiucheng</creator><creator>Jiang, Yifan</creator><creator>Wu, Junru</creator><creator>Kulikov, Victor</creator><creator>Goel, Vidit</creator><creator>Orlov, Nikita</creator><creator>Shi, Humphrey</creator><creator>Wang, Zhangyang</creator><creator>Chang, Shiyu</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0003-1026-8783</orcidid><orcidid>https://orcid.org/0000-0002-2050-5693</orcidid><orcidid>https://orcid.org/0000-0002-7113-305X</orcidid><orcidid>https://orcid.org/0000-0001-8483-6363</orcidid><orcidid>https://orcid.org/0000-0003-4443-0873</orcidid><orcidid>https://orcid.org/0000-0002-2922-5663</orcidid></search><sort><creationdate>2023</creationdate><title>Broad Spectrum Image Deblurring via an Adaptive Super-Network</title><author>Wu, Qiucheng ; Jiang, Yifan ; Wu, Junru ; Kulikov, Victor ; Goel, Vidit ; Orlov, Nikita ; Shi, Humphrey ; Wang, Zhangyang ; Chang, Shiyu</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c278t-81397019db7ebe96d8d92bf25ee8039b9bd6d9fe72534f81edb5f1ea925d6fe93</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Adaptation models</topic><topic>adaptive network</topic><topic>Cameras</topic><topic>Computer architecture</topic><topic>Image processing</topic><topic>Image reconstruction</topic><topic>Image restoration</topic><topic>Kernel</topic><topic>Network architecture</topic><topic>Shaking</topic><topic>Task analysis</topic><topic>Training</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Wu, Qiucheng</creatorcontrib><creatorcontrib>Jiang, Yifan</creatorcontrib><creatorcontrib>Wu, Junru</creatorcontrib><creatorcontrib>Kulikov, Victor</creatorcontrib><creatorcontrib>Goel, Vidit</creatorcontrib><creatorcontrib>Orlov, Nikita</creatorcontrib><creatorcontrib>Shi, Humphrey</creatorcontrib><creatorcontrib>Wang, Zhangyang</creatorcontrib><creatorcontrib>Chang, Shiyu</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>MEDLINE - Academic</collection><jtitle>IEEE transactions on image processing</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Wu, Qiucheng</au><au>Jiang, Yifan</au><au>Wu, Junru</au><au>Kulikov, Victor</au><au>Goel, Vidit</au><au>Orlov, Nikita</au><au>Shi, Humphrey</au><au>Wang, Zhangyang</au><au>Chang, Shiyu</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Broad Spectrum Image Deblurring via an Adaptive Super-Network</atitle><jtitle>IEEE transactions on image processing</jtitle><stitle>TIP</stitle><date>2023</date><risdate>2023</risdate><volume>32</volume><spage>5270</spage><epage>5282</epage><pages>5270-5282</pages><issn>1057-7149</issn><eissn>1941-0042</eissn><coden>IIPRE4</coden><abstract>In blurry images, the degree of image blurs may vary drastically due to different factors, such as varying speeds of shaking cameras and moving objects, as well as defects of the camera lens. However, current end-to-end models failed to explicitly take into account such diversity of blurs. This unawareness compromises the specialization at each blur level, yielding sub-optimal deblurred images as well as redundant post-processing. Therefore, how to specialize one model simultaneously at different blur levels, while still ensuring coverage and generalization, becomes an emerging challenge. In this work, we propose Ada-Deblur, a super-network that can be applied to a "broad spectrum" of blur levels with no re-training on novel blurs. To balance between individual blur level specialization and wide-range blur levels coverage, the key idea is to dynamically adapt the network architectures from a single well-trained super-network structure, targeting flexible image processing with different deblurring capacities at test time. Extensive experiments demonstrate that our work outperforms strong baselines by demonstrating better reconstruction accuracy while incurring minimal computational overhead. Besides, we show that our method is effective for both synthetic and realistic blurs compared to these baselines. The performance gap between our model and the state-of-the-art becomes more prominent when testing with unseen and strong blur levels. Specifically, our model demonstrates surprising deblurring performance on these images with PSNR improvements of around 1 dB. Our code is publicly available at https://github.com/wuqiuche/Ada-Deblur .</abstract><cop>New York</cop><pub>IEEE</pub><pmid>37721872</pmid><doi>10.1109/TIP.2023.3312912</doi><tpages>13</tpages><orcidid>https://orcid.org/0000-0003-1026-8783</orcidid><orcidid>https://orcid.org/0000-0002-2050-5693</orcidid><orcidid>https://orcid.org/0000-0002-7113-305X</orcidid><orcidid>https://orcid.org/0000-0001-8483-6363</orcidid><orcidid>https://orcid.org/0000-0003-4443-0873</orcidid><orcidid>https://orcid.org/0000-0002-2922-5663</orcidid></addata></record>
fulltext fulltext_linktorsrc
identifier ISSN: 1057-7149
ispartof IEEE transactions on image processing, 2023, Vol.32, p.5270-5282
issn 1057-7149
1941-0042
language eng
recordid cdi_proquest_journals_2867396276
source IEEE Electronic Library (IEL)
subjects Adaptation models
adaptive network
Cameras
Computer architecture
Image processing
Image reconstruction
Image restoration
Kernel
Network architecture
Shaking
Task analysis
Training
title Broad Spectrum Image Deblurring via an Adaptive Super-Network
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-12T12%3A09%3A23IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Broad%20Spectrum%20Image%20Deblurring%20via%20an%20Adaptive%20Super-Network&rft.jtitle=IEEE%20transactions%20on%20image%20processing&rft.au=Wu,%20Qiucheng&rft.date=2023&rft.volume=32&rft.spage=5270&rft.epage=5282&rft.pages=5270-5282&rft.issn=1057-7149&rft.eissn=1941-0042&rft.coden=IIPRE4&rft_id=info:doi/10.1109/TIP.2023.3312912&rft_dat=%3Cproquest_RIE%3E2867396276%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2867396276&rft_id=info:pmid/37721872&rft_ieee_id=10254493&rfr_iscdi=true