Trainable ISTA for Sparse Signal Recovery

In this paper, we propose a novel sparse signal recovery algorithm called the trainable iterative soft thresholding algorithm (TISTA). The proposed algorithm consists of two estimation units: a linear estimation unit and a minimum mean squared error (MMSE) estimator based shrinkage unit. The error v...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on signal processing 2019-06, Vol.67 (12), p.3113-3125
Hauptverfasser: Ito, Daisuke, Takabe, Satoshi, Wadayama, Tadashi
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 3125
container_issue 12
container_start_page 3113
container_title IEEE transactions on signal processing
container_volume 67
creator Ito, Daisuke
Takabe, Satoshi
Wadayama, Tadashi
description In this paper, we propose a novel sparse signal recovery algorithm called the trainable iterative soft thresholding algorithm (TISTA). The proposed algorithm consists of two estimation units: a linear estimation unit and a minimum mean squared error (MMSE) estimator based shrinkage unit. The error variance required in the MMSE shrinkage unit is precisely estimated from a tentative estimate of the original signal. The remarkable feature of the proposed scheme is that TISTA includes adjustable variables that control step size and the error variance for the MMSE shrinkage function. The variables are adjusted by standard deep learning techniques. The number of trainable variables of TISTA is nearly equal to the number of iteration rounds and is much smaller than that of known learnable sparse signal recovery algorithms. This feature leads to highly stable and fast training processes of TISTA. Computer experiments show that TISTA is applicable to various classes of sensing matrices, such as Gaussian matrices, binary matrices, and matrices with large condition numbers. Numerical results also demonstrate that, in many cases, TISTA provides significantly faster convergence than approximate message passing (AMP) and the learned iterative shrinkage thresholding algorithm and also outperforms orthogonal AMP in the NMSE performance.
doi_str_mv 10.1109/TSP.2019.2912879
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1109_TSP_2019_2912879</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>8695874</ieee_id><sourcerecordid>2220394151</sourcerecordid><originalsourceid>FETCH-LOGICAL-c377t-630266ca81264481852658d50e907df95dcb6e8d117710fe4b39416086466bc53</originalsourceid><addsrcrecordid>eNo9kM9LAzEQhYMoWKt3wcuCJw9bZ7LJJDmW4o9CQXFX8Bayu1nZUrs1qUL_e1NaPM0cvvd4fIxdI0wQwdxX5euEA5oJN8i1MidshEZgDkLRafpBFrnU6uOcXcS4BEAhDI3YXRVcv3b1ymfzsppm3RCycuNC9FnZf67dKnvzzfDrw-6SnXVuFf3V8Y7Z--NDNXvOFy9P89l0kTeFUtucCuBEjdPISQiNWnKSupXgDai2M7JtavK6RVQKofOiLtJMAk2CqG5kMWa3h95NGL5_fNza5fAT0pJoOeewpyUmCg5UE4YYg-_sJvRfLuwsgt0LsUmI3QuxRyEpcnOI9N77f1yTSVpE8QcSwliR</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2220394151</pqid></control><display><type>article</type><title>Trainable ISTA for Sparse Signal Recovery</title><source>IEEE Electronic Library (IEL)</source><creator>Ito, Daisuke ; Takabe, Satoshi ; Wadayama, Tadashi</creator><creatorcontrib>Ito, Daisuke ; Takabe, Satoshi ; Wadayama, Tadashi</creatorcontrib><description>In this paper, we propose a novel sparse signal recovery algorithm called the trainable iterative soft thresholding algorithm (TISTA). The proposed algorithm consists of two estimation units: a linear estimation unit and a minimum mean squared error (MMSE) estimator based shrinkage unit. The error variance required in the MMSE shrinkage unit is precisely estimated from a tentative estimate of the original signal. The remarkable feature of the proposed scheme is that TISTA includes adjustable variables that control step size and the error variance for the MMSE shrinkage function. The variables are adjusted by standard deep learning techniques. The number of trainable variables of TISTA is nearly equal to the number of iteration rounds and is much smaller than that of known learnable sparse signal recovery algorithms. This feature leads to highly stable and fast training processes of TISTA. Computer experiments show that TISTA is applicable to various classes of sensing matrices, such as Gaussian matrices, binary matrices, and matrices with large condition numbers. Numerical results also demonstrate that, in many cases, TISTA provides significantly faster convergence than approximate message passing (AMP) and the learned iterative shrinkage thresholding algorithm and also outperforms orthogonal AMP in the NMSE performance.</description><identifier>ISSN: 1053-587X</identifier><identifier>EISSN: 1941-0476</identifier><identifier>DOI: 10.1109/TSP.2019.2912879</identifier><identifier>CODEN: ITPRED</identifier><language>eng</language><publisher>New York: IEEE</publisher><subject>Algorithms ; Compressed sensing ; Convergence ; Errors ; Estimation ; Iterative methods ; Machine learning ; Message passing ; Probability density function ; Random variables ; Recovery ; Sensors ; Shrinkage ; Signal processing algorithms ; Signal reconstruction ; Sparse matrices ; supervised learning ; Variance</subject><ispartof>IEEE transactions on signal processing, 2019-06, Vol.67 (12), p.3113-3125</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2019</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c377t-630266ca81264481852658d50e907df95dcb6e8d117710fe4b39416086466bc53</citedby><cites>FETCH-LOGICAL-c377t-630266ca81264481852658d50e907df95dcb6e8d117710fe4b39416086466bc53</cites><orcidid>0000-0002-5755-2231 ; 0000-0003-4391-4294</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/8695874$$EHTML$$P50$$Gieee$$Hfree_for_read</linktohtml><link.rule.ids>314,780,784,796,27924,27925,54758</link.rule.ids></links><search><creatorcontrib>Ito, Daisuke</creatorcontrib><creatorcontrib>Takabe, Satoshi</creatorcontrib><creatorcontrib>Wadayama, Tadashi</creatorcontrib><title>Trainable ISTA for Sparse Signal Recovery</title><title>IEEE transactions on signal processing</title><addtitle>TSP</addtitle><description>In this paper, we propose a novel sparse signal recovery algorithm called the trainable iterative soft thresholding algorithm (TISTA). The proposed algorithm consists of two estimation units: a linear estimation unit and a minimum mean squared error (MMSE) estimator based shrinkage unit. The error variance required in the MMSE shrinkage unit is precisely estimated from a tentative estimate of the original signal. The remarkable feature of the proposed scheme is that TISTA includes adjustable variables that control step size and the error variance for the MMSE shrinkage function. The variables are adjusted by standard deep learning techniques. The number of trainable variables of TISTA is nearly equal to the number of iteration rounds and is much smaller than that of known learnable sparse signal recovery algorithms. This feature leads to highly stable and fast training processes of TISTA. Computer experiments show that TISTA is applicable to various classes of sensing matrices, such as Gaussian matrices, binary matrices, and matrices with large condition numbers. Numerical results also demonstrate that, in many cases, TISTA provides significantly faster convergence than approximate message passing (AMP) and the learned iterative shrinkage thresholding algorithm and also outperforms orthogonal AMP in the NMSE performance.</description><subject>Algorithms</subject><subject>Compressed sensing</subject><subject>Convergence</subject><subject>Errors</subject><subject>Estimation</subject><subject>Iterative methods</subject><subject>Machine learning</subject><subject>Message passing</subject><subject>Probability density function</subject><subject>Random variables</subject><subject>Recovery</subject><subject>Sensors</subject><subject>Shrinkage</subject><subject>Signal processing algorithms</subject><subject>Signal reconstruction</subject><subject>Sparse matrices</subject><subject>supervised learning</subject><subject>Variance</subject><issn>1053-587X</issn><issn>1941-0476</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><sourceid>ESBDL</sourceid><sourceid>RIE</sourceid><recordid>eNo9kM9LAzEQhYMoWKt3wcuCJw9bZ7LJJDmW4o9CQXFX8Bayu1nZUrs1qUL_e1NaPM0cvvd4fIxdI0wQwdxX5euEA5oJN8i1MidshEZgDkLRafpBFrnU6uOcXcS4BEAhDI3YXRVcv3b1ymfzsppm3RCycuNC9FnZf67dKnvzzfDrw-6SnXVuFf3V8Y7Z--NDNXvOFy9P89l0kTeFUtucCuBEjdPISQiNWnKSupXgDai2M7JtavK6RVQKofOiLtJMAk2CqG5kMWa3h95NGL5_fNza5fAT0pJoOeewpyUmCg5UE4YYg-_sJvRfLuwsgt0LsUmI3QuxRyEpcnOI9N77f1yTSVpE8QcSwliR</recordid><startdate>20190615</startdate><enddate>20190615</enddate><creator>Ito, Daisuke</creator><creator>Takabe, Satoshi</creator><creator>Wadayama, Tadashi</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>ESBDL</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0002-5755-2231</orcidid><orcidid>https://orcid.org/0000-0003-4391-4294</orcidid></search><sort><creationdate>20190615</creationdate><title>Trainable ISTA for Sparse Signal Recovery</title><author>Ito, Daisuke ; Takabe, Satoshi ; Wadayama, Tadashi</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c377t-630266ca81264481852658d50e907df95dcb6e8d117710fe4b39416086466bc53</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Algorithms</topic><topic>Compressed sensing</topic><topic>Convergence</topic><topic>Errors</topic><topic>Estimation</topic><topic>Iterative methods</topic><topic>Machine learning</topic><topic>Message passing</topic><topic>Probability density function</topic><topic>Random variables</topic><topic>Recovery</topic><topic>Sensors</topic><topic>Shrinkage</topic><topic>Signal processing algorithms</topic><topic>Signal reconstruction</topic><topic>Sparse matrices</topic><topic>supervised learning</topic><topic>Variance</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Ito, Daisuke</creatorcontrib><creatorcontrib>Takabe, Satoshi</creatorcontrib><creatorcontrib>Wadayama, Tadashi</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005–Present</collection><collection>IEEE Open Access Journals</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE transactions on signal processing</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Ito, Daisuke</au><au>Takabe, Satoshi</au><au>Wadayama, Tadashi</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Trainable ISTA for Sparse Signal Recovery</atitle><jtitle>IEEE transactions on signal processing</jtitle><stitle>TSP</stitle><date>2019-06-15</date><risdate>2019</risdate><volume>67</volume><issue>12</issue><spage>3113</spage><epage>3125</epage><pages>3113-3125</pages><issn>1053-587X</issn><eissn>1941-0476</eissn><coden>ITPRED</coden><abstract>In this paper, we propose a novel sparse signal recovery algorithm called the trainable iterative soft thresholding algorithm (TISTA). The proposed algorithm consists of two estimation units: a linear estimation unit and a minimum mean squared error (MMSE) estimator based shrinkage unit. The error variance required in the MMSE shrinkage unit is precisely estimated from a tentative estimate of the original signal. The remarkable feature of the proposed scheme is that TISTA includes adjustable variables that control step size and the error variance for the MMSE shrinkage function. The variables are adjusted by standard deep learning techniques. The number of trainable variables of TISTA is nearly equal to the number of iteration rounds and is much smaller than that of known learnable sparse signal recovery algorithms. This feature leads to highly stable and fast training processes of TISTA. Computer experiments show that TISTA is applicable to various classes of sensing matrices, such as Gaussian matrices, binary matrices, and matrices with large condition numbers. Numerical results also demonstrate that, in many cases, TISTA provides significantly faster convergence than approximate message passing (AMP) and the learned iterative shrinkage thresholding algorithm and also outperforms orthogonal AMP in the NMSE performance.</abstract><cop>New York</cop><pub>IEEE</pub><doi>10.1109/TSP.2019.2912879</doi><tpages>13</tpages><orcidid>https://orcid.org/0000-0002-5755-2231</orcidid><orcidid>https://orcid.org/0000-0003-4391-4294</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 1053-587X
ispartof IEEE transactions on signal processing, 2019-06, Vol.67 (12), p.3113-3125
issn 1053-587X
1941-0476
language eng
recordid cdi_crossref_primary_10_1109_TSP_2019_2912879
source IEEE Electronic Library (IEL)
subjects Algorithms
Compressed sensing
Convergence
Errors
Estimation
Iterative methods
Machine learning
Message passing
Probability density function
Random variables
Recovery
Sensors
Shrinkage
Signal processing algorithms
Signal reconstruction
Sparse matrices
supervised learning
Variance
title Trainable ISTA for Sparse Signal Recovery
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T19%3A17%3A03IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Trainable%20ISTA%20for%20Sparse%20Signal%20Recovery&rft.jtitle=IEEE%20transactions%20on%20signal%20processing&rft.au=Ito,%20Daisuke&rft.date=2019-06-15&rft.volume=67&rft.issue=12&rft.spage=3113&rft.epage=3125&rft.pages=3113-3125&rft.issn=1053-587X&rft.eissn=1941-0476&rft.coden=ITPRED&rft_id=info:doi/10.1109/TSP.2019.2912879&rft_dat=%3Cproquest_cross%3E2220394151%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2220394151&rft_id=info:pmid/&rft_ieee_id=8695874&rfr_iscdi=true