An inexact successive quadratic approximation method for L-1 regularized optimization
We study a Newton-like method for the minimization of an objective function ϕ that is the sum of a smooth function and an ℓ 1 regularization term. This method, which is sometimes referred to in the literature as a proximal Newton method, computes a step by minimizing a piecewise quadratic model q k...
Gespeichert in:
| Veröffentlicht in: | Mathematical programming 2016-06, Vol.157 (2), p.375-396 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 396 |
|---|---|
| container_issue | 2 |
| container_start_page | 375 |
| container_title | Mathematical programming |
| container_volume | 157 |
| creator | Byrd, Richard H. Nocedal, Jorge Oztoprak, Figen |
| description | We study a Newton-like method for the minimization of an objective function
ϕ
that is the sum of a smooth function and an
ℓ
1
regularization term. This method, which is sometimes referred to in the literature as a proximal Newton method, computes a step by minimizing a piecewise quadratic model
q
k
of the objective function
ϕ
. In order to make this approach efficient in practice, it is imperative to perform this inner minimization inexactly. In this paper, we give inexactness conditions that guarantee global convergence and that can be used to control the local rate of convergence of the iteration. Our inexactness conditions are based on a semi-smooth function that represents a (continuous) measure of the optimality conditions of the problem, and that embodies the soft-thresholding iteration. We give careful consideration to the algorithm employed for the inner minimization, and report numerical results on two test sets originating in machine learning. |
| doi_str_mv | 10.1007/s10107-015-0941-y |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_sprin</sourceid><recordid>TN_cdi_proquest_miscellaneous_1825468261</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>1825468261</sourcerecordid><originalsourceid>FETCH-LOGICAL-p189t-df01da09810a8fb253497f3ab2d82313a5aca767b7873c635488ced3026dc97d3</originalsourceid><addsrcrecordid>eNpd0E1LAzEQgOEgCtbqD_AW8OIlOpPsJtljKX5BwYs9L2mSrSn71c2utP31bq0H8TQMPAzDS8gtwgMCqMeIgKAYYMogS5Dtz8gEEyFZIhN5TiYAPGWpRLgkVzFuAACF1hOynNU01H5nbE_jYK2PMXx5uh2M60wfLDVt2zW7UI1LU9PK95-No0XT0QVD2vn1UJouHLyjTduHKhx-3DW5KEwZ_c3vnJLl89PH_JUt3l_e5rMFa1FnPXMFoDOQaQSjixVPRZKpQpgVd5oLFCY11iipVkorYaVIE62tdwK4dDZTTkzJ_enu-ON28LHPqxCtL0tT-2aIOWqeJlJziSO9-0c3zdDV43c5qgy5gESmo-InFdsu1Gvf_VGQH0vnp9L5WDo_ls734hvNo3Ga</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1791230465</pqid></control><display><type>article</type><title>An inexact successive quadratic approximation method for L-1 regularized optimization</title><source>SpringerNature Journals</source><source>EBSCOhost Business Source Complete</source><creator>Byrd, Richard H. ; Nocedal, Jorge ; Oztoprak, Figen</creator><creatorcontrib>Byrd, Richard H. ; Nocedal, Jorge ; Oztoprak, Figen</creatorcontrib><description>We study a Newton-like method for the minimization of an objective function
ϕ
that is the sum of a smooth function and an
ℓ
1
regularization term. This method, which is sometimes referred to in the literature as a proximal Newton method, computes a step by minimizing a piecewise quadratic model
q
k
of the objective function
ϕ
. In order to make this approach efficient in practice, it is imperative to perform this inner minimization inexactly. In this paper, we give inexactness conditions that guarantee global convergence and that can be used to control the local rate of convergence of the iteration. Our inexactness conditions are based on a semi-smooth function that represents a (continuous) measure of the optimality conditions of the problem, and that embodies the soft-thresholding iteration. We give careful consideration to the algorithm employed for the inner minimization, and report numerical results on two test sets originating in machine learning.</description><identifier>ISSN: 0025-5610</identifier><identifier>EISSN: 1436-4646</identifier><identifier>DOI: 10.1007/s10107-015-0941-y</identifier><identifier>CODEN: MHPGA4</identifier><language>eng</language><publisher>Berlin/Heidelberg: Springer Berlin Heidelberg</publisher><subject>Algorithms ; Approximation ; Artificial intelligence ; Calculus of Variations and Optimal Control; Optimization ; Combinatorics ; Convergence ; Full Length Paper ; Industrial engineering ; Iterative methods ; Mathematical analysis ; Mathematical and Computational Physics ; Mathematical Methods in Physics ; Mathematical models ; Mathematics ; Mathematics and Statistics ; Mathematics of Computing ; Minimization ; Newton methods ; Numerical Analysis ; Optimization ; Studies ; Test sets ; Theoretical</subject><ispartof>Mathematical programming, 2016-06, Vol.157 (2), p.375-396</ispartof><rights>Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2015</rights><rights>Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2016</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s10107-015-0941-y$$EPDF$$P50$$Gspringer$$H</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s10107-015-0941-y$$EHTML$$P50$$Gspringer$$H</linktohtml><link.rule.ids>314,780,784,27924,27925,41488,42557,51319</link.rule.ids></links><search><creatorcontrib>Byrd, Richard H.</creatorcontrib><creatorcontrib>Nocedal, Jorge</creatorcontrib><creatorcontrib>Oztoprak, Figen</creatorcontrib><title>An inexact successive quadratic approximation method for L-1 regularized optimization</title><title>Mathematical programming</title><addtitle>Math. Program</addtitle><description>We study a Newton-like method for the minimization of an objective function
ϕ
that is the sum of a smooth function and an
ℓ
1
regularization term. This method, which is sometimes referred to in the literature as a proximal Newton method, computes a step by minimizing a piecewise quadratic model
q
k
of the objective function
ϕ
. In order to make this approach efficient in practice, it is imperative to perform this inner minimization inexactly. In this paper, we give inexactness conditions that guarantee global convergence and that can be used to control the local rate of convergence of the iteration. Our inexactness conditions are based on a semi-smooth function that represents a (continuous) measure of the optimality conditions of the problem, and that embodies the soft-thresholding iteration. We give careful consideration to the algorithm employed for the inner minimization, and report numerical results on two test sets originating in machine learning.</description><subject>Algorithms</subject><subject>Approximation</subject><subject>Artificial intelligence</subject><subject>Calculus of Variations and Optimal Control; Optimization</subject><subject>Combinatorics</subject><subject>Convergence</subject><subject>Full Length Paper</subject><subject>Industrial engineering</subject><subject>Iterative methods</subject><subject>Mathematical analysis</subject><subject>Mathematical and Computational Physics</subject><subject>Mathematical Methods in Physics</subject><subject>Mathematical models</subject><subject>Mathematics</subject><subject>Mathematics and Statistics</subject><subject>Mathematics of Computing</subject><subject>Minimization</subject><subject>Newton methods</subject><subject>Numerical Analysis</subject><subject>Optimization</subject><subject>Studies</subject><subject>Test sets</subject><subject>Theoretical</subject><issn>0025-5610</issn><issn>1436-4646</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2016</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><recordid>eNpd0E1LAzEQgOEgCtbqD_AW8OIlOpPsJtljKX5BwYs9L2mSrSn71c2utP31bq0H8TQMPAzDS8gtwgMCqMeIgKAYYMogS5Dtz8gEEyFZIhN5TiYAPGWpRLgkVzFuAACF1hOynNU01H5nbE_jYK2PMXx5uh2M60wfLDVt2zW7UI1LU9PK95-No0XT0QVD2vn1UJouHLyjTduHKhx-3DW5KEwZ_c3vnJLl89PH_JUt3l_e5rMFa1FnPXMFoDOQaQSjixVPRZKpQpgVd5oLFCY11iipVkorYaVIE62tdwK4dDZTTkzJ_enu-ON28LHPqxCtL0tT-2aIOWqeJlJziSO9-0c3zdDV43c5qgy5gESmo-InFdsu1Gvf_VGQH0vnp9L5WDo_ls734hvNo3Ga</recordid><startdate>20160601</startdate><enddate>20160601</enddate><creator>Byrd, Richard H.</creator><creator>Nocedal, Jorge</creator><creator>Oztoprak, Figen</creator><general>Springer Berlin Heidelberg</general><general>Springer Nature B.V</general><scope>3V.</scope><scope>7SC</scope><scope>7WY</scope><scope>7WZ</scope><scope>7XB</scope><scope>87Z</scope><scope>88I</scope><scope>8AL</scope><scope>8AO</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>8FK</scope><scope>8FL</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BEZIV</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FRNLG</scope><scope>F~G</scope><scope>GNUQQ</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K60</scope><scope>K6~</scope><scope>K7-</scope><scope>L.-</scope><scope>L6V</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>M0C</scope><scope>M0N</scope><scope>M2P</scope><scope>M7S</scope><scope>P5Z</scope><scope>P62</scope><scope>PQBIZ</scope><scope>PQBZA</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PTHSS</scope><scope>Q9U</scope></search><sort><creationdate>20160601</creationdate><title>An inexact successive quadratic approximation method for L-1 regularized optimization</title><author>Byrd, Richard H. ; Nocedal, Jorge ; Oztoprak, Figen</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-p189t-df01da09810a8fb253497f3ab2d82313a5aca767b7873c635488ced3026dc97d3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2016</creationdate><topic>Algorithms</topic><topic>Approximation</topic><topic>Artificial intelligence</topic><topic>Calculus of Variations and Optimal Control; Optimization</topic><topic>Combinatorics</topic><topic>Convergence</topic><topic>Full Length Paper</topic><topic>Industrial engineering</topic><topic>Iterative methods</topic><topic>Mathematical analysis</topic><topic>Mathematical and Computational Physics</topic><topic>Mathematical Methods in Physics</topic><topic>Mathematical models</topic><topic>Mathematics</topic><topic>Mathematics and Statistics</topic><topic>Mathematics of Computing</topic><topic>Minimization</topic><topic>Newton methods</topic><topic>Numerical Analysis</topic><topic>Optimization</topic><topic>Studies</topic><topic>Test sets</topic><topic>Theoretical</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Byrd, Richard H.</creatorcontrib><creatorcontrib>Nocedal, Jorge</creatorcontrib><creatorcontrib>Oztoprak, Figen</creatorcontrib><collection>ProQuest Central (Corporate)</collection><collection>Computer and Information Systems Abstracts</collection><collection>Access via ABI/INFORM (ProQuest)</collection><collection>ABI/INFORM Global (PDF only)</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>ABI/INFORM Global (Alumni Edition)</collection><collection>Science Database (Alumni Edition)</collection><collection>Computing Database (Alumni Edition)</collection><collection>ProQuest Pharma Collection</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ABI/INFORM Collection (Alumni Edition)</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies & Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Business Premium Collection</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>Business Premium Collection (Alumni)</collection><collection>ABI/INFORM Global (Corporate)</collection><collection>ProQuest Central Student</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>ProQuest Business Collection (Alumni Edition)</collection><collection>ProQuest Business Collection</collection><collection>Computer Science Database</collection><collection>ABI/INFORM Professional Advanced</collection><collection>ProQuest Engineering Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>ABI/INFORM Global</collection><collection>Computing Database</collection><collection>Science Database</collection><collection>Engineering Database</collection><collection>Advanced Technologies & Aerospace Database</collection><collection>ProQuest Advanced Technologies & Aerospace Collection</collection><collection>ProQuest One Business</collection><collection>ProQuest One Business (Alumni)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>Engineering Collection</collection><collection>ProQuest Central Basic</collection><jtitle>Mathematical programming</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Byrd, Richard H.</au><au>Nocedal, Jorge</au><au>Oztoprak, Figen</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>An inexact successive quadratic approximation method for L-1 regularized optimization</atitle><jtitle>Mathematical programming</jtitle><stitle>Math. Program</stitle><date>2016-06-01</date><risdate>2016</risdate><volume>157</volume><issue>2</issue><spage>375</spage><epage>396</epage><pages>375-396</pages><issn>0025-5610</issn><eissn>1436-4646</eissn><coden>MHPGA4</coden><abstract>We study a Newton-like method for the minimization of an objective function
ϕ
that is the sum of a smooth function and an
ℓ
1
regularization term. This method, which is sometimes referred to in the literature as a proximal Newton method, computes a step by minimizing a piecewise quadratic model
q
k
of the objective function
ϕ
. In order to make this approach efficient in practice, it is imperative to perform this inner minimization inexactly. In this paper, we give inexactness conditions that guarantee global convergence and that can be used to control the local rate of convergence of the iteration. Our inexactness conditions are based on a semi-smooth function that represents a (continuous) measure of the optimality conditions of the problem, and that embodies the soft-thresholding iteration. We give careful consideration to the algorithm employed for the inner minimization, and report numerical results on two test sets originating in machine learning.</abstract><cop>Berlin/Heidelberg</cop><pub>Springer Berlin Heidelberg</pub><doi>10.1007/s10107-015-0941-y</doi><tpages>22</tpages></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0025-5610 |
| ispartof | Mathematical programming, 2016-06, Vol.157 (2), p.375-396 |
| issn | 0025-5610 1436-4646 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_1825468261 |
| source | SpringerNature Journals; EBSCOhost Business Source Complete |
| subjects | Algorithms Approximation Artificial intelligence Calculus of Variations and Optimal Control Optimization Combinatorics Convergence Full Length Paper Industrial engineering Iterative methods Mathematical analysis Mathematical and Computational Physics Mathematical Methods in Physics Mathematical models Mathematics Mathematics and Statistics Mathematics of Computing Minimization Newton methods Numerical Analysis Optimization Studies Test sets Theoretical |
| title | An inexact successive quadratic approximation method for L-1 regularized optimization |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T02%3A58%3A08IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_sprin&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=An%20inexact%20successive%20quadratic%20approximation%20method%20for%20L-1%20regularized%20optimization&rft.jtitle=Mathematical%20programming&rft.au=Byrd,%20Richard%20H.&rft.date=2016-06-01&rft.volume=157&rft.issue=2&rft.spage=375&rft.epage=396&rft.pages=375-396&rft.issn=0025-5610&rft.eissn=1436-4646&rft.coden=MHPGA4&rft_id=info:doi/10.1007/s10107-015-0941-y&rft_dat=%3Cproquest_sprin%3E1825468261%3C/proquest_sprin%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1791230465&rft_id=info:pmid/&rfr_iscdi=true |