The Inexact Cyclic Block Proximal Gradient Method and Properties of Inexact Proximal Maps
This paper expands the cyclic block proximal gradient method for block separable composite minimization by allowing for inexactly computed gradients and pre-conditioned proximal maps. The resultant algorithm, the inexact cyclic block proximal gradient (I-CBPG) method, shares the same convergence rat...
Gespeichert in:
Veröffentlicht in: | Journal of optimization theory and applications 2024-05, Vol.201 (2), p.668-698 |
---|---|
Hauptverfasser: | , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 698 |
---|---|
container_issue | 2 |
container_start_page | 668 |
container_title | Journal of optimization theory and applications |
container_volume | 201 |
creator | Farias Maia, Leandro Gutman, David Huckleberry Hughes, Ryan Christopher |
description | This paper expands the cyclic block proximal gradient method for block separable composite minimization by allowing for inexactly computed gradients and pre-conditioned proximal maps. The resultant algorithm, the inexact cyclic block proximal gradient (I-CBPG) method, shares the same convergence rate as its exactly computed analogue provided the allowable errors decrease sufficiently quickly or are pre-selected to be sufficiently small. We provide numerical experiments that showcase the practical computational advantage of I-CBPG for certain fixed tolerances of approximation error and for a dynamically decreasing error tolerance regime in particular. Our experimental results indicate that cyclic methods with dynamically decreasing error tolerance regimes can actually outpace their randomized siblings with fixed error tolerance regimes. We establish a tight relationship between inexact pre-conditioned proximal map evaluations and
δ
-subgradients in our
(
δ
,
B
)
-Second Prox theorem. This theorem forms the foundation of our convergence analysis and enables us to show that inexact gradient computations can be subsumed within a single unifying framework. |
doi_str_mv | 10.1007/s10957-024-02404-7 |
format | Article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_3052919675</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3052919675</sourcerecordid><originalsourceid>FETCH-LOGICAL-c270t-a9d4ffccc9a4638cd7cc938b1623f17d0b237d8cb801a5e55b275a1e5de692813</originalsourceid><addsrcrecordid>eNp9kDFPwzAQhS0EEqXwB5gsMQfOdhzbI1TQVmoFQxmYLMd2aEpIgp1K7b8nJahsDKc76d737vQQuiZwSwDEXSSguEiApoeCNBEnaES4YAmVQp6iEQClCaNMnaOLGDcAoKRIR-httfZ4XvudsR2e7G1VWvxQNfYDv4RmV36aCk-DcaWvO7z03bpx2NTusGx96EofcVMc-SOyNG28RGeFqaK_-u1j9Pr0uJrMksXzdD65XySWCugSo1xaFNZaZdKMSetEPzKZk4yygggHOWXCSZtLIIZ7znMquCGeO58pKgkbo5vBtw3N19bHTm-abaj7k5oBp4qoTPBeRQeVDU2MwRe6Df2rYa8J6EOEeohQ9_Hpnwi16CE2QLEX1-8-_Fn_Q30DMY5zzg</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3052919675</pqid></control><display><type>article</type><title>The Inexact Cyclic Block Proximal Gradient Method and Properties of Inexact Proximal Maps</title><source>SpringerLink Journals - AutoHoldings</source><creator>Farias Maia, Leandro ; Gutman, David Huckleberry ; Hughes, Ryan Christopher</creator><creatorcontrib>Farias Maia, Leandro ; Gutman, David Huckleberry ; Hughes, Ryan Christopher</creatorcontrib><description>This paper expands the cyclic block proximal gradient method for block separable composite minimization by allowing for inexactly computed gradients and pre-conditioned proximal maps. The resultant algorithm, the inexact cyclic block proximal gradient (I-CBPG) method, shares the same convergence rate as its exactly computed analogue provided the allowable errors decrease sufficiently quickly or are pre-selected to be sufficiently small. We provide numerical experiments that showcase the practical computational advantage of I-CBPG for certain fixed tolerances of approximation error and for a dynamically decreasing error tolerance regime in particular. Our experimental results indicate that cyclic methods with dynamically decreasing error tolerance regimes can actually outpace their randomized siblings with fixed error tolerance regimes. We establish a tight relationship between inexact pre-conditioned proximal map evaluations and
δ
-subgradients in our
(
δ
,
B
)
-Second Prox theorem. This theorem forms the foundation of our convergence analysis and enables us to show that inexact gradient computations can be subsumed within a single unifying framework.</description><identifier>ISSN: 0022-3239</identifier><identifier>EISSN: 1573-2878</identifier><identifier>DOI: 10.1007/s10957-024-02404-7</identifier><language>eng</language><publisher>New York: Springer US</publisher><subject>Algorithms ; Applications of Mathematics ; Approximation ; Calculus of Variations and Optimal Control; Optimization ; Computation ; Convergence ; Descent ; Engineering ; Experiments ; Gradients ; Mathematics ; Mathematics and Statistics ; Methods ; Operations Research/Decision Theory ; Optimization ; Theorems ; Theory of Computation</subject><ispartof>Journal of optimization theory and applications, 2024-05, Vol.201 (2), p.668-698</ispartof><rights>The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c270t-a9d4ffccc9a4638cd7cc938b1623f17d0b237d8cb801a5e55b275a1e5de692813</cites><orcidid>0000-0003-3130-0076</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s10957-024-02404-7$$EPDF$$P50$$Gspringer$$H</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s10957-024-02404-7$$EHTML$$P50$$Gspringer$$H</linktohtml><link.rule.ids>314,777,781,27905,27906,41469,42538,51300</link.rule.ids></links><search><creatorcontrib>Farias Maia, Leandro</creatorcontrib><creatorcontrib>Gutman, David Huckleberry</creatorcontrib><creatorcontrib>Hughes, Ryan Christopher</creatorcontrib><title>The Inexact Cyclic Block Proximal Gradient Method and Properties of Inexact Proximal Maps</title><title>Journal of optimization theory and applications</title><addtitle>J Optim Theory Appl</addtitle><description>This paper expands the cyclic block proximal gradient method for block separable composite minimization by allowing for inexactly computed gradients and pre-conditioned proximal maps. The resultant algorithm, the inexact cyclic block proximal gradient (I-CBPG) method, shares the same convergence rate as its exactly computed analogue provided the allowable errors decrease sufficiently quickly or are pre-selected to be sufficiently small. We provide numerical experiments that showcase the practical computational advantage of I-CBPG for certain fixed tolerances of approximation error and for a dynamically decreasing error tolerance regime in particular. Our experimental results indicate that cyclic methods with dynamically decreasing error tolerance regimes can actually outpace their randomized siblings with fixed error tolerance regimes. We establish a tight relationship between inexact pre-conditioned proximal map evaluations and
δ
-subgradients in our
(
δ
,
B
)
-Second Prox theorem. This theorem forms the foundation of our convergence analysis and enables us to show that inexact gradient computations can be subsumed within a single unifying framework.</description><subject>Algorithms</subject><subject>Applications of Mathematics</subject><subject>Approximation</subject><subject>Calculus of Variations and Optimal Control; Optimization</subject><subject>Computation</subject><subject>Convergence</subject><subject>Descent</subject><subject>Engineering</subject><subject>Experiments</subject><subject>Gradients</subject><subject>Mathematics</subject><subject>Mathematics and Statistics</subject><subject>Methods</subject><subject>Operations Research/Decision Theory</subject><subject>Optimization</subject><subject>Theorems</subject><subject>Theory of Computation</subject><issn>0022-3239</issn><issn>1573-2878</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><recordid>eNp9kDFPwzAQhS0EEqXwB5gsMQfOdhzbI1TQVmoFQxmYLMd2aEpIgp1K7b8nJahsDKc76d737vQQuiZwSwDEXSSguEiApoeCNBEnaES4YAmVQp6iEQClCaNMnaOLGDcAoKRIR-httfZ4XvudsR2e7G1VWvxQNfYDv4RmV36aCk-DcaWvO7z03bpx2NTusGx96EofcVMc-SOyNG28RGeFqaK_-u1j9Pr0uJrMksXzdD65XySWCugSo1xaFNZaZdKMSetEPzKZk4yygggHOWXCSZtLIIZ7znMquCGeO58pKgkbo5vBtw3N19bHTm-abaj7k5oBp4qoTPBeRQeVDU2MwRe6Df2rYa8J6EOEeohQ9_Hpnwi16CE2QLEX1-8-_Fn_Q30DMY5zzg</recordid><startdate>20240501</startdate><enddate>20240501</enddate><creator>Farias Maia, Leandro</creator><creator>Gutman, David Huckleberry</creator><creator>Hughes, Ryan Christopher</creator><general>Springer US</general><general>Springer Nature B.V</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7TB</scope><scope>8FD</scope><scope>FR3</scope><scope>JQ2</scope><scope>KR7</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0003-3130-0076</orcidid></search><sort><creationdate>20240501</creationdate><title>The Inexact Cyclic Block Proximal Gradient Method and Properties of Inexact Proximal Maps</title><author>Farias Maia, Leandro ; Gutman, David Huckleberry ; Hughes, Ryan Christopher</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c270t-a9d4ffccc9a4638cd7cc938b1623f17d0b237d8cb801a5e55b275a1e5de692813</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Algorithms</topic><topic>Applications of Mathematics</topic><topic>Approximation</topic><topic>Calculus of Variations and Optimal Control; Optimization</topic><topic>Computation</topic><topic>Convergence</topic><topic>Descent</topic><topic>Engineering</topic><topic>Experiments</topic><topic>Gradients</topic><topic>Mathematics</topic><topic>Mathematics and Statistics</topic><topic>Methods</topic><topic>Operations Research/Decision Theory</topic><topic>Optimization</topic><topic>Theorems</topic><topic>Theory of Computation</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Farias Maia, Leandro</creatorcontrib><creatorcontrib>Gutman, David Huckleberry</creatorcontrib><creatorcontrib>Hughes, Ryan Christopher</creatorcontrib><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Mechanical & Transportation Engineering Abstracts</collection><collection>Technology Research Database</collection><collection>Engineering Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Civil Engineering Abstracts</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>Journal of optimization theory and applications</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Farias Maia, Leandro</au><au>Gutman, David Huckleberry</au><au>Hughes, Ryan Christopher</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>The Inexact Cyclic Block Proximal Gradient Method and Properties of Inexact Proximal Maps</atitle><jtitle>Journal of optimization theory and applications</jtitle><stitle>J Optim Theory Appl</stitle><date>2024-05-01</date><risdate>2024</risdate><volume>201</volume><issue>2</issue><spage>668</spage><epage>698</epage><pages>668-698</pages><issn>0022-3239</issn><eissn>1573-2878</eissn><abstract>This paper expands the cyclic block proximal gradient method for block separable composite minimization by allowing for inexactly computed gradients and pre-conditioned proximal maps. The resultant algorithm, the inexact cyclic block proximal gradient (I-CBPG) method, shares the same convergence rate as its exactly computed analogue provided the allowable errors decrease sufficiently quickly or are pre-selected to be sufficiently small. We provide numerical experiments that showcase the practical computational advantage of I-CBPG for certain fixed tolerances of approximation error and for a dynamically decreasing error tolerance regime in particular. Our experimental results indicate that cyclic methods with dynamically decreasing error tolerance regimes can actually outpace their randomized siblings with fixed error tolerance regimes. We establish a tight relationship between inexact pre-conditioned proximal map evaluations and
δ
-subgradients in our
(
δ
,
B
)
-Second Prox theorem. This theorem forms the foundation of our convergence analysis and enables us to show that inexact gradient computations can be subsumed within a single unifying framework.</abstract><cop>New York</cop><pub>Springer US</pub><doi>10.1007/s10957-024-02404-7</doi><tpages>31</tpages><orcidid>https://orcid.org/0000-0003-3130-0076</orcidid></addata></record> |
fulltext | fulltext |
identifier | ISSN: 0022-3239 |
ispartof | Journal of optimization theory and applications, 2024-05, Vol.201 (2), p.668-698 |
issn | 0022-3239 1573-2878 |
language | eng |
recordid | cdi_proquest_journals_3052919675 |
source | SpringerLink Journals - AutoHoldings |
subjects | Algorithms Applications of Mathematics Approximation Calculus of Variations and Optimal Control Optimization Computation Convergence Descent Engineering Experiments Gradients Mathematics Mathematics and Statistics Methods Operations Research/Decision Theory Optimization Theorems Theory of Computation |
title | The Inexact Cyclic Block Proximal Gradient Method and Properties of Inexact Proximal Maps |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-18T23%3A25%3A27IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=The%20Inexact%20Cyclic%20Block%20Proximal%20Gradient%20Method%20and%20Properties%20of%20Inexact%20Proximal%20Maps&rft.jtitle=Journal%20of%20optimization%20theory%20and%20applications&rft.au=Farias%C2%A0Maia,%20Leandro&rft.date=2024-05-01&rft.volume=201&rft.issue=2&rft.spage=668&rft.epage=698&rft.pages=668-698&rft.issn=0022-3239&rft.eissn=1573-2878&rft_id=info:doi/10.1007/s10957-024-02404-7&rft_dat=%3Cproquest_cross%3E3052919675%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3052919675&rft_id=info:pmid/&rfr_iscdi=true |