Sizing and Least-Change Secant Methods

Oren and Luenberger introduced in 1974 a strategy for replacing Hessian approximations by their scalar multiples and then performing quasi-Newton updates, generally least-change secant updates such as the BFGS or DFP updates [Oren and Luenberger, Management Sci., 20 (1974), pp. 845-862]. In this pap...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:SIAM journal on numerical analysis 1993-10, Vol.30 (5), p.1291-1314
Hauptverfasser: Dennis, J. E., Wolkowicz, H.
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 1314
container_issue 5
container_start_page 1291
container_title SIAM journal on numerical analysis
container_volume 30
creator Dennis, J. E.
Wolkowicz, H.
description Oren and Luenberger introduced in 1974 a strategy for replacing Hessian approximations by their scalar multiples and then performing quasi-Newton updates, generally least-change secant updates such as the BFGS or DFP updates [Oren and Luenberger, Management Sci., 20 (1974), pp. 845-862]. In this paper, the function$\omega(A) = \big(\frac{\operatorname{trace}(A)/n}{\operatorname{det} (A)^{1/n}}\big)$is shown to be a measure of change with a direct connection to the Oren-Luenberger strategy. This measure is interesting because it is related to the ℓ2condition number, but it takes all the eigenvalues of A into account rather than just the extremes. If the class of possible updates is restricted to the Broyden class, i.e., scalar premultiples are not allowed, then the optimal update depends on the dimension of the problem. It may, or may not, be in the convex class, but it becomes the BFGS update as the dimension increases. This seems to be yet another explanation for why the optimally conditioned updates are not significantly better than the BFGS update. The theory results in several new interesting updates including a self-scaling, hereditarily positive definite, update in the Broyden class which is not necessarily in the convex class. This update, in conjunction with the Oren-Luenberger scaling strategy at the first iteration only, was consistently the best in numerical tests.
doi_str_mv 10.1137/0730067
format Article
fullrecord <record><control><sourceid>jstor_proqu</sourceid><recordid>TN_cdi_proquest_journals_923054284</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><jstor_id>2158238</jstor_id><sourcerecordid>2158238</sourcerecordid><originalsourceid>FETCH-LOGICAL-c265t-7e48df29e073e13f0bf23b5a43368e1faf685f231e1dc2475961708b8c37bc513</originalsourceid><addsrcrecordid>eNo9kE9LxDAQxYMoWFfxC3goInqqZjJNkxyl-A8qHlbPJU2T3Za1XZPuQT-9WXbZ0zAzP957PEIugd4DoHigAiktxBFJgCqeCRD0mCSUYpFBztQpOQuhp3GXgAm5nXd_3bBI9dCmldVhysqlHhY2nVujhyl9t9NybMM5OXF6FezFfs7I1_PTZ_maVR8vb-VjlRlW8CkTNpetY8rGEBbQ0cYxbLjOEQtpwWlXSB5PYKE1LBdcFTGebKRB0RgOOCPXO921H382Nkx1P278EC1rxZDynMk8Qnc7yPgxBG9dvfbdt_a_NdB620G97yCSN3s5HYxeOa8H04UDjqgUY1vXqx3Wh2n0hzcDLhlK_AeBoWB7</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>923054284</pqid></control><display><type>article</type><title>Sizing and Least-Change Secant Methods</title><source>SIAM Journals Online</source><source>JSTOR Mathematics &amp; Statistics</source><source>JSTOR Archive Collection A-Z Listing</source><creator>Dennis, J. E. ; Wolkowicz, H.</creator><creatorcontrib>Dennis, J. E. ; Wolkowicz, H.</creatorcontrib><description>Oren and Luenberger introduced in 1974 a strategy for replacing Hessian approximations by their scalar multiples and then performing quasi-Newton updates, generally least-change secant updates such as the BFGS or DFP updates [Oren and Luenberger, Management Sci., 20 (1974), pp. 845-862]. In this paper, the function$\omega(A) = \big(\frac{\operatorname{trace}(A)/n}{\operatorname{det} (A)^{1/n}}\big)$is shown to be a measure of change with a direct connection to the Oren-Luenberger strategy. This measure is interesting because it is related to the ℓ2condition number, but it takes all the eigenvalues of A into account rather than just the extremes. If the class of possible updates is restricted to the Broyden class, i.e., scalar premultiples are not allowed, then the optimal update depends on the dimension of the problem. It may, or may not, be in the convex class, but it becomes the BFGS update as the dimension increases. This seems to be yet another explanation for why the optimally conditioned updates are not significantly better than the BFGS update. The theory results in several new interesting updates including a self-scaling, hereditarily positive definite, update in the Broyden class which is not necessarily in the convex class. This update, in conjunction with the Oren-Luenberger scaling strategy at the first iteration only, was consistently the best in numerical tests.</description><identifier>ISSN: 0036-1429</identifier><identifier>EISSN: 1095-7170</identifier><identifier>DOI: 10.1137/0730067</identifier><identifier>CODEN: SJNAEQ</identifier><language>eng</language><publisher>Philadelphia, PA: Society for Industrial and Applied Mathematics</publisher><subject>Algorithms ; Applied mathematics ; Applied sciences ; Approximation ; Combinatorics ; Curvature ; Eigenvalues ; Exact sciences and technology ; Hessian matrices ; Lagrange multipliers ; Matrices ; Operational research and scientific management ; Operational research. Management science ; Optimization ; Optimization. Search problems ; Secant function ; Spectral theory</subject><ispartof>SIAM journal on numerical analysis, 1993-10, Vol.30 (5), p.1291-1314</ispartof><rights>Copyright 1993 Society for Industrial and Applied Mathematics</rights><rights>1995 INIST-CNRS</rights><rights>[Copyright] © 1993 Society for Industrial and Applied Mathematics</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c265t-7e48df29e073e13f0bf23b5a43368e1faf685f231e1dc2475961708b8c37bc513</citedby><cites>FETCH-LOGICAL-c265t-7e48df29e073e13f0bf23b5a43368e1faf685f231e1dc2475961708b8c37bc513</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.jstor.org/stable/pdf/2158238$$EPDF$$P50$$Gjstor$$H</linktopdf><linktohtml>$$Uhttps://www.jstor.org/stable/2158238$$EHTML$$P50$$Gjstor$$H</linktohtml><link.rule.ids>314,780,784,803,832,3184,27924,27925,58017,58021,58250,58254</link.rule.ids><backlink>$$Uhttp://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&amp;idt=3399221$$DView record in Pascal Francis$$Hfree_for_read</backlink></links><search><creatorcontrib>Dennis, J. E.</creatorcontrib><creatorcontrib>Wolkowicz, H.</creatorcontrib><title>Sizing and Least-Change Secant Methods</title><title>SIAM journal on numerical analysis</title><description>Oren and Luenberger introduced in 1974 a strategy for replacing Hessian approximations by their scalar multiples and then performing quasi-Newton updates, generally least-change secant updates such as the BFGS or DFP updates [Oren and Luenberger, Management Sci., 20 (1974), pp. 845-862]. In this paper, the function$\omega(A) = \big(\frac{\operatorname{trace}(A)/n}{\operatorname{det} (A)^{1/n}}\big)$is shown to be a measure of change with a direct connection to the Oren-Luenberger strategy. This measure is interesting because it is related to the ℓ2condition number, but it takes all the eigenvalues of A into account rather than just the extremes. If the class of possible updates is restricted to the Broyden class, i.e., scalar premultiples are not allowed, then the optimal update depends on the dimension of the problem. It may, or may not, be in the convex class, but it becomes the BFGS update as the dimension increases. This seems to be yet another explanation for why the optimally conditioned updates are not significantly better than the BFGS update. The theory results in several new interesting updates including a self-scaling, hereditarily positive definite, update in the Broyden class which is not necessarily in the convex class. This update, in conjunction with the Oren-Luenberger scaling strategy at the first iteration only, was consistently the best in numerical tests.</description><subject>Algorithms</subject><subject>Applied mathematics</subject><subject>Applied sciences</subject><subject>Approximation</subject><subject>Combinatorics</subject><subject>Curvature</subject><subject>Eigenvalues</subject><subject>Exact sciences and technology</subject><subject>Hessian matrices</subject><subject>Lagrange multipliers</subject><subject>Matrices</subject><subject>Operational research and scientific management</subject><subject>Operational research. Management science</subject><subject>Optimization</subject><subject>Optimization. Search problems</subject><subject>Secant function</subject><subject>Spectral theory</subject><issn>0036-1429</issn><issn>1095-7170</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>1993</creationdate><recordtype>article</recordtype><sourceid>8G5</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><recordid>eNo9kE9LxDAQxYMoWFfxC3goInqqZjJNkxyl-A8qHlbPJU2T3Za1XZPuQT-9WXbZ0zAzP957PEIugd4DoHigAiktxBFJgCqeCRD0mCSUYpFBztQpOQuhp3GXgAm5nXd_3bBI9dCmldVhysqlHhY2nVujhyl9t9NybMM5OXF6FezFfs7I1_PTZ_maVR8vb-VjlRlW8CkTNpetY8rGEBbQ0cYxbLjOEQtpwWlXSB5PYKE1LBdcFTGebKRB0RgOOCPXO921H382Nkx1P278EC1rxZDynMk8Qnc7yPgxBG9dvfbdt_a_NdB620G97yCSN3s5HYxeOa8H04UDjqgUY1vXqx3Wh2n0hzcDLhlK_AeBoWB7</recordid><startdate>19931001</startdate><enddate>19931001</enddate><creator>Dennis, J. E.</creator><creator>Wolkowicz, H.</creator><general>Society for Industrial and Applied Mathematics</general><scope>IQODW</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7WY</scope><scope>7WZ</scope><scope>7X2</scope><scope>7XB</scope><scope>87Z</scope><scope>88A</scope><scope>88F</scope><scope>88I</scope><scope>88K</scope><scope>8AL</scope><scope>8FE</scope><scope>8FG</scope><scope>8FH</scope><scope>8FK</scope><scope>8FL</scope><scope>8G5</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>ATCPS</scope><scope>AZQEC</scope><scope>BBNVY</scope><scope>BENPR</scope><scope>BEZIV</scope><scope>BGLVJ</scope><scope>BHPHI</scope><scope>CCPQU</scope><scope>D1I</scope><scope>DWQXO</scope><scope>FRNLG</scope><scope>F~G</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K60</scope><scope>K6~</scope><scope>K7-</scope><scope>KB.</scope><scope>L.-</scope><scope>L6V</scope><scope>LK8</scope><scope>M0C</scope><scope>M0K</scope><scope>M0N</scope><scope>M1Q</scope><scope>M2O</scope><scope>M2P</scope><scope>M2T</scope><scope>M7P</scope><scope>M7S</scope><scope>MBDVC</scope><scope>P5Z</scope><scope>P62</scope><scope>PATMY</scope><scope>PDBOC</scope><scope>PQBIZ</scope><scope>PQBZA</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PTHSS</scope><scope>PYCSY</scope><scope>Q9U</scope></search><sort><creationdate>19931001</creationdate><title>Sizing and Least-Change Secant Methods</title><author>Dennis, J. E. ; Wolkowicz, H.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c265t-7e48df29e073e13f0bf23b5a43368e1faf685f231e1dc2475961708b8c37bc513</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>1993</creationdate><topic>Algorithms</topic><topic>Applied mathematics</topic><topic>Applied sciences</topic><topic>Approximation</topic><topic>Combinatorics</topic><topic>Curvature</topic><topic>Eigenvalues</topic><topic>Exact sciences and technology</topic><topic>Hessian matrices</topic><topic>Lagrange multipliers</topic><topic>Matrices</topic><topic>Operational research and scientific management</topic><topic>Operational research. Management science</topic><topic>Optimization</topic><topic>Optimization. Search problems</topic><topic>Secant function</topic><topic>Spectral theory</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Dennis, J. E.</creatorcontrib><creatorcontrib>Wolkowicz, H.</creatorcontrib><collection>Pascal-Francis</collection><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Access via ABI/INFORM (ProQuest)</collection><collection>ABI/INFORM Global (PDF only)</collection><collection>Agricultural Science Collection</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>ABI/INFORM Global (Alumni Edition)</collection><collection>Biology Database (Alumni Edition)</collection><collection>Military Database (Alumni Edition)</collection><collection>Science Database (Alumni Edition)</collection><collection>Telecommunications (Alumni Edition)</collection><collection>Computing Database (Alumni Edition)</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Natural Science Collection</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ABI/INFORM Collection (Alumni Edition)</collection><collection>Research Library (Alumni Edition)</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies &amp; Aerospace Collection</collection><collection>Agricultural &amp; Environmental Science Collection</collection><collection>ProQuest Central Essentials</collection><collection>Biological Science Collection</collection><collection>ProQuest Central</collection><collection>Business Premium Collection</collection><collection>Technology Collection</collection><collection>Natural Science Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Materials Science Collection</collection><collection>ProQuest Central Korea</collection><collection>Business Premium Collection (Alumni)</collection><collection>ABI/INFORM Global (Corporate)</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>ProQuest Business Collection (Alumni Edition)</collection><collection>ProQuest Business Collection</collection><collection>Computer Science Database</collection><collection>Materials Science Database</collection><collection>ABI/INFORM Professional Advanced</collection><collection>ProQuest Engineering Collection</collection><collection>ProQuest Biological Science Collection</collection><collection>ABI/INFORM Global</collection><collection>Agricultural Science Database</collection><collection>Computing Database</collection><collection>Military Database</collection><collection>Research Library</collection><collection>Science Database</collection><collection>Telecommunications Database</collection><collection>Biological Science Database</collection><collection>Engineering Database</collection><collection>Research Library (Corporate)</collection><collection>Advanced Technologies &amp; Aerospace Database</collection><collection>ProQuest Advanced Technologies &amp; Aerospace Collection</collection><collection>Environmental Science Database</collection><collection>Materials Science Collection</collection><collection>ProQuest One Business</collection><collection>ProQuest One Business (Alumni)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>Engineering Collection</collection><collection>Environmental Science Collection</collection><collection>ProQuest Central Basic</collection><jtitle>SIAM journal on numerical analysis</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Dennis, J. E.</au><au>Wolkowicz, H.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Sizing and Least-Change Secant Methods</atitle><jtitle>SIAM journal on numerical analysis</jtitle><date>1993-10-01</date><risdate>1993</risdate><volume>30</volume><issue>5</issue><spage>1291</spage><epage>1314</epage><pages>1291-1314</pages><issn>0036-1429</issn><eissn>1095-7170</eissn><coden>SJNAEQ</coden><abstract>Oren and Luenberger introduced in 1974 a strategy for replacing Hessian approximations by their scalar multiples and then performing quasi-Newton updates, generally least-change secant updates such as the BFGS or DFP updates [Oren and Luenberger, Management Sci., 20 (1974), pp. 845-862]. In this paper, the function$\omega(A) = \big(\frac{\operatorname{trace}(A)/n}{\operatorname{det} (A)^{1/n}}\big)$is shown to be a measure of change with a direct connection to the Oren-Luenberger strategy. This measure is interesting because it is related to the ℓ2condition number, but it takes all the eigenvalues of A into account rather than just the extremes. If the class of possible updates is restricted to the Broyden class, i.e., scalar premultiples are not allowed, then the optimal update depends on the dimension of the problem. It may, or may not, be in the convex class, but it becomes the BFGS update as the dimension increases. This seems to be yet another explanation for why the optimally conditioned updates are not significantly better than the BFGS update. The theory results in several new interesting updates including a self-scaling, hereditarily positive definite, update in the Broyden class which is not necessarily in the convex class. This update, in conjunction with the Oren-Luenberger scaling strategy at the first iteration only, was consistently the best in numerical tests.</abstract><cop>Philadelphia, PA</cop><pub>Society for Industrial and Applied Mathematics</pub><doi>10.1137/0730067</doi><tpages>24</tpages><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 0036-1429
ispartof SIAM journal on numerical analysis, 1993-10, Vol.30 (5), p.1291-1314
issn 0036-1429
1095-7170
language eng
recordid cdi_proquest_journals_923054284
source SIAM Journals Online; JSTOR Mathematics & Statistics; JSTOR Archive Collection A-Z Listing
subjects Algorithms
Applied mathematics
Applied sciences
Approximation
Combinatorics
Curvature
Eigenvalues
Exact sciences and technology
Hessian matrices
Lagrange multipliers
Matrices
Operational research and scientific management
Operational research. Management science
Optimization
Optimization. Search problems
Secant function
Spectral theory
title Sizing and Least-Change Secant Methods
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-23T15%3A52%3A01IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-jstor_proqu&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Sizing%20and%20Least-Change%20Secant%20Methods&rft.jtitle=SIAM%20journal%20on%20numerical%20analysis&rft.au=Dennis,%20J.%20E.&rft.date=1993-10-01&rft.volume=30&rft.issue=5&rft.spage=1291&rft.epage=1314&rft.pages=1291-1314&rft.issn=0036-1429&rft.eissn=1095-7170&rft.coden=SJNAEQ&rft_id=info:doi/10.1137/0730067&rft_dat=%3Cjstor_proqu%3E2158238%3C/jstor_proqu%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=923054284&rft_id=info:pmid/&rft_jstor_id=2158238&rfr_iscdi=true