Adaptive step-length selection in gradient boosting for Gaussian location and scale models

Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive models for location, scale and shape (GAMLSS), imbalanced updates of predictors, where so...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Computational statistics 2022-11, Vol.37 (5), p.2295-2332
Hauptverfasser: Zhang, Boyao, Hepp, Tobias, Greven, Sonja, Bergherr, Elisabeth
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 2332
container_issue 5
container_start_page 2295
container_title Computational statistics
container_volume 37
creator Zhang, Boyao
Hepp, Tobias
Greven, Sonja
Bergherr, Elisabeth
description Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive models for location, scale and shape (GAMLSS), imbalanced updates of predictors, where some distribution parameters are updated more frequently than others, can be a problem that prevents some submodels to be appropriately fitted within a limited number of boosting iterations. We propose an approach using adaptive step-length (ASL) determination within a non-cyclical boosting algorithm for Gaussian location and scale models, as an important special case of the wider class of GAMLSS, to prevent such imbalance. Moreover, we discuss properties of the ASL and derive a semi-analytical form of the ASL that avoids manual selection of the search interval and numerical optimization to find the optimal step-length, and consequently improves computational efficiency. We show competitive behavior of the proposed approaches compared to penalized maximum likelihood and boosting with a fixed step-length for Gaussian location and scale models in two simulations and two applications, in particular for cases of large variance and/or more variables than observations. In addition, the underlying concept of the ASL is also applicable to the whole GAMLSS framework and to other models with more than one predictor like zero-inflated count models, and brings up insights into the choice of the reasonable defaults for the step-length in the simpler special case of (Gaussian) additive models.
doi_str_mv 10.1007/s00180-022-01199-3
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2719231098</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2719231098</sourcerecordid><originalsourceid>FETCH-LOGICAL-c363t-953578a75784b7c25e53883f0a7fcc55a67ce46cc2bed52452f0029f6aa0ec913</originalsourceid><addsrcrecordid>eNp9kE1LAzEQhoMoWKt_wFPAczQfm83mWIpWoeBFL15Cmp1dt2yTmqSC_97YFbx5mbk87zvDg9A1o7eMUnWXKGUNJZRzQhnTmogTNGM1E0TXsjlFM6orQSpa83N0kdKWFlJxNkNvi9bu8_AJOGXYkxF8n99xghFcHoLHg8d9tO0APuNNCCkPvsddiHhlDykN1uMxOHtErW9xcnYEvAstjOkSnXV2THD1u-fo9eH-ZflI1s-rp-ViTZyoRSZaCqkaq8qoNspxCVI0jeioVZ1zUtpaOahq5_gGWskrybvyve5qayk4zcQc3Uy9-xg-DpCy2YZD9OWk4YppLhjVTaH4RLkYUorQmX0cdjZ-GUbNj0MzOTTFjDk6NKKExBRKBfY9xL_qf1LfdCZ0nA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2719231098</pqid></control><display><type>article</type><title>Adaptive step-length selection in gradient boosting for Gaussian location and scale models</title><source>SpringerLink Journals - AutoHoldings</source><creator>Zhang, Boyao ; Hepp, Tobias ; Greven, Sonja ; Bergherr, Elisabeth</creator><creatorcontrib>Zhang, Boyao ; Hepp, Tobias ; Greven, Sonja ; Bergherr, Elisabeth</creatorcontrib><description>Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive models for location, scale and shape (GAMLSS), imbalanced updates of predictors, where some distribution parameters are updated more frequently than others, can be a problem that prevents some submodels to be appropriately fitted within a limited number of boosting iterations. We propose an approach using adaptive step-length (ASL) determination within a non-cyclical boosting algorithm for Gaussian location and scale models, as an important special case of the wider class of GAMLSS, to prevent such imbalance. Moreover, we discuss properties of the ASL and derive a semi-analytical form of the ASL that avoids manual selection of the search interval and numerical optimization to find the optimal step-length, and consequently improves computational efficiency. We show competitive behavior of the proposed approaches compared to penalized maximum likelihood and boosting with a fixed step-length for Gaussian location and scale models in two simulations and two applications, in particular for cases of large variance and/or more variables than observations. In addition, the underlying concept of the ASL is also applicable to the whole GAMLSS framework and to other models with more than one predictor like zero-inflated count models, and brings up insights into the choice of the reasonable defaults for the step-length in the simpler special case of (Gaussian) additive models.</description><identifier>ISSN: 0943-4062</identifier><identifier>EISSN: 1613-9658</identifier><identifier>DOI: 10.1007/s00180-022-01199-3</identifier><language>eng</language><publisher>Berlin/Heidelberg: Springer Berlin Heidelberg</publisher><subject>Algorithms ; Economic Theory/Quantitative Economics/Mathematical Methods ; Mathematics and Statistics ; Optimization ; Original Paper ; Probability and Statistics in Computer Science ; Probability Theory and Stochastic Processes ; Scale models ; Statistics</subject><ispartof>Computational statistics, 2022-11, Vol.37 (5), p.2295-2332</ispartof><rights>The Author(s) 2022</rights><rights>The Author(s) 2022. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c363t-953578a75784b7c25e53883f0a7fcc55a67ce46cc2bed52452f0029f6aa0ec913</citedby><cites>FETCH-LOGICAL-c363t-953578a75784b7c25e53883f0a7fcc55a67ce46cc2bed52452f0029f6aa0ec913</cites><orcidid>0000-0001-9161-921X</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s00180-022-01199-3$$EPDF$$P50$$Gspringer$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s00180-022-01199-3$$EHTML$$P50$$Gspringer$$Hfree_for_read</linktohtml><link.rule.ids>314,780,784,27924,27925,41488,42557,51319</link.rule.ids></links><search><creatorcontrib>Zhang, Boyao</creatorcontrib><creatorcontrib>Hepp, Tobias</creatorcontrib><creatorcontrib>Greven, Sonja</creatorcontrib><creatorcontrib>Bergherr, Elisabeth</creatorcontrib><title>Adaptive step-length selection in gradient boosting for Gaussian location and scale models</title><title>Computational statistics</title><addtitle>Comput Stat</addtitle><description>Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive models for location, scale and shape (GAMLSS), imbalanced updates of predictors, where some distribution parameters are updated more frequently than others, can be a problem that prevents some submodels to be appropriately fitted within a limited number of boosting iterations. We propose an approach using adaptive step-length (ASL) determination within a non-cyclical boosting algorithm for Gaussian location and scale models, as an important special case of the wider class of GAMLSS, to prevent such imbalance. Moreover, we discuss properties of the ASL and derive a semi-analytical form of the ASL that avoids manual selection of the search interval and numerical optimization to find the optimal step-length, and consequently improves computational efficiency. We show competitive behavior of the proposed approaches compared to penalized maximum likelihood and boosting with a fixed step-length for Gaussian location and scale models in two simulations and two applications, in particular for cases of large variance and/or more variables than observations. In addition, the underlying concept of the ASL is also applicable to the whole GAMLSS framework and to other models with more than one predictor like zero-inflated count models, and brings up insights into the choice of the reasonable defaults for the step-length in the simpler special case of (Gaussian) additive models.</description><subject>Algorithms</subject><subject>Economic Theory/Quantitative Economics/Mathematical Methods</subject><subject>Mathematics and Statistics</subject><subject>Optimization</subject><subject>Original Paper</subject><subject>Probability and Statistics in Computer Science</subject><subject>Probability Theory and Stochastic Processes</subject><subject>Scale models</subject><subject>Statistics</subject><issn>0943-4062</issn><issn>1613-9658</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>C6C</sourceid><sourceid>8G5</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><recordid>eNp9kE1LAzEQhoMoWKt_wFPAczQfm83mWIpWoeBFL15Cmp1dt2yTmqSC_97YFbx5mbk87zvDg9A1o7eMUnWXKGUNJZRzQhnTmogTNGM1E0TXsjlFM6orQSpa83N0kdKWFlJxNkNvi9bu8_AJOGXYkxF8n99xghFcHoLHg8d9tO0APuNNCCkPvsddiHhlDykN1uMxOHtErW9xcnYEvAstjOkSnXV2THD1u-fo9eH-ZflI1s-rp-ViTZyoRSZaCqkaq8qoNspxCVI0jeioVZ1zUtpaOahq5_gGWskrybvyve5qayk4zcQc3Uy9-xg-DpCy2YZD9OWk4YppLhjVTaH4RLkYUorQmX0cdjZ-GUbNj0MzOTTFjDk6NKKExBRKBfY9xL_qf1LfdCZ0nA</recordid><startdate>20221101</startdate><enddate>20221101</enddate><creator>Zhang, Boyao</creator><creator>Hepp, Tobias</creator><creator>Greven, Sonja</creator><creator>Bergherr, Elisabeth</creator><general>Springer Berlin Heidelberg</general><general>Springer Nature B.V</general><scope>C6C</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7SC</scope><scope>7TB</scope><scope>7WY</scope><scope>7WZ</scope><scope>7XB</scope><scope>87Z</scope><scope>88I</scope><scope>8AL</scope><scope>8C1</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>8FK</scope><scope>8FL</scope><scope>8G5</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BEZIV</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FR3</scope><scope>FRNLG</scope><scope>FYUFA</scope><scope>F~G</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K60</scope><scope>K6~</scope><scope>K7-</scope><scope>KR7</scope><scope>L.-</scope><scope>L6V</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>M0C</scope><scope>M0N</scope><scope>M2O</scope><scope>M2P</scope><scope>M7S</scope><scope>MBDVC</scope><scope>P5Z</scope><scope>P62</scope><scope>PQBIZ</scope><scope>PQBZA</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PTHSS</scope><scope>Q9U</scope><orcidid>https://orcid.org/0000-0001-9161-921X</orcidid></search><sort><creationdate>20221101</creationdate><title>Adaptive step-length selection in gradient boosting for Gaussian location and scale models</title><author>Zhang, Boyao ; Hepp, Tobias ; Greven, Sonja ; Bergherr, Elisabeth</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c363t-953578a75784b7c25e53883f0a7fcc55a67ce46cc2bed52452f0029f6aa0ec913</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Algorithms</topic><topic>Economic Theory/Quantitative Economics/Mathematical Methods</topic><topic>Mathematics and Statistics</topic><topic>Optimization</topic><topic>Original Paper</topic><topic>Probability and Statistics in Computer Science</topic><topic>Probability Theory and Stochastic Processes</topic><topic>Scale models</topic><topic>Statistics</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Zhang, Boyao</creatorcontrib><creatorcontrib>Hepp, Tobias</creatorcontrib><creatorcontrib>Greven, Sonja</creatorcontrib><creatorcontrib>Bergherr, Elisabeth</creatorcontrib><collection>Springer Nature OA Free Journals</collection><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Computer and Information Systems Abstracts</collection><collection>Mechanical &amp; Transportation Engineering Abstracts</collection><collection>ABI/INFORM Collection</collection><collection>ABI/INFORM Global (PDF only)</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>ABI/INFORM Global (Alumni Edition)</collection><collection>Science Database (Alumni Edition)</collection><collection>Computing Database (Alumni Edition)</collection><collection>Public Health Database</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ABI/INFORM Collection (Alumni Edition)</collection><collection>Research Library (Alumni Edition)</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies &amp; Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Business Premium Collection</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>Engineering Research Database</collection><collection>Business Premium Collection (Alumni)</collection><collection>Health Research Premium Collection</collection><collection>ABI/INFORM Global (Corporate)</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>ProQuest Business Collection (Alumni Edition)</collection><collection>ProQuest Business Collection</collection><collection>Computer Science Database</collection><collection>Civil Engineering Abstracts</collection><collection>ABI/INFORM Professional Advanced</collection><collection>ProQuest Engineering Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>ABI/INFORM Global</collection><collection>Computing Database</collection><collection>Research Library</collection><collection>Science Database</collection><collection>Engineering Database</collection><collection>Research Library (Corporate)</collection><collection>Advanced Technologies &amp; Aerospace Database</collection><collection>ProQuest Advanced Technologies &amp; Aerospace Collection</collection><collection>ProQuest One Business</collection><collection>ProQuest One Business (Alumni)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>Engineering Collection</collection><collection>ProQuest Central Basic</collection><jtitle>Computational statistics</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Zhang, Boyao</au><au>Hepp, Tobias</au><au>Greven, Sonja</au><au>Bergherr, Elisabeth</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Adaptive step-length selection in gradient boosting for Gaussian location and scale models</atitle><jtitle>Computational statistics</jtitle><stitle>Comput Stat</stitle><date>2022-11-01</date><risdate>2022</risdate><volume>37</volume><issue>5</issue><spage>2295</spage><epage>2332</epage><pages>2295-2332</pages><issn>0943-4062</issn><eissn>1613-9658</eissn><abstract>Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive models for location, scale and shape (GAMLSS), imbalanced updates of predictors, where some distribution parameters are updated more frequently than others, can be a problem that prevents some submodels to be appropriately fitted within a limited number of boosting iterations. We propose an approach using adaptive step-length (ASL) determination within a non-cyclical boosting algorithm for Gaussian location and scale models, as an important special case of the wider class of GAMLSS, to prevent such imbalance. Moreover, we discuss properties of the ASL and derive a semi-analytical form of the ASL that avoids manual selection of the search interval and numerical optimization to find the optimal step-length, and consequently improves computational efficiency. We show competitive behavior of the proposed approaches compared to penalized maximum likelihood and boosting with a fixed step-length for Gaussian location and scale models in two simulations and two applications, in particular for cases of large variance and/or more variables than observations. In addition, the underlying concept of the ASL is also applicable to the whole GAMLSS framework and to other models with more than one predictor like zero-inflated count models, and brings up insights into the choice of the reasonable defaults for the step-length in the simpler special case of (Gaussian) additive models.</abstract><cop>Berlin/Heidelberg</cop><pub>Springer Berlin Heidelberg</pub><doi>10.1007/s00180-022-01199-3</doi><tpages>38</tpages><orcidid>https://orcid.org/0000-0001-9161-921X</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 0943-4062
ispartof Computational statistics, 2022-11, Vol.37 (5), p.2295-2332
issn 0943-4062
1613-9658
language eng
recordid cdi_proquest_journals_2719231098
source SpringerLink Journals - AutoHoldings
subjects Algorithms
Economic Theory/Quantitative Economics/Mathematical Methods
Mathematics and Statistics
Optimization
Original Paper
Probability and Statistics in Computer Science
Probability Theory and Stochastic Processes
Scale models
Statistics
title Adaptive step-length selection in gradient boosting for Gaussian location and scale models
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-06T12%3A40%3A27IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Adaptive%20step-length%20selection%20in%20gradient%20boosting%20for%20Gaussian%20location%20and%20scale%20models&rft.jtitle=Computational%20statistics&rft.au=Zhang,%20Boyao&rft.date=2022-11-01&rft.volume=37&rft.issue=5&rft.spage=2295&rft.epage=2332&rft.pages=2295-2332&rft.issn=0943-4062&rft.eissn=1613-9658&rft_id=info:doi/10.1007/s00180-022-01199-3&rft_dat=%3Cproquest_cross%3E2719231098%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2719231098&rft_id=info:pmid/&rfr_iscdi=true