Adaptive step-length selection in gradient boosting for Gaussian location and scale models
Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive models for location, scale and shape (GAMLSS), imbalanced updates of predictors, where so...
Gespeichert in:
Veröffentlicht in: | Computational statistics 2022-11, Vol.37 (5), p.2295-2332 |
---|---|
Hauptverfasser: | , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 2332 |
---|---|
container_issue | 5 |
container_start_page | 2295 |
container_title | Computational statistics |
container_volume | 37 |
creator | Zhang, Boyao Hepp, Tobias Greven, Sonja Bergherr, Elisabeth |
description | Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive models for location, scale and shape (GAMLSS), imbalanced updates of predictors, where some distribution parameters are updated more frequently than others, can be a problem that prevents some submodels to be appropriately fitted within a limited number of boosting iterations. We propose an approach using adaptive step-length (ASL) determination within a non-cyclical boosting algorithm for Gaussian location and scale models, as an important special case of the wider class of GAMLSS, to prevent such imbalance. Moreover, we discuss properties of the ASL and derive a semi-analytical form of the ASL that avoids manual selection of the search interval and numerical optimization to find the optimal step-length, and consequently improves computational efficiency. We show competitive behavior of the proposed approaches compared to penalized maximum likelihood and boosting with a fixed step-length for Gaussian location and scale models in two simulations and two applications, in particular for cases of large variance and/or more variables than observations. In addition, the underlying concept of the ASL is also applicable to the whole GAMLSS framework and to other models with more than one predictor like zero-inflated count models, and brings up insights into the choice of the reasonable defaults for the step-length in the simpler special case of (Gaussian) additive models. |
doi_str_mv | 10.1007/s00180-022-01199-3 |
format | Article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2719231098</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2719231098</sourcerecordid><originalsourceid>FETCH-LOGICAL-c363t-953578a75784b7c25e53883f0a7fcc55a67ce46cc2bed52452f0029f6aa0ec913</originalsourceid><addsrcrecordid>eNp9kE1LAzEQhoMoWKt_wFPAczQfm83mWIpWoeBFL15Cmp1dt2yTmqSC_97YFbx5mbk87zvDg9A1o7eMUnWXKGUNJZRzQhnTmogTNGM1E0TXsjlFM6orQSpa83N0kdKWFlJxNkNvi9bu8_AJOGXYkxF8n99xghFcHoLHg8d9tO0APuNNCCkPvsddiHhlDykN1uMxOHtErW9xcnYEvAstjOkSnXV2THD1u-fo9eH-ZflI1s-rp-ViTZyoRSZaCqkaq8qoNspxCVI0jeioVZ1zUtpaOahq5_gGWskrybvyve5qayk4zcQc3Uy9-xg-DpCy2YZD9OWk4YppLhjVTaH4RLkYUorQmX0cdjZ-GUbNj0MzOTTFjDk6NKKExBRKBfY9xL_qf1LfdCZ0nA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2719231098</pqid></control><display><type>article</type><title>Adaptive step-length selection in gradient boosting for Gaussian location and scale models</title><source>SpringerLink Journals - AutoHoldings</source><creator>Zhang, Boyao ; Hepp, Tobias ; Greven, Sonja ; Bergherr, Elisabeth</creator><creatorcontrib>Zhang, Boyao ; Hepp, Tobias ; Greven, Sonja ; Bergherr, Elisabeth</creatorcontrib><description>Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive models for location, scale and shape (GAMLSS), imbalanced updates of predictors, where some distribution parameters are updated more frequently than others, can be a problem that prevents some submodels to be appropriately fitted within a limited number of boosting iterations. We propose an approach using adaptive step-length (ASL) determination within a non-cyclical boosting algorithm for Gaussian location and scale models, as an important special case of the wider class of GAMLSS, to prevent such imbalance. Moreover, we discuss properties of the ASL and derive a semi-analytical form of the ASL that avoids manual selection of the search interval and numerical optimization to find the optimal step-length, and consequently improves computational efficiency. We show competitive behavior of the proposed approaches compared to penalized maximum likelihood and boosting with a fixed step-length for Gaussian location and scale models in two simulations and two applications, in particular for cases of large variance and/or more variables than observations. In addition, the underlying concept of the ASL is also applicable to the whole GAMLSS framework and to other models with more than one predictor like zero-inflated count models, and brings up insights into the choice of the reasonable defaults for the step-length in the simpler special case of (Gaussian) additive models.</description><identifier>ISSN: 0943-4062</identifier><identifier>EISSN: 1613-9658</identifier><identifier>DOI: 10.1007/s00180-022-01199-3</identifier><language>eng</language><publisher>Berlin/Heidelberg: Springer Berlin Heidelberg</publisher><subject>Algorithms ; Economic Theory/Quantitative Economics/Mathematical Methods ; Mathematics and Statistics ; Optimization ; Original Paper ; Probability and Statistics in Computer Science ; Probability Theory and Stochastic Processes ; Scale models ; Statistics</subject><ispartof>Computational statistics, 2022-11, Vol.37 (5), p.2295-2332</ispartof><rights>The Author(s) 2022</rights><rights>The Author(s) 2022. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c363t-953578a75784b7c25e53883f0a7fcc55a67ce46cc2bed52452f0029f6aa0ec913</citedby><cites>FETCH-LOGICAL-c363t-953578a75784b7c25e53883f0a7fcc55a67ce46cc2bed52452f0029f6aa0ec913</cites><orcidid>0000-0001-9161-921X</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s00180-022-01199-3$$EPDF$$P50$$Gspringer$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s00180-022-01199-3$$EHTML$$P50$$Gspringer$$Hfree_for_read</linktohtml><link.rule.ids>314,780,784,27924,27925,41488,42557,51319</link.rule.ids></links><search><creatorcontrib>Zhang, Boyao</creatorcontrib><creatorcontrib>Hepp, Tobias</creatorcontrib><creatorcontrib>Greven, Sonja</creatorcontrib><creatorcontrib>Bergherr, Elisabeth</creatorcontrib><title>Adaptive step-length selection in gradient boosting for Gaussian location and scale models</title><title>Computational statistics</title><addtitle>Comput Stat</addtitle><description>Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive models for location, scale and shape (GAMLSS), imbalanced updates of predictors, where some distribution parameters are updated more frequently than others, can be a problem that prevents some submodels to be appropriately fitted within a limited number of boosting iterations. We propose an approach using adaptive step-length (ASL) determination within a non-cyclical boosting algorithm for Gaussian location and scale models, as an important special case of the wider class of GAMLSS, to prevent such imbalance. Moreover, we discuss properties of the ASL and derive a semi-analytical form of the ASL that avoids manual selection of the search interval and numerical optimization to find the optimal step-length, and consequently improves computational efficiency. We show competitive behavior of the proposed approaches compared to penalized maximum likelihood and boosting with a fixed step-length for Gaussian location and scale models in two simulations and two applications, in particular for cases of large variance and/or more variables than observations. In addition, the underlying concept of the ASL is also applicable to the whole GAMLSS framework and to other models with more than one predictor like zero-inflated count models, and brings up insights into the choice of the reasonable defaults for the step-length in the simpler special case of (Gaussian) additive models.</description><subject>Algorithms</subject><subject>Economic Theory/Quantitative Economics/Mathematical Methods</subject><subject>Mathematics and Statistics</subject><subject>Optimization</subject><subject>Original Paper</subject><subject>Probability and Statistics in Computer Science</subject><subject>Probability Theory and Stochastic Processes</subject><subject>Scale models</subject><subject>Statistics</subject><issn>0943-4062</issn><issn>1613-9658</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>C6C</sourceid><sourceid>8G5</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><recordid>eNp9kE1LAzEQhoMoWKt_wFPAczQfm83mWIpWoeBFL15Cmp1dt2yTmqSC_97YFbx5mbk87zvDg9A1o7eMUnWXKGUNJZRzQhnTmogTNGM1E0TXsjlFM6orQSpa83N0kdKWFlJxNkNvi9bu8_AJOGXYkxF8n99xghFcHoLHg8d9tO0APuNNCCkPvsddiHhlDykN1uMxOHtErW9xcnYEvAstjOkSnXV2THD1u-fo9eH-ZflI1s-rp-ViTZyoRSZaCqkaq8qoNspxCVI0jeioVZ1zUtpaOahq5_gGWskrybvyve5qayk4zcQc3Uy9-xg-DpCy2YZD9OWk4YppLhjVTaH4RLkYUorQmX0cdjZ-GUbNj0MzOTTFjDk6NKKExBRKBfY9xL_qf1LfdCZ0nA</recordid><startdate>20221101</startdate><enddate>20221101</enddate><creator>Zhang, Boyao</creator><creator>Hepp, Tobias</creator><creator>Greven, Sonja</creator><creator>Bergherr, Elisabeth</creator><general>Springer Berlin Heidelberg</general><general>Springer Nature B.V</general><scope>C6C</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7SC</scope><scope>7TB</scope><scope>7WY</scope><scope>7WZ</scope><scope>7XB</scope><scope>87Z</scope><scope>88I</scope><scope>8AL</scope><scope>8C1</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>8FK</scope><scope>8FL</scope><scope>8G5</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BEZIV</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FR3</scope><scope>FRNLG</scope><scope>FYUFA</scope><scope>F~G</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K60</scope><scope>K6~</scope><scope>K7-</scope><scope>KR7</scope><scope>L.-</scope><scope>L6V</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>M0C</scope><scope>M0N</scope><scope>M2O</scope><scope>M2P</scope><scope>M7S</scope><scope>MBDVC</scope><scope>P5Z</scope><scope>P62</scope><scope>PQBIZ</scope><scope>PQBZA</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PTHSS</scope><scope>Q9U</scope><orcidid>https://orcid.org/0000-0001-9161-921X</orcidid></search><sort><creationdate>20221101</creationdate><title>Adaptive step-length selection in gradient boosting for Gaussian location and scale models</title><author>Zhang, Boyao ; Hepp, Tobias ; Greven, Sonja ; Bergherr, Elisabeth</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c363t-953578a75784b7c25e53883f0a7fcc55a67ce46cc2bed52452f0029f6aa0ec913</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Algorithms</topic><topic>Economic Theory/Quantitative Economics/Mathematical Methods</topic><topic>Mathematics and Statistics</topic><topic>Optimization</topic><topic>Original Paper</topic><topic>Probability and Statistics in Computer Science</topic><topic>Probability Theory and Stochastic Processes</topic><topic>Scale models</topic><topic>Statistics</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Zhang, Boyao</creatorcontrib><creatorcontrib>Hepp, Tobias</creatorcontrib><creatorcontrib>Greven, Sonja</creatorcontrib><creatorcontrib>Bergherr, Elisabeth</creatorcontrib><collection>Springer Nature OA Free Journals</collection><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Computer and Information Systems Abstracts</collection><collection>Mechanical & Transportation Engineering Abstracts</collection><collection>ABI/INFORM Collection</collection><collection>ABI/INFORM Global (PDF only)</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>ABI/INFORM Global (Alumni Edition)</collection><collection>Science Database (Alumni Edition)</collection><collection>Computing Database (Alumni Edition)</collection><collection>Public Health Database</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ABI/INFORM Collection (Alumni Edition)</collection><collection>Research Library (Alumni Edition)</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies & Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Business Premium Collection</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>Engineering Research Database</collection><collection>Business Premium Collection (Alumni)</collection><collection>Health Research Premium Collection</collection><collection>ABI/INFORM Global (Corporate)</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>ProQuest Business Collection (Alumni Edition)</collection><collection>ProQuest Business Collection</collection><collection>Computer Science Database</collection><collection>Civil Engineering Abstracts</collection><collection>ABI/INFORM Professional Advanced</collection><collection>ProQuest Engineering Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>ABI/INFORM Global</collection><collection>Computing Database</collection><collection>Research Library</collection><collection>Science Database</collection><collection>Engineering Database</collection><collection>Research Library (Corporate)</collection><collection>Advanced Technologies & Aerospace Database</collection><collection>ProQuest Advanced Technologies & Aerospace Collection</collection><collection>ProQuest One Business</collection><collection>ProQuest One Business (Alumni)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>Engineering Collection</collection><collection>ProQuest Central Basic</collection><jtitle>Computational statistics</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Zhang, Boyao</au><au>Hepp, Tobias</au><au>Greven, Sonja</au><au>Bergherr, Elisabeth</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Adaptive step-length selection in gradient boosting for Gaussian location and scale models</atitle><jtitle>Computational statistics</jtitle><stitle>Comput Stat</stitle><date>2022-11-01</date><risdate>2022</risdate><volume>37</volume><issue>5</issue><spage>2295</spage><epage>2332</epage><pages>2295-2332</pages><issn>0943-4062</issn><eissn>1613-9658</eissn><abstract>Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive models for location, scale and shape (GAMLSS), imbalanced updates of predictors, where some distribution parameters are updated more frequently than others, can be a problem that prevents some submodels to be appropriately fitted within a limited number of boosting iterations. We propose an approach using adaptive step-length (ASL) determination within a non-cyclical boosting algorithm for Gaussian location and scale models, as an important special case of the wider class of GAMLSS, to prevent such imbalance. Moreover, we discuss properties of the ASL and derive a semi-analytical form of the ASL that avoids manual selection of the search interval and numerical optimization to find the optimal step-length, and consequently improves computational efficiency. We show competitive behavior of the proposed approaches compared to penalized maximum likelihood and boosting with a fixed step-length for Gaussian location and scale models in two simulations and two applications, in particular for cases of large variance and/or more variables than observations. In addition, the underlying concept of the ASL is also applicable to the whole GAMLSS framework and to other models with more than one predictor like zero-inflated count models, and brings up insights into the choice of the reasonable defaults for the step-length in the simpler special case of (Gaussian) additive models.</abstract><cop>Berlin/Heidelberg</cop><pub>Springer Berlin Heidelberg</pub><doi>10.1007/s00180-022-01199-3</doi><tpages>38</tpages><orcidid>https://orcid.org/0000-0001-9161-921X</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 0943-4062 |
ispartof | Computational statistics, 2022-11, Vol.37 (5), p.2295-2332 |
issn | 0943-4062 1613-9658 |
language | eng |
recordid | cdi_proquest_journals_2719231098 |
source | SpringerLink Journals - AutoHoldings |
subjects | Algorithms Economic Theory/Quantitative Economics/Mathematical Methods Mathematics and Statistics Optimization Original Paper Probability and Statistics in Computer Science Probability Theory and Stochastic Processes Scale models Statistics |
title | Adaptive step-length selection in gradient boosting for Gaussian location and scale models |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-06T12%3A40%3A27IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Adaptive%20step-length%20selection%20in%20gradient%20boosting%20for%20Gaussian%20location%20and%20scale%20models&rft.jtitle=Computational%20statistics&rft.au=Zhang,%20Boyao&rft.date=2022-11-01&rft.volume=37&rft.issue=5&rft.spage=2295&rft.epage=2332&rft.pages=2295-2332&rft.issn=0943-4062&rft.eissn=1613-9658&rft_id=info:doi/10.1007/s00180-022-01199-3&rft_dat=%3Cproquest_cross%3E2719231098%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2719231098&rft_id=info:pmid/&rfr_iscdi=true |