CatBoost: unbiased boosting with categorical features

CatBoost: unbiased boosting with categorical features

This paper presents the key algorithmic techniques behind CatBoost, a new gradient boosting toolkit. Their combination leads to CatBoost outperforming other publicly available boosting implementations in terms of quality on a variety of datasets. Two critical algorithmic advances introduced in CatBo...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Prokhorenkova, Liudmila, Gusev, Gleb, Vorobev, Aleksandr, Dorogush, Anna Veronika, Gulin, Andrey
Format: Artikel
Sprache:eng
Schlagworte:
Computer Science - Learning
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Prokhorenkova, Liudmila
Gusev, Gleb
Vorobev, Aleksandr
Dorogush, Anna Veronika
Gulin, Andrey
description This paper presents the key algorithmic techniques behind CatBoost, a new gradient boosting toolkit. Their combination leads to CatBoost outperforming other publicly available boosting implementations in terms of quality on a variety of datasets. Two critical algorithmic advances introduced in CatBoost are the implementation of ordered boosting, a permutation-driven alternative to the classic algorithm, and an innovative algorithm for processing categorical features. Both techniques were created to fight a prediction shift caused by a special kind of target leakage present in all currently existing implementations of gradient boosting algorithms. In this paper, we provide a detailed analysis of this problem and demonstrate that proposed algorithms solve it effectively, leading to excellent empirical results.
doi_str_mv 10.48550/arxiv.1706.09516
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_1706_09516</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>1706_09516</sourcerecordid><originalsourceid>FETCH-LOGICAL-a1636-95946ace704d9f928b6e4c1b99cac5b5e22bb8b6b55122fe8bfb1798eae898aa3</originalsourceid><addsrcrecordid>eNotj8tOwzAQRb3pAhU-gBX-gQTbiR0PuxLxkip1U9bRjDsulkqDHJfH30NLV1f3Lo7uEeJaq7r11qpbzN_ps9adcrUCq92FsD2W-3Gcyp087CnhxBtJx572W_mVypsMWHg75hRwJyNjOWSeLsUs4m7iq3POxevjw7p_rparp5d-saxQu8ZVYKF1GLhT7QYiGE-O26AJIGCwZNkYor-RrNXGRPYUSXfgGdmDR2zm4uafezo-fOT0jvlnOAoMJ4HmF0DPQRM</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>CatBoost: unbiased boosting with categorical features</title><source>arXiv.org</source><creator>Prokhorenkova, Liudmila ; Gusev, Gleb ; Vorobev, Aleksandr ; Dorogush, Anna Veronika ; Gulin, Andrey</creator><creatorcontrib>Prokhorenkova, Liudmila ; Gusev, Gleb ; Vorobev, Aleksandr ; Dorogush, Anna Veronika ; Gulin, Andrey</creatorcontrib><description>This paper presents the key algorithmic techniques behind CatBoost, a new gradient boosting toolkit. Their combination leads to CatBoost outperforming other publicly available boosting implementations in terms of quality on a variety of datasets. Two critical algorithmic advances introduced in CatBoost are the implementation of ordered boosting, a permutation-driven alternative to the classic algorithm, and an innovative algorithm for processing categorical features. Both techniques were created to fight a prediction shift caused by a special kind of target leakage present in all currently existing implementations of gradient boosting algorithms. In this paper, we provide a detailed analysis of this problem and demonstrate that proposed algorithms solve it effectively, leading to excellent empirical results.</description><identifier>DOI: 10.48550/arxiv.1706.09516</identifier><language>eng</language><subject>Computer Science - Learning</subject><creationdate>2017-06</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-a1636-95946ace704d9f928b6e4c1b99cac5b5e22bb8b6b55122fe8bfb1798eae898aa3</citedby></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/1706.09516$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.1706.09516$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Prokhorenkova, Liudmila</creatorcontrib><creatorcontrib>Gusev, Gleb</creatorcontrib><creatorcontrib>Vorobev, Aleksandr</creatorcontrib><creatorcontrib>Dorogush, Anna Veronika</creatorcontrib><creatorcontrib>Gulin, Andrey</creatorcontrib><title>CatBoost: unbiased boosting with categorical features</title><description>This paper presents the key algorithmic techniques behind CatBoost, a new gradient boosting toolkit. Their combination leads to CatBoost outperforming other publicly available boosting implementations in terms of quality on a variety of datasets. Two critical algorithmic advances introduced in CatBoost are the implementation of ordered boosting, a permutation-driven alternative to the classic algorithm, and an innovative algorithm for processing categorical features. Both techniques were created to fight a prediction shift caused by a special kind of target leakage present in all currently existing implementations of gradient boosting algorithms. In this paper, we provide a detailed analysis of this problem and demonstrate that proposed algorithms solve it effectively, leading to excellent empirical results.</description><subject>Computer Science - Learning</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2017</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotj8tOwzAQRb3pAhU-gBX-gQTbiR0PuxLxkip1U9bRjDsulkqDHJfH30NLV1f3Lo7uEeJaq7r11qpbzN_ps9adcrUCq92FsD2W-3Gcyp087CnhxBtJx572W_mVypsMWHg75hRwJyNjOWSeLsUs4m7iq3POxevjw7p_rparp5d-saxQu8ZVYKF1GLhT7QYiGE-O26AJIGCwZNkYor-RrNXGRPYUSXfgGdmDR2zm4uafezo-fOT0jvlnOAoMJ4HmF0DPQRM</recordid><startdate>20170628</startdate><enddate>20170628</enddate><creator>Prokhorenkova, Liudmila</creator><creator>Gusev, Gleb</creator><creator>Vorobev, Aleksandr</creator><creator>Dorogush, Anna Veronika</creator><creator>Gulin, Andrey</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20170628</creationdate><title>CatBoost: unbiased boosting with categorical features</title><author>Prokhorenkova, Liudmila ; Gusev, Gleb ; Vorobev, Aleksandr ; Dorogush, Anna Veronika ; Gulin, Andrey</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a1636-95946ace704d9f928b6e4c1b99cac5b5e22bb8b6b55122fe8bfb1798eae898aa3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2017</creationdate><topic>Computer Science - Learning</topic><toplevel>online_resources</toplevel><creatorcontrib>Prokhorenkova, Liudmila</creatorcontrib><creatorcontrib>Gusev, Gleb</creatorcontrib><creatorcontrib>Vorobev, Aleksandr</creatorcontrib><creatorcontrib>Dorogush, Anna Veronika</creatorcontrib><creatorcontrib>Gulin, Andrey</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Prokhorenkova, Liudmila</au><au>Gusev, Gleb</au><au>Vorobev, Aleksandr</au><au>Dorogush, Anna Veronika</au><au>Gulin, Andrey</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>CatBoost: unbiased boosting with categorical features</atitle><date>2017-06-28</date><risdate>2017</risdate><abstract>This paper presents the key algorithmic techniques behind CatBoost, a new gradient boosting toolkit. Their combination leads to CatBoost outperforming other publicly available boosting implementations in terms of quality on a variety of datasets. Two critical algorithmic advances introduced in CatBoost are the implementation of ordered boosting, a permutation-driven alternative to the classic algorithm, and an innovative algorithm for processing categorical features. Both techniques were created to fight a prediction shift caused by a special kind of target leakage present in all currently existing implementations of gradient boosting algorithms. In this paper, we provide a detailed analysis of this problem and demonstrate that proposed algorithms solve it effectively, leading to excellent empirical results.</abstract><doi>10.48550/arxiv.1706.09516</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.1706.09516
ispartof
issn
language eng
recordid cdi_arxiv_primary_1706_09516
source arXiv.org
subjects Computer Science - Learning
title CatBoost: unbiased boosting with categorical features
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-06T23%3A10%3A34IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=CatBoost:%20unbiased%20boosting%20with%20categorical%20features&rft.au=Prokhorenkova,%20Liudmila&rft.date=2017-06-28&rft_id=info:doi/10.48550/arxiv.1706.09516&rft_dat=%3Carxiv_GOX%3E1706_09516%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true