The Balanced Accuracy and Its Posterior Distribution
Evaluating the performance of a classification algorithm critically requires a measure of the degree to which unseen examples have been identified with their correct class labels. In practice, generalizability is frequently estimated by averaging the accuracies obtained on individual cross-validatio...
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Tagungsbericht |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 3124 |
|---|---|
| container_issue | |
| container_start_page | 3121 |
| container_title | |
| container_volume | |
| creator | Brodersen, Kay H Cheng Soon Ong Stephan, Klaas E Buhmann, Joachim M |
| description | Evaluating the performance of a classification algorithm critically requires a measure of the degree to which unseen examples have been identified with their correct class labels. In practice, generalizability is frequently estimated by averaging the accuracies obtained on individual cross-validation folds. This procedure, however, is problematic in two ways. First, it does not allow for the derivation of meaningful confidence intervals. Second, it leads to an optimistic estimate when a biased classifier is tested on an imbalanced dataset. We show that both problems can be overcome by replacing the conventional point estimate of accuracy by an estimate of the posterior distribution of the balanced accuracy. |
| doi_str_mv | 10.1109/ICPR.2010.764 |
| format | Conference Proceeding |
| fullrecord | <record><control><sourceid>ieee_6IE</sourceid><recordid>TN_cdi_ieee_primary_5597285</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>5597285</ieee_id><sourcerecordid>5597285</sourcerecordid><originalsourceid>FETCH-LOGICAL-c244t-523c94cea8298a4dde9725c7bfd1f9ff53e3378285337bc914196f81b229e1173</originalsourceid><addsrcrecordid>eNo1jctOwzAURM1LIpQsWbHxD6TkXtuxvSwpj0iVqFBZV45jC0slQXa66N9jCViNjkZzhpA7qJcAtX7o2u37EuuMsuFnpNRSAUfOpeDAz0mBikElM16Qm_8C8ZIUUAuoeCPgmpQphb7GRjZSCFEQvvt09NEczGjdQFfWHqOxJ2rGgXZzotspzS6GKdJ1SHMM_XEO03hLrrw5JFf-5YJ8PD_t2tdq8_bStatNZfP5XAlkVnPrjEKtDB8GpyUKK3s_gNfeC-YYkwqVyNFbDRx04xX0iNoBSLYg97_e4Jzbf8fwZeJpL0TW5M0PsrhIJQ</addsrcrecordid><sourcetype>Publisher</sourcetype><iscdi>true</iscdi><recordtype>conference_proceeding</recordtype></control><display><type>conference_proceeding</type><title>The Balanced Accuracy and Its Posterior Distribution</title><source>IEEE Electronic Library (IEL) Conference Proceedings</source><creator>Brodersen, Kay H ; Cheng Soon Ong ; Stephan, Klaas E ; Buhmann, Joachim M</creator><creatorcontrib>Brodersen, Kay H ; Cheng Soon Ong ; Stephan, Klaas E ; Buhmann, Joachim M</creatorcontrib><description>Evaluating the performance of a classification algorithm critically requires a measure of the degree to which unseen examples have been identified with their correct class labels. In practice, generalizability is frequently estimated by averaging the accuracies obtained on individual cross-validation folds. This procedure, however, is problematic in two ways. First, it does not allow for the derivation of meaningful confidence intervals. Second, it leads to an optimistic estimate when a biased classifier is tested on an imbalanced dataset. We show that both problems can be overcome by replacing the conventional point estimate of accuracy by an estimate of the posterior distribution of the balanced accuracy.</description><identifier>ISSN: 1051-4651</identifier><identifier>ISBN: 1424475422</identifier><identifier>ISBN: 9781424475421</identifier><identifier>EISSN: 2831-7475</identifier><identifier>EISBN: 9781424475414</identifier><identifier>EISBN: 9780769541099</identifier><identifier>EISBN: 1424475414</identifier><identifier>EISBN: 0769541097</identifier><identifier>DOI: 10.1109/ICPR.2010.764</identifier><language>eng</language><publisher>IEEE</publisher><subject>Accuracy ; Approximation algorithms ; bias ; class imbalance ; classification performance ; generalizability ; Inference algorithms ; Machine learning ; Prediction algorithms ; Probabilistic logic ; Training</subject><ispartof>2010 20th International Conference on Pattern Recognition, 2010, p.3121-3124</ispartof><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c244t-523c94cea8298a4dde9725c7bfd1f9ff53e3378285337bc914196f81b229e1173</citedby></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/5597285$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>309,310,780,784,789,790,2058,27925,54920</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/5597285$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Brodersen, Kay H</creatorcontrib><creatorcontrib>Cheng Soon Ong</creatorcontrib><creatorcontrib>Stephan, Klaas E</creatorcontrib><creatorcontrib>Buhmann, Joachim M</creatorcontrib><title>The Balanced Accuracy and Its Posterior Distribution</title><title>2010 20th International Conference on Pattern Recognition</title><addtitle>ICPR</addtitle><description>Evaluating the performance of a classification algorithm critically requires a measure of the degree to which unseen examples have been identified with their correct class labels. In practice, generalizability is frequently estimated by averaging the accuracies obtained on individual cross-validation folds. This procedure, however, is problematic in two ways. First, it does not allow for the derivation of meaningful confidence intervals. Second, it leads to an optimistic estimate when a biased classifier is tested on an imbalanced dataset. We show that both problems can be overcome by replacing the conventional point estimate of accuracy by an estimate of the posterior distribution of the balanced accuracy.</description><subject>Accuracy</subject><subject>Approximation algorithms</subject><subject>bias</subject><subject>class imbalance</subject><subject>classification performance</subject><subject>generalizability</subject><subject>Inference algorithms</subject><subject>Machine learning</subject><subject>Prediction algorithms</subject><subject>Probabilistic logic</subject><subject>Training</subject><issn>1051-4651</issn><issn>2831-7475</issn><isbn>1424475422</isbn><isbn>9781424475421</isbn><isbn>9781424475414</isbn><isbn>9780769541099</isbn><isbn>1424475414</isbn><isbn>0769541097</isbn><fulltext>true</fulltext><rsrctype>conference_proceeding</rsrctype><creationdate>2010</creationdate><recordtype>conference_proceeding</recordtype><sourceid>6IE</sourceid><sourceid>RIE</sourceid><recordid>eNo1jctOwzAURM1LIpQsWbHxD6TkXtuxvSwpj0iVqFBZV45jC0slQXa66N9jCViNjkZzhpA7qJcAtX7o2u37EuuMsuFnpNRSAUfOpeDAz0mBikElM16Qm_8C8ZIUUAuoeCPgmpQphb7GRjZSCFEQvvt09NEczGjdQFfWHqOxJ2rGgXZzotspzS6GKdJ1SHMM_XEO03hLrrw5JFf-5YJ8PD_t2tdq8_bStatNZfP5XAlkVnPrjEKtDB8GpyUKK3s_gNfeC-YYkwqVyNFbDRx04xX0iNoBSLYg97_e4Jzbf8fwZeJpL0TW5M0PsrhIJQ</recordid><startdate>201008</startdate><enddate>201008</enddate><creator>Brodersen, Kay H</creator><creator>Cheng Soon Ong</creator><creator>Stephan, Klaas E</creator><creator>Buhmann, Joachim M</creator><general>IEEE</general><scope>6IE</scope><scope>6IL</scope><scope>CBEJK</scope><scope>RIE</scope><scope>RIL</scope></search><sort><creationdate>201008</creationdate><title>The Balanced Accuracy and Its Posterior Distribution</title><author>Brodersen, Kay H ; Cheng Soon Ong ; Stephan, Klaas E ; Buhmann, Joachim M</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c244t-523c94cea8298a4dde9725c7bfd1f9ff53e3378285337bc914196f81b229e1173</frbrgroupid><rsrctype>conference_proceedings</rsrctype><prefilter>conference_proceedings</prefilter><language>eng</language><creationdate>2010</creationdate><topic>Accuracy</topic><topic>Approximation algorithms</topic><topic>bias</topic><topic>class imbalance</topic><topic>classification performance</topic><topic>generalizability</topic><topic>Inference algorithms</topic><topic>Machine learning</topic><topic>Prediction algorithms</topic><topic>Probabilistic logic</topic><topic>Training</topic><toplevel>online_resources</toplevel><creatorcontrib>Brodersen, Kay H</creatorcontrib><creatorcontrib>Cheng Soon Ong</creatorcontrib><creatorcontrib>Stephan, Klaas E</creatorcontrib><creatorcontrib>Buhmann, Joachim M</creatorcontrib><collection>IEEE Electronic Library (IEL) Conference Proceedings</collection><collection>IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume</collection><collection>IEEE Xplore All Conference Proceedings</collection><collection>IEEE Electronic Library (IEL)</collection><collection>IEEE Proceedings Order Plans (POP All) 1998-Present</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Brodersen, Kay H</au><au>Cheng Soon Ong</au><au>Stephan, Klaas E</au><au>Buhmann, Joachim M</au><format>book</format><genre>proceeding</genre><ristype>CONF</ristype><atitle>The Balanced Accuracy and Its Posterior Distribution</atitle><btitle>2010 20th International Conference on Pattern Recognition</btitle><stitle>ICPR</stitle><date>2010-08</date><risdate>2010</risdate><spage>3121</spage><epage>3124</epage><pages>3121-3124</pages><issn>1051-4651</issn><eissn>2831-7475</eissn><isbn>1424475422</isbn><isbn>9781424475421</isbn><eisbn>9781424475414</eisbn><eisbn>9780769541099</eisbn><eisbn>1424475414</eisbn><eisbn>0769541097</eisbn><abstract>Evaluating the performance of a classification algorithm critically requires a measure of the degree to which unseen examples have been identified with their correct class labels. In practice, generalizability is frequently estimated by averaging the accuracies obtained on individual cross-validation folds. This procedure, however, is problematic in two ways. First, it does not allow for the derivation of meaningful confidence intervals. Second, it leads to an optimistic estimate when a biased classifier is tested on an imbalanced dataset. We show that both problems can be overcome by replacing the conventional point estimate of accuracy by an estimate of the posterior distribution of the balanced accuracy.</abstract><pub>IEEE</pub><doi>10.1109/ICPR.2010.764</doi><tpages>4</tpages></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | ISSN: 1051-4651 |
| ispartof | 2010 20th International Conference on Pattern Recognition, 2010, p.3121-3124 |
| issn | 1051-4651 2831-7475 |
| language | eng |
| recordid | cdi_ieee_primary_5597285 |
| source | IEEE Electronic Library (IEL) Conference Proceedings |
| subjects | Accuracy Approximation algorithms bias class imbalance classification performance generalizability Inference algorithms Machine learning Prediction algorithms Probabilistic logic Training |
| title | The Balanced Accuracy and Its Posterior Distribution |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-22T08%3A57%3A51IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-ieee_6IE&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=proceeding&rft.atitle=The%20Balanced%20Accuracy%20and%20Its%20Posterior%20Distribution&rft.btitle=2010%2020th%20International%20Conference%20on%20Pattern%20Recognition&rft.au=Brodersen,%20Kay%20H&rft.date=2010-08&rft.spage=3121&rft.epage=3124&rft.pages=3121-3124&rft.issn=1051-4651&rft.eissn=2831-7475&rft.isbn=1424475422&rft.isbn_list=9781424475421&rft_id=info:doi/10.1109/ICPR.2010.764&rft_dat=%3Cieee_6IE%3E5597285%3C/ieee_6IE%3E%3Curl%3E%3C/url%3E&rft.eisbn=9781424475414&rft.eisbn_list=9780769541099&rft.eisbn_list=1424475414&rft.eisbn_list=0769541097&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rft_ieee_id=5597285&rfr_iscdi=true |