Logical assessment formula and its principles for evaluations with inaccurate ground-truth labels
Evaluations with accurate ground-truth labels (AGTLs) have been widely employed to assess predictive models for artificial intelligence applications. However, in some specific fields, such as medical histopathology whole slide image analysis, it is quite usual the situation that AGTLs are difficult...
Gespeichert in:
| Veröffentlicht in: | Knowledge and information systems 2024-04, Vol.66 (4), p.2561-2573 |
|---|---|
| 1. Verfasser: | |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 2573 |
|---|---|
| container_issue | 4 |
| container_start_page | 2561 |
| container_title | Knowledge and information systems |
| container_volume | 66 |
| creator | Yang, Yongquan |
| description | Evaluations with accurate ground-truth labels (AGTLs) have been widely employed to assess predictive models for artificial intelligence applications. However, in some specific fields, such as medical histopathology whole slide image analysis, it is quite usual the situation that AGTLs are difficult to be precisely defined or even do not exist. To alleviate this situation, we propose logical assessment formula (LAF) and reveal its principles for evaluations with inaccurate ground-truth labels (IAGTLs) via logical reasoning under uncertainty. From the revealed principles of LAF, we summarize the practicability of LAF: (1) LAF can be applied for evaluations with IAGTLs on a more difficult task, able to act like usual strategies for evaluations with AGTLs reasonably; (2) LAF can be applied for evaluations with IAGTLs from the logical perspective on an easier task, unable to act like usual strategies for evaluations with AGTLs confidently. |
| doi_str_mv | 10.1007/s10115-023-02047-6 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_3030969172</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3030969172</sourcerecordid><originalsourceid>FETCH-LOGICAL-c319t-decee0d48f4c765ffb305bc601b37245f952cfa8130cb2153086bdf443dc82b03</originalsourceid><addsrcrecordid>eNp9kE1PwzAMhiMEEmPwBzhF4lywkzZpj2jiS5rEBc5RmiajU9eOOAXx7-koEjcOli3b72v5YewS4RoB9A0hIBYZCDkF5DpTR2wBAqtMIqrj3xql1qfsjGgLgFohLphdD5vW2Y5bIk-0833iYYi7sbPc9g1vE_F9bHvX7jtPhxH3H7YbbWqHnvhnm95421vnxmiT55s4jH2TpThO_c7WvqNzdhJsR_7iNy_Z6_3dy-oxWz8_PK1u15mTWKWs8c57aPIy5E6rIoRaQlE7BVhLLfIiVIVwwZYowdUCCwmlqpuQ57JxpahBLtnV7LuPw_voKZntMMZ-OmkkSKhUhVpMW2LecnEgij6Y6budjV8GwRxQmhmlmVCaH5RGTSI5i-iAYuPjn_U_qm-IvHhd</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3030969172</pqid></control><display><type>article</type><title>Logical assessment formula and its principles for evaluations with inaccurate ground-truth labels</title><source>Springer Journals</source><creator>Yang, Yongquan</creator><creatorcontrib>Yang, Yongquan</creatorcontrib><description>Evaluations with accurate ground-truth labels (AGTLs) have been widely employed to assess predictive models for artificial intelligence applications. However, in some specific fields, such as medical histopathology whole slide image analysis, it is quite usual the situation that AGTLs are difficult to be precisely defined or even do not exist. To alleviate this situation, we propose logical assessment formula (LAF) and reveal its principles for evaluations with inaccurate ground-truth labels (IAGTLs) via logical reasoning under uncertainty. From the revealed principles of LAF, we summarize the practicability of LAF: (1) LAF can be applied for evaluations with IAGTLs on a more difficult task, able to act like usual strategies for evaluations with AGTLs reasonably; (2) LAF can be applied for evaluations with IAGTLs from the logical perspective on an easier task, unable to act like usual strategies for evaluations with AGTLs confidently.</description><identifier>ISSN: 0219-1377</identifier><identifier>EISSN: 0219-3116</identifier><identifier>DOI: 10.1007/s10115-023-02047-6</identifier><language>eng</language><publisher>London: Springer London</publisher><subject>Artificial intelligence ; Computer Science ; Data Mining and Knowledge Discovery ; Database Management ; Evaluation ; Image analysis ; Information Storage and Retrieval ; Information Systems and Communication Service ; Information Systems Applications (incl.Internet) ; IT in Business ; Labels ; Prediction models ; Principles ; Regular Paper ; Uncertainty</subject><ispartof>Knowledge and information systems, 2024-04, Vol.66 (4), p.2561-2573</ispartof><rights>The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c319t-decee0d48f4c765ffb305bc601b37245f952cfa8130cb2153086bdf443dc82b03</citedby><cites>FETCH-LOGICAL-c319t-decee0d48f4c765ffb305bc601b37245f952cfa8130cb2153086bdf443dc82b03</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s10115-023-02047-6$$EPDF$$P50$$Gspringer$$H</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s10115-023-02047-6$$EHTML$$P50$$Gspringer$$H</linktohtml><link.rule.ids>314,776,780,27901,27902,41464,42533,51294</link.rule.ids></links><search><creatorcontrib>Yang, Yongquan</creatorcontrib><title>Logical assessment formula and its principles for evaluations with inaccurate ground-truth labels</title><title>Knowledge and information systems</title><addtitle>Knowl Inf Syst</addtitle><description>Evaluations with accurate ground-truth labels (AGTLs) have been widely employed to assess predictive models for artificial intelligence applications. However, in some specific fields, such as medical histopathology whole slide image analysis, it is quite usual the situation that AGTLs are difficult to be precisely defined or even do not exist. To alleviate this situation, we propose logical assessment formula (LAF) and reveal its principles for evaluations with inaccurate ground-truth labels (IAGTLs) via logical reasoning under uncertainty. From the revealed principles of LAF, we summarize the practicability of LAF: (1) LAF can be applied for evaluations with IAGTLs on a more difficult task, able to act like usual strategies for evaluations with AGTLs reasonably; (2) LAF can be applied for evaluations with IAGTLs from the logical perspective on an easier task, unable to act like usual strategies for evaluations with AGTLs confidently.</description><subject>Artificial intelligence</subject><subject>Computer Science</subject><subject>Data Mining and Knowledge Discovery</subject><subject>Database Management</subject><subject>Evaluation</subject><subject>Image analysis</subject><subject>Information Storage and Retrieval</subject><subject>Information Systems and Communication Service</subject><subject>Information Systems Applications (incl.Internet)</subject><subject>IT in Business</subject><subject>Labels</subject><subject>Prediction models</subject><subject>Principles</subject><subject>Regular Paper</subject><subject>Uncertainty</subject><issn>0219-1377</issn><issn>0219-3116</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><recordid>eNp9kE1PwzAMhiMEEmPwBzhF4lywkzZpj2jiS5rEBc5RmiajU9eOOAXx7-koEjcOli3b72v5YewS4RoB9A0hIBYZCDkF5DpTR2wBAqtMIqrj3xql1qfsjGgLgFohLphdD5vW2Y5bIk-0833iYYi7sbPc9g1vE_F9bHvX7jtPhxH3H7YbbWqHnvhnm95421vnxmiT55s4jH2TpThO_c7WvqNzdhJsR_7iNy_Z6_3dy-oxWz8_PK1u15mTWKWs8c57aPIy5E6rIoRaQlE7BVhLLfIiVIVwwZYowdUCCwmlqpuQ57JxpahBLtnV7LuPw_voKZntMMZ-OmkkSKhUhVpMW2LecnEgij6Y6budjV8GwRxQmhmlmVCaH5RGTSI5i-iAYuPjn_U_qm-IvHhd</recordid><startdate>20240401</startdate><enddate>20240401</enddate><creator>Yang, Yongquan</creator><general>Springer London</general><general>Springer Nature B.V</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope></search><sort><creationdate>20240401</creationdate><title>Logical assessment formula and its principles for evaluations with inaccurate ground-truth labels</title><author>Yang, Yongquan</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c319t-decee0d48f4c765ffb305bc601b37245f952cfa8130cb2153086bdf443dc82b03</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Artificial intelligence</topic><topic>Computer Science</topic><topic>Data Mining and Knowledge Discovery</topic><topic>Database Management</topic><topic>Evaluation</topic><topic>Image analysis</topic><topic>Information Storage and Retrieval</topic><topic>Information Systems and Communication Service</topic><topic>Information Systems Applications (incl.Internet)</topic><topic>IT in Business</topic><topic>Labels</topic><topic>Prediction models</topic><topic>Principles</topic><topic>Regular Paper</topic><topic>Uncertainty</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Yang, Yongquan</creatorcontrib><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>Knowledge and information systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Yang, Yongquan</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Logical assessment formula and its principles for evaluations with inaccurate ground-truth labels</atitle><jtitle>Knowledge and information systems</jtitle><stitle>Knowl Inf Syst</stitle><date>2024-04-01</date><risdate>2024</risdate><volume>66</volume><issue>4</issue><spage>2561</spage><epage>2573</epage><pages>2561-2573</pages><issn>0219-1377</issn><eissn>0219-3116</eissn><abstract>Evaluations with accurate ground-truth labels (AGTLs) have been widely employed to assess predictive models for artificial intelligence applications. However, in some specific fields, such as medical histopathology whole slide image analysis, it is quite usual the situation that AGTLs are difficult to be precisely defined or even do not exist. To alleviate this situation, we propose logical assessment formula (LAF) and reveal its principles for evaluations with inaccurate ground-truth labels (IAGTLs) via logical reasoning under uncertainty. From the revealed principles of LAF, we summarize the practicability of LAF: (1) LAF can be applied for evaluations with IAGTLs on a more difficult task, able to act like usual strategies for evaluations with AGTLs reasonably; (2) LAF can be applied for evaluations with IAGTLs from the logical perspective on an easier task, unable to act like usual strategies for evaluations with AGTLs confidently.</abstract><cop>London</cop><pub>Springer London</pub><doi>10.1007/s10115-023-02047-6</doi><tpages>13</tpages></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0219-1377 |
| ispartof | Knowledge and information systems, 2024-04, Vol.66 (4), p.2561-2573 |
| issn | 0219-1377 0219-3116 |
| language | eng |
| recordid | cdi_proquest_journals_3030969172 |
| source | Springer Journals |
| subjects | Artificial intelligence Computer Science Data Mining and Knowledge Discovery Database Management Evaluation Image analysis Information Storage and Retrieval Information Systems and Communication Service Information Systems Applications (incl.Internet) IT in Business Labels Prediction models Principles Regular Paper Uncertainty |
| title | Logical assessment formula and its principles for evaluations with inaccurate ground-truth labels |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-12T06%3A46%3A01IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Logical%20assessment%20formula%20and%20its%20principles%20for%20evaluations%20with%20inaccurate%20ground-truth%20labels&rft.jtitle=Knowledge%20and%20information%20systems&rft.au=Yang,%20Yongquan&rft.date=2024-04-01&rft.volume=66&rft.issue=4&rft.spage=2561&rft.epage=2573&rft.pages=2561-2573&rft.issn=0219-1377&rft.eissn=0219-3116&rft_id=info:doi/10.1007/s10115-023-02047-6&rft_dat=%3Cproquest_cross%3E3030969172%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3030969172&rft_id=info:pmid/&rfr_iscdi=true |