Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?

Fact-checking pipelines increasingly adopt the Decompose-Then-Verify paradigm, where texts are broken down into smaller claims for individual verification and subsequently combined for a veracity decision. While decomposition is widely-adopted in such pipelines, its effects on final fact-checking pe...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Hu, Qisheng, Long, Quanyu, Wang, Wenya
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Hu, Qisheng
Long, Quanyu
Wang, Wenya
description Fact-checking pipelines increasingly adopt the Decompose-Then-Verify paradigm, where texts are broken down into smaller claims for individual verification and subsequently combined for a veracity decision. While decomposition is widely-adopted in such pipelines, its effects on final fact-checking performance remain underexplored. Some studies have reported improvements from decompostition, while others have observed performance declines, indicating its inconsistent impact. To date, no comprehensive analysis has been conducted to understand this variability. To address this gap, we present an in-depth analysis that explicitly examines the impact of decomposition on downstream verification performance. Through error case inspection and experiments, we introduce a categorization of decomposition errors and reveal a trade-off between accuracy gains and the noise introduced through decomposition. Our analysis provides new insights into understanding current system's instability and offers guidance for future studies toward improving claim decomposition in fact-checking pipelines.
doi_str_mv 10.48550/arxiv.2411.02400
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2411_02400</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2411_02400</sourcerecordid><originalsourceid>FETCH-arxiv_primary_2411_024003</originalsourceid><addsrcrecordid>eNpjYJA0NNAzsTA1NdBPLKrILNMzMjE01DMwMjEw4GSIdklNzs8tyC_OLMnMz1NwycxJzc1NLLZScMlPLVZwzknMzFVAVeKUn19copBfpOBUWpSSmqfglphcouuckZqcnZmXrhCQWpSWX5SbmJecas_DwJqWmFOcyguluRnk3VxDnD10wc6ILyjKzE0sqowHOSce7BxjwioAzSpAKg</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?</title><source>arXiv.org</source><creator>Hu, Qisheng ; Long, Quanyu ; Wang, Wenya</creator><creatorcontrib>Hu, Qisheng ; Long, Quanyu ; Wang, Wenya</creatorcontrib><description>Fact-checking pipelines increasingly adopt the Decompose-Then-Verify paradigm, where texts are broken down into smaller claims for individual verification and subsequently combined for a veracity decision. While decomposition is widely-adopted in such pipelines, its effects on final fact-checking performance remain underexplored. Some studies have reported improvements from decompostition, while others have observed performance declines, indicating its inconsistent impact. To date, no comprehensive analysis has been conducted to understand this variability. To address this gap, we present an in-depth analysis that explicitly examines the impact of decomposition on downstream verification performance. Through error case inspection and experiments, we introduce a categorization of decomposition errors and reveal a trade-off between accuracy gains and the noise introduced through decomposition. Our analysis provides new insights into understanding current system's instability and offers guidance for future studies toward improving claim decomposition in fact-checking pipelines.</description><identifier>DOI: 10.48550/arxiv.2411.02400</identifier><language>eng</language><subject>Computer Science - Artificial Intelligence ; Computer Science - Computation and Language ; Computer Science - Information Retrieval</subject><creationdate>2024-10</creationdate><rights>http://creativecommons.org/licenses/by-nc-sa/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,776,881</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2411.02400$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2411.02400$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Hu, Qisheng</creatorcontrib><creatorcontrib>Long, Quanyu</creatorcontrib><creatorcontrib>Wang, Wenya</creatorcontrib><title>Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?</title><description>Fact-checking pipelines increasingly adopt the Decompose-Then-Verify paradigm, where texts are broken down into smaller claims for individual verification and subsequently combined for a veracity decision. While decomposition is widely-adopted in such pipelines, its effects on final fact-checking performance remain underexplored. Some studies have reported improvements from decompostition, while others have observed performance declines, indicating its inconsistent impact. To date, no comprehensive analysis has been conducted to understand this variability. To address this gap, we present an in-depth analysis that explicitly examines the impact of decomposition on downstream verification performance. Through error case inspection and experiments, we introduce a categorization of decomposition errors and reveal a trade-off between accuracy gains and the noise introduced through decomposition. Our analysis provides new insights into understanding current system's instability and offers guidance for future studies toward improving claim decomposition in fact-checking pipelines.</description><subject>Computer Science - Artificial Intelligence</subject><subject>Computer Science - Computation and Language</subject><subject>Computer Science - Information Retrieval</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNpjYJA0NNAzsTA1NdBPLKrILNMzMjE01DMwMjEw4GSIdklNzs8tyC_OLMnMz1NwycxJzc1NLLZScMlPLVZwzknMzFVAVeKUn19copBfpOBUWpSSmqfglphcouuckZqcnZmXrhCQWpSWX5SbmJecas_DwJqWmFOcyguluRnk3VxDnD10wc6ILyjKzE0sqowHOSce7BxjwioAzSpAKg</recordid><startdate>20241017</startdate><enddate>20241017</enddate><creator>Hu, Qisheng</creator><creator>Long, Quanyu</creator><creator>Wang, Wenya</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20241017</creationdate><title>Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?</title><author>Hu, Qisheng ; Long, Quanyu ; Wang, Wenya</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-arxiv_primary_2411_024003</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Computer Science - Artificial Intelligence</topic><topic>Computer Science - Computation and Language</topic><topic>Computer Science - Information Retrieval</topic><toplevel>online_resources</toplevel><creatorcontrib>Hu, Qisheng</creatorcontrib><creatorcontrib>Long, Quanyu</creatorcontrib><creatorcontrib>Wang, Wenya</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Hu, Qisheng</au><au>Long, Quanyu</au><au>Wang, Wenya</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?</atitle><date>2024-10-17</date><risdate>2024</risdate><abstract>Fact-checking pipelines increasingly adopt the Decompose-Then-Verify paradigm, where texts are broken down into smaller claims for individual verification and subsequently combined for a veracity decision. While decomposition is widely-adopted in such pipelines, its effects on final fact-checking performance remain underexplored. Some studies have reported improvements from decompostition, while others have observed performance declines, indicating its inconsistent impact. To date, no comprehensive analysis has been conducted to understand this variability. To address this gap, we present an in-depth analysis that explicitly examines the impact of decomposition on downstream verification performance. Through error case inspection and experiments, we introduce a categorization of decomposition errors and reveal a trade-off between accuracy gains and the noise introduced through decomposition. Our analysis provides new insights into understanding current system's instability and offers guidance for future studies toward improving claim decomposition in fact-checking pipelines.</abstract><doi>10.48550/arxiv.2411.02400</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2411.02400
ispartof
issn
language eng
recordid cdi_arxiv_primary_2411_02400
source arXiv.org
subjects Computer Science - Artificial Intelligence
Computer Science - Computation and Language
Computer Science - Information Retrieval
title Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-22T19%3A45%3A30IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Decomposition%20Dilemmas:%20Does%20Claim%20Decomposition%20Boost%20or%20Burden%20Fact-Checking%20Performance?&rft.au=Hu,%20Qisheng&rft.date=2024-10-17&rft_id=info:doi/10.48550/arxiv.2411.02400&rft_dat=%3Carxiv_GOX%3E2411_02400%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true