Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph

In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE access 2024, Vol.12, p.136502-136514
Hauptverfasser: Li, Tingpeng, Wang, Lei, Peng, Danhua, Liao, Jun, Liu, Li, Liu, Zhendong
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 136514
container_issue
container_start_page 136502
container_title IEEE access
container_volume 12
creator Li, Tingpeng
Wang, Lei
Peng, Danhua
Liao, Jun
Liu, Li
Liu, Zhendong
description In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To address this, we propose a novel method for evaluating causal discovery algorithms without needing real causal graphs. Specifically, our method employs deep learning evaluation strategies and ensemble learning techniques to robustly assess the performance of causal discovery methods. To elaborate, our approach emulates deep learning validation strategies by dividing the data into training and testing sets. We perform causal discovery on the training set and subsequently use the testing set to conduct Markov blanket tests on the node set and causal direction determination on the edge set. Moreover, we employ multiple ensemble strategies to ensure a comprehensive evaluation of the algorithms. Furthermore, experiments on both synthetic and real datasets demonstrate our method's effectiveness in accurately and comprehensively validating causal discovery algorithms. Our results show that our proposed method can reflect the performance of causal discovery methods in practice with reasonable error.
doi_str_mv 10.1109/ACCESS.2024.3456233
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1109_ACCESS_2024_3456233</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10669554</ieee_id><doaj_id>oai_doaj_org_article_0d9c60d322634aca864dcaa6e72f88f9</doaj_id><sourcerecordid>3112224369</sourcerecordid><originalsourceid>FETCH-LOGICAL-c289t-f851d3992c8133b8fb17ab1640c753bfa87d8edd2bd30aec6238131735054a393</originalsourceid><addsrcrecordid>eNpNkVFLwzAUhYsoOOZ-gT4EfO5Mcts0fRx1m4Igoj6H2yR1nVszk3ayf29nh5iXhMM5597wRdE1o1PGaH43K4r56-uUU55MIUkFBziLRpyJPIYUxPm_92U0CWFN-yN7Kc1G0UuBXcANua-DdnvrD2S-x02Hbe0asvC4td_Of5K6Ie3KklkZbKMtcRVZetc1Jn7zXbsip5Klx93qKrqocBPs5HSPo_fF_K14iJ-el4_F7CnWXOZtXMmUGchzriUDKGVVsgxLJhKqsxTKCmVmpDWGlwYoWt1_qzeyDFKaJgg5jKPHodc4XKudr7foD8phrX4F5z8U-rbWG6uoybWgBjgXkKBGKRKjEYXNeCVldey6Hbp23n11NrRq7Trf9OsrYIxznoA4umBwae9C8Lb6m8qoOqJQAwp1RKFOKPrUzZCqrbX_EqIHkCbwA5rOg6o</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3112224369</pqid></control><display><type>article</type><title>Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph</title><source>DOAJ Directory of Open Access Journals</source><source>EZB-FREE-00999 freely available EZB journals</source><source>IEEE Xplore Open Access Journals</source><creator>Li, Tingpeng ; Wang, Lei ; Peng, Danhua ; Liao, Jun ; Liu, Li ; Liu, Zhendong</creator><creatorcontrib>Li, Tingpeng ; Wang, Lei ; Peng, Danhua ; Liao, Jun ; Liu, Li ; Liu, Zhendong</creatorcontrib><description>In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To address this, we propose a novel method for evaluating causal discovery algorithms without needing real causal graphs. Specifically, our method employs deep learning evaluation strategies and ensemble learning techniques to robustly assess the performance of causal discovery methods. To elaborate, our approach emulates deep learning validation strategies by dividing the data into training and testing sets. We perform causal discovery on the training set and subsequently use the testing set to conduct Markov blanket tests on the node set and causal direction determination on the edge set. Moreover, we employ multiple ensemble strategies to ensure a comprehensive evaluation of the algorithms. Furthermore, experiments on both synthetic and real datasets demonstrate our method's effectiveness in accurately and comprehensively validating causal discovery algorithms. Our results show that our proposed method can reflect the performance of causal discovery methods in practice with reasonable error.</description><identifier>ISSN: 2169-3536</identifier><identifier>EISSN: 2169-3536</identifier><identifier>DOI: 10.1109/ACCESS.2024.3456233</identifier><identifier>CODEN: IAECCG</identifier><language>eng</language><publisher>Piscataway: IEEE</publisher><subject>Accuracy ; Algorithms ; Bayes methods ; Benchmark testing ; Biological system modeling ; Causal discovery ; causal graphical models ; cause effect identification ; condition independence testing ; Correlation ; Datasets ; Deep learning ; Ensemble learning ; Machine learning ; Markov blanket ; Performance evaluation ; Synthetic data ; Training</subject><ispartof>IEEE access, 2024, Vol.12, p.136502-136514</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c289t-f851d3992c8133b8fb17ab1640c753bfa87d8edd2bd30aec6238131735054a393</cites><orcidid>0000-0002-0865-3079 ; 0000-0002-4776-5292 ; 0009-0009-4538-3387</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10669554$$EHTML$$P50$$Gieee$$Hfree_for_read</linktohtml><link.rule.ids>315,781,785,865,2103,4025,27637,27927,27928,27929,54937</link.rule.ids></links><search><creatorcontrib>Li, Tingpeng</creatorcontrib><creatorcontrib>Wang, Lei</creatorcontrib><creatorcontrib>Peng, Danhua</creatorcontrib><creatorcontrib>Liao, Jun</creatorcontrib><creatorcontrib>Liu, Li</creatorcontrib><creatorcontrib>Liu, Zhendong</creatorcontrib><title>Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph</title><title>IEEE access</title><addtitle>Access</addtitle><description>In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To address this, we propose a novel method for evaluating causal discovery algorithms without needing real causal graphs. Specifically, our method employs deep learning evaluation strategies and ensemble learning techniques to robustly assess the performance of causal discovery methods. To elaborate, our approach emulates deep learning validation strategies by dividing the data into training and testing sets. We perform causal discovery on the training set and subsequently use the testing set to conduct Markov blanket tests on the node set and causal direction determination on the edge set. Moreover, we employ multiple ensemble strategies to ensure a comprehensive evaluation of the algorithms. Furthermore, experiments on both synthetic and real datasets demonstrate our method's effectiveness in accurately and comprehensively validating causal discovery algorithms. Our results show that our proposed method can reflect the performance of causal discovery methods in practice with reasonable error.</description><subject>Accuracy</subject><subject>Algorithms</subject><subject>Bayes methods</subject><subject>Benchmark testing</subject><subject>Biological system modeling</subject><subject>Causal discovery</subject><subject>causal graphical models</subject><subject>cause effect identification</subject><subject>condition independence testing</subject><subject>Correlation</subject><subject>Datasets</subject><subject>Deep learning</subject><subject>Ensemble learning</subject><subject>Machine learning</subject><subject>Markov blanket</subject><subject>Performance evaluation</subject><subject>Synthetic data</subject><subject>Training</subject><issn>2169-3536</issn><issn>2169-3536</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>ESBDL</sourceid><sourceid>RIE</sourceid><sourceid>DOA</sourceid><recordid>eNpNkVFLwzAUhYsoOOZ-gT4EfO5Mcts0fRx1m4Igoj6H2yR1nVszk3ayf29nh5iXhMM5597wRdE1o1PGaH43K4r56-uUU55MIUkFBziLRpyJPIYUxPm_92U0CWFN-yN7Kc1G0UuBXcANua-DdnvrD2S-x02Hbe0asvC4td_Of5K6Ie3KklkZbKMtcRVZetc1Jn7zXbsip5Klx93qKrqocBPs5HSPo_fF_K14iJ-el4_F7CnWXOZtXMmUGchzriUDKGVVsgxLJhKqsxTKCmVmpDWGlwYoWt1_qzeyDFKaJgg5jKPHodc4XKudr7foD8phrX4F5z8U-rbWG6uoybWgBjgXkKBGKRKjEYXNeCVldey6Hbp23n11NrRq7Trf9OsrYIxznoA4umBwae9C8Lb6m8qoOqJQAwp1RKFOKPrUzZCqrbX_EqIHkCbwA5rOg6o</recordid><startdate>2024</startdate><enddate>2024</enddate><creator>Li, Tingpeng</creator><creator>Wang, Lei</creator><creator>Peng, Danhua</creator><creator>Liao, Jun</creator><creator>Liu, Li</creator><creator>Liu, Zhendong</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>ESBDL</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>7SR</scope><scope>8BQ</scope><scope>8FD</scope><scope>JG9</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0002-0865-3079</orcidid><orcidid>https://orcid.org/0000-0002-4776-5292</orcidid><orcidid>https://orcid.org/0009-0009-4538-3387</orcidid></search><sort><creationdate>2024</creationdate><title>Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph</title><author>Li, Tingpeng ; Wang, Lei ; Peng, Danhua ; Liao, Jun ; Liu, Li ; Liu, Zhendong</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c289t-f851d3992c8133b8fb17ab1640c753bfa87d8edd2bd30aec6238131735054a393</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Accuracy</topic><topic>Algorithms</topic><topic>Bayes methods</topic><topic>Benchmark testing</topic><topic>Biological system modeling</topic><topic>Causal discovery</topic><topic>causal graphical models</topic><topic>cause effect identification</topic><topic>condition independence testing</topic><topic>Correlation</topic><topic>Datasets</topic><topic>Deep learning</topic><topic>Ensemble learning</topic><topic>Machine learning</topic><topic>Markov blanket</topic><topic>Performance evaluation</topic><topic>Synthetic data</topic><topic>Training</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Li, Tingpeng</creatorcontrib><creatorcontrib>Wang, Lei</creatorcontrib><creatorcontrib>Peng, Danhua</creatorcontrib><creatorcontrib>Liao, Jun</creatorcontrib><creatorcontrib>Liu, Li</creatorcontrib><creatorcontrib>Liu, Zhendong</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE Xplore Open Access Journals</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Engineered Materials Abstracts</collection><collection>METADEX</collection><collection>Technology Research Database</collection><collection>Materials Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>IEEE access</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Li, Tingpeng</au><au>Wang, Lei</au><au>Peng, Danhua</au><au>Liao, Jun</au><au>Liu, Li</au><au>Liu, Zhendong</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph</atitle><jtitle>IEEE access</jtitle><stitle>Access</stitle><date>2024</date><risdate>2024</risdate><volume>12</volume><spage>136502</spage><epage>136514</epage><pages>136502-136514</pages><issn>2169-3536</issn><eissn>2169-3536</eissn><coden>IAECCG</coden><abstract>In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To address this, we propose a novel method for evaluating causal discovery algorithms without needing real causal graphs. Specifically, our method employs deep learning evaluation strategies and ensemble learning techniques to robustly assess the performance of causal discovery methods. To elaborate, our approach emulates deep learning validation strategies by dividing the data into training and testing sets. We perform causal discovery on the training set and subsequently use the testing set to conduct Markov blanket tests on the node set and causal direction determination on the edge set. Moreover, we employ multiple ensemble strategies to ensure a comprehensive evaluation of the algorithms. Furthermore, experiments on both synthetic and real datasets demonstrate our method's effectiveness in accurately and comprehensively validating causal discovery algorithms. Our results show that our proposed method can reflect the performance of causal discovery methods in practice with reasonable error.</abstract><cop>Piscataway</cop><pub>IEEE</pub><doi>10.1109/ACCESS.2024.3456233</doi><tpages>13</tpages><orcidid>https://orcid.org/0000-0002-0865-3079</orcidid><orcidid>https://orcid.org/0000-0002-4776-5292</orcidid><orcidid>https://orcid.org/0009-0009-4538-3387</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 2169-3536
ispartof IEEE access, 2024, Vol.12, p.136502-136514
issn 2169-3536
2169-3536
language eng
recordid cdi_crossref_primary_10_1109_ACCESS_2024_3456233
source DOAJ Directory of Open Access Journals; EZB-FREE-00999 freely available EZB journals; IEEE Xplore Open Access Journals
subjects Accuracy
Algorithms
Bayes methods
Benchmark testing
Biological system modeling
Causal discovery
causal graphical models
cause effect identification
condition independence testing
Correlation
Datasets
Deep learning
Ensemble learning
Machine learning
Markov blanket
Performance evaluation
Synthetic data
Training
title Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-16T14%3A54%3A29IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Causal%20Discovery%20Evaluation%20Framework%20in%20the%20Absence%20of%20Ground-Truth%20Causal%20Graph&rft.jtitle=IEEE%20access&rft.au=Li,%20Tingpeng&rft.date=2024&rft.volume=12&rft.spage=136502&rft.epage=136514&rft.pages=136502-136514&rft.issn=2169-3536&rft.eissn=2169-3536&rft.coden=IAECCG&rft_id=info:doi/10.1109/ACCESS.2024.3456233&rft_dat=%3Cproquest_cross%3E3112224369%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3112224369&rft_id=info:pmid/&rft_ieee_id=10669554&rft_doaj_id=oai_doaj_org_article_0d9c60d322634aca864dcaa6e72f88f9&rfr_iscdi=true