Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph
In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To...
Gespeichert in:
Veröffentlicht in: | IEEE access 2024, Vol.12, p.136502-136514 |
---|---|
Hauptverfasser: | , , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 136514 |
---|---|
container_issue | |
container_start_page | 136502 |
container_title | IEEE access |
container_volume | 12 |
creator | Li, Tingpeng Wang, Lei Peng, Danhua Liao, Jun Liu, Li Liu, Zhendong |
description | In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To address this, we propose a novel method for evaluating causal discovery algorithms without needing real causal graphs. Specifically, our method employs deep learning evaluation strategies and ensemble learning techniques to robustly assess the performance of causal discovery methods. To elaborate, our approach emulates deep learning validation strategies by dividing the data into training and testing sets. We perform causal discovery on the training set and subsequently use the testing set to conduct Markov blanket tests on the node set and causal direction determination on the edge set. Moreover, we employ multiple ensemble strategies to ensure a comprehensive evaluation of the algorithms. Furthermore, experiments on both synthetic and real datasets demonstrate our method's effectiveness in accurately and comprehensively validating causal discovery algorithms. Our results show that our proposed method can reflect the performance of causal discovery methods in practice with reasonable error. |
doi_str_mv | 10.1109/ACCESS.2024.3456233 |
format | Article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1109_ACCESS_2024_3456233</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10669554</ieee_id><doaj_id>oai_doaj_org_article_0d9c60d322634aca864dcaa6e72f88f9</doaj_id><sourcerecordid>3112224369</sourcerecordid><originalsourceid>FETCH-LOGICAL-c289t-f851d3992c8133b8fb17ab1640c753bfa87d8edd2bd30aec6238131735054a393</originalsourceid><addsrcrecordid>eNpNkVFLwzAUhYsoOOZ-gT4EfO5Mcts0fRx1m4Igoj6H2yR1nVszk3ayf29nh5iXhMM5597wRdE1o1PGaH43K4r56-uUU55MIUkFBziLRpyJPIYUxPm_92U0CWFN-yN7Kc1G0UuBXcANua-DdnvrD2S-x02Hbe0asvC4td_Of5K6Ie3KklkZbKMtcRVZetc1Jn7zXbsip5Klx93qKrqocBPs5HSPo_fF_K14iJ-el4_F7CnWXOZtXMmUGchzriUDKGVVsgxLJhKqsxTKCmVmpDWGlwYoWt1_qzeyDFKaJgg5jKPHodc4XKudr7foD8phrX4F5z8U-rbWG6uoybWgBjgXkKBGKRKjEYXNeCVldey6Hbp23n11NrRq7Trf9OsrYIxznoA4umBwae9C8Lb6m8qoOqJQAwp1RKFOKPrUzZCqrbX_EqIHkCbwA5rOg6o</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3112224369</pqid></control><display><type>article</type><title>Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph</title><source>DOAJ Directory of Open Access Journals</source><source>EZB-FREE-00999 freely available EZB journals</source><source>IEEE Xplore Open Access Journals</source><creator>Li, Tingpeng ; Wang, Lei ; Peng, Danhua ; Liao, Jun ; Liu, Li ; Liu, Zhendong</creator><creatorcontrib>Li, Tingpeng ; Wang, Lei ; Peng, Danhua ; Liao, Jun ; Liu, Li ; Liu, Zhendong</creatorcontrib><description>In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To address this, we propose a novel method for evaluating causal discovery algorithms without needing real causal graphs. Specifically, our method employs deep learning evaluation strategies and ensemble learning techniques to robustly assess the performance of causal discovery methods. To elaborate, our approach emulates deep learning validation strategies by dividing the data into training and testing sets. We perform causal discovery on the training set and subsequently use the testing set to conduct Markov blanket tests on the node set and causal direction determination on the edge set. Moreover, we employ multiple ensemble strategies to ensure a comprehensive evaluation of the algorithms. Furthermore, experiments on both synthetic and real datasets demonstrate our method's effectiveness in accurately and comprehensively validating causal discovery algorithms. Our results show that our proposed method can reflect the performance of causal discovery methods in practice with reasonable error.</description><identifier>ISSN: 2169-3536</identifier><identifier>EISSN: 2169-3536</identifier><identifier>DOI: 10.1109/ACCESS.2024.3456233</identifier><identifier>CODEN: IAECCG</identifier><language>eng</language><publisher>Piscataway: IEEE</publisher><subject>Accuracy ; Algorithms ; Bayes methods ; Benchmark testing ; Biological system modeling ; Causal discovery ; causal graphical models ; cause effect identification ; condition independence testing ; Correlation ; Datasets ; Deep learning ; Ensemble learning ; Machine learning ; Markov blanket ; Performance evaluation ; Synthetic data ; Training</subject><ispartof>IEEE access, 2024, Vol.12, p.136502-136514</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c289t-f851d3992c8133b8fb17ab1640c753bfa87d8edd2bd30aec6238131735054a393</cites><orcidid>0000-0002-0865-3079 ; 0000-0002-4776-5292 ; 0009-0009-4538-3387</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10669554$$EHTML$$P50$$Gieee$$Hfree_for_read</linktohtml><link.rule.ids>315,781,785,865,2103,4025,27637,27927,27928,27929,54937</link.rule.ids></links><search><creatorcontrib>Li, Tingpeng</creatorcontrib><creatorcontrib>Wang, Lei</creatorcontrib><creatorcontrib>Peng, Danhua</creatorcontrib><creatorcontrib>Liao, Jun</creatorcontrib><creatorcontrib>Liu, Li</creatorcontrib><creatorcontrib>Liu, Zhendong</creatorcontrib><title>Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph</title><title>IEEE access</title><addtitle>Access</addtitle><description>In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To address this, we propose a novel method for evaluating causal discovery algorithms without needing real causal graphs. Specifically, our method employs deep learning evaluation strategies and ensemble learning techniques to robustly assess the performance of causal discovery methods. To elaborate, our approach emulates deep learning validation strategies by dividing the data into training and testing sets. We perform causal discovery on the training set and subsequently use the testing set to conduct Markov blanket tests on the node set and causal direction determination on the edge set. Moreover, we employ multiple ensemble strategies to ensure a comprehensive evaluation of the algorithms. Furthermore, experiments on both synthetic and real datasets demonstrate our method's effectiveness in accurately and comprehensively validating causal discovery algorithms. Our results show that our proposed method can reflect the performance of causal discovery methods in practice with reasonable error.</description><subject>Accuracy</subject><subject>Algorithms</subject><subject>Bayes methods</subject><subject>Benchmark testing</subject><subject>Biological system modeling</subject><subject>Causal discovery</subject><subject>causal graphical models</subject><subject>cause effect identification</subject><subject>condition independence testing</subject><subject>Correlation</subject><subject>Datasets</subject><subject>Deep learning</subject><subject>Ensemble learning</subject><subject>Machine learning</subject><subject>Markov blanket</subject><subject>Performance evaluation</subject><subject>Synthetic data</subject><subject>Training</subject><issn>2169-3536</issn><issn>2169-3536</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>ESBDL</sourceid><sourceid>RIE</sourceid><sourceid>DOA</sourceid><recordid>eNpNkVFLwzAUhYsoOOZ-gT4EfO5Mcts0fRx1m4Igoj6H2yR1nVszk3ayf29nh5iXhMM5597wRdE1o1PGaH43K4r56-uUU55MIUkFBziLRpyJPIYUxPm_92U0CWFN-yN7Kc1G0UuBXcANua-DdnvrD2S-x02Hbe0asvC4td_Of5K6Ie3KklkZbKMtcRVZetc1Jn7zXbsip5Klx93qKrqocBPs5HSPo_fF_K14iJ-el4_F7CnWXOZtXMmUGchzriUDKGVVsgxLJhKqsxTKCmVmpDWGlwYoWt1_qzeyDFKaJgg5jKPHodc4XKudr7foD8phrX4F5z8U-rbWG6uoybWgBjgXkKBGKRKjEYXNeCVldey6Hbp23n11NrRq7Trf9OsrYIxznoA4umBwae9C8Lb6m8qoOqJQAwp1RKFOKPrUzZCqrbX_EqIHkCbwA5rOg6o</recordid><startdate>2024</startdate><enddate>2024</enddate><creator>Li, Tingpeng</creator><creator>Wang, Lei</creator><creator>Peng, Danhua</creator><creator>Liao, Jun</creator><creator>Liu, Li</creator><creator>Liu, Zhendong</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>ESBDL</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>7SR</scope><scope>8BQ</scope><scope>8FD</scope><scope>JG9</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0002-0865-3079</orcidid><orcidid>https://orcid.org/0000-0002-4776-5292</orcidid><orcidid>https://orcid.org/0009-0009-4538-3387</orcidid></search><sort><creationdate>2024</creationdate><title>Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph</title><author>Li, Tingpeng ; Wang, Lei ; Peng, Danhua ; Liao, Jun ; Liu, Li ; Liu, Zhendong</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c289t-f851d3992c8133b8fb17ab1640c753bfa87d8edd2bd30aec6238131735054a393</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Accuracy</topic><topic>Algorithms</topic><topic>Bayes methods</topic><topic>Benchmark testing</topic><topic>Biological system modeling</topic><topic>Causal discovery</topic><topic>causal graphical models</topic><topic>cause effect identification</topic><topic>condition independence testing</topic><topic>Correlation</topic><topic>Datasets</topic><topic>Deep learning</topic><topic>Ensemble learning</topic><topic>Machine learning</topic><topic>Markov blanket</topic><topic>Performance evaluation</topic><topic>Synthetic data</topic><topic>Training</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Li, Tingpeng</creatorcontrib><creatorcontrib>Wang, Lei</creatorcontrib><creatorcontrib>Peng, Danhua</creatorcontrib><creatorcontrib>Liao, Jun</creatorcontrib><creatorcontrib>Liu, Li</creatorcontrib><creatorcontrib>Liu, Zhendong</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE Xplore Open Access Journals</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Engineered Materials Abstracts</collection><collection>METADEX</collection><collection>Technology Research Database</collection><collection>Materials Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>IEEE access</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Li, Tingpeng</au><au>Wang, Lei</au><au>Peng, Danhua</au><au>Liao, Jun</au><au>Liu, Li</au><au>Liu, Zhendong</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph</atitle><jtitle>IEEE access</jtitle><stitle>Access</stitle><date>2024</date><risdate>2024</risdate><volume>12</volume><spage>136502</spage><epage>136514</epage><pages>136502-136514</pages><issn>2169-3536</issn><eissn>2169-3536</eissn><coden>IAECCG</coden><abstract>In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To address this, we propose a novel method for evaluating causal discovery algorithms without needing real causal graphs. Specifically, our method employs deep learning evaluation strategies and ensemble learning techniques to robustly assess the performance of causal discovery methods. To elaborate, our approach emulates deep learning validation strategies by dividing the data into training and testing sets. We perform causal discovery on the training set and subsequently use the testing set to conduct Markov blanket tests on the node set and causal direction determination on the edge set. Moreover, we employ multiple ensemble strategies to ensure a comprehensive evaluation of the algorithms. Furthermore, experiments on both synthetic and real datasets demonstrate our method's effectiveness in accurately and comprehensively validating causal discovery algorithms. Our results show that our proposed method can reflect the performance of causal discovery methods in practice with reasonable error.</abstract><cop>Piscataway</cop><pub>IEEE</pub><doi>10.1109/ACCESS.2024.3456233</doi><tpages>13</tpages><orcidid>https://orcid.org/0000-0002-0865-3079</orcidid><orcidid>https://orcid.org/0000-0002-4776-5292</orcidid><orcidid>https://orcid.org/0009-0009-4538-3387</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 2169-3536 |
ispartof | IEEE access, 2024, Vol.12, p.136502-136514 |
issn | 2169-3536 2169-3536 |
language | eng |
recordid | cdi_crossref_primary_10_1109_ACCESS_2024_3456233 |
source | DOAJ Directory of Open Access Journals; EZB-FREE-00999 freely available EZB journals; IEEE Xplore Open Access Journals |
subjects | Accuracy Algorithms Bayes methods Benchmark testing Biological system modeling Causal discovery causal graphical models cause effect identification condition independence testing Correlation Datasets Deep learning Ensemble learning Machine learning Markov blanket Performance evaluation Synthetic data Training |
title | Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-16T14%3A54%3A29IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Causal%20Discovery%20Evaluation%20Framework%20in%20the%20Absence%20of%20Ground-Truth%20Causal%20Graph&rft.jtitle=IEEE%20access&rft.au=Li,%20Tingpeng&rft.date=2024&rft.volume=12&rft.spage=136502&rft.epage=136514&rft.pages=136502-136514&rft.issn=2169-3536&rft.eissn=2169-3536&rft.coden=IAECCG&rft_id=info:doi/10.1109/ACCESS.2024.3456233&rft_dat=%3Cproquest_cross%3E3112224369%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3112224369&rft_id=info:pmid/&rft_ieee_id=10669554&rft_doaj_id=oai_doaj_org_article_0d9c60d322634aca864dcaa6e72f88f9&rfr_iscdi=true |