Comparing transit spectroscopy pipelines at the catalogue level: evidence for systematic differences
ABSTRACT The challenge of inconsistent results from different data pipelines, even when starting from identical data, is a recognized concern in exoplanetary science. As we transition into the JWST era and prepare for the ARIEL space mission, addressing this issue becomes paramount because of its im...
Gespeichert in:
Veröffentlicht in: | Monthly notices of the Royal Astronomical Society 2024-06, Vol.531 (1), p.35-51 |
---|---|
Hauptverfasser: | , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 51 |
---|---|
container_issue | 1 |
container_start_page | 35 |
container_title | Monthly notices of the Royal Astronomical Society |
container_volume | 531 |
creator | Mugnai, Lorenzo V Swain, Mark R Estrela, Raissa Roudier, Gael M |
description | ABSTRACT
The challenge of inconsistent results from different data pipelines, even when starting from identical data, is a recognized concern in exoplanetary science. As we transition into the JWST era and prepare for the ARIEL space mission, addressing this issue becomes paramount because of its implications on our understanding of exoplanets. Although comparing pipeline results for individual exoplanets has become more common, this study is the first to compare pipeline results at the catalogue level. We present a comprehensive framework to statistically compare the outcomes of data analysis reduction on a population of exoplanets and we leverage the large number of observations conducted using the same instrument configured with HST-WFC3. We employ three independent pipelines: Iraclis, excalibur, and CASCADe. Our combined findings reveal that these pipelines, despite starting from the same data and planet system parameters, yield substantially different spectra in some cases. However, the most significant manifestations of pipeline differences are observed in the compositional trends of the resulting exoplanet catalogues. We conclude that pipeline-induced differences lead to biases in the retrieved information, which are not reflected in the retrieved uncertainties. Our findings underscore the critical need to confront these pipeline differences to ensure the reproducibility, accuracy, and reliability of results in exoplanetary research. Our results demonstrate the need to understand the potential for population-level bias that pipelines may inject, which could compromise our understanding of exoplanets as a class of objects. |
doi_str_mv | 10.1093/mnras/stae1073 |
format | Article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_3133545079</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><oup_id>10.1093/mnras/stae1073</oup_id><sourcerecordid>3133545079</sourcerecordid><originalsourceid>FETCH-LOGICAL-c296t-b0a7e79fe7508c6ff202ade659bc1749c9c45bf300cf1a067f95055667f763c3</originalsourceid><addsrcrecordid>eNqFkL1PwzAQxS0EEuVjZbbExJD2HMd2w4YqvqRKLN0j1zkXV2lsbKdS_3tSCjPTnXTv3b37EXLHYMqg5rNdH3WapayRgeJnZMK4FEVZS3lOJgBcFHPF2CW5SmkLABUv5YS0C78LOrp-Q3PUfXKZpoAmR5-MDwcaXMDO9ZiozjR_IjU6685vBqQd7rF7pLh3LfYGqfWRpkPKuNPZGdo6azEeJ-mGXFjdJbz9rddk9fK8WrwVy4_X98XTsjBjylysQStUtUUlYG6ktSWUukUp6rVhqqpNbSqxthzAWKZBKlsLEEKOjZLc8Gtyf1obov8aMOVm64fYjxcbzjgXlQBVj6rpSWXGH1NE24TodjoeGgbNEWTzA7L5AzkaHk4GP4T_tN8B53jK</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3133545079</pqid></control><display><type>article</type><title>Comparing transit spectroscopy pipelines at the catalogue level: evidence for systematic differences</title><source>Oxford Journals Open Access Collection</source><source>DOAJ Directory of Open Access Journals</source><creator>Mugnai, Lorenzo V ; Swain, Mark R ; Estrela, Raissa ; Roudier, Gael M</creator><creatorcontrib>Mugnai, Lorenzo V ; Swain, Mark R ; Estrela, Raissa ; Roudier, Gael M</creatorcontrib><description>ABSTRACT
The challenge of inconsistent results from different data pipelines, even when starting from identical data, is a recognized concern in exoplanetary science. As we transition into the JWST era and prepare for the ARIEL space mission, addressing this issue becomes paramount because of its implications on our understanding of exoplanets. Although comparing pipeline results for individual exoplanets has become more common, this study is the first to compare pipeline results at the catalogue level. We present a comprehensive framework to statistically compare the outcomes of data analysis reduction on a population of exoplanets and we leverage the large number of observations conducted using the same instrument configured with HST-WFC3. We employ three independent pipelines: Iraclis, excalibur, and CASCADe. Our combined findings reveal that these pipelines, despite starting from the same data and planet system parameters, yield substantially different spectra in some cases. However, the most significant manifestations of pipeline differences are observed in the compositional trends of the resulting exoplanet catalogues. We conclude that pipeline-induced differences lead to biases in the retrieved information, which are not reflected in the retrieved uncertainties. Our findings underscore the critical need to confront these pipeline differences to ensure the reproducibility, accuracy, and reliability of results in exoplanetary research. Our results demonstrate the need to understand the potential for population-level bias that pipelines may inject, which could compromise our understanding of exoplanets as a class of objects.</description><identifier>ISSN: 0035-8711</identifier><identifier>EISSN: 1365-2966</identifier><identifier>DOI: 10.1093/mnras/stae1073</identifier><language>eng</language><publisher>London: Oxford University Press</publisher><subject>Bias ; Data analysis ; Extrasolar planets ; Information retrieval ; Pipelines ; Population (statistical) ; Space missions ; Spectrum analysis</subject><ispartof>Monthly notices of the Royal Astronomical Society, 2024-06, Vol.531 (1), p.35-51</ispartof><rights>2024 The Author(s). Published by Oxford University Press on behalf of Royal Astronomical Society. 2024</rights><rights>2024 The Author(s). Published by Oxford University Press on behalf of Royal Astronomical Society.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c296t-b0a7e79fe7508c6ff202ade659bc1749c9c45bf300cf1a067f95055667f763c3</cites><orcidid>0000-0002-9007-9802</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,860,1598,27903,27904</link.rule.ids></links><search><creatorcontrib>Mugnai, Lorenzo V</creatorcontrib><creatorcontrib>Swain, Mark R</creatorcontrib><creatorcontrib>Estrela, Raissa</creatorcontrib><creatorcontrib>Roudier, Gael M</creatorcontrib><title>Comparing transit spectroscopy pipelines at the catalogue level: evidence for systematic differences</title><title>Monthly notices of the Royal Astronomical Society</title><description>ABSTRACT
The challenge of inconsistent results from different data pipelines, even when starting from identical data, is a recognized concern in exoplanetary science. As we transition into the JWST era and prepare for the ARIEL space mission, addressing this issue becomes paramount because of its implications on our understanding of exoplanets. Although comparing pipeline results for individual exoplanets has become more common, this study is the first to compare pipeline results at the catalogue level. We present a comprehensive framework to statistically compare the outcomes of data analysis reduction on a population of exoplanets and we leverage the large number of observations conducted using the same instrument configured with HST-WFC3. We employ three independent pipelines: Iraclis, excalibur, and CASCADe. Our combined findings reveal that these pipelines, despite starting from the same data and planet system parameters, yield substantially different spectra in some cases. However, the most significant manifestations of pipeline differences are observed in the compositional trends of the resulting exoplanet catalogues. We conclude that pipeline-induced differences lead to biases in the retrieved information, which are not reflected in the retrieved uncertainties. Our findings underscore the critical need to confront these pipeline differences to ensure the reproducibility, accuracy, and reliability of results in exoplanetary research. Our results demonstrate the need to understand the potential for population-level bias that pipelines may inject, which could compromise our understanding of exoplanets as a class of objects.</description><subject>Bias</subject><subject>Data analysis</subject><subject>Extrasolar planets</subject><subject>Information retrieval</subject><subject>Pipelines</subject><subject>Population (statistical)</subject><subject>Space missions</subject><subject>Spectrum analysis</subject><issn>0035-8711</issn><issn>1365-2966</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>TOX</sourceid><recordid>eNqFkL1PwzAQxS0EEuVjZbbExJD2HMd2w4YqvqRKLN0j1zkXV2lsbKdS_3tSCjPTnXTv3b37EXLHYMqg5rNdH3WapayRgeJnZMK4FEVZS3lOJgBcFHPF2CW5SmkLABUv5YS0C78LOrp-Q3PUfXKZpoAmR5-MDwcaXMDO9ZiozjR_IjU6685vBqQd7rF7pLh3LfYGqfWRpkPKuNPZGdo6azEeJ-mGXFjdJbz9rddk9fK8WrwVy4_X98XTsjBjylysQStUtUUlYG6ktSWUukUp6rVhqqpNbSqxthzAWKZBKlsLEEKOjZLc8Gtyf1obov8aMOVm64fYjxcbzjgXlQBVj6rpSWXGH1NE24TodjoeGgbNEWTzA7L5AzkaHk4GP4T_tN8B53jK</recordid><startdate>20240601</startdate><enddate>20240601</enddate><creator>Mugnai, Lorenzo V</creator><creator>Swain, Mark R</creator><creator>Estrela, Raissa</creator><creator>Roudier, Gael M</creator><general>Oxford University Press</general><scope>TOX</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>8FD</scope><scope>H8D</scope><scope>L7M</scope><orcidid>https://orcid.org/0000-0002-9007-9802</orcidid></search><sort><creationdate>20240601</creationdate><title>Comparing transit spectroscopy pipelines at the catalogue level: evidence for systematic differences</title><author>Mugnai, Lorenzo V ; Swain, Mark R ; Estrela, Raissa ; Roudier, Gael M</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c296t-b0a7e79fe7508c6ff202ade659bc1749c9c45bf300cf1a067f95055667f763c3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Bias</topic><topic>Data analysis</topic><topic>Extrasolar planets</topic><topic>Information retrieval</topic><topic>Pipelines</topic><topic>Population (statistical)</topic><topic>Space missions</topic><topic>Spectrum analysis</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Mugnai, Lorenzo V</creatorcontrib><creatorcontrib>Swain, Mark R</creatorcontrib><creatorcontrib>Estrela, Raissa</creatorcontrib><creatorcontrib>Roudier, Gael M</creatorcontrib><collection>Oxford Journals Open Access Collection</collection><collection>CrossRef</collection><collection>Technology Research Database</collection><collection>Aerospace Database</collection><collection>Advanced Technologies Database with Aerospace</collection><jtitle>Monthly notices of the Royal Astronomical Society</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Mugnai, Lorenzo V</au><au>Swain, Mark R</au><au>Estrela, Raissa</au><au>Roudier, Gael M</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Comparing transit spectroscopy pipelines at the catalogue level: evidence for systematic differences</atitle><jtitle>Monthly notices of the Royal Astronomical Society</jtitle><date>2024-06-01</date><risdate>2024</risdate><volume>531</volume><issue>1</issue><spage>35</spage><epage>51</epage><pages>35-51</pages><issn>0035-8711</issn><eissn>1365-2966</eissn><abstract>ABSTRACT
The challenge of inconsistent results from different data pipelines, even when starting from identical data, is a recognized concern in exoplanetary science. As we transition into the JWST era and prepare for the ARIEL space mission, addressing this issue becomes paramount because of its implications on our understanding of exoplanets. Although comparing pipeline results for individual exoplanets has become more common, this study is the first to compare pipeline results at the catalogue level. We present a comprehensive framework to statistically compare the outcomes of data analysis reduction on a population of exoplanets and we leverage the large number of observations conducted using the same instrument configured with HST-WFC3. We employ three independent pipelines: Iraclis, excalibur, and CASCADe. Our combined findings reveal that these pipelines, despite starting from the same data and planet system parameters, yield substantially different spectra in some cases. However, the most significant manifestations of pipeline differences are observed in the compositional trends of the resulting exoplanet catalogues. We conclude that pipeline-induced differences lead to biases in the retrieved information, which are not reflected in the retrieved uncertainties. Our findings underscore the critical need to confront these pipeline differences to ensure the reproducibility, accuracy, and reliability of results in exoplanetary research. Our results demonstrate the need to understand the potential for population-level bias that pipelines may inject, which could compromise our understanding of exoplanets as a class of objects.</abstract><cop>London</cop><pub>Oxford University Press</pub><doi>10.1093/mnras/stae1073</doi><tpages>17</tpages><orcidid>https://orcid.org/0000-0002-9007-9802</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 0035-8711 |
ispartof | Monthly notices of the Royal Astronomical Society, 2024-06, Vol.531 (1), p.35-51 |
issn | 0035-8711 1365-2966 |
language | eng |
recordid | cdi_proquest_journals_3133545079 |
source | Oxford Journals Open Access Collection; DOAJ Directory of Open Access Journals |
subjects | Bias Data analysis Extrasolar planets Information retrieval Pipelines Population (statistical) Space missions Spectrum analysis |
title | Comparing transit spectroscopy pipelines at the catalogue level: evidence for systematic differences |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-27T05%3A57%3A35IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Comparing%20transit%20spectroscopy%20pipelines%20at%20the%20catalogue%20level:%20evidence%20for%20systematic%20differences&rft.jtitle=Monthly%20notices%20of%20the%20Royal%20Astronomical%20Society&rft.au=Mugnai,%20Lorenzo%20V&rft.date=2024-06-01&rft.volume=531&rft.issue=1&rft.spage=35&rft.epage=51&rft.pages=35-51&rft.issn=0035-8711&rft.eissn=1365-2966&rft_id=info:doi/10.1093/mnras/stae1073&rft_dat=%3Cproquest_cross%3E3133545079%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3133545079&rft_id=info:pmid/&rft_oup_id=10.1093/mnras/stae1073&rfr_iscdi=true |