SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey

We present SoFiA 2, the fully automated 3D source finding pipeline for the WALLABY extragalactic HI survey with the Australian SKA Pathfinder (ASKAP). SoFiA 2 is a reimplementation of parts of the original SoFiA pipeline in the C programming language and makes use of OpenMP for multi-threading of th...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:arXiv.org 2021-06
Hauptverfasser: Westmeier, T, Kitaeff, S, Pallot, D, Serra, P, J M van der Hulst, Jurek, R J, Elagali, A, B -Q, Kleiner, D, Koribalski, B S, Lee-Waddell, K, Mould, J R, Reynolds, T N, Rhee, J, Staveley-Smith, L
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Westmeier, T
Kitaeff, S
Pallot, D
Serra, P
J M van der Hulst
Jurek, R J
Elagali, A
B -Q
Kleiner, D
Koribalski, B S
Lee-Waddell, K
Mould, J R
Reynolds, T N
Rhee, J
Staveley-Smith, L
description We present SoFiA 2, the fully automated 3D source finding pipeline for the WALLABY extragalactic HI survey with the Australian SKA Pathfinder (ASKAP). SoFiA 2 is a reimplementation of parts of the original SoFiA pipeline in the C programming language and makes use of OpenMP for multi-threading of the most time-critical algorithms. In addition, we have developed a parallel framework called SoFiA-X that allows the processing of large data cubes to be split across multiple computing nodes. As a result of these efforts, SoFiA 2 is substantially faster and comes with a much reduced memory footprint compared to its predecessor, thus allowing the large WALLABY data volumes of hundreds of gigabytes of imaging data per epoch to be processed in real-time. The source code has been made publicly available to the entire community under an open-source licence. Performance tests using mock galaxies injected into genuine ASKAP data suggest that in the absence of significant imaging artefacts SoFiA 2 is capable of achieving near-100% completeness and reliability above an integrated signal-to-noise ratio of about 5-6. We also demonstrate that SoFiA 2 generally recovers the location, integrated flux and w20 line width of galaxies with high accuracy. Other parameters, including the peak flux density and w50 line width, are more strongly biased due to the influence of the noise on the measurement. In addition, very faint galaxies below an integrated signal-to-noise ratio of about 10 may get broken up into multiple components, thus requiring a strategy to identify fragmented sources and ensure that they do not affect the integrity of any scientific analysis based on the SoFiA 2 output.
doi_str_mv 10.48550/arxiv.2106.15789
format Article
fullrecord <record><control><sourceid>proquest_arxiv</sourceid><recordid>TN_cdi_arxiv_primary_2106_15789</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2547170618</sourcerecordid><originalsourceid>FETCH-LOGICAL-a528-95ffcce4f979455d900a97ba26ef70b099fc1fbc1a9baaacc4c7a8c142132f233</originalsourceid><addsrcrecordid>eNotz1FLwzAUBeAgCI65H-CTAV9NTdKkaR7rcG5Q8GED8ancpolmdG1N2-H-vXXz6cDhcLkfQneMRiKVkj5B-PHHiDOaREyqVF-hGY9jRlLB-Q1a9P2eUsoTxaWMZ2i7bVc-wxwTgrMGwzi0Bxhs9Yg7CFDXtsbrDe7bMRiLnW8q33zizne29s1UtAEPXxa_Z3mePX_gfgxHe7pF1w7q3i7-c452q5fdck3yt9fNMssJSJ4SLZ0zxgqnlRZSVppS0KoEnlinaEm1doa50jDQJQAYI4yC1DDBWczdRJqj-8vZs7jogj9AOBV_8uIsnxYPl0UX2u_R9kOxnyDN9FPBpVBM0YSl8S_Czlod</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2547170618</pqid></control><display><type>article</type><title>SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey</title><source>arXiv.org</source><source>Free E- Journals</source><creator>Westmeier, T ; Kitaeff, S ; Pallot, D ; Serra, P ; J M van der Hulst ; Jurek, R J ; Elagali, A ; B -Q ; Kleiner, D ; Koribalski, B S ; Lee-Waddell, K ; Mould, J R ; Reynolds, T N ; Rhee, J ; Staveley-Smith, L</creator><creatorcontrib>Westmeier, T ; Kitaeff, S ; Pallot, D ; Serra, P ; J M van der Hulst ; Jurek, R J ; Elagali, A ; B -Q ; Kleiner, D ; Koribalski, B S ; Lee-Waddell, K ; Mould, J R ; Reynolds, T N ; Rhee, J ; Staveley-Smith, L</creatorcontrib><description>We present SoFiA 2, the fully automated 3D source finding pipeline for the WALLABY extragalactic HI survey with the Australian SKA Pathfinder (ASKAP). SoFiA 2 is a reimplementation of parts of the original SoFiA pipeline in the C programming language and makes use of OpenMP for multi-threading of the most time-critical algorithms. In addition, we have developed a parallel framework called SoFiA-X that allows the processing of large data cubes to be split across multiple computing nodes. As a result of these efforts, SoFiA 2 is substantially faster and comes with a much reduced memory footprint compared to its predecessor, thus allowing the large WALLABY data volumes of hundreds of gigabytes of imaging data per epoch to be processed in real-time. The source code has been made publicly available to the entire community under an open-source licence. Performance tests using mock galaxies injected into genuine ASKAP data suggest that in the absence of significant imaging artefacts SoFiA 2 is capable of achieving near-100% completeness and reliability above an integrated signal-to-noise ratio of about 5-6. We also demonstrate that SoFiA 2 generally recovers the location, integrated flux and w20 line width of galaxies with high accuracy. Other parameters, including the peak flux density and w50 line width, are more strongly biased due to the influence of the noise on the measurement. In addition, very faint galaxies below an integrated signal-to-noise ratio of about 10 may get broken up into multiple components, thus requiring a strategy to identify fragmented sources and ensure that they do not affect the integrity of any scientific analysis based on the SoFiA 2 output.</description><identifier>EISSN: 2331-8422</identifier><identifier>DOI: 10.48550/arxiv.2106.15789</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Algorithms ; Automation ; C (programming language) ; Cubes ; Flux density ; Galaxies ; Noise measurement ; Performance tests ; Physics - Instrumentation and Methods for Astrophysics ; Pipelines ; Signal to noise ratio ; Source code</subject><ispartof>arXiv.org, 2021-06</ispartof><rights>2021. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,776,780,881,27904</link.rule.ids><backlink>$$Uhttps://doi.org/10.1093/mnras/stab1881$$DView published paper (Access to full text may be restricted)$$Hfree_for_read</backlink><backlink>$$Uhttps://doi.org/10.48550/arXiv.2106.15789$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Westmeier, T</creatorcontrib><creatorcontrib>Kitaeff, S</creatorcontrib><creatorcontrib>Pallot, D</creatorcontrib><creatorcontrib>Serra, P</creatorcontrib><creatorcontrib>J M van der Hulst</creatorcontrib><creatorcontrib>Jurek, R J</creatorcontrib><creatorcontrib>Elagali, A</creatorcontrib><creatorcontrib>B -Q</creatorcontrib><creatorcontrib>Kleiner, D</creatorcontrib><creatorcontrib>Koribalski, B S</creatorcontrib><creatorcontrib>Lee-Waddell, K</creatorcontrib><creatorcontrib>Mould, J R</creatorcontrib><creatorcontrib>Reynolds, T N</creatorcontrib><creatorcontrib>Rhee, J</creatorcontrib><creatorcontrib>Staveley-Smith, L</creatorcontrib><title>SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey</title><title>arXiv.org</title><description>We present SoFiA 2, the fully automated 3D source finding pipeline for the WALLABY extragalactic HI survey with the Australian SKA Pathfinder (ASKAP). SoFiA 2 is a reimplementation of parts of the original SoFiA pipeline in the C programming language and makes use of OpenMP for multi-threading of the most time-critical algorithms. In addition, we have developed a parallel framework called SoFiA-X that allows the processing of large data cubes to be split across multiple computing nodes. As a result of these efforts, SoFiA 2 is substantially faster and comes with a much reduced memory footprint compared to its predecessor, thus allowing the large WALLABY data volumes of hundreds of gigabytes of imaging data per epoch to be processed in real-time. The source code has been made publicly available to the entire community under an open-source licence. Performance tests using mock galaxies injected into genuine ASKAP data suggest that in the absence of significant imaging artefacts SoFiA 2 is capable of achieving near-100% completeness and reliability above an integrated signal-to-noise ratio of about 5-6. We also demonstrate that SoFiA 2 generally recovers the location, integrated flux and w20 line width of galaxies with high accuracy. Other parameters, including the peak flux density and w50 line width, are more strongly biased due to the influence of the noise on the measurement. In addition, very faint galaxies below an integrated signal-to-noise ratio of about 10 may get broken up into multiple components, thus requiring a strategy to identify fragmented sources and ensure that they do not affect the integrity of any scientific analysis based on the SoFiA 2 output.</description><subject>Algorithms</subject><subject>Automation</subject><subject>C (programming language)</subject><subject>Cubes</subject><subject>Flux density</subject><subject>Galaxies</subject><subject>Noise measurement</subject><subject>Performance tests</subject><subject>Physics - Instrumentation and Methods for Astrophysics</subject><subject>Pipelines</subject><subject>Signal to noise ratio</subject><subject>Source code</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GOX</sourceid><recordid>eNotz1FLwzAUBeAgCI65H-CTAV9NTdKkaR7rcG5Q8GED8ancpolmdG1N2-H-vXXz6cDhcLkfQneMRiKVkj5B-PHHiDOaREyqVF-hGY9jRlLB-Q1a9P2eUsoTxaWMZ2i7bVc-wxwTgrMGwzi0Bxhs9Yg7CFDXtsbrDe7bMRiLnW8q33zizne29s1UtAEPXxa_Z3mePX_gfgxHe7pF1w7q3i7-c452q5fdck3yt9fNMssJSJ4SLZ0zxgqnlRZSVppS0KoEnlinaEm1doa50jDQJQAYI4yC1DDBWczdRJqj-8vZs7jogj9AOBV_8uIsnxYPl0UX2u_R9kOxnyDN9FPBpVBM0YSl8S_Czlod</recordid><startdate>20210630</startdate><enddate>20210630</enddate><creator>Westmeier, T</creator><creator>Kitaeff, S</creator><creator>Pallot, D</creator><creator>Serra, P</creator><creator>J M van der Hulst</creator><creator>Jurek, R J</creator><creator>Elagali, A</creator><creator>B -Q</creator><creator>Kleiner, D</creator><creator>Koribalski, B S</creator><creator>Lee-Waddell, K</creator><creator>Mould, J R</creator><creator>Reynolds, T N</creator><creator>Rhee, J</creator><creator>Staveley-Smith, L</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope><scope>GOX</scope></search><sort><creationdate>20210630</creationdate><title>SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey</title><author>Westmeier, T ; Kitaeff, S ; Pallot, D ; Serra, P ; J M van der Hulst ; Jurek, R J ; Elagali, A ; B -Q ; Kleiner, D ; Koribalski, B S ; Lee-Waddell, K ; Mould, J R ; Reynolds, T N ; Rhee, J ; Staveley-Smith, L</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a528-95ffcce4f979455d900a97ba26ef70b099fc1fbc1a9baaacc4c7a8c142132f233</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Algorithms</topic><topic>Automation</topic><topic>C (programming language)</topic><topic>Cubes</topic><topic>Flux density</topic><topic>Galaxies</topic><topic>Noise measurement</topic><topic>Performance tests</topic><topic>Physics - Instrumentation and Methods for Astrophysics</topic><topic>Pipelines</topic><topic>Signal to noise ratio</topic><topic>Source code</topic><toplevel>online_resources</toplevel><creatorcontrib>Westmeier, T</creatorcontrib><creatorcontrib>Kitaeff, S</creatorcontrib><creatorcontrib>Pallot, D</creatorcontrib><creatorcontrib>Serra, P</creatorcontrib><creatorcontrib>J M van der Hulst</creatorcontrib><creatorcontrib>Jurek, R J</creatorcontrib><creatorcontrib>Elagali, A</creatorcontrib><creatorcontrib>B -Q</creatorcontrib><creatorcontrib>Kleiner, D</creatorcontrib><creatorcontrib>Koribalski, B S</creatorcontrib><creatorcontrib>Lee-Waddell, K</creatorcontrib><creatorcontrib>Mould, J R</creatorcontrib><creatorcontrib>Reynolds, T N</creatorcontrib><creatorcontrib>Rhee, J</creatorcontrib><creatorcontrib>Staveley-Smith, L</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection><collection>arXiv.org</collection><jtitle>arXiv.org</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Westmeier, T</au><au>Kitaeff, S</au><au>Pallot, D</au><au>Serra, P</au><au>J M van der Hulst</au><au>Jurek, R J</au><au>Elagali, A</au><au>B -Q</au><au>Kleiner, D</au><au>Koribalski, B S</au><au>Lee-Waddell, K</au><au>Mould, J R</au><au>Reynolds, T N</au><au>Rhee, J</au><au>Staveley-Smith, L</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey</atitle><jtitle>arXiv.org</jtitle><date>2021-06-30</date><risdate>2021</risdate><eissn>2331-8422</eissn><abstract>We present SoFiA 2, the fully automated 3D source finding pipeline for the WALLABY extragalactic HI survey with the Australian SKA Pathfinder (ASKAP). SoFiA 2 is a reimplementation of parts of the original SoFiA pipeline in the C programming language and makes use of OpenMP for multi-threading of the most time-critical algorithms. In addition, we have developed a parallel framework called SoFiA-X that allows the processing of large data cubes to be split across multiple computing nodes. As a result of these efforts, SoFiA 2 is substantially faster and comes with a much reduced memory footprint compared to its predecessor, thus allowing the large WALLABY data volumes of hundreds of gigabytes of imaging data per epoch to be processed in real-time. The source code has been made publicly available to the entire community under an open-source licence. Performance tests using mock galaxies injected into genuine ASKAP data suggest that in the absence of significant imaging artefacts SoFiA 2 is capable of achieving near-100% completeness and reliability above an integrated signal-to-noise ratio of about 5-6. We also demonstrate that SoFiA 2 generally recovers the location, integrated flux and w20 line width of galaxies with high accuracy. Other parameters, including the peak flux density and w50 line width, are more strongly biased due to the influence of the noise on the measurement. In addition, very faint galaxies below an integrated signal-to-noise ratio of about 10 may get broken up into multiple components, thus requiring a strategy to identify fragmented sources and ensure that they do not affect the integrity of any scientific analysis based on the SoFiA 2 output.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><doi>10.48550/arxiv.2106.15789</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2021-06
issn 2331-8422
language eng
recordid cdi_arxiv_primary_2106_15789
source arXiv.org; Free E- Journals
subjects Algorithms
Automation
C (programming language)
Cubes
Flux density
Galaxies
Noise measurement
Performance tests
Physics - Instrumentation and Methods for Astrophysics
Pipelines
Signal to noise ratio
Source code
title SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-25T21%3A20%3A00IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_arxiv&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=SoFiA%202%20--%20An%20automated,%20parallel%20HI%20source%20finding%20pipeline%20for%20the%20WALLABY%20survey&rft.jtitle=arXiv.org&rft.au=Westmeier,%20T&rft.date=2021-06-30&rft.eissn=2331-8422&rft_id=info:doi/10.48550/arxiv.2106.15789&rft_dat=%3Cproquest_arxiv%3E2547170618%3C/proquest_arxiv%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2547170618&rft_id=info:pmid/&rfr_iscdi=true