SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey
We present SoFiA 2, the fully automated 3D source finding pipeline for the WALLABY extragalactic HI survey with the Australian SKA Pathfinder (ASKAP). SoFiA 2 is a reimplementation of parts of the original SoFiA pipeline in the C programming language and makes use of OpenMP for multi-threading of th...
Gespeichert in:
Veröffentlicht in: | arXiv.org 2021-06 |
---|---|
Hauptverfasser: | , , , , , , , , , , , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | |
---|---|
container_issue | |
container_start_page | |
container_title | arXiv.org |
container_volume | |
creator | Westmeier, T Kitaeff, S Pallot, D Serra, P J M van der Hulst Jurek, R J Elagali, A B -Q Kleiner, D Koribalski, B S Lee-Waddell, K Mould, J R Reynolds, T N Rhee, J Staveley-Smith, L |
description | We present SoFiA 2, the fully automated 3D source finding pipeline for the WALLABY extragalactic HI survey with the Australian SKA Pathfinder (ASKAP). SoFiA 2 is a reimplementation of parts of the original SoFiA pipeline in the C programming language and makes use of OpenMP for multi-threading of the most time-critical algorithms. In addition, we have developed a parallel framework called SoFiA-X that allows the processing of large data cubes to be split across multiple computing nodes. As a result of these efforts, SoFiA 2 is substantially faster and comes with a much reduced memory footprint compared to its predecessor, thus allowing the large WALLABY data volumes of hundreds of gigabytes of imaging data per epoch to be processed in real-time. The source code has been made publicly available to the entire community under an open-source licence. Performance tests using mock galaxies injected into genuine ASKAP data suggest that in the absence of significant imaging artefacts SoFiA 2 is capable of achieving near-100% completeness and reliability above an integrated signal-to-noise ratio of about 5-6. We also demonstrate that SoFiA 2 generally recovers the location, integrated flux and w20 line width of galaxies with high accuracy. Other parameters, including the peak flux density and w50 line width, are more strongly biased due to the influence of the noise on the measurement. In addition, very faint galaxies below an integrated signal-to-noise ratio of about 10 may get broken up into multiple components, thus requiring a strategy to identify fragmented sources and ensure that they do not affect the integrity of any scientific analysis based on the SoFiA 2 output. |
doi_str_mv | 10.48550/arxiv.2106.15789 |
format | Article |
fullrecord | <record><control><sourceid>proquest_arxiv</sourceid><recordid>TN_cdi_arxiv_primary_2106_15789</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2547170618</sourcerecordid><originalsourceid>FETCH-LOGICAL-a528-95ffcce4f979455d900a97ba26ef70b099fc1fbc1a9baaacc4c7a8c142132f233</originalsourceid><addsrcrecordid>eNotz1FLwzAUBeAgCI65H-CTAV9NTdKkaR7rcG5Q8GED8ancpolmdG1N2-H-vXXz6cDhcLkfQneMRiKVkj5B-PHHiDOaREyqVF-hGY9jRlLB-Q1a9P2eUsoTxaWMZ2i7bVc-wxwTgrMGwzi0Bxhs9Yg7CFDXtsbrDe7bMRiLnW8q33zizne29s1UtAEPXxa_Z3mePX_gfgxHe7pF1w7q3i7-c452q5fdck3yt9fNMssJSJ4SLZ0zxgqnlRZSVppS0KoEnlinaEm1doa50jDQJQAYI4yC1DDBWczdRJqj-8vZs7jogj9AOBV_8uIsnxYPl0UX2u_R9kOxnyDN9FPBpVBM0YSl8S_Czlod</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2547170618</pqid></control><display><type>article</type><title>SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey</title><source>arXiv.org</source><source>Free E- Journals</source><creator>Westmeier, T ; Kitaeff, S ; Pallot, D ; Serra, P ; J M van der Hulst ; Jurek, R J ; Elagali, A ; B -Q ; Kleiner, D ; Koribalski, B S ; Lee-Waddell, K ; Mould, J R ; Reynolds, T N ; Rhee, J ; Staveley-Smith, L</creator><creatorcontrib>Westmeier, T ; Kitaeff, S ; Pallot, D ; Serra, P ; J M van der Hulst ; Jurek, R J ; Elagali, A ; B -Q ; Kleiner, D ; Koribalski, B S ; Lee-Waddell, K ; Mould, J R ; Reynolds, T N ; Rhee, J ; Staveley-Smith, L</creatorcontrib><description>We present SoFiA 2, the fully automated 3D source finding pipeline for the WALLABY extragalactic HI survey with the Australian SKA Pathfinder (ASKAP). SoFiA 2 is a reimplementation of parts of the original SoFiA pipeline in the C programming language and makes use of OpenMP for multi-threading of the most time-critical algorithms. In addition, we have developed a parallel framework called SoFiA-X that allows the processing of large data cubes to be split across multiple computing nodes. As a result of these efforts, SoFiA 2 is substantially faster and comes with a much reduced memory footprint compared to its predecessor, thus allowing the large WALLABY data volumes of hundreds of gigabytes of imaging data per epoch to be processed in real-time. The source code has been made publicly available to the entire community under an open-source licence. Performance tests using mock galaxies injected into genuine ASKAP data suggest that in the absence of significant imaging artefacts SoFiA 2 is capable of achieving near-100% completeness and reliability above an integrated signal-to-noise ratio of about 5-6. We also demonstrate that SoFiA 2 generally recovers the location, integrated flux and w20 line width of galaxies with high accuracy. Other parameters, including the peak flux density and w50 line width, are more strongly biased due to the influence of the noise on the measurement. In addition, very faint galaxies below an integrated signal-to-noise ratio of about 10 may get broken up into multiple components, thus requiring a strategy to identify fragmented sources and ensure that they do not affect the integrity of any scientific analysis based on the SoFiA 2 output.</description><identifier>EISSN: 2331-8422</identifier><identifier>DOI: 10.48550/arxiv.2106.15789</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Algorithms ; Automation ; C (programming language) ; Cubes ; Flux density ; Galaxies ; Noise measurement ; Performance tests ; Physics - Instrumentation and Methods for Astrophysics ; Pipelines ; Signal to noise ratio ; Source code</subject><ispartof>arXiv.org, 2021-06</ispartof><rights>2021. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,776,780,881,27904</link.rule.ids><backlink>$$Uhttps://doi.org/10.1093/mnras/stab1881$$DView published paper (Access to full text may be restricted)$$Hfree_for_read</backlink><backlink>$$Uhttps://doi.org/10.48550/arXiv.2106.15789$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Westmeier, T</creatorcontrib><creatorcontrib>Kitaeff, S</creatorcontrib><creatorcontrib>Pallot, D</creatorcontrib><creatorcontrib>Serra, P</creatorcontrib><creatorcontrib>J M van der Hulst</creatorcontrib><creatorcontrib>Jurek, R J</creatorcontrib><creatorcontrib>Elagali, A</creatorcontrib><creatorcontrib>B -Q</creatorcontrib><creatorcontrib>Kleiner, D</creatorcontrib><creatorcontrib>Koribalski, B S</creatorcontrib><creatorcontrib>Lee-Waddell, K</creatorcontrib><creatorcontrib>Mould, J R</creatorcontrib><creatorcontrib>Reynolds, T N</creatorcontrib><creatorcontrib>Rhee, J</creatorcontrib><creatorcontrib>Staveley-Smith, L</creatorcontrib><title>SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey</title><title>arXiv.org</title><description>We present SoFiA 2, the fully automated 3D source finding pipeline for the WALLABY extragalactic HI survey with the Australian SKA Pathfinder (ASKAP). SoFiA 2 is a reimplementation of parts of the original SoFiA pipeline in the C programming language and makes use of OpenMP for multi-threading of the most time-critical algorithms. In addition, we have developed a parallel framework called SoFiA-X that allows the processing of large data cubes to be split across multiple computing nodes. As a result of these efforts, SoFiA 2 is substantially faster and comes with a much reduced memory footprint compared to its predecessor, thus allowing the large WALLABY data volumes of hundreds of gigabytes of imaging data per epoch to be processed in real-time. The source code has been made publicly available to the entire community under an open-source licence. Performance tests using mock galaxies injected into genuine ASKAP data suggest that in the absence of significant imaging artefacts SoFiA 2 is capable of achieving near-100% completeness and reliability above an integrated signal-to-noise ratio of about 5-6. We also demonstrate that SoFiA 2 generally recovers the location, integrated flux and w20 line width of galaxies with high accuracy. Other parameters, including the peak flux density and w50 line width, are more strongly biased due to the influence of the noise on the measurement. In addition, very faint galaxies below an integrated signal-to-noise ratio of about 10 may get broken up into multiple components, thus requiring a strategy to identify fragmented sources and ensure that they do not affect the integrity of any scientific analysis based on the SoFiA 2 output.</description><subject>Algorithms</subject><subject>Automation</subject><subject>C (programming language)</subject><subject>Cubes</subject><subject>Flux density</subject><subject>Galaxies</subject><subject>Noise measurement</subject><subject>Performance tests</subject><subject>Physics - Instrumentation and Methods for Astrophysics</subject><subject>Pipelines</subject><subject>Signal to noise ratio</subject><subject>Source code</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GOX</sourceid><recordid>eNotz1FLwzAUBeAgCI65H-CTAV9NTdKkaR7rcG5Q8GED8ancpolmdG1N2-H-vXXz6cDhcLkfQneMRiKVkj5B-PHHiDOaREyqVF-hGY9jRlLB-Q1a9P2eUsoTxaWMZ2i7bVc-wxwTgrMGwzi0Bxhs9Yg7CFDXtsbrDe7bMRiLnW8q33zizne29s1UtAEPXxa_Z3mePX_gfgxHe7pF1w7q3i7-c452q5fdck3yt9fNMssJSJ4SLZ0zxgqnlRZSVppS0KoEnlinaEm1doa50jDQJQAYI4yC1DDBWczdRJqj-8vZs7jogj9AOBV_8uIsnxYPl0UX2u_R9kOxnyDN9FPBpVBM0YSl8S_Czlod</recordid><startdate>20210630</startdate><enddate>20210630</enddate><creator>Westmeier, T</creator><creator>Kitaeff, S</creator><creator>Pallot, D</creator><creator>Serra, P</creator><creator>J M van der Hulst</creator><creator>Jurek, R J</creator><creator>Elagali, A</creator><creator>B -Q</creator><creator>Kleiner, D</creator><creator>Koribalski, B S</creator><creator>Lee-Waddell, K</creator><creator>Mould, J R</creator><creator>Reynolds, T N</creator><creator>Rhee, J</creator><creator>Staveley-Smith, L</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope><scope>GOX</scope></search><sort><creationdate>20210630</creationdate><title>SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey</title><author>Westmeier, T ; Kitaeff, S ; Pallot, D ; Serra, P ; J M van der Hulst ; Jurek, R J ; Elagali, A ; B -Q ; Kleiner, D ; Koribalski, B S ; Lee-Waddell, K ; Mould, J R ; Reynolds, T N ; Rhee, J ; Staveley-Smith, L</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a528-95ffcce4f979455d900a97ba26ef70b099fc1fbc1a9baaacc4c7a8c142132f233</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Algorithms</topic><topic>Automation</topic><topic>C (programming language)</topic><topic>Cubes</topic><topic>Flux density</topic><topic>Galaxies</topic><topic>Noise measurement</topic><topic>Performance tests</topic><topic>Physics - Instrumentation and Methods for Astrophysics</topic><topic>Pipelines</topic><topic>Signal to noise ratio</topic><topic>Source code</topic><toplevel>online_resources</toplevel><creatorcontrib>Westmeier, T</creatorcontrib><creatorcontrib>Kitaeff, S</creatorcontrib><creatorcontrib>Pallot, D</creatorcontrib><creatorcontrib>Serra, P</creatorcontrib><creatorcontrib>J M van der Hulst</creatorcontrib><creatorcontrib>Jurek, R J</creatorcontrib><creatorcontrib>Elagali, A</creatorcontrib><creatorcontrib>B -Q</creatorcontrib><creatorcontrib>Kleiner, D</creatorcontrib><creatorcontrib>Koribalski, B S</creatorcontrib><creatorcontrib>Lee-Waddell, K</creatorcontrib><creatorcontrib>Mould, J R</creatorcontrib><creatorcontrib>Reynolds, T N</creatorcontrib><creatorcontrib>Rhee, J</creatorcontrib><creatorcontrib>Staveley-Smith, L</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection><collection>arXiv.org</collection><jtitle>arXiv.org</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Westmeier, T</au><au>Kitaeff, S</au><au>Pallot, D</au><au>Serra, P</au><au>J M van der Hulst</au><au>Jurek, R J</au><au>Elagali, A</au><au>B -Q</au><au>Kleiner, D</au><au>Koribalski, B S</au><au>Lee-Waddell, K</au><au>Mould, J R</au><au>Reynolds, T N</au><au>Rhee, J</au><au>Staveley-Smith, L</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey</atitle><jtitle>arXiv.org</jtitle><date>2021-06-30</date><risdate>2021</risdate><eissn>2331-8422</eissn><abstract>We present SoFiA 2, the fully automated 3D source finding pipeline for the WALLABY extragalactic HI survey with the Australian SKA Pathfinder (ASKAP). SoFiA 2 is a reimplementation of parts of the original SoFiA pipeline in the C programming language and makes use of OpenMP for multi-threading of the most time-critical algorithms. In addition, we have developed a parallel framework called SoFiA-X that allows the processing of large data cubes to be split across multiple computing nodes. As a result of these efforts, SoFiA 2 is substantially faster and comes with a much reduced memory footprint compared to its predecessor, thus allowing the large WALLABY data volumes of hundreds of gigabytes of imaging data per epoch to be processed in real-time. The source code has been made publicly available to the entire community under an open-source licence. Performance tests using mock galaxies injected into genuine ASKAP data suggest that in the absence of significant imaging artefacts SoFiA 2 is capable of achieving near-100% completeness and reliability above an integrated signal-to-noise ratio of about 5-6. We also demonstrate that SoFiA 2 generally recovers the location, integrated flux and w20 line width of galaxies with high accuracy. Other parameters, including the peak flux density and w50 line width, are more strongly biased due to the influence of the noise on the measurement. In addition, very faint galaxies below an integrated signal-to-noise ratio of about 10 may get broken up into multiple components, thus requiring a strategy to identify fragmented sources and ensure that they do not affect the integrity of any scientific analysis based on the SoFiA 2 output.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><doi>10.48550/arxiv.2106.15789</doi><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | EISSN: 2331-8422 |
ispartof | arXiv.org, 2021-06 |
issn | 2331-8422 |
language | eng |
recordid | cdi_arxiv_primary_2106_15789 |
source | arXiv.org; Free E- Journals |
subjects | Algorithms Automation C (programming language) Cubes Flux density Galaxies Noise measurement Performance tests Physics - Instrumentation and Methods for Astrophysics Pipelines Signal to noise ratio Source code |
title | SoFiA 2 -- An automated, parallel HI source finding pipeline for the WALLABY survey |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-25T21%3A20%3A00IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_arxiv&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=SoFiA%202%20--%20An%20automated,%20parallel%20HI%20source%20finding%20pipeline%20for%20the%20WALLABY%20survey&rft.jtitle=arXiv.org&rft.au=Westmeier,%20T&rft.date=2021-06-30&rft.eissn=2331-8422&rft_id=info:doi/10.48550/arxiv.2106.15789&rft_dat=%3Cproquest_arxiv%3E2547170618%3C/proquest_arxiv%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2547170618&rft_id=info:pmid/&rfr_iscdi=true |