Deep Multi-Look Sequence Processing for Synthetic Aperture Sonar Image Segmentation

Deep learning has enabled significant improvements in semantic image segmentation, especially in underwater imaging domains such as side scan sonar (SSS). In this work, we apply deep learning to synthetic aperture sonar (SAS) imagery, which has an advantage over traditional SSS in which SAS produces...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on geoscience and remote sensing 2023, Vol.61, p.1-15
Hauptverfasser: Gerg, Isaac D., Monga, Vishal
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 15
container_issue
container_start_page 1
container_title IEEE transactions on geoscience and remote sensing
container_volume 61
creator Gerg, Isaac D.
Monga, Vishal
description Deep learning has enabled significant improvements in semantic image segmentation, especially in underwater imaging domains such as side scan sonar (SSS). In this work, we apply deep learning to synthetic aperture sonar (SAS) imagery, which has an advantage over traditional SSS in which SAS produces coherent high- and constant-resolution imagery. Despite the successes of deep learning, one drawback is the need for abundant labeled training data to enable success. Such abundant labeled data are not always available as in the case of SAS where collections are expensive and obtaining quality ground-truth labels may require diver intervention. To overcome these challenges, we propose a domain-specific deep learning network architecture utilizing a unique property to complex-valued SAS imagery: the ability to resolve angle-of-arrival (AoA) of acoustic returns through k -space processing. By sweeping through consecutive incrementally advanced AoA bandpass filters (a process sometimes referred to as multi-look processing), this technique generates a sequence of images emphasizing angle-dependent seafloor scattering and motion from biologics along the seafloor or in the water column. Our proposal, which we call multi-look sequence processing network (MLSP-Net), is a domain-enriched deep neural network architecture that models the multi-look image sequence using a recurrent neural network (RNN) to extract robust features suitable for semantic segmentation of the seafloor without the need for abundant training data. Unlike previous segmentation works in SAS, our model ingests a complex-valued SAS image and affords the ability to learn the AoA filters in k -space as part of the training procedure. We show the results on a challenging real-world SAS database, and despite the lack of abundant training data, our proposed method shows superior results over state-of-the-art techniques.
doi_str_mv 10.1109/TGRS.2023.3234229
format Article
fullrecord <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_crossref_primary_10_1109_TGRS_2023_3234229</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10008947</ieee_id><sourcerecordid>2766612090</sourcerecordid><originalsourceid>FETCH-LOGICAL-c294t-97d6e9f26e7fe95469b26875b859d898c28e8d993ebbcaf3699a2d2550a336803</originalsourceid><addsrcrecordid>eNpNkD1PwzAQhi0EEqXwA5AYLDGn-CN2fGNVoFQqApEyW05yKSltHJxk6L8nURmY7obnfe_0EHLL2YxzBg-b5Uc6E0zImRQyFgLOyIQrZSKm4_icTBgHHQkD4pJcte2OMR4rnkxI-ojY0Nd-31XR2vtvmuJPj3WO9D34HNu2qre09IGmx7r7wq7K6bzB0PUBaeprF-jq4LbDjtsD1p3rKl9fk4vS7Vu8-ZtT8vn8tFm8ROu35WoxX0e5gLiLICk0Qik0JiWCijVkQptEZUZBYcDkwqApACRmWe5KqQGcKIRSzEmpDZNTcn_qbYIfnm47u_N9qIeTViRaay4YjBQ_UXnwbRuwtE2oDi4cLWd2dGdHd3Z0Z__cDZm7U6ZCxH88YwbiRP4Cz9hqaw</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2766612090</pqid></control><display><type>article</type><title>Deep Multi-Look Sequence Processing for Synthetic Aperture Sonar Image Segmentation</title><source>IEEE Electronic Library (IEL)</source><creator>Gerg, Isaac D. ; Monga, Vishal</creator><creatorcontrib>Gerg, Isaac D. ; Monga, Vishal</creatorcontrib><description><![CDATA[Deep learning has enabled significant improvements in semantic image segmentation, especially in underwater imaging domains such as side scan sonar (SSS). In this work, we apply deep learning to synthetic aperture sonar (SAS) imagery, which has an advantage over traditional SSS in which SAS produces coherent high- and constant-resolution imagery. Despite the successes of deep learning, one drawback is the need for abundant labeled training data to enable success. Such abundant labeled data are not always available as in the case of SAS where collections are expensive and obtaining quality ground-truth labels may require diver intervention. To overcome these challenges, we propose a domain-specific deep learning network architecture utilizing a unique property to complex-valued SAS imagery: the ability to resolve angle-of-arrival (AoA) of acoustic returns through <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-space processing. By sweeping through consecutive incrementally advanced AoA bandpass filters (a process sometimes referred to as multi-look processing), this technique generates a sequence of images emphasizing angle-dependent seafloor scattering and motion from biologics along the seafloor or in the water column. Our proposal, which we call multi-look sequence processing network (MLSP-Net), is a domain-enriched deep neural network architecture that models the multi-look image sequence using a recurrent neural network (RNN) to extract robust features suitable for semantic segmentation of the seafloor without the need for abundant training data. Unlike previous segmentation works in SAS, our model ingests a complex-valued SAS image and affords the ability to learn the AoA filters in <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-space as part of the training procedure. We show the results on a challenging real-world SAS database, and despite the lack of abundant training data, our proposed method shows superior results over state-of-the-art techniques.]]></description><identifier>ISSN: 0196-2892</identifier><identifier>EISSN: 1558-0644</identifier><identifier>DOI: 10.1109/TGRS.2023.3234229</identifier><identifier>CODEN: IGRSD2</identifier><language>eng</language><publisher>New York: IEEE</publisher><subject>Acoustics ; Artificial neural networks ; Bandpass filters ; Collections ; Computer architecture ; Deep learning ; Feature extraction ; Filters ; Image processing ; Image segmentation ; Image sequences ; Imagery ; Machine learning ; Neural networks ; Ocean floor ; Recurrent neural networks ; recurrent neural networks (RNNs) ; Semantic segmentation ; Semantics ; Sequencing ; Side scan sonar ; Sonar ; Space processing ; Synthetic aperture sonar ; synthetic aperture sonar (SAS) ; Synthetic apertures ; Training ; Training data ; Water circulation ; Water column</subject><ispartof>IEEE transactions on geoscience and remote sensing, 2023, Vol.61, p.1-15</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c294t-97d6e9f26e7fe95469b26875b859d898c28e8d993ebbcaf3699a2d2550a336803</citedby><cites>FETCH-LOGICAL-c294t-97d6e9f26e7fe95469b26875b859d898c28e8d993ebbcaf3699a2d2550a336803</cites><orcidid>0000-0002-3352-6864 ; 0000-0002-5100-2263</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10008947$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,777,781,793,4010,27904,27905,27906,54739</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/10008947$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Gerg, Isaac D.</creatorcontrib><creatorcontrib>Monga, Vishal</creatorcontrib><title>Deep Multi-Look Sequence Processing for Synthetic Aperture Sonar Image Segmentation</title><title>IEEE transactions on geoscience and remote sensing</title><addtitle>TGRS</addtitle><description><![CDATA[Deep learning has enabled significant improvements in semantic image segmentation, especially in underwater imaging domains such as side scan sonar (SSS). In this work, we apply deep learning to synthetic aperture sonar (SAS) imagery, which has an advantage over traditional SSS in which SAS produces coherent high- and constant-resolution imagery. Despite the successes of deep learning, one drawback is the need for abundant labeled training data to enable success. Such abundant labeled data are not always available as in the case of SAS where collections are expensive and obtaining quality ground-truth labels may require diver intervention. To overcome these challenges, we propose a domain-specific deep learning network architecture utilizing a unique property to complex-valued SAS imagery: the ability to resolve angle-of-arrival (AoA) of acoustic returns through <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-space processing. By sweeping through consecutive incrementally advanced AoA bandpass filters (a process sometimes referred to as multi-look processing), this technique generates a sequence of images emphasizing angle-dependent seafloor scattering and motion from biologics along the seafloor or in the water column. Our proposal, which we call multi-look sequence processing network (MLSP-Net), is a domain-enriched deep neural network architecture that models the multi-look image sequence using a recurrent neural network (RNN) to extract robust features suitable for semantic segmentation of the seafloor without the need for abundant training data. Unlike previous segmentation works in SAS, our model ingests a complex-valued SAS image and affords the ability to learn the AoA filters in <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-space as part of the training procedure. We show the results on a challenging real-world SAS database, and despite the lack of abundant training data, our proposed method shows superior results over state-of-the-art techniques.]]></description><subject>Acoustics</subject><subject>Artificial neural networks</subject><subject>Bandpass filters</subject><subject>Collections</subject><subject>Computer architecture</subject><subject>Deep learning</subject><subject>Feature extraction</subject><subject>Filters</subject><subject>Image processing</subject><subject>Image segmentation</subject><subject>Image sequences</subject><subject>Imagery</subject><subject>Machine learning</subject><subject>Neural networks</subject><subject>Ocean floor</subject><subject>Recurrent neural networks</subject><subject>recurrent neural networks (RNNs)</subject><subject>Semantic segmentation</subject><subject>Semantics</subject><subject>Sequencing</subject><subject>Side scan sonar</subject><subject>Sonar</subject><subject>Space processing</subject><subject>Synthetic aperture sonar</subject><subject>synthetic aperture sonar (SAS)</subject><subject>Synthetic apertures</subject><subject>Training</subject><subject>Training data</subject><subject>Water circulation</subject><subject>Water column</subject><issn>0196-2892</issn><issn>1558-0644</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNpNkD1PwzAQhi0EEqXwA5AYLDGn-CN2fGNVoFQqApEyW05yKSltHJxk6L8nURmY7obnfe_0EHLL2YxzBg-b5Uc6E0zImRQyFgLOyIQrZSKm4_icTBgHHQkD4pJcte2OMR4rnkxI-ojY0Nd-31XR2vtvmuJPj3WO9D34HNu2qre09IGmx7r7wq7K6bzB0PUBaeprF-jq4LbDjtsD1p3rKl9fk4vS7Vu8-ZtT8vn8tFm8ROu35WoxX0e5gLiLICk0Qik0JiWCijVkQptEZUZBYcDkwqApACRmWe5KqQGcKIRSzEmpDZNTcn_qbYIfnm47u_N9qIeTViRaay4YjBQ_UXnwbRuwtE2oDi4cLWd2dGdHd3Z0Z__cDZm7U6ZCxH88YwbiRP4Cz9hqaw</recordid><startdate>2023</startdate><enddate>2023</enddate><creator>Gerg, Isaac D.</creator><creator>Monga, Vishal</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7UA</scope><scope>8FD</scope><scope>C1K</scope><scope>F1W</scope><scope>FR3</scope><scope>H8D</scope><scope>H96</scope><scope>KR7</scope><scope>L.G</scope><scope>L7M</scope><orcidid>https://orcid.org/0000-0002-3352-6864</orcidid><orcidid>https://orcid.org/0000-0002-5100-2263</orcidid></search><sort><creationdate>2023</creationdate><title>Deep Multi-Look Sequence Processing for Synthetic Aperture Sonar Image Segmentation</title><author>Gerg, Isaac D. ; Monga, Vishal</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c294t-97d6e9f26e7fe95469b26875b859d898c28e8d993ebbcaf3699a2d2550a336803</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Acoustics</topic><topic>Artificial neural networks</topic><topic>Bandpass filters</topic><topic>Collections</topic><topic>Computer architecture</topic><topic>Deep learning</topic><topic>Feature extraction</topic><topic>Filters</topic><topic>Image processing</topic><topic>Image segmentation</topic><topic>Image sequences</topic><topic>Imagery</topic><topic>Machine learning</topic><topic>Neural networks</topic><topic>Ocean floor</topic><topic>Recurrent neural networks</topic><topic>recurrent neural networks (RNNs)</topic><topic>Semantic segmentation</topic><topic>Semantics</topic><topic>Sequencing</topic><topic>Side scan sonar</topic><topic>Sonar</topic><topic>Space processing</topic><topic>Synthetic aperture sonar</topic><topic>synthetic aperture sonar (SAS)</topic><topic>Synthetic apertures</topic><topic>Training</topic><topic>Training data</topic><topic>Water circulation</topic><topic>Water column</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Gerg, Isaac D.</creatorcontrib><creatorcontrib>Monga, Vishal</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Water Resources Abstracts</collection><collection>Technology Research Database</collection><collection>Environmental Sciences and Pollution Management</collection><collection>ASFA: Aquatic Sciences and Fisheries Abstracts</collection><collection>Engineering Research Database</collection><collection>Aerospace Database</collection><collection>Aquatic Science &amp; Fisheries Abstracts (ASFA) 2: Ocean Technology, Policy &amp; Non-Living Resources</collection><collection>Civil Engineering Abstracts</collection><collection>Aquatic Science &amp; Fisheries Abstracts (ASFA) Professional</collection><collection>Advanced Technologies Database with Aerospace</collection><jtitle>IEEE transactions on geoscience and remote sensing</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Gerg, Isaac D.</au><au>Monga, Vishal</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Deep Multi-Look Sequence Processing for Synthetic Aperture Sonar Image Segmentation</atitle><jtitle>IEEE transactions on geoscience and remote sensing</jtitle><stitle>TGRS</stitle><date>2023</date><risdate>2023</risdate><volume>61</volume><spage>1</spage><epage>15</epage><pages>1-15</pages><issn>0196-2892</issn><eissn>1558-0644</eissn><coden>IGRSD2</coden><abstract><![CDATA[Deep learning has enabled significant improvements in semantic image segmentation, especially in underwater imaging domains such as side scan sonar (SSS). In this work, we apply deep learning to synthetic aperture sonar (SAS) imagery, which has an advantage over traditional SSS in which SAS produces coherent high- and constant-resolution imagery. Despite the successes of deep learning, one drawback is the need for abundant labeled training data to enable success. Such abundant labeled data are not always available as in the case of SAS where collections are expensive and obtaining quality ground-truth labels may require diver intervention. To overcome these challenges, we propose a domain-specific deep learning network architecture utilizing a unique property to complex-valued SAS imagery: the ability to resolve angle-of-arrival (AoA) of acoustic returns through <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-space processing. By sweeping through consecutive incrementally advanced AoA bandpass filters (a process sometimes referred to as multi-look processing), this technique generates a sequence of images emphasizing angle-dependent seafloor scattering and motion from biologics along the seafloor or in the water column. Our proposal, which we call multi-look sequence processing network (MLSP-Net), is a domain-enriched deep neural network architecture that models the multi-look image sequence using a recurrent neural network (RNN) to extract robust features suitable for semantic segmentation of the seafloor without the need for abundant training data. Unlike previous segmentation works in SAS, our model ingests a complex-valued SAS image and affords the ability to learn the AoA filters in <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-space as part of the training procedure. We show the results on a challenging real-world SAS database, and despite the lack of abundant training data, our proposed method shows superior results over state-of-the-art techniques.]]></abstract><cop>New York</cop><pub>IEEE</pub><doi>10.1109/TGRS.2023.3234229</doi><tpages>15</tpages><orcidid>https://orcid.org/0000-0002-3352-6864</orcidid><orcidid>https://orcid.org/0000-0002-5100-2263</orcidid></addata></record>
fulltext fulltext_linktorsrc
identifier ISSN: 0196-2892
ispartof IEEE transactions on geoscience and remote sensing, 2023, Vol.61, p.1-15
issn 0196-2892
1558-0644
language eng
recordid cdi_crossref_primary_10_1109_TGRS_2023_3234229
source IEEE Electronic Library (IEL)
subjects Acoustics
Artificial neural networks
Bandpass filters
Collections
Computer architecture
Deep learning
Feature extraction
Filters
Image processing
Image segmentation
Image sequences
Imagery
Machine learning
Neural networks
Ocean floor
Recurrent neural networks
recurrent neural networks (RNNs)
Semantic segmentation
Semantics
Sequencing
Side scan sonar
Sonar
Space processing
Synthetic aperture sonar
synthetic aperture sonar (SAS)
Synthetic apertures
Training
Training data
Water circulation
Water column
title Deep Multi-Look Sequence Processing for Synthetic Aperture Sonar Image Segmentation
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-20T02%3A30%3A05IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Deep%20Multi-Look%20Sequence%20Processing%20for%20Synthetic%20Aperture%20Sonar%20Image%20Segmentation&rft.jtitle=IEEE%20transactions%20on%20geoscience%20and%20remote%20sensing&rft.au=Gerg,%20Isaac%20D.&rft.date=2023&rft.volume=61&rft.spage=1&rft.epage=15&rft.pages=1-15&rft.issn=0196-2892&rft.eissn=1558-0644&rft.coden=IGRSD2&rft_id=info:doi/10.1109/TGRS.2023.3234229&rft_dat=%3Cproquest_RIE%3E2766612090%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2766612090&rft_id=info:pmid/&rft_ieee_id=10008947&rfr_iscdi=true