A Topological Loss Function for Deep-Learning Based Image Segmentation Using Persistent Homology

We introduce a method for training neural networks to perform image or volume segmentation in which prior knowledge about the topology of the segmented object can be explicitly provided and then incorporated into the training process. By using the differentiable properties of persistent homology, a...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence 2022-12, Vol.44 (12), p.8766-8778
Hauptverfasser: Clough, James R., Byrne, Nicholas, Oksuz, Ilkay, Zimmer, Veronika A., Schnabel, Julia A., King, Andrew P.
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 8778
container_issue 12
container_start_page 8766
container_title IEEE transactions on pattern analysis and machine intelligence
container_volume 44
creator Clough, James R.
Byrne, Nicholas
Oksuz, Ilkay
Zimmer, Veronika A.
Schnabel, Julia A.
King, Andrew P.
description We introduce a method for training neural networks to perform image or volume segmentation in which prior knowledge about the topology of the segmented object can be explicitly provided and then incorporated into the training process. By using the differentiable properties of persistent homology, a concept used in topological data analysis, we can specify the desired topology of segmented objects in terms of their Betti numbers and then drive the proposed segmentations to contain the specified topological features. Importantly this process does not require any ground-truth labels, just prior knowledge of the topology of the structure being segmented. We demonstrate our approach in four experiments: one on MNIST image denoising and digit recognition, one on left ventricular myocardium segmentation from magnetic resonance imaging data from the UK Biobank, one on the ACDC public challenge dataset and one on placenta segmentation from 3-D ultrasound. We find that embedding explicit prior knowledge in neural network segmentation tasks is most beneficial when the segmentation task is especially challenging and that it can be used in either a semi-supervised or post-processing context to extract a useful training gradient from images without pixelwise labels.
doi_str_mv 10.1109/TPAMI.2020.3013679
format Article
fullrecord <record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_pubmed_primary_32886606</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9186664</ieee_id><sourcerecordid>2440471578</sourcerecordid><originalsourceid>FETCH-LOGICAL-c450t-5206482fe3a7b8993322bdaf1e5580e07c7d2ac9311c2fa7a7188593c61268693</originalsourceid><addsrcrecordid>eNpdkUtvEzEUhS0EomnhD4CELLFhM8GP8WuDFAqlkYKoRLo2jnNncDUzDvYMUv99nSZEwMqSz3ePzr0HoVeUzCkl5v36ZvF1OWeEkTknlEtlnqAZNdxUXHDzFM0IlazSmukzdJ7zHSG0FoQ_R2ecaS0lkTP0Y4HXcRe72AbvOryKOeOrafBjiANuYsKfAHbVClwawtDijy7DFi971wL-Dm0Pw-ge0du8l28g5ZDH8ouvY793vX-BnjWuy_Dy-F6g26vP68vravXty_Jysap8yTRWghFZa9YAd2qjjeGcsc3WNRSE0ASI8mrLnDecUs8ap5yiWgvDvaRMamn4Bfpw8N1Nmx62vmRIrrO7FHqX7m10wf6rDOGnbeNvaxSjgsli8O5okOKvCfJo-5A9dJ0bIE7ZsromtaJC6YK-_Q-9i1MaynqWKV5zLZRUhWIHyqdy1QTNKQwldl-gfSzQ7gu0xwLL0Ju_1ziN_GmsAK8PQACAk2xoUWXNHwCMM59U</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2734385767</pqid></control><display><type>article</type><title>A Topological Loss Function for Deep-Learning Based Image Segmentation Using Persistent Homology</title><source>MEDLINE</source><source>IEEE Electronic Library (IEL)</source><creator>Clough, James R. ; Byrne, Nicholas ; Oksuz, Ilkay ; Zimmer, Veronika A. ; Schnabel, Julia A. ; King, Andrew P.</creator><creatorcontrib>Clough, James R. ; Byrne, Nicholas ; Oksuz, Ilkay ; Zimmer, Veronika A. ; Schnabel, Julia A. ; King, Andrew P.</creatorcontrib><description>We introduce a method for training neural networks to perform image or volume segmentation in which prior knowledge about the topology of the segmented object can be explicitly provided and then incorporated into the training process. By using the differentiable properties of persistent homology, a concept used in topological data analysis, we can specify the desired topology of segmented objects in terms of their Betti numbers and then drive the proposed segmentations to contain the specified topological features. Importantly this process does not require any ground-truth labels, just prior knowledge of the topology of the structure being segmented. We demonstrate our approach in four experiments: one on MNIST image denoising and digit recognition, one on left ventricular myocardium segmentation from magnetic resonance imaging data from the UK Biobank, one on the ACDC public challenge dataset and one on placenta segmentation from 3-D ultrasound. We find that embedding explicit prior knowledge in neural network segmentation tasks is most beneficial when the segmentation task is especially challenging and that it can be used in either a semi-supervised or post-processing context to extract a useful training gradient from images without pixelwise labels.</description><identifier>ISSN: 0162-8828</identifier><identifier>EISSN: 1939-3539</identifier><identifier>EISSN: 2160-9292</identifier><identifier>DOI: 10.1109/TPAMI.2020.3013679</identifier><identifier>PMID: 32886606</identifier><identifier>CODEN: ITPIDJ</identifier><language>eng</language><publisher>United States: IEEE</publisher><subject>Algorithms ; convolutional neural networks ; Data analysis ; Deep Learning ; Homology ; Image Processing, Computer-Assisted - methods ; Image segmentation ; Labels ; Loss measurement ; Magnetic resonance imaging ; Magnetic Resonance Imaging - methods ; medical imaging ; Myocardium ; Network topologies ; Network topology ; Neural networks ; Neural Networks, Computer ; Object recognition ; persistent homology ; Segmentation ; Shape ; Topology ; Training</subject><ispartof>IEEE transactions on pattern analysis and machine intelligence, 2022-12, Vol.44 (12), p.8766-8778</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022</rights><rights>2020 Author</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c450t-5206482fe3a7b8993322bdaf1e5580e07c7d2ac9311c2fa7a7188593c61268693</citedby><cites>FETCH-LOGICAL-c450t-5206482fe3a7b8993322bdaf1e5580e07c7d2ac9311c2fa7a7188593c61268693</cites><orcidid>0000-0001-6107-3009 ; 0000-0003-3401-9570 ; 0000-0002-9965-7015 ; 0000-0001-6478-0534 ; 0000-0002-9135-0545 ; 0000-0002-5093-5854</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9186664$$EHTML$$P50$$Gieee$$Hfree_for_read</linktohtml><link.rule.ids>230,314,780,784,796,885,27923,27924,54757</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/32886606$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Clough, James R.</creatorcontrib><creatorcontrib>Byrne, Nicholas</creatorcontrib><creatorcontrib>Oksuz, Ilkay</creatorcontrib><creatorcontrib>Zimmer, Veronika A.</creatorcontrib><creatorcontrib>Schnabel, Julia A.</creatorcontrib><creatorcontrib>King, Andrew P.</creatorcontrib><title>A Topological Loss Function for Deep-Learning Based Image Segmentation Using Persistent Homology</title><title>IEEE transactions on pattern analysis and machine intelligence</title><addtitle>TPAMI</addtitle><addtitle>IEEE Trans Pattern Anal Mach Intell</addtitle><description>We introduce a method for training neural networks to perform image or volume segmentation in which prior knowledge about the topology of the segmented object can be explicitly provided and then incorporated into the training process. By using the differentiable properties of persistent homology, a concept used in topological data analysis, we can specify the desired topology of segmented objects in terms of their Betti numbers and then drive the proposed segmentations to contain the specified topological features. Importantly this process does not require any ground-truth labels, just prior knowledge of the topology of the structure being segmented. We demonstrate our approach in four experiments: one on MNIST image denoising and digit recognition, one on left ventricular myocardium segmentation from magnetic resonance imaging data from the UK Biobank, one on the ACDC public challenge dataset and one on placenta segmentation from 3-D ultrasound. We find that embedding explicit prior knowledge in neural network segmentation tasks is most beneficial when the segmentation task is especially challenging and that it can be used in either a semi-supervised or post-processing context to extract a useful training gradient from images without pixelwise labels.</description><subject>Algorithms</subject><subject>convolutional neural networks</subject><subject>Data analysis</subject><subject>Deep Learning</subject><subject>Homology</subject><subject>Image Processing, Computer-Assisted - methods</subject><subject>Image segmentation</subject><subject>Labels</subject><subject>Loss measurement</subject><subject>Magnetic resonance imaging</subject><subject>Magnetic Resonance Imaging - methods</subject><subject>medical imaging</subject><subject>Myocardium</subject><subject>Network topologies</subject><subject>Network topology</subject><subject>Neural networks</subject><subject>Neural Networks, Computer</subject><subject>Object recognition</subject><subject>persistent homology</subject><subject>Segmentation</subject><subject>Shape</subject><subject>Topology</subject><subject>Training</subject><issn>0162-8828</issn><issn>1939-3539</issn><issn>2160-9292</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>ESBDL</sourceid><sourceid>RIE</sourceid><sourceid>EIF</sourceid><recordid>eNpdkUtvEzEUhS0EomnhD4CELLFhM8GP8WuDFAqlkYKoRLo2jnNncDUzDvYMUv99nSZEwMqSz3ePzr0HoVeUzCkl5v36ZvF1OWeEkTknlEtlnqAZNdxUXHDzFM0IlazSmukzdJ7zHSG0FoQ_R2ecaS0lkTP0Y4HXcRe72AbvOryKOeOrafBjiANuYsKfAHbVClwawtDijy7DFi971wL-Dm0Pw-ge0du8l28g5ZDH8ouvY793vX-BnjWuy_Dy-F6g26vP68vravXty_Jysap8yTRWghFZa9YAd2qjjeGcsc3WNRSE0ASI8mrLnDecUs8ap5yiWgvDvaRMamn4Bfpw8N1Nmx62vmRIrrO7FHqX7m10wf6rDOGnbeNvaxSjgsli8O5okOKvCfJo-5A9dJ0bIE7ZsromtaJC6YK-_Q-9i1MaynqWKV5zLZRUhWIHyqdy1QTNKQwldl-gfSzQ7gu0xwLL0Ju_1ziN_GmsAK8PQACAk2xoUWXNHwCMM59U</recordid><startdate>20221201</startdate><enddate>20221201</enddate><creator>Clough, James R.</creator><creator>Byrne, Nicholas</creator><creator>Oksuz, Ilkay</creator><creator>Zimmer, Veronika A.</creator><creator>Schnabel, Julia A.</creator><creator>King, Andrew P.</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>ESBDL</scope><scope>RIA</scope><scope>RIE</scope><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>7X8</scope><scope>5PM</scope><orcidid>https://orcid.org/0000-0001-6107-3009</orcidid><orcidid>https://orcid.org/0000-0003-3401-9570</orcidid><orcidid>https://orcid.org/0000-0002-9965-7015</orcidid><orcidid>https://orcid.org/0000-0001-6478-0534</orcidid><orcidid>https://orcid.org/0000-0002-9135-0545</orcidid><orcidid>https://orcid.org/0000-0002-5093-5854</orcidid></search><sort><creationdate>20221201</creationdate><title>A Topological Loss Function for Deep-Learning Based Image Segmentation Using Persistent Homology</title><author>Clough, James R. ; Byrne, Nicholas ; Oksuz, Ilkay ; Zimmer, Veronika A. ; Schnabel, Julia A. ; King, Andrew P.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c450t-5206482fe3a7b8993322bdaf1e5580e07c7d2ac9311c2fa7a7188593c61268693</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Algorithms</topic><topic>convolutional neural networks</topic><topic>Data analysis</topic><topic>Deep Learning</topic><topic>Homology</topic><topic>Image Processing, Computer-Assisted - methods</topic><topic>Image segmentation</topic><topic>Labels</topic><topic>Loss measurement</topic><topic>Magnetic resonance imaging</topic><topic>Magnetic Resonance Imaging - methods</topic><topic>medical imaging</topic><topic>Myocardium</topic><topic>Network topologies</topic><topic>Network topology</topic><topic>Neural networks</topic><topic>Neural Networks, Computer</topic><topic>Object recognition</topic><topic>persistent homology</topic><topic>Segmentation</topic><topic>Shape</topic><topic>Topology</topic><topic>Training</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Clough, James R.</creatorcontrib><creatorcontrib>Byrne, Nicholas</creatorcontrib><creatorcontrib>Oksuz, Ilkay</creatorcontrib><creatorcontrib>Zimmer, Veronika A.</creatorcontrib><creatorcontrib>Schnabel, Julia A.</creatorcontrib><creatorcontrib>King, Andrew P.</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE Open Access Journals</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><jtitle>IEEE transactions on pattern analysis and machine intelligence</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Clough, James R.</au><au>Byrne, Nicholas</au><au>Oksuz, Ilkay</au><au>Zimmer, Veronika A.</au><au>Schnabel, Julia A.</au><au>King, Andrew P.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A Topological Loss Function for Deep-Learning Based Image Segmentation Using Persistent Homology</atitle><jtitle>IEEE transactions on pattern analysis and machine intelligence</jtitle><stitle>TPAMI</stitle><addtitle>IEEE Trans Pattern Anal Mach Intell</addtitle><date>2022-12-01</date><risdate>2022</risdate><volume>44</volume><issue>12</issue><spage>8766</spage><epage>8778</epage><pages>8766-8778</pages><issn>0162-8828</issn><eissn>1939-3539</eissn><eissn>2160-9292</eissn><coden>ITPIDJ</coden><abstract>We introduce a method for training neural networks to perform image or volume segmentation in which prior knowledge about the topology of the segmented object can be explicitly provided and then incorporated into the training process. By using the differentiable properties of persistent homology, a concept used in topological data analysis, we can specify the desired topology of segmented objects in terms of their Betti numbers and then drive the proposed segmentations to contain the specified topological features. Importantly this process does not require any ground-truth labels, just prior knowledge of the topology of the structure being segmented. We demonstrate our approach in four experiments: one on MNIST image denoising and digit recognition, one on left ventricular myocardium segmentation from magnetic resonance imaging data from the UK Biobank, one on the ACDC public challenge dataset and one on placenta segmentation from 3-D ultrasound. We find that embedding explicit prior knowledge in neural network segmentation tasks is most beneficial when the segmentation task is especially challenging and that it can be used in either a semi-supervised or post-processing context to extract a useful training gradient from images without pixelwise labels.</abstract><cop>United States</cop><pub>IEEE</pub><pmid>32886606</pmid><doi>10.1109/TPAMI.2020.3013679</doi><tpages>13</tpages><orcidid>https://orcid.org/0000-0001-6107-3009</orcidid><orcidid>https://orcid.org/0000-0003-3401-9570</orcidid><orcidid>https://orcid.org/0000-0002-9965-7015</orcidid><orcidid>https://orcid.org/0000-0001-6478-0534</orcidid><orcidid>https://orcid.org/0000-0002-9135-0545</orcidid><orcidid>https://orcid.org/0000-0002-5093-5854</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 0162-8828
ispartof IEEE transactions on pattern analysis and machine intelligence, 2022-12, Vol.44 (12), p.8766-8778
issn 0162-8828
1939-3539
2160-9292
language eng
recordid cdi_pubmed_primary_32886606
source MEDLINE; IEEE Electronic Library (IEL)
subjects Algorithms
convolutional neural networks
Data analysis
Deep Learning
Homology
Image Processing, Computer-Assisted - methods
Image segmentation
Labels
Loss measurement
Magnetic resonance imaging
Magnetic Resonance Imaging - methods
medical imaging
Myocardium
Network topologies
Network topology
Neural networks
Neural Networks, Computer
Object recognition
persistent homology
Segmentation
Shape
Topology
Training
title A Topological Loss Function for Deep-Learning Based Image Segmentation Using Persistent Homology
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-08T19%3A20%3A24IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20Topological%20Loss%20Function%20for%20Deep-Learning%20Based%20Image%20Segmentation%20Using%20Persistent%20Homology&rft.jtitle=IEEE%20transactions%20on%20pattern%20analysis%20and%20machine%20intelligence&rft.au=Clough,%20James%20R.&rft.date=2022-12-01&rft.volume=44&rft.issue=12&rft.spage=8766&rft.epage=8778&rft.pages=8766-8778&rft.issn=0162-8828&rft.eissn=1939-3539&rft.coden=ITPIDJ&rft_id=info:doi/10.1109/TPAMI.2020.3013679&rft_dat=%3Cproquest_pubme%3E2440471578%3C/proquest_pubme%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2734385767&rft_id=info:pmid/32886606&rft_ieee_id=9186664&rfr_iscdi=true