HIGT: Hierarchical Interaction Graph-Transformer for Whole Slide Image Analysis

In computation pathology, the pyramid structure of gigapixel Whole Slide Images (WSIs) has recently been studied for capturing various information from individual cell interactions to tissue microenvironments. This hierarchical structure is believed to be beneficial for cancer diagnosis and prognosi...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Guo, Ziyu, Zhao, Weiqin, Wang, Shujun, Yu, Lequan
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Guo, Ziyu
Zhao, Weiqin
Wang, Shujun
Yu, Lequan
description In computation pathology, the pyramid structure of gigapixel Whole Slide Images (WSIs) has recently been studied for capturing various information from individual cell interactions to tissue microenvironments. This hierarchical structure is believed to be beneficial for cancer diagnosis and prognosis tasks. However, most previous hierarchical WSI analysis works (1) only characterize local or global correlations within the WSI pyramids and (2) use only unidirectional interaction between different resolutions, leading to an incomplete picture of WSI pyramids. To this end, this paper presents a novel Hierarchical Interaction Graph-Transformer (i.e., HIGT) for WSI analysis. With Graph Neural Network and Transformer as the building commons, HIGT can learn both short-range local information and long-range global representation of the WSI pyramids. Considering that the information from different resolutions is complementary and can benefit each other during the learning process, we further design a novel Bidirectional Interaction block to establish communication between different levels within the WSI pyramids. Finally, we aggregate both coarse-grained and fine-grained features learned from different levels together for slide-level prediction. We evaluate our methods on two public WSI datasets from TCGA projects, i.e., kidney carcinoma (KICA) and esophageal carcinoma (ESCA). Experimental results show that our HIGT outperforms both hierarchical and non-hierarchical state-of-the-art methods on both tumor subtyping and staging tasks.
doi_str_mv 10.48550/arxiv.2309.07400
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2309_07400</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2309_07400</sourcerecordid><originalsourceid>FETCH-LOGICAL-a670-badcc4edc87143420d0189eb477cd9ce8c51d0dfd92b7d68445878bb384372613</originalsourceid><addsrcrecordid>eNotz71qwzAYhWEtHULaC8hU3YCdT5Zsyd1CSG1DIEMMHc2nH9cC2Q5yKM3dN007vZzlwEPIhkEqVJ7DFuO3_0ozDmUKUgCsyKluqvaN1t5FjGbwBgNtput9maufJ1pFvAxJG3Fa-jmOLtJ76McwB0fPwVtHmxE_Hd1NGG6LX57JU49hcS__XZP2_dDu6-R4qpr97phgISHRaI0RzholmeAiAwtMlU4LKY0tjVMmZxZsb8tMS1soIXIlldZcCS6zgvE1ef27fYi6S_Qjxlv3K-seMv4DiOlIiw</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>HIGT: Hierarchical Interaction Graph-Transformer for Whole Slide Image Analysis</title><source>arXiv.org</source><creator>Guo, Ziyu ; Zhao, Weiqin ; Wang, Shujun ; Yu, Lequan</creator><creatorcontrib>Guo, Ziyu ; Zhao, Weiqin ; Wang, Shujun ; Yu, Lequan</creatorcontrib><description>In computation pathology, the pyramid structure of gigapixel Whole Slide Images (WSIs) has recently been studied for capturing various information from individual cell interactions to tissue microenvironments. This hierarchical structure is believed to be beneficial for cancer diagnosis and prognosis tasks. However, most previous hierarchical WSI analysis works (1) only characterize local or global correlations within the WSI pyramids and (2) use only unidirectional interaction between different resolutions, leading to an incomplete picture of WSI pyramids. To this end, this paper presents a novel Hierarchical Interaction Graph-Transformer (i.e., HIGT) for WSI analysis. With Graph Neural Network and Transformer as the building commons, HIGT can learn both short-range local information and long-range global representation of the WSI pyramids. Considering that the information from different resolutions is complementary and can benefit each other during the learning process, we further design a novel Bidirectional Interaction block to establish communication between different levels within the WSI pyramids. Finally, we aggregate both coarse-grained and fine-grained features learned from different levels together for slide-level prediction. We evaluate our methods on two public WSI datasets from TCGA projects, i.e., kidney carcinoma (KICA) and esophageal carcinoma (ESCA). Experimental results show that our HIGT outperforms both hierarchical and non-hierarchical state-of-the-art methods on both tumor subtyping and staging tasks.</description><identifier>DOI: 10.48550/arxiv.2309.07400</identifier><language>eng</language><subject>Computer Science - Computer Vision and Pattern Recognition</subject><creationdate>2023-09</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2309.07400$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2309.07400$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Guo, Ziyu</creatorcontrib><creatorcontrib>Zhao, Weiqin</creatorcontrib><creatorcontrib>Wang, Shujun</creatorcontrib><creatorcontrib>Yu, Lequan</creatorcontrib><title>HIGT: Hierarchical Interaction Graph-Transformer for Whole Slide Image Analysis</title><description>In computation pathology, the pyramid structure of gigapixel Whole Slide Images (WSIs) has recently been studied for capturing various information from individual cell interactions to tissue microenvironments. This hierarchical structure is believed to be beneficial for cancer diagnosis and prognosis tasks. However, most previous hierarchical WSI analysis works (1) only characterize local or global correlations within the WSI pyramids and (2) use only unidirectional interaction between different resolutions, leading to an incomplete picture of WSI pyramids. To this end, this paper presents a novel Hierarchical Interaction Graph-Transformer (i.e., HIGT) for WSI analysis. With Graph Neural Network and Transformer as the building commons, HIGT can learn both short-range local information and long-range global representation of the WSI pyramids. Considering that the information from different resolutions is complementary and can benefit each other during the learning process, we further design a novel Bidirectional Interaction block to establish communication between different levels within the WSI pyramids. Finally, we aggregate both coarse-grained and fine-grained features learned from different levels together for slide-level prediction. We evaluate our methods on two public WSI datasets from TCGA projects, i.e., kidney carcinoma (KICA) and esophageal carcinoma (ESCA). Experimental results show that our HIGT outperforms both hierarchical and non-hierarchical state-of-the-art methods on both tumor subtyping and staging tasks.</description><subject>Computer Science - Computer Vision and Pattern Recognition</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotz71qwzAYhWEtHULaC8hU3YCdT5Zsyd1CSG1DIEMMHc2nH9cC2Q5yKM3dN007vZzlwEPIhkEqVJ7DFuO3_0ozDmUKUgCsyKluqvaN1t5FjGbwBgNtput9maufJ1pFvAxJG3Fa-jmOLtJ76McwB0fPwVtHmxE_Hd1NGG6LX57JU49hcS__XZP2_dDu6-R4qpr97phgISHRaI0RzholmeAiAwtMlU4LKY0tjVMmZxZsb8tMS1soIXIlldZcCS6zgvE1ef27fYi6S_Qjxlv3K-seMv4DiOlIiw</recordid><startdate>20230913</startdate><enddate>20230913</enddate><creator>Guo, Ziyu</creator><creator>Zhao, Weiqin</creator><creator>Wang, Shujun</creator><creator>Yu, Lequan</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20230913</creationdate><title>HIGT: Hierarchical Interaction Graph-Transformer for Whole Slide Image Analysis</title><author>Guo, Ziyu ; Zhao, Weiqin ; Wang, Shujun ; Yu, Lequan</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a670-badcc4edc87143420d0189eb477cd9ce8c51d0dfd92b7d68445878bb384372613</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Computer Science - Computer Vision and Pattern Recognition</topic><toplevel>online_resources</toplevel><creatorcontrib>Guo, Ziyu</creatorcontrib><creatorcontrib>Zhao, Weiqin</creatorcontrib><creatorcontrib>Wang, Shujun</creatorcontrib><creatorcontrib>Yu, Lequan</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Guo, Ziyu</au><au>Zhao, Weiqin</au><au>Wang, Shujun</au><au>Yu, Lequan</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>HIGT: Hierarchical Interaction Graph-Transformer for Whole Slide Image Analysis</atitle><date>2023-09-13</date><risdate>2023</risdate><abstract>In computation pathology, the pyramid structure of gigapixel Whole Slide Images (WSIs) has recently been studied for capturing various information from individual cell interactions to tissue microenvironments. This hierarchical structure is believed to be beneficial for cancer diagnosis and prognosis tasks. However, most previous hierarchical WSI analysis works (1) only characterize local or global correlations within the WSI pyramids and (2) use only unidirectional interaction between different resolutions, leading to an incomplete picture of WSI pyramids. To this end, this paper presents a novel Hierarchical Interaction Graph-Transformer (i.e., HIGT) for WSI analysis. With Graph Neural Network and Transformer as the building commons, HIGT can learn both short-range local information and long-range global representation of the WSI pyramids. Considering that the information from different resolutions is complementary and can benefit each other during the learning process, we further design a novel Bidirectional Interaction block to establish communication between different levels within the WSI pyramids. Finally, we aggregate both coarse-grained and fine-grained features learned from different levels together for slide-level prediction. We evaluate our methods on two public WSI datasets from TCGA projects, i.e., kidney carcinoma (KICA) and esophageal carcinoma (ESCA). Experimental results show that our HIGT outperforms both hierarchical and non-hierarchical state-of-the-art methods on both tumor subtyping and staging tasks.</abstract><doi>10.48550/arxiv.2309.07400</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2309.07400
ispartof
issn
language eng
recordid cdi_arxiv_primary_2309_07400
source arXiv.org
subjects Computer Science - Computer Vision and Pattern Recognition
title HIGT: Hierarchical Interaction Graph-Transformer for Whole Slide Image Analysis
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-02T13%3A10%3A08IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=HIGT:%20Hierarchical%20Interaction%20Graph-Transformer%20for%20Whole%20Slide%20Image%20Analysis&rft.au=Guo,%20Ziyu&rft.date=2023-09-13&rft_id=info:doi/10.48550/arxiv.2309.07400&rft_dat=%3Carxiv_GOX%3E2309_07400%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true