DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images

Three-dimensional microscopy is often limited by anisotropic spatial resolution, resulting in lower axial resolution than lateral resolution. Current State-of-The-Art (SoTA) isotropic reconstruction methods utilizing deep neural networks can achieve impressive super-resolution performance in fixed i...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Pan, Mingjie, Gan, Yulu, Zhou, Fangxu, Liu, Jiaming, Wang, Aimin, Zhang, Shanghang, Li, Dawei
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Pan, Mingjie
Gan, Yulu
Zhou, Fangxu
Liu, Jiaming
Wang, Aimin
Zhang, Shanghang
Li, Dawei
description Three-dimensional microscopy is often limited by anisotropic spatial resolution, resulting in lower axial resolution than lateral resolution. Current State-of-The-Art (SoTA) isotropic reconstruction methods utilizing deep neural networks can achieve impressive super-resolution performance in fixed imaging settings. However, their generality in practical use is limited by degraded performance caused by artifacts and blurring when facing unseen anisotropic factors. To address these issues, we propose DiffuseIR, an unsupervised method for isotropic reconstruction based on diffusion models. First, we pre-train a diffusion model to learn the structural distribution of biological tissue from lateral microscopic images, resulting in generating naturally high-resolution images. Then we use low-axial-resolution microscopy images to condition the generation process of the diffusion model and generate high-axial-resolution reconstruction results. Since the diffusion model learns the universal structural distribution of biological tissues, which is independent of the axial resolution, DiffuseIR can reconstruct authentic images with unseen low-axial resolutions into a high-axial resolution without requiring re-training. The proposed DiffuseIR achieves SoTA performance in experiments on EM data and can even compete with supervised methods.
doi_str_mv 10.48550/arxiv.2306.12109
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2306_12109</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2306_12109</sourcerecordid><originalsourceid>FETCH-LOGICAL-a679-cf50c4c93a2e3cf24d12300676754895b2b9c79f3bb3a80ba322eae1c33557093</originalsourceid><addsrcrecordid>eNotj0FuwjAURL1hUdEeoKv6Aklt_ziOu0NQSiQQCLGP7I9dWQKM7FC1ty8EVjOLp9E8Ql45K6tGSvZu0m_4KQWwuuSCM_1ENrPg_SW7dvtxbyGe6Cru3SHTeUy0zbFP8RyQbh3GU-7TBfsbEz2FGV0FTDHjALRH8-3yMxl5c8ju5ZFjspt_7qaLYrn-aqeTZWFqpQv0kmGFGoxwgF5Ue359xWpVK1k1WlphNSrtwVowDbMGhHDGcQSQUjENY_J2nx2UunMKR5P-uptaN6jBPw6BSOc</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images</title><source>arXiv.org</source><creator>Pan, Mingjie ; Gan, Yulu ; Zhou, Fangxu ; Liu, Jiaming ; Wang, Aimin ; Zhang, Shanghang ; Li, Dawei</creator><creatorcontrib>Pan, Mingjie ; Gan, Yulu ; Zhou, Fangxu ; Liu, Jiaming ; Wang, Aimin ; Zhang, Shanghang ; Li, Dawei</creatorcontrib><description>Three-dimensional microscopy is often limited by anisotropic spatial resolution, resulting in lower axial resolution than lateral resolution. Current State-of-The-Art (SoTA) isotropic reconstruction methods utilizing deep neural networks can achieve impressive super-resolution performance in fixed imaging settings. However, their generality in practical use is limited by degraded performance caused by artifacts and blurring when facing unseen anisotropic factors. To address these issues, we propose DiffuseIR, an unsupervised method for isotropic reconstruction based on diffusion models. First, we pre-train a diffusion model to learn the structural distribution of biological tissue from lateral microscopic images, resulting in generating naturally high-resolution images. Then we use low-axial-resolution microscopy images to condition the generation process of the diffusion model and generate high-axial-resolution reconstruction results. Since the diffusion model learns the universal structural distribution of biological tissues, which is independent of the axial resolution, DiffuseIR can reconstruct authentic images with unseen low-axial resolutions into a high-axial resolution without requiring re-training. The proposed DiffuseIR achieves SoTA performance in experiments on EM data and can even compete with supervised methods.</description><identifier>DOI: 10.48550/arxiv.2306.12109</identifier><language>eng</language><subject>Computer Science - Computer Vision and Pattern Recognition</subject><creationdate>2023-06</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,776,881</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2306.12109$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2306.12109$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Pan, Mingjie</creatorcontrib><creatorcontrib>Gan, Yulu</creatorcontrib><creatorcontrib>Zhou, Fangxu</creatorcontrib><creatorcontrib>Liu, Jiaming</creatorcontrib><creatorcontrib>Wang, Aimin</creatorcontrib><creatorcontrib>Zhang, Shanghang</creatorcontrib><creatorcontrib>Li, Dawei</creatorcontrib><title>DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images</title><description>Three-dimensional microscopy is often limited by anisotropic spatial resolution, resulting in lower axial resolution than lateral resolution. Current State-of-The-Art (SoTA) isotropic reconstruction methods utilizing deep neural networks can achieve impressive super-resolution performance in fixed imaging settings. However, their generality in practical use is limited by degraded performance caused by artifacts and blurring when facing unseen anisotropic factors. To address these issues, we propose DiffuseIR, an unsupervised method for isotropic reconstruction based on diffusion models. First, we pre-train a diffusion model to learn the structural distribution of biological tissue from lateral microscopic images, resulting in generating naturally high-resolution images. Then we use low-axial-resolution microscopy images to condition the generation process of the diffusion model and generate high-axial-resolution reconstruction results. Since the diffusion model learns the universal structural distribution of biological tissues, which is independent of the axial resolution, DiffuseIR can reconstruct authentic images with unseen low-axial resolutions into a high-axial resolution without requiring re-training. The proposed DiffuseIR achieves SoTA performance in experiments on EM data and can even compete with supervised methods.</description><subject>Computer Science - Computer Vision and Pattern Recognition</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotj0FuwjAURL1hUdEeoKv6Aklt_ziOu0NQSiQQCLGP7I9dWQKM7FC1ty8EVjOLp9E8Ql45K6tGSvZu0m_4KQWwuuSCM_1ENrPg_SW7dvtxbyGe6Cru3SHTeUy0zbFP8RyQbh3GU-7TBfsbEz2FGV0FTDHjALRH8-3yMxl5c8ju5ZFjspt_7qaLYrn-aqeTZWFqpQv0kmGFGoxwgF5Ue359xWpVK1k1WlphNSrtwVowDbMGhHDGcQSQUjENY_J2nx2UunMKR5P-uptaN6jBPw6BSOc</recordid><startdate>20230621</startdate><enddate>20230621</enddate><creator>Pan, Mingjie</creator><creator>Gan, Yulu</creator><creator>Zhou, Fangxu</creator><creator>Liu, Jiaming</creator><creator>Wang, Aimin</creator><creator>Zhang, Shanghang</creator><creator>Li, Dawei</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20230621</creationdate><title>DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images</title><author>Pan, Mingjie ; Gan, Yulu ; Zhou, Fangxu ; Liu, Jiaming ; Wang, Aimin ; Zhang, Shanghang ; Li, Dawei</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a679-cf50c4c93a2e3cf24d12300676754895b2b9c79f3bb3a80ba322eae1c33557093</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Computer Science - Computer Vision and Pattern Recognition</topic><toplevel>online_resources</toplevel><creatorcontrib>Pan, Mingjie</creatorcontrib><creatorcontrib>Gan, Yulu</creatorcontrib><creatorcontrib>Zhou, Fangxu</creatorcontrib><creatorcontrib>Liu, Jiaming</creatorcontrib><creatorcontrib>Wang, Aimin</creatorcontrib><creatorcontrib>Zhang, Shanghang</creatorcontrib><creatorcontrib>Li, Dawei</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Pan, Mingjie</au><au>Gan, Yulu</au><au>Zhou, Fangxu</au><au>Liu, Jiaming</au><au>Wang, Aimin</au><au>Zhang, Shanghang</au><au>Li, Dawei</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images</atitle><date>2023-06-21</date><risdate>2023</risdate><abstract>Three-dimensional microscopy is often limited by anisotropic spatial resolution, resulting in lower axial resolution than lateral resolution. Current State-of-The-Art (SoTA) isotropic reconstruction methods utilizing deep neural networks can achieve impressive super-resolution performance in fixed imaging settings. However, their generality in practical use is limited by degraded performance caused by artifacts and blurring when facing unseen anisotropic factors. To address these issues, we propose DiffuseIR, an unsupervised method for isotropic reconstruction based on diffusion models. First, we pre-train a diffusion model to learn the structural distribution of biological tissue from lateral microscopic images, resulting in generating naturally high-resolution images. Then we use low-axial-resolution microscopy images to condition the generation process of the diffusion model and generate high-axial-resolution reconstruction results. Since the diffusion model learns the universal structural distribution of biological tissues, which is independent of the axial resolution, DiffuseIR can reconstruct authentic images with unseen low-axial resolutions into a high-axial resolution without requiring re-training. The proposed DiffuseIR achieves SoTA performance in experiments on EM data and can even compete with supervised methods.</abstract><doi>10.48550/arxiv.2306.12109</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2306.12109
ispartof
issn
language eng
recordid cdi_arxiv_primary_2306_12109
source arXiv.org
subjects Computer Science - Computer Vision and Pattern Recognition
title DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-23T06%3A52%3A09IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=DiffuseIR:Diffusion%20Models%20For%20Isotropic%20Reconstruction%20of%203D%20Microscopic%20Images&rft.au=Pan,%20Mingjie&rft.date=2023-06-21&rft_id=info:doi/10.48550/arxiv.2306.12109&rft_dat=%3Carxiv_GOX%3E2306_12109%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true