NeurTV: Total Variation on the Neural Domain

Recently, we have witnessed the success of total variation (TV) for many imaging applications. However, traditional TV is defined on the original pixel domain, which limits its potential. In this work, we suggest a new TV regularization defined on the neural domain. Concretely, the discrete data is...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Luo, Yisi, Zhao, Xile, Ye, Kai, Meng, Deyu
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Luo, Yisi
Zhao, Xile
Ye, Kai
Meng, Deyu
description Recently, we have witnessed the success of total variation (TV) for many imaging applications. However, traditional TV is defined on the original pixel domain, which limits its potential. In this work, we suggest a new TV regularization defined on the neural domain. Concretely, the discrete data is implicitly and continuously represented by a deep neural network (DNN), and we use the derivatives of DNN outputs w.r.t. input coordinates to capture local correlations of data. As compared with classical TV on the original domain, the proposed TV on the neural domain (termed NeurTV) enjoys the following advantages. First, NeurTV is free of discretization error induced by the discrete difference operator. Second, NeurTV is not limited to meshgrid but is suitable for both meshgrid and non-meshgrid data. Third, NeurTV can more exactly capture local correlations across data for any direction and any order of derivatives attributed to the implicit and continuous nature of neural domain. We theoretically reinterpret NeurTV under the variational approximation framework, which allows us to build the connection between NeurTV and classical TV and inspires us to develop variants (e.g., space-variant NeurTV). Extensive numerical experiments with meshgrid data (e.g., color and hyperspectral images) and non-meshgrid data (e.g., point clouds and spatial transcriptomics) showcase the effectiveness of the proposed methods.
doi_str_mv 10.48550/arxiv.2405.17241
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2405_17241</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2405_17241</sourcerecordid><originalsourceid>FETCH-LOGICAL-a671-da9993077b9201fc173effa5f79e13222bbbd39c753d03870eb194383d78c9693</originalsourceid><addsrcrecordid>eNotjs2KwjAUhbNxMagPMCv7ALYmuU1vr7tB5w9EN8VtuWkTDFQ7xCozbz_qDBw4cD44fEI8K5nlpTFywfE7XDOdS5Mp1Ll6EvOtu8Rqv0yqfuAu2XMMPIT-lNwyHFxyx7d93R85nCZi5Lk7u-l_j0X19lqtPtLN7v1z9bJJuUCVtkxEIBEtaal8oxCc92w8klOgtbbWtkANGmgllCidVZRDCS2WDRUEYzH7u33o1l8xHDn-1Hft-qENv2B7Ovw</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>NeurTV: Total Variation on the Neural Domain</title><source>arXiv.org</source><creator>Luo, Yisi ; Zhao, Xile ; Ye, Kai ; Meng, Deyu</creator><creatorcontrib>Luo, Yisi ; Zhao, Xile ; Ye, Kai ; Meng, Deyu</creatorcontrib><description>Recently, we have witnessed the success of total variation (TV) for many imaging applications. However, traditional TV is defined on the original pixel domain, which limits its potential. In this work, we suggest a new TV regularization defined on the neural domain. Concretely, the discrete data is implicitly and continuously represented by a deep neural network (DNN), and we use the derivatives of DNN outputs w.r.t. input coordinates to capture local correlations of data. As compared with classical TV on the original domain, the proposed TV on the neural domain (termed NeurTV) enjoys the following advantages. First, NeurTV is free of discretization error induced by the discrete difference operator. Second, NeurTV is not limited to meshgrid but is suitable for both meshgrid and non-meshgrid data. Third, NeurTV can more exactly capture local correlations across data for any direction and any order of derivatives attributed to the implicit and continuous nature of neural domain. We theoretically reinterpret NeurTV under the variational approximation framework, which allows us to build the connection between NeurTV and classical TV and inspires us to develop variants (e.g., space-variant NeurTV). Extensive numerical experiments with meshgrid data (e.g., color and hyperspectral images) and non-meshgrid data (e.g., point clouds and spatial transcriptomics) showcase the effectiveness of the proposed methods.</description><identifier>DOI: 10.48550/arxiv.2405.17241</identifier><language>eng</language><subject>Computer Science - Computer Vision and Pattern Recognition</subject><creationdate>2024-05</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,778,883</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2405.17241$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2405.17241$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Luo, Yisi</creatorcontrib><creatorcontrib>Zhao, Xile</creatorcontrib><creatorcontrib>Ye, Kai</creatorcontrib><creatorcontrib>Meng, Deyu</creatorcontrib><title>NeurTV: Total Variation on the Neural Domain</title><description>Recently, we have witnessed the success of total variation (TV) for many imaging applications. However, traditional TV is defined on the original pixel domain, which limits its potential. In this work, we suggest a new TV regularization defined on the neural domain. Concretely, the discrete data is implicitly and continuously represented by a deep neural network (DNN), and we use the derivatives of DNN outputs w.r.t. input coordinates to capture local correlations of data. As compared with classical TV on the original domain, the proposed TV on the neural domain (termed NeurTV) enjoys the following advantages. First, NeurTV is free of discretization error induced by the discrete difference operator. Second, NeurTV is not limited to meshgrid but is suitable for both meshgrid and non-meshgrid data. Third, NeurTV can more exactly capture local correlations across data for any direction and any order of derivatives attributed to the implicit and continuous nature of neural domain. We theoretically reinterpret NeurTV under the variational approximation framework, which allows us to build the connection between NeurTV and classical TV and inspires us to develop variants (e.g., space-variant NeurTV). Extensive numerical experiments with meshgrid data (e.g., color and hyperspectral images) and non-meshgrid data (e.g., point clouds and spatial transcriptomics) showcase the effectiveness of the proposed methods.</description><subject>Computer Science - Computer Vision and Pattern Recognition</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotjs2KwjAUhbNxMagPMCv7ALYmuU1vr7tB5w9EN8VtuWkTDFQ7xCozbz_qDBw4cD44fEI8K5nlpTFywfE7XDOdS5Mp1Ll6EvOtu8Rqv0yqfuAu2XMMPIT-lNwyHFxyx7d93R85nCZi5Lk7u-l_j0X19lqtPtLN7v1z9bJJuUCVtkxEIBEtaal8oxCc92w8klOgtbbWtkANGmgllCidVZRDCS2WDRUEYzH7u33o1l8xHDn-1Hft-qENv2B7Ovw</recordid><startdate>20240527</startdate><enddate>20240527</enddate><creator>Luo, Yisi</creator><creator>Zhao, Xile</creator><creator>Ye, Kai</creator><creator>Meng, Deyu</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20240527</creationdate><title>NeurTV: Total Variation on the Neural Domain</title><author>Luo, Yisi ; Zhao, Xile ; Ye, Kai ; Meng, Deyu</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a671-da9993077b9201fc173effa5f79e13222bbbd39c753d03870eb194383d78c9693</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Computer Science - Computer Vision and Pattern Recognition</topic><toplevel>online_resources</toplevel><creatorcontrib>Luo, Yisi</creatorcontrib><creatorcontrib>Zhao, Xile</creatorcontrib><creatorcontrib>Ye, Kai</creatorcontrib><creatorcontrib>Meng, Deyu</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Luo, Yisi</au><au>Zhao, Xile</au><au>Ye, Kai</au><au>Meng, Deyu</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>NeurTV: Total Variation on the Neural Domain</atitle><date>2024-05-27</date><risdate>2024</risdate><abstract>Recently, we have witnessed the success of total variation (TV) for many imaging applications. However, traditional TV is defined on the original pixel domain, which limits its potential. In this work, we suggest a new TV regularization defined on the neural domain. Concretely, the discrete data is implicitly and continuously represented by a deep neural network (DNN), and we use the derivatives of DNN outputs w.r.t. input coordinates to capture local correlations of data. As compared with classical TV on the original domain, the proposed TV on the neural domain (termed NeurTV) enjoys the following advantages. First, NeurTV is free of discretization error induced by the discrete difference operator. Second, NeurTV is not limited to meshgrid but is suitable for both meshgrid and non-meshgrid data. Third, NeurTV can more exactly capture local correlations across data for any direction and any order of derivatives attributed to the implicit and continuous nature of neural domain. We theoretically reinterpret NeurTV under the variational approximation framework, which allows us to build the connection between NeurTV and classical TV and inspires us to develop variants (e.g., space-variant NeurTV). Extensive numerical experiments with meshgrid data (e.g., color and hyperspectral images) and non-meshgrid data (e.g., point clouds and spatial transcriptomics) showcase the effectiveness of the proposed methods.</abstract><doi>10.48550/arxiv.2405.17241</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2405.17241
ispartof
issn
language eng
recordid cdi_arxiv_primary_2405_17241
source arXiv.org
subjects Computer Science - Computer Vision and Pattern Recognition
title NeurTV: Total Variation on the Neural Domain
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-15T11%3A19%3A38IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=NeurTV:%20Total%20Variation%20on%20the%20Neural%20Domain&rft.au=Luo,%20Yisi&rft.date=2024-05-27&rft_id=info:doi/10.48550/arxiv.2405.17241&rft_dat=%3Carxiv_GOX%3E2405_17241%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true