TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation

TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation

Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture....

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Berthelot, David, Autef, Arnaud, Lin, Jierui, Yap, Dian Ang, Zhai, Shuangfei, Hu, Siyuan, Zheng, Daniel, Talbott, Walter, Gu, Eric
Format: Artikel
Sprache:eng
Schlagworte:
Computer Science - Computer Vision and Pattern Recognition
Computer Science - Learning
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Berthelot, David
Autef, Arnaud
Lin, Jierui
Yap, Dian Ang
Zhai, Shuangfei
Hu, Siyuan
Zheng, Daniel
Talbott, Walter
Gu, Eric
description Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture. In this paper, we introduce TRAnsitive Closure Time-distillation (TRACT), a new method that extends BTD. For single step diffusion,TRACT improves FID by up to 2.4x on the same architecture, and achieves new single-step Denoising Diffusion Implicit Models (DDIM) state-of-the-art FID (7.4 for ImageNet64, 3.8 for CIFAR10). Finally we tease apart the method through extended ablations. The PyTorch implementation will be released soon.
doi_str_mv 10.48550/arxiv.2303.04248
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2303_04248</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2303_04248</sourcerecordid><originalsourceid>FETCH-LOGICAL-a678-1e2142fcbe416fd0bfd7e67a0e90cb1699152f09f3eb481a52cc4dc63ce5c25f3</originalsourceid><addsrcrecordid>eNotz0tOwzAUhWFPGKDCAhjVG0jwOwmzKuElipAqzyPHuYYruQmy0wK7BwqjM_qP9BFyxVmpaq3ZtUufeCyFZLJkSqj6nDzZ3aa1N7SDacaM0yvtMIRDxnmiz_MIMdMPXN6oTW7KuOARaBvnfEhALe6h6DAvGKNbfoILchZczHD5vyti725t-1BsX-4f2822cKaqCw6CKxH8AIqbMLIhjBWYyjFomB-4aRquRWBNkDComjstvFejN9KD9kIHuSLrv9uTpn9PuHfpq_9V9SeV_AZ-k0gy</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation</title><source>arXiv.org</source><creator>Berthelot, David ; Autef, Arnaud ; Lin, Jierui ; Yap, Dian Ang ; Zhai, Shuangfei ; Hu, Siyuan ; Zheng, Daniel ; Talbott, Walter ; Gu, Eric</creator><creatorcontrib>Berthelot, David ; Autef, Arnaud ; Lin, Jierui ; Yap, Dian Ang ; Zhai, Shuangfei ; Hu, Siyuan ; Zheng, Daniel ; Talbott, Walter ; Gu, Eric</creatorcontrib><description>Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture. In this paper, we introduce TRAnsitive Closure Time-distillation (TRACT), a new method that extends BTD. For single step diffusion,TRACT improves FID by up to 2.4x on the same architecture, and achieves new single-step Denoising Diffusion Implicit Models (DDIM) state-of-the-art FID (7.4 for ImageNet64, 3.8 for CIFAR10). Finally we tease apart the method through extended ablations. The PyTorch implementation will be released soon.</description><identifier>DOI: 10.48550/arxiv.2303.04248</identifier><language>eng</language><subject>Computer Science - Computer Vision and Pattern Recognition ; Computer Science - Learning</subject><creationdate>2023-03</creationdate><rights>http://creativecommons.org/licenses/by/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,776,881</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2303.04248$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2303.04248$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Berthelot, David</creatorcontrib><creatorcontrib>Autef, Arnaud</creatorcontrib><creatorcontrib>Lin, Jierui</creatorcontrib><creatorcontrib>Yap, Dian Ang</creatorcontrib><creatorcontrib>Zhai, Shuangfei</creatorcontrib><creatorcontrib>Hu, Siyuan</creatorcontrib><creatorcontrib>Zheng, Daniel</creatorcontrib><creatorcontrib>Talbott, Walter</creatorcontrib><creatorcontrib>Gu, Eric</creatorcontrib><title>TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation</title><description>Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture. In this paper, we introduce TRAnsitive Closure Time-distillation (TRACT), a new method that extends BTD. For single step diffusion,TRACT improves FID by up to 2.4x on the same architecture, and achieves new single-step Denoising Diffusion Implicit Models (DDIM) state-of-the-art FID (7.4 for ImageNet64, 3.8 for CIFAR10). Finally we tease apart the method through extended ablations. The PyTorch implementation will be released soon.</description><subject>Computer Science - Computer Vision and Pattern Recognition</subject><subject>Computer Science - Learning</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotz0tOwzAUhWFPGKDCAhjVG0jwOwmzKuElipAqzyPHuYYruQmy0wK7BwqjM_qP9BFyxVmpaq3ZtUufeCyFZLJkSqj6nDzZ3aa1N7SDacaM0yvtMIRDxnmiz_MIMdMPXN6oTW7KuOARaBvnfEhALe6h6DAvGKNbfoILchZczHD5vyti725t-1BsX-4f2822cKaqCw6CKxH8AIqbMLIhjBWYyjFomB-4aRquRWBNkDComjstvFejN9KD9kIHuSLrv9uTpn9PuHfpq_9V9SeV_AZ-k0gy</recordid><startdate>20230307</startdate><enddate>20230307</enddate><creator>Berthelot, David</creator><creator>Autef, Arnaud</creator><creator>Lin, Jierui</creator><creator>Yap, Dian Ang</creator><creator>Zhai, Shuangfei</creator><creator>Hu, Siyuan</creator><creator>Zheng, Daniel</creator><creator>Talbott, Walter</creator><creator>Gu, Eric</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20230307</creationdate><title>TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation</title><author>Berthelot, David ; Autef, Arnaud ; Lin, Jierui ; Yap, Dian Ang ; Zhai, Shuangfei ; Hu, Siyuan ; Zheng, Daniel ; Talbott, Walter ; Gu, Eric</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a678-1e2142fcbe416fd0bfd7e67a0e90cb1699152f09f3eb481a52cc4dc63ce5c25f3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Computer Science - Computer Vision and Pattern Recognition</topic><topic>Computer Science - Learning</topic><toplevel>online_resources</toplevel><creatorcontrib>Berthelot, David</creatorcontrib><creatorcontrib>Autef, Arnaud</creatorcontrib><creatorcontrib>Lin, Jierui</creatorcontrib><creatorcontrib>Yap, Dian Ang</creatorcontrib><creatorcontrib>Zhai, Shuangfei</creatorcontrib><creatorcontrib>Hu, Siyuan</creatorcontrib><creatorcontrib>Zheng, Daniel</creatorcontrib><creatorcontrib>Talbott, Walter</creatorcontrib><creatorcontrib>Gu, Eric</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Berthelot, David</au><au>Autef, Arnaud</au><au>Lin, Jierui</au><au>Yap, Dian Ang</au><au>Zhai, Shuangfei</au><au>Hu, Siyuan</au><au>Zheng, Daniel</au><au>Talbott, Walter</au><au>Gu, Eric</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation</atitle><date>2023-03-07</date><risdate>2023</risdate><abstract>Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture. In this paper, we introduce TRAnsitive Closure Time-distillation (TRACT), a new method that extends BTD. For single step diffusion,TRACT improves FID by up to 2.4x on the same architecture, and achieves new single-step Denoising Diffusion Implicit Models (DDIM) state-of-the-art FID (7.4 for ImageNet64, 3.8 for CIFAR10). Finally we tease apart the method through extended ablations. The PyTorch implementation will be released soon.</abstract><doi>10.48550/arxiv.2303.04248</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2303.04248
ispartof
issn
language eng
recordid cdi_arxiv_primary_2303_04248
source arXiv.org
subjects Computer Science - Computer Vision and Pattern Recognition
Computer Science - Learning
title TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-30T03%3A12%3A13IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=TRACT:%20Denoising%20Diffusion%20Models%20with%20Transitive%20Closure%20Time-Distillation&rft.au=Berthelot,%20David&rft.date=2023-03-07&rft_id=info:doi/10.48550/arxiv.2303.04248&rft_dat=%3Carxiv_GOX%3E2303_04248%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true