Whitening Convergence Rate of Coupling-based Normalizing Flows
Coupling-based normalizing flows (e.g. RealNVP) are a popular family of normalizing flow architectures that work surprisingly well in practice. This calls for theoretical understanding. Existing work shows that such flows weakly converge to arbitrary data distributions. However, they make no stateme...
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | |
| container_volume | |
| creator | Draxler, Felix Schnörr, Christoph Köthe, Ullrich |
| description | Coupling-based normalizing flows (e.g. RealNVP) are a popular family of
normalizing flow architectures that work surprisingly well in practice. This
calls for theoretical understanding. Existing work shows that such flows weakly
converge to arbitrary data distributions. However, they make no statement about
the stricter convergence criterion used in practice, the maximum likelihood
loss. For the first time, we make a quantitative statement about this kind of
convergence: We prove that all coupling-based normalizing flows perform
whitening of the data distribution (i.e. diagonalize the covariance matrix) and
derive corresponding convergence bounds that show a linear convergence rate in
the depth of the flow. Numerical experiments demonstrate the implications of
our theory and point at open questions. |
| doi_str_mv | 10.48550/arxiv.2210.14032 |
| format | Article |
| fullrecord | <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2210_14032</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2210_14032</sourcerecordid><originalsourceid>FETCH-LOGICAL-a672-2a50db2cf8c4f1b4cdc3ccfbaabf8a1e186c16b2cd3eaca3eee88722a9fb3a923</originalsourceid><addsrcrecordid>eNotj8uKwkAQRXszC3H8AFfmB-Kkq5PYboQh-AJxYBBchupKtTbERDo-5-snPlYX7j1cOEL0ZTSMdZJEX-hv7jIEaAsZRwo6YrLduxNXrtoFWV1d2O-4Ig5-8cRBbdvufCzbMTTYcBGsa3_A0v098FlZX5tP8WGxbLj3zq7YzKabbBGufubL7HsVYjqCEDCJCgNkNcVWmpgKUkTWIBqrUbLUKcm0BQrFSKiYWesRAI6tUTgG1RWD1-1TID96d0B_zx8i-VNE_QMAi0UG</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Whitening Convergence Rate of Coupling-based Normalizing Flows</title><source>arXiv.org</source><creator>Draxler, Felix ; Schnörr, Christoph ; Köthe, Ullrich</creator><creatorcontrib>Draxler, Felix ; Schnörr, Christoph ; Köthe, Ullrich</creatorcontrib><description>Coupling-based normalizing flows (e.g. RealNVP) are a popular family of
normalizing flow architectures that work surprisingly well in practice. This
calls for theoretical understanding. Existing work shows that such flows weakly
converge to arbitrary data distributions. However, they make no statement about
the stricter convergence criterion used in practice, the maximum likelihood
loss. For the first time, we make a quantitative statement about this kind of
convergence: We prove that all coupling-based normalizing flows perform
whitening of the data distribution (i.e. diagonalize the covariance matrix) and
derive corresponding convergence bounds that show a linear convergence rate in
the depth of the flow. Numerical experiments demonstrate the implications of
our theory and point at open questions.</description><identifier>DOI: 10.48550/arxiv.2210.14032</identifier><language>eng</language><subject>Computer Science - Learning ; Statistics - Machine Learning</subject><creationdate>2022-10</creationdate><rights>http://creativecommons.org/licenses/by-sa/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2210.14032$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2210.14032$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Draxler, Felix</creatorcontrib><creatorcontrib>Schnörr, Christoph</creatorcontrib><creatorcontrib>Köthe, Ullrich</creatorcontrib><title>Whitening Convergence Rate of Coupling-based Normalizing Flows</title><description>Coupling-based normalizing flows (e.g. RealNVP) are a popular family of
normalizing flow architectures that work surprisingly well in practice. This
calls for theoretical understanding. Existing work shows that such flows weakly
converge to arbitrary data distributions. However, they make no statement about
the stricter convergence criterion used in practice, the maximum likelihood
loss. For the first time, we make a quantitative statement about this kind of
convergence: We prove that all coupling-based normalizing flows perform
whitening of the data distribution (i.e. diagonalize the covariance matrix) and
derive corresponding convergence bounds that show a linear convergence rate in
the depth of the flow. Numerical experiments demonstrate the implications of
our theory and point at open questions.</description><subject>Computer Science - Learning</subject><subject>Statistics - Machine Learning</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotj8uKwkAQRXszC3H8AFfmB-Kkq5PYboQh-AJxYBBchupKtTbERDo-5-snPlYX7j1cOEL0ZTSMdZJEX-hv7jIEaAsZRwo6YrLduxNXrtoFWV1d2O-4Ig5-8cRBbdvufCzbMTTYcBGsa3_A0v098FlZX5tP8WGxbLj3zq7YzKabbBGufubL7HsVYjqCEDCJCgNkNcVWmpgKUkTWIBqrUbLUKcm0BQrFSKiYWesRAI6tUTgG1RWD1-1TID96d0B_zx8i-VNE_QMAi0UG</recordid><startdate>20221025</startdate><enddate>20221025</enddate><creator>Draxler, Felix</creator><creator>Schnörr, Christoph</creator><creator>Köthe, Ullrich</creator><scope>AKY</scope><scope>EPD</scope><scope>GOX</scope></search><sort><creationdate>20221025</creationdate><title>Whitening Convergence Rate of Coupling-based Normalizing Flows</title><author>Draxler, Felix ; Schnörr, Christoph ; Köthe, Ullrich</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a672-2a50db2cf8c4f1b4cdc3ccfbaabf8a1e186c16b2cd3eaca3eee88722a9fb3a923</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Computer Science - Learning</topic><topic>Statistics - Machine Learning</topic><toplevel>online_resources</toplevel><creatorcontrib>Draxler, Felix</creatorcontrib><creatorcontrib>Schnörr, Christoph</creatorcontrib><creatorcontrib>Köthe, Ullrich</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv Statistics</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Draxler, Felix</au><au>Schnörr, Christoph</au><au>Köthe, Ullrich</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Whitening Convergence Rate of Coupling-based Normalizing Flows</atitle><date>2022-10-25</date><risdate>2022</risdate><abstract>Coupling-based normalizing flows (e.g. RealNVP) are a popular family of
normalizing flow architectures that work surprisingly well in practice. This
calls for theoretical understanding. Existing work shows that such flows weakly
converge to arbitrary data distributions. However, they make no statement about
the stricter convergence criterion used in practice, the maximum likelihood
loss. For the first time, we make a quantitative statement about this kind of
convergence: We prove that all coupling-based normalizing flows perform
whitening of the data distribution (i.e. diagonalize the covariance matrix) and
derive corresponding convergence bounds that show a linear convergence rate in
the depth of the flow. Numerical experiments demonstrate the implications of
our theory and point at open questions.</abstract><doi>10.48550/arxiv.2210.14032</doi><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | DOI: 10.48550/arxiv.2210.14032 |
| ispartof | |
| issn | |
| language | eng |
| recordid | cdi_arxiv_primary_2210_14032 |
| source | arXiv.org |
| subjects | Computer Science - Learning Statistics - Machine Learning |
| title | Whitening Convergence Rate of Coupling-based Normalizing Flows |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-12T01%3A42%3A28IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Whitening%20Convergence%20Rate%20of%20Coupling-based%20Normalizing%20Flows&rft.au=Draxler,%20Felix&rft.date=2022-10-25&rft_id=info:doi/10.48550/arxiv.2210.14032&rft_dat=%3Carxiv_GOX%3E2210_14032%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |