MusicLM: Generating Music From Text

MusicLM: Generating Music From Text

We introduce MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody backed by a distorted guitar riff". MusicLM casts the process of conditional music generation as a hierarchical sequence-to-sequence modeling task, and it generates music at...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:arXiv.org 2023-01
Hauptverfasser: Agostinelli, Andrea, Denk, Timo I, Borsos, Zalán, Engel, Jesse, Verzetti, Mauro, Caillon, Antoine, Huang, Qingqing, Jansen, Aren, Roberts, Adam, Tagliasacchi, Marco, Sharifi, Matt, Zeghidour, Neil, Frank, Christian
Format: Artikel
Sprache:eng
Schlagworte:
Descriptions
Music
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Agostinelli, Andrea
Denk, Timo I
Borsos, Zalán
Engel, Jesse
Verzetti, Mauro
Caillon, Antoine
Huang, Qingqing
Jansen, Aren
Roberts, Adam
Tagliasacchi, Marco
Sharifi, Matt
Zeghidour, Neil
Frank, Christian
description We introduce MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody backed by a distorted guitar riff". MusicLM casts the process of conditional music generation as a hierarchical sequence-to-sequence modeling task, and it generates music at 24 kHz that remains consistent over several minutes. Our experiments show that MusicLM outperforms previous systems both in audio quality and adherence to the text description. Moreover, we demonstrate that MusicLM can be conditioned on both text and a melody in that it can transform whistled and hummed melodies according to the style described in a text caption. To support future research, we publicly release MusicCaps, a dataset composed of 5.5k music-text pairs, with rich text descriptions provided by human experts.
format Article
fullrecord <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_2770179214</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2770179214</sourcerecordid><originalsourceid>FETCH-proquest_journals_27701792143</originalsourceid><addsrcrecordid>eNpjYuA0MjY21LUwMTLiYOAtLs4yMDAwMjM3MjU15mRQ9i0tzkz28bVScE_NSy1KLMnMS1cAiym4FeXnKoSkVpTwMLCmJeYUp_JCaW4GZTfXEGcP3YKi_MLS1OKS-Kz80qI8oFS8kbm5gaG5pZGhiTFxqgBKUS2K</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2770179214</pqid></control><display><type>article</type><title>MusicLM: Generating Music From Text</title><source>Free E- Journals</source><creator>Agostinelli, Andrea ; Denk, Timo I ; Borsos, Zalán ; Engel, Jesse ; Verzetti, Mauro ; Caillon, Antoine ; Huang, Qingqing ; Jansen, Aren ; Roberts, Adam ; Tagliasacchi, Marco ; Sharifi, Matt ; Zeghidour, Neil ; Frank, Christian</creator><creatorcontrib>Agostinelli, Andrea ; Denk, Timo I ; Borsos, Zalán ; Engel, Jesse ; Verzetti, Mauro ; Caillon, Antoine ; Huang, Qingqing ; Jansen, Aren ; Roberts, Adam ; Tagliasacchi, Marco ; Sharifi, Matt ; Zeghidour, Neil ; Frank, Christian</creatorcontrib><description>We introduce MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody backed by a distorted guitar riff". MusicLM casts the process of conditional music generation as a hierarchical sequence-to-sequence modeling task, and it generates music at 24 kHz that remains consistent over several minutes. Our experiments show that MusicLM outperforms previous systems both in audio quality and adherence to the text description. Moreover, we demonstrate that MusicLM can be conditioned on both text and a melody in that it can transform whistled and hummed melodies according to the style described in a text caption. To support future research, we publicly release MusicCaps, a dataset composed of 5.5k music-text pairs, with rich text descriptions provided by human experts.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Descriptions ; Music</subject><ispartof>arXiv.org, 2023-01</ispartof><rights>2023. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>776,780</link.rule.ids></links><search><creatorcontrib>Agostinelli, Andrea</creatorcontrib><creatorcontrib>Denk, Timo I</creatorcontrib><creatorcontrib>Borsos, Zalán</creatorcontrib><creatorcontrib>Engel, Jesse</creatorcontrib><creatorcontrib>Verzetti, Mauro</creatorcontrib><creatorcontrib>Caillon, Antoine</creatorcontrib><creatorcontrib>Huang, Qingqing</creatorcontrib><creatorcontrib>Jansen, Aren</creatorcontrib><creatorcontrib>Roberts, Adam</creatorcontrib><creatorcontrib>Tagliasacchi, Marco</creatorcontrib><creatorcontrib>Sharifi, Matt</creatorcontrib><creatorcontrib>Zeghidour, Neil</creatorcontrib><creatorcontrib>Frank, Christian</creatorcontrib><title>MusicLM: Generating Music From Text</title><title>arXiv.org</title><description>We introduce MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody backed by a distorted guitar riff". MusicLM casts the process of conditional music generation as a hierarchical sequence-to-sequence modeling task, and it generates music at 24 kHz that remains consistent over several minutes. Our experiments show that MusicLM outperforms previous systems both in audio quality and adherence to the text description. Moreover, we demonstrate that MusicLM can be conditioned on both text and a melody in that it can transform whistled and hummed melodies according to the style described in a text caption. To support future research, we publicly release MusicCaps, a dataset composed of 5.5k music-text pairs, with rich text descriptions provided by human experts.</description><subject>Descriptions</subject><subject>Music</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>BENPR</sourceid><recordid>eNpjYuA0MjY21LUwMTLiYOAtLs4yMDAwMjM3MjU15mRQ9i0tzkz28bVScE_NSy1KLMnMS1cAiym4FeXnKoSkVpTwMLCmJeYUp_JCaW4GZTfXEGcP3YKi_MLS1OKS-Kz80qI8oFS8kbm5gaG5pZGhiTFxqgBKUS2K</recordid><startdate>20230126</startdate><enddate>20230126</enddate><creator>Agostinelli, Andrea</creator><creator>Denk, Timo I</creator><creator>Borsos, Zalán</creator><creator>Engel, Jesse</creator><creator>Verzetti, Mauro</creator><creator>Caillon, Antoine</creator><creator>Huang, Qingqing</creator><creator>Jansen, Aren</creator><creator>Roberts, Adam</creator><creator>Tagliasacchi, Marco</creator><creator>Sharifi, Matt</creator><creator>Zeghidour, Neil</creator><creator>Frank, Christian</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20230126</creationdate><title>MusicLM: Generating Music From Text</title><author>Agostinelli, Andrea ; Denk, Timo I ; Borsos, Zalán ; Engel, Jesse ; Verzetti, Mauro ; Caillon, Antoine ; Huang, Qingqing ; Jansen, Aren ; Roberts, Adam ; Tagliasacchi, Marco ; Sharifi, Matt ; Zeghidour, Neil ; Frank, Christian</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_27701792143</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Descriptions</topic><topic>Music</topic><toplevel>online_resources</toplevel><creatorcontrib>Agostinelli, Andrea</creatorcontrib><creatorcontrib>Denk, Timo I</creatorcontrib><creatorcontrib>Borsos, Zalán</creatorcontrib><creatorcontrib>Engel, Jesse</creatorcontrib><creatorcontrib>Verzetti, Mauro</creatorcontrib><creatorcontrib>Caillon, Antoine</creatorcontrib><creatorcontrib>Huang, Qingqing</creatorcontrib><creatorcontrib>Jansen, Aren</creatorcontrib><creatorcontrib>Roberts, Adam</creatorcontrib><creatorcontrib>Tagliasacchi, Marco</creatorcontrib><creatorcontrib>Sharifi, Matt</creatorcontrib><creatorcontrib>Zeghidour, Neil</creatorcontrib><creatorcontrib>Frank, Christian</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Agostinelli, Andrea</au><au>Denk, Timo I</au><au>Borsos, Zalán</au><au>Engel, Jesse</au><au>Verzetti, Mauro</au><au>Caillon, Antoine</au><au>Huang, Qingqing</au><au>Jansen, Aren</au><au>Roberts, Adam</au><au>Tagliasacchi, Marco</au><au>Sharifi, Matt</au><au>Zeghidour, Neil</au><au>Frank, Christian</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>MusicLM: Generating Music From Text</atitle><jtitle>arXiv.org</jtitle><date>2023-01-26</date><risdate>2023</risdate><eissn>2331-8422</eissn><abstract>We introduce MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody backed by a distorted guitar riff". MusicLM casts the process of conditional music generation as a hierarchical sequence-to-sequence modeling task, and it generates music at 24 kHz that remains consistent over several minutes. Our experiments show that MusicLM outperforms previous systems both in audio quality and adherence to the text description. Moreover, we demonstrate that MusicLM can be conditioned on both text and a melody in that it can transform whistled and hummed melodies according to the style described in a text caption. To support future research, we publicly release MusicCaps, a dataset composed of 5.5k music-text pairs, with rich text descriptions provided by human experts.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2023-01
issn 2331-8422
language eng
recordid cdi_proquest_journals_2770179214
source Free E- Journals
subjects Descriptions
Music
title MusicLM: Generating Music From Text
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-06T21%3A24%3A20IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=MusicLM:%20Generating%20Music%20From%20Text&rft.jtitle=arXiv.org&rft.au=Agostinelli,%20Andrea&rft.date=2023-01-26&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E2770179214%3C/proquest%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2770179214&rft_id=info:pmid/&rfr_iscdi=true