Blank Language Models
We propose Blank Language Model (BLM), a model that generates sequences by dynamically creating and filling in blanks. The blanks control which part of the sequence to expand, making BLM ideal for a variety of text editing and rewriting tasks. The model can start from a single blank or partially com...
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | |
| container_volume | |
| creator | Shen, Tianxiao Quach, Victor Barzilay, Regina Jaakkola, Tommi |
| description | We propose Blank Language Model (BLM), a model that generates sequences by
dynamically creating and filling in blanks. The blanks control which part of
the sequence to expand, making BLM ideal for a variety of text editing and
rewriting tasks. The model can start from a single blank or partially completed
text with blanks at specified locations. It iteratively determines which word
to place in a blank and whether to insert new blanks, and stops generating when
no blanks are left to fill. BLM can be efficiently trained using a lower bound
of the marginal data likelihood. On the task of filling missing text snippets,
BLM significantly outperforms all other baselines in terms of both accuracy and
fluency. Experiments on style transfer and damaged ancient text restoration
demonstrate the potential of this framework for a wide range of applications. |
| doi_str_mv | 10.48550/arxiv.2002.03079 |
| format | Article |
| fullrecord | <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2002_03079</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2002_03079</sourcerecordid><originalsourceid>FETCH-LOGICAL-a679-7b4037a0333c62452dc89e0dfb9b41272ab55ced6c1807464fcef59b68ea7563</originalsourceid><addsrcrecordid>eNotzrkOgkAUQNFpLAxaWljJD4CPWZlSiVuCsdCevFkgREQD0ejfG5fqdjeHkGkCMU-FgDl2z_oRUwAaAwOlh2SybLA9hzm21R0rH-6vzjf9iAxKbHo__jcgx_XqlG2j_LDZZYs8Qql0pAwHphAYY1ZSLqizqfbgSqMNT6iiaISw3kmbpKC45KX1pdBGph6VkCwgs9_1yypuXX3B7lV8eMWXx94qoDLh</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Blank Language Models</title><source>arXiv.org</source><creator>Shen, Tianxiao ; Quach, Victor ; Barzilay, Regina ; Jaakkola, Tommi</creator><creatorcontrib>Shen, Tianxiao ; Quach, Victor ; Barzilay, Regina ; Jaakkola, Tommi</creatorcontrib><description>We propose Blank Language Model (BLM), a model that generates sequences by
dynamically creating and filling in blanks. The blanks control which part of
the sequence to expand, making BLM ideal for a variety of text editing and
rewriting tasks. The model can start from a single blank or partially completed
text with blanks at specified locations. It iteratively determines which word
to place in a blank and whether to insert new blanks, and stops generating when
no blanks are left to fill. BLM can be efficiently trained using a lower bound
of the marginal data likelihood. On the task of filling missing text snippets,
BLM significantly outperforms all other baselines in terms of both accuracy and
fluency. Experiments on style transfer and damaged ancient text restoration
demonstrate the potential of this framework for a wide range of applications.</description><identifier>DOI: 10.48550/arxiv.2002.03079</identifier><language>eng</language><subject>Computer Science - Computation and Language ; Computer Science - Learning</subject><creationdate>2020-02</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,776,881</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2002.03079$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2002.03079$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Shen, Tianxiao</creatorcontrib><creatorcontrib>Quach, Victor</creatorcontrib><creatorcontrib>Barzilay, Regina</creatorcontrib><creatorcontrib>Jaakkola, Tommi</creatorcontrib><title>Blank Language Models</title><description>We propose Blank Language Model (BLM), a model that generates sequences by
dynamically creating and filling in blanks. The blanks control which part of
the sequence to expand, making BLM ideal for a variety of text editing and
rewriting tasks. The model can start from a single blank or partially completed
text with blanks at specified locations. It iteratively determines which word
to place in a blank and whether to insert new blanks, and stops generating when
no blanks are left to fill. BLM can be efficiently trained using a lower bound
of the marginal data likelihood. On the task of filling missing text snippets,
BLM significantly outperforms all other baselines in terms of both accuracy and
fluency. Experiments on style transfer and damaged ancient text restoration
demonstrate the potential of this framework for a wide range of applications.</description><subject>Computer Science - Computation and Language</subject><subject>Computer Science - Learning</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotzrkOgkAUQNFpLAxaWljJD4CPWZlSiVuCsdCevFkgREQD0ejfG5fqdjeHkGkCMU-FgDl2z_oRUwAaAwOlh2SybLA9hzm21R0rH-6vzjf9iAxKbHo__jcgx_XqlG2j_LDZZYs8Qql0pAwHphAYY1ZSLqizqfbgSqMNT6iiaISw3kmbpKC45KX1pdBGph6VkCwgs9_1yypuXX3B7lV8eMWXx94qoDLh</recordid><startdate>20200207</startdate><enddate>20200207</enddate><creator>Shen, Tianxiao</creator><creator>Quach, Victor</creator><creator>Barzilay, Regina</creator><creator>Jaakkola, Tommi</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20200207</creationdate><title>Blank Language Models</title><author>Shen, Tianxiao ; Quach, Victor ; Barzilay, Regina ; Jaakkola, Tommi</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a679-7b4037a0333c62452dc89e0dfb9b41272ab55ced6c1807464fcef59b68ea7563</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Computer Science - Computation and Language</topic><topic>Computer Science - Learning</topic><toplevel>online_resources</toplevel><creatorcontrib>Shen, Tianxiao</creatorcontrib><creatorcontrib>Quach, Victor</creatorcontrib><creatorcontrib>Barzilay, Regina</creatorcontrib><creatorcontrib>Jaakkola, Tommi</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Shen, Tianxiao</au><au>Quach, Victor</au><au>Barzilay, Regina</au><au>Jaakkola, Tommi</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Blank Language Models</atitle><date>2020-02-07</date><risdate>2020</risdate><abstract>We propose Blank Language Model (BLM), a model that generates sequences by
dynamically creating and filling in blanks. The blanks control which part of
the sequence to expand, making BLM ideal for a variety of text editing and
rewriting tasks. The model can start from a single blank or partially completed
text with blanks at specified locations. It iteratively determines which word
to place in a blank and whether to insert new blanks, and stops generating when
no blanks are left to fill. BLM can be efficiently trained using a lower bound
of the marginal data likelihood. On the task of filling missing text snippets,
BLM significantly outperforms all other baselines in terms of both accuracy and
fluency. Experiments on style transfer and damaged ancient text restoration
demonstrate the potential of this framework for a wide range of applications.</abstract><doi>10.48550/arxiv.2002.03079</doi><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | DOI: 10.48550/arxiv.2002.03079 |
| ispartof | |
| issn | |
| language | eng |
| recordid | cdi_arxiv_primary_2002_03079 |
| source | arXiv.org |
| subjects | Computer Science - Computation and Language Computer Science - Learning |
| title | Blank Language Models |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-28T05%3A25%3A43IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Blank%20Language%20Models&rft.au=Shen,%20Tianxiao&rft.date=2020-02-07&rft_id=info:doi/10.48550/arxiv.2002.03079&rft_dat=%3Carxiv_GOX%3E2002_03079%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |