Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences
Current work on image-based story generation suffers from the fact that the existing image sequence collections do not have coherent plots behind them. We improve visual story generation by producing a new image-grounded dataset, Visual Writing Prompts (VWP). VWP contains almost 2K selected sequence...
Gespeichert in:
| Veröffentlicht in: | Transactions of the Association for Computational Linguistics 2023-06, Vol.11, p.565-581 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 581 |
|---|---|
| container_issue | |
| container_start_page | 565 |
| container_title | Transactions of the Association for Computational Linguistics |
| container_volume | 11 |
| creator | Hong, Xudong Sayeed, Asad Mehra, Khushboo Demberg, Vera Schiele, Bernt |
| description | Current work on image-based story generation suffers from the fact that the existing image sequence collections do not have coherent plots behind them. We improve visual story generation by producing a new image-grounded dataset, Visual Writing Prompts (VWP). VWP contains almost 2K selected sequences of movie shots, each including 5-10 images. The image sequences are aligned with a total of 12K stories which were collected via crowdsourcing given the image sequences and a set of grounded characters from the corresponding image sequence. Our new image sequence collection and filtering process has allowed us to obtain stories that are more coherent, diverse, and visually grounded compared to previous work. We also propose a character-based story generation model driven by coherence as a strong baseline. Evaluations show that our generated stories are more coherent, visually grounded, and diverse than stories generated with the current state-of-the-art model. Our code, image features, annotations and collected stories are available at
. |
| doi_str_mv | 10.1162/tacl_a_00553 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2893946864</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><doaj_id>oai_doaj_org_article_ee386cf2c0d44ad7b338b95c4efd7dcc</doaj_id><sourcerecordid>2893946864</sourcerecordid><originalsourceid>FETCH-LOGICAL-c450t-87bcafb7a1bb914c203ddae8cca15cfba5a4432dc3b0787f5b76575ce7bc0e5a3</originalsourceid><addsrcrecordid>eNptkVtrFDEUgIeiYKl96w8Y8MUHR3OdZHwQZKnrQkGhavsWTi4zzbI7GZOMpf56046UVfp0LvnOFw6nqs4weotxS95lMDsFCiHO6VF1TCgSDZXi-tlB_qI6TWmLEMISS9SS4-r6h08z7Oqr6LMfh_prDPspp_f16gYimOxis45hHq2z9WUO8a5eu9FFyD6M9a3PN_VqLlV53exhcPWl-zm70bj0snrewy6507_xpPr-6fzb6nNz8WW9WX28aAzjKDdSaAO9FoC17jAzBFFrwUljAHPTa-DAGCXWUI2EFD3XouWCG1fmkONAT6rN4rUBtmqKfg_xTgXw6qER4qAgZm92TjlHZWt6YpBlDKzQlErdccNcb4U1priaxZVu3TTrf2zDPKnSGmaVnKJE0q4r_KuFn2Ioa6estmGOY1lXEdnRjrWyZYV6s1AmhpSi6x-9GKn706nD0xX8w4Lv_YHvHvmFsaJMtGWMIELVQ1S__fS_4PUTgif_-gNIN7EI</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2893946864</pqid></control><display><type>article</type><title>Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences</title><source>DOAJ Directory of Open Access Journals</source><source>Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals</source><source>ProQuest Central</source><creator>Hong, Xudong ; Sayeed, Asad ; Mehra, Khushboo ; Demberg, Vera ; Schiele, Bernt</creator><creatorcontrib>Hong, Xudong ; Sayeed, Asad ; Mehra, Khushboo ; Demberg, Vera ; Schiele, Bernt</creatorcontrib><description>Current work on image-based story generation suffers from the fact that the existing image sequence collections do not have coherent plots behind them. We improve visual story generation by producing a new image-grounded dataset, Visual Writing Prompts (VWP). VWP contains almost 2K selected sequences of movie shots, each including 5-10 images. The image sequences are aligned with a total of 12K stories which were collected via crowdsourcing given the image sequences and a set of grounded characters from the corresponding image sequence. Our new image sequence collection and filtering process has allowed us to obtain stories that are more coherent, diverse, and visually grounded compared to previous work. We also propose a character-based story generation model driven by coherence as a strong baseline. Evaluations show that our generated stories are more coherent, visually grounded, and diverse than stories generated with the current state-of-the-art model. Our code, image features, annotations and collected stories are available at
.</description><identifier>ISSN: 2307-387X</identifier><identifier>EISSN: 2307-387X</identifier><identifier>DOI: 10.1162/tacl_a_00553</identifier><language>eng</language><publisher>One Broadway, 12th Floor, Cambridge, Massachusetts 02142, USA: MIT Press</publisher><subject>Annotations ; Coherence ; Comparative Language Studies and Linguistics ; Computational linguistics ; Datasets ; Image filters ; Image sequencing ; Informatics ; Jämförande språkvetenskap och allmän lingvistik ; Linguistics ; Motion pictures ; Narratives ; Short stories ; Storytelling ; Writing</subject><ispartof>Transactions of the Association for Computational Linguistics, 2023-06, Vol.11, p.565-581</ispartof><rights>2023. This work is published under https://creativecommons.org/licenses/by/4.0/legalcode (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c450t-87bcafb7a1bb914c203ddae8cca15cfba5a4432dc3b0787f5b76575ce7bc0e5a3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/2893946864?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>230,314,776,780,860,881,2096,21367,27901,27902,33721,43781</link.rule.ids><backlink>$$Uhttps://gup.ub.gu.se/publication/328399$$DView record from Swedish Publication Index$$Hfree_for_read</backlink></links><search><creatorcontrib>Hong, Xudong</creatorcontrib><creatorcontrib>Sayeed, Asad</creatorcontrib><creatorcontrib>Mehra, Khushboo</creatorcontrib><creatorcontrib>Demberg, Vera</creatorcontrib><creatorcontrib>Schiele, Bernt</creatorcontrib><title>Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences</title><title>Transactions of the Association for Computational Linguistics</title><description>Current work on image-based story generation suffers from the fact that the existing image sequence collections do not have coherent plots behind them. We improve visual story generation by producing a new image-grounded dataset, Visual Writing Prompts (VWP). VWP contains almost 2K selected sequences of movie shots, each including 5-10 images. The image sequences are aligned with a total of 12K stories which were collected via crowdsourcing given the image sequences and a set of grounded characters from the corresponding image sequence. Our new image sequence collection and filtering process has allowed us to obtain stories that are more coherent, diverse, and visually grounded compared to previous work. We also propose a character-based story generation model driven by coherence as a strong baseline. Evaluations show that our generated stories are more coherent, visually grounded, and diverse than stories generated with the current state-of-the-art model. Our code, image features, annotations and collected stories are available at
.</description><subject>Annotations</subject><subject>Coherence</subject><subject>Comparative Language Studies and Linguistics</subject><subject>Computational linguistics</subject><subject>Datasets</subject><subject>Image filters</subject><subject>Image sequencing</subject><subject>Informatics</subject><subject>Jämförande språkvetenskap och allmän lingvistik</subject><subject>Linguistics</subject><subject>Motion pictures</subject><subject>Narratives</subject><subject>Short stories</subject><subject>Storytelling</subject><subject>Writing</subject><issn>2307-387X</issn><issn>2307-387X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>BENPR</sourceid><sourceid>DOA</sourceid><recordid>eNptkVtrFDEUgIeiYKl96w8Y8MUHR3OdZHwQZKnrQkGhavsWTi4zzbI7GZOMpf56046UVfp0LvnOFw6nqs4weotxS95lMDsFCiHO6VF1TCgSDZXi-tlB_qI6TWmLEMISS9SS4-r6h08z7Oqr6LMfh_prDPspp_f16gYimOxis45hHq2z9WUO8a5eu9FFyD6M9a3PN_VqLlV53exhcPWl-zm70bj0snrewy6507_xpPr-6fzb6nNz8WW9WX28aAzjKDdSaAO9FoC17jAzBFFrwUljAHPTa-DAGCXWUI2EFD3XouWCG1fmkONAT6rN4rUBtmqKfg_xTgXw6qER4qAgZm92TjlHZWt6YpBlDKzQlErdccNcb4U1priaxZVu3TTrf2zDPKnSGmaVnKJE0q4r_KuFn2Ioa6estmGOY1lXEdnRjrWyZYV6s1AmhpSi6x-9GKn706nD0xX8w4Lv_YHvHvmFsaJMtGWMIELVQ1S__fS_4PUTgif_-gNIN7EI</recordid><startdate>20230620</startdate><enddate>20230620</enddate><creator>Hong, Xudong</creator><creator>Sayeed, Asad</creator><creator>Mehra, Khushboo</creator><creator>Demberg, Vera</creator><creator>Schiele, Bernt</creator><general>MIT Press</general><general>MIT Press Journals, The</general><general>The MIT Press</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7T9</scope><scope>8FE</scope><scope>8FG</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ALSLI</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>CPGLG</scope><scope>CRLPW</scope><scope>DWQXO</scope><scope>GNUQQ</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K7-</scope><scope>P5Z</scope><scope>P62</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>ADTPV</scope><scope>AOWAS</scope><scope>F1U</scope><scope>DOA</scope></search><sort><creationdate>20230620</creationdate><title>Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences</title><author>Hong, Xudong ; Sayeed, Asad ; Mehra, Khushboo ; Demberg, Vera ; Schiele, Bernt</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c450t-87bcafb7a1bb914c203ddae8cca15cfba5a4432dc3b0787f5b76575ce7bc0e5a3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Annotations</topic><topic>Coherence</topic><topic>Comparative Language Studies and Linguistics</topic><topic>Computational linguistics</topic><topic>Datasets</topic><topic>Image filters</topic><topic>Image sequencing</topic><topic>Informatics</topic><topic>Jämförande språkvetenskap och allmän lingvistik</topic><topic>Linguistics</topic><topic>Motion pictures</topic><topic>Narratives</topic><topic>Short stories</topic><topic>Storytelling</topic><topic>Writing</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Hong, Xudong</creatorcontrib><creatorcontrib>Sayeed, Asad</creatorcontrib><creatorcontrib>Mehra, Khushboo</creatorcontrib><creatorcontrib>Demberg, Vera</creatorcontrib><creatorcontrib>Schiele, Bernt</creatorcontrib><collection>CrossRef</collection><collection>Linguistics and Language Behavior Abstracts (LLBA)</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Social Science Premium Collection</collection><collection>Advanced Technologies & Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>Linguistics Collection</collection><collection>Linguistics Database</collection><collection>ProQuest Central Korea</collection><collection>ProQuest Central Student</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>Computer Science Database</collection><collection>Advanced Technologies & Aerospace Database</collection><collection>ProQuest Advanced Technologies & Aerospace Collection</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>SwePub</collection><collection>SwePub Articles</collection><collection>SWEPUB Göteborgs universitet</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>Transactions of the Association for Computational Linguistics</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Hong, Xudong</au><au>Sayeed, Asad</au><au>Mehra, Khushboo</au><au>Demberg, Vera</au><au>Schiele, Bernt</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences</atitle><jtitle>Transactions of the Association for Computational Linguistics</jtitle><date>2023-06-20</date><risdate>2023</risdate><volume>11</volume><spage>565</spage><epage>581</epage><pages>565-581</pages><issn>2307-387X</issn><eissn>2307-387X</eissn><abstract>Current work on image-based story generation suffers from the fact that the existing image sequence collections do not have coherent plots behind them. We improve visual story generation by producing a new image-grounded dataset, Visual Writing Prompts (VWP). VWP contains almost 2K selected sequences of movie shots, each including 5-10 images. The image sequences are aligned with a total of 12K stories which were collected via crowdsourcing given the image sequences and a set of grounded characters from the corresponding image sequence. Our new image sequence collection and filtering process has allowed us to obtain stories that are more coherent, diverse, and visually grounded compared to previous work. We also propose a character-based story generation model driven by coherence as a strong baseline. Evaluations show that our generated stories are more coherent, visually grounded, and diverse than stories generated with the current state-of-the-art model. Our code, image features, annotations and collected stories are available at
.</abstract><cop>One Broadway, 12th Floor, Cambridge, Massachusetts 02142, USA</cop><pub>MIT Press</pub><doi>10.1162/tacl_a_00553</doi><tpages>17</tpages><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 2307-387X |
| ispartof | Transactions of the Association for Computational Linguistics, 2023-06, Vol.11, p.565-581 |
| issn | 2307-387X 2307-387X |
| language | eng |
| recordid | cdi_proquest_journals_2893946864 |
| source | DOAJ Directory of Open Access Journals; Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals; ProQuest Central |
| subjects | Annotations Coherence Comparative Language Studies and Linguistics Computational linguistics Datasets Image filters Image sequencing Informatics Jämförande språkvetenskap och allmän lingvistik Linguistics Motion pictures Narratives Short stories Storytelling Writing |
| title | Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-28T13%3A23%3A10IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Visual%20Writing%20Prompts:%20Character-Grounded%20Story%20Generation%20with%20Curated%20Image%20Sequences&rft.jtitle=Transactions%20of%20the%20Association%20for%20Computational%20Linguistics&rft.au=Hong,%20Xudong&rft.date=2023-06-20&rft.volume=11&rft.spage=565&rft.epage=581&rft.pages=565-581&rft.issn=2307-387X&rft.eissn=2307-387X&rft_id=info:doi/10.1162/tacl_a_00553&rft_dat=%3Cproquest_cross%3E2893946864%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2893946864&rft_id=info:pmid/&rft_doaj_id=oai_doaj_org_article_ee386cf2c0d44ad7b338b95c4efd7dcc&rfr_iscdi=true |