Automatic Chinese Meme Generation Using Deep Neural Networks
Internet memes have become widely used by people for online communication and interaction, particularly through social media. Interest in meme-generation research has been increasing rapidly. In this study, we address the problem of meme generation as an image captioning task, which uses an encoder-...
Gespeichert in:
| Veröffentlicht in: | IEEE access 2021, Vol.9, p.152657-152667 |
|---|---|
| Hauptverfasser: | , , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 152667 |
|---|---|
| container_issue | |
| container_start_page | 152657 |
| container_title | IEEE access |
| container_volume | 9 |
| creator | Wang, Lin Zhang, Qimeng Kim, Youngbin Wu, Ruizheng Jin, Hongyu Deng, Haoke Luo, Pengchu Kim, Chang-Hun |
| description | Internet memes have become widely used by people for online communication and interaction, particularly through social media. Interest in meme-generation research has been increasing rapidly. In this study, we address the problem of meme generation as an image captioning task, which uses an encoder-decoder architecture to generate Chinese meme texts that match image content. First, to train the model on the characteristics of Chinese memes, we collected a dataset of 3,000 meme images with 30,000 corresponding humorous Chinese meme texts. Second, we introduced a Chinese meme generation system that can generate humorous and relevant texts from any given image. Our system used a pre-trained ResNet-50 for image feature extraction and a state-of-the-art transformer-based GPT-2 model to generate Chinese meme texts. Finally, we combined the generated text and images to form common image memes. We performed qualitative evaluations of the generated Chinese meme texts through different user studies. The evaluation results revealed that the Chinese memes generated by our model were indistinguishable from real ones. |
| doi_str_mv | 10.1109/ACCESS.2021.3127324 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2599203649</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9611242</ieee_id><doaj_id>oai_doaj_org_article_2d90e3c311a84b9f9e40afd641a880ae</doaj_id><sourcerecordid>2599203649</sourcerecordid><originalsourceid>FETCH-LOGICAL-c358t-99d907f1b21d8c418fb168fc29408eec4e65c47650262d81e9eb3689776815f23</originalsourceid><addsrcrecordid>eNqNkUtPxCAUhRujiUb9BW6auDQzcoFSSNxM6jPxsVDXhNKLdpwpI7SZ-O9lrFGXsrlwc8658JFlR0CmAESdzqrq4vFxSgmFKQNaMsq3sj0KQk1YwcT2n_1udhjjnKQlU6so97Kz2dD7pelbm1evbYcR8ztcYn6FHYbU9l3-HNvuJT9HXOX3OASzSKVf-_AWD7IdZxYRD7_rfvZ8efFUXU9uH65uqtntxLJC9hOlGkVKBzWFRloO0tUgpLNUcSIRLUdRWF6KglBBGwmosGZCqrIUEgpH2X52M-Y23sz1KrRLEz60N63-avjwok1IT1igpmkUMssAjOS1cgo5Ma4RPJ0lMZiyjsesVfDvA8Zez_0QunR9TQulKGGCq6Rio8oGH2NA9zMViN5Q1yN1vaGuv6knlxxda6y9i7bFzuKPM1EvKSmAsc0HQNX2X3wrP3R9sp7835rUR6O6RfxVKQFAOWWfEnWb1Q</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2599203649</pqid></control><display><type>article</type><title>Automatic Chinese Meme Generation Using Deep Neural Networks</title><source>IEEE Open Access Journals</source><source>DOAJ Directory of Open Access Journals</source><source>Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals</source><source>Web of Science - Science Citation Index Expanded - 2021<img src="https://exlibris-pub.s3.amazonaws.com/fromwos-v2.jpg" /></source><creator>Wang, Lin ; Zhang, Qimeng ; Kim, Youngbin ; Wu, Ruizheng ; Jin, Hongyu ; Deng, Haoke ; Luo, Pengchu ; Kim, Chang-Hun</creator><creatorcontrib>Wang, Lin ; Zhang, Qimeng ; Kim, Youngbin ; Wu, Ruizheng ; Jin, Hongyu ; Deng, Haoke ; Luo, Pengchu ; Kim, Chang-Hun</creatorcontrib><description>Internet memes have become widely used by people for online communication and interaction, particularly through social media. Interest in meme-generation research has been increasing rapidly. In this study, we address the problem of meme generation as an image captioning task, which uses an encoder-decoder architecture to generate Chinese meme texts that match image content. First, to train the model on the characteristics of Chinese memes, we collected a dataset of 3,000 meme images with 30,000 corresponding humorous Chinese meme texts. Second, we introduced a Chinese meme generation system that can generate humorous and relevant texts from any given image. Our system used a pre-trained ResNet-50 for image feature extraction and a state-of-the-art transformer-based GPT-2 model to generate Chinese meme texts. Finally, we combined the generated text and images to form common image memes. We performed qualitative evaluations of the generated Chinese meme texts through different user studies. The evaluation results revealed that the Chinese memes generated by our model were indistinguishable from real ones.</description><identifier>ISSN: 2169-3536</identifier><identifier>EISSN: 2169-3536</identifier><identifier>DOI: 10.1109/ACCESS.2021.3127324</identifier><identifier>CODEN: IAECCG</identifier><language>eng</language><publisher>PISCATAWAY: IEEE</publisher><subject>Artificial neural networks ; Coders ; Computer architecture ; Computer Science ; Computer Science, Information Systems ; computer vision ; Decoding ; Deep learning ; Engineering ; Engineering, Electrical & Electronic ; Feature extraction ; image captioning ; Internet ; internet meme ; meme generation ; Science & Technology ; Social networking (online) ; Task analysis ; Technology ; Telecommunications ; Texts ; Transformers</subject><ispartof>IEEE access, 2021, Vol.9, p.152657-152667</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>true</woscitedreferencessubscribed><woscitedreferencescount>0</woscitedreferencescount><woscitedreferencesoriginalsourcerecordid>wos000720513300001</woscitedreferencesoriginalsourcerecordid><cites>FETCH-LOGICAL-c358t-99d907f1b21d8c418fb168fc29408eec4e65c47650262d81e9eb3689776815f23</cites><orcidid>0000-0003-4679-7919 ; 0000-0002-2114-0120 ; 0000-0002-9630-9031 ; 0000-0002-5189-7562</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9611242$$EHTML$$P50$$Gieee$$Hfree_for_read</linktohtml><link.rule.ids>315,781,785,865,2103,2115,4025,27638,27928,27929,27930,39263,54938</link.rule.ids></links><search><creatorcontrib>Wang, Lin</creatorcontrib><creatorcontrib>Zhang, Qimeng</creatorcontrib><creatorcontrib>Kim, Youngbin</creatorcontrib><creatorcontrib>Wu, Ruizheng</creatorcontrib><creatorcontrib>Jin, Hongyu</creatorcontrib><creatorcontrib>Deng, Haoke</creatorcontrib><creatorcontrib>Luo, Pengchu</creatorcontrib><creatorcontrib>Kim, Chang-Hun</creatorcontrib><title>Automatic Chinese Meme Generation Using Deep Neural Networks</title><title>IEEE access</title><addtitle>Access</addtitle><addtitle>IEEE ACCESS</addtitle><description>Internet memes have become widely used by people for online communication and interaction, particularly through social media. Interest in meme-generation research has been increasing rapidly. In this study, we address the problem of meme generation as an image captioning task, which uses an encoder-decoder architecture to generate Chinese meme texts that match image content. First, to train the model on the characteristics of Chinese memes, we collected a dataset of 3,000 meme images with 30,000 corresponding humorous Chinese meme texts. Second, we introduced a Chinese meme generation system that can generate humorous and relevant texts from any given image. Our system used a pre-trained ResNet-50 for image feature extraction and a state-of-the-art transformer-based GPT-2 model to generate Chinese meme texts. Finally, we combined the generated text and images to form common image memes. We performed qualitative evaluations of the generated Chinese meme texts through different user studies. The evaluation results revealed that the Chinese memes generated by our model were indistinguishable from real ones.</description><subject>Artificial neural networks</subject><subject>Coders</subject><subject>Computer architecture</subject><subject>Computer Science</subject><subject>Computer Science, Information Systems</subject><subject>computer vision</subject><subject>Decoding</subject><subject>Deep learning</subject><subject>Engineering</subject><subject>Engineering, Electrical & Electronic</subject><subject>Feature extraction</subject><subject>image captioning</subject><subject>Internet</subject><subject>internet meme</subject><subject>meme generation</subject><subject>Science & Technology</subject><subject>Social networking (online)</subject><subject>Task analysis</subject><subject>Technology</subject><subject>Telecommunications</subject><subject>Texts</subject><subject>Transformers</subject><issn>2169-3536</issn><issn>2169-3536</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>ESBDL</sourceid><sourceid>RIE</sourceid><sourceid>HGBXW</sourceid><sourceid>DOA</sourceid><recordid>eNqNkUtPxCAUhRujiUb9BW6auDQzcoFSSNxM6jPxsVDXhNKLdpwpI7SZ-O9lrFGXsrlwc8658JFlR0CmAESdzqrq4vFxSgmFKQNaMsq3sj0KQk1YwcT2n_1udhjjnKQlU6so97Kz2dD7pelbm1evbYcR8ztcYn6FHYbU9l3-HNvuJT9HXOX3OASzSKVf-_AWD7IdZxYRD7_rfvZ8efFUXU9uH65uqtntxLJC9hOlGkVKBzWFRloO0tUgpLNUcSIRLUdRWF6KglBBGwmosGZCqrIUEgpH2X52M-Y23sz1KrRLEz60N63-avjwok1IT1igpmkUMssAjOS1cgo5Ma4RPJ0lMZiyjsesVfDvA8Zez_0QunR9TQulKGGCq6Rio8oGH2NA9zMViN5Q1yN1vaGuv6knlxxda6y9i7bFzuKPM1EvKSmAsc0HQNX2X3wrP3R9sp7835rUR6O6RfxVKQFAOWWfEnWb1Q</recordid><startdate>2021</startdate><enddate>2021</enddate><creator>Wang, Lin</creator><creator>Zhang, Qimeng</creator><creator>Kim, Youngbin</creator><creator>Wu, Ruizheng</creator><creator>Jin, Hongyu</creator><creator>Deng, Haoke</creator><creator>Luo, Pengchu</creator><creator>Kim, Chang-Hun</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>ESBDL</scope><scope>RIA</scope><scope>RIE</scope><scope>BLEPL</scope><scope>DTL</scope><scope>HGBXW</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>7SR</scope><scope>8BQ</scope><scope>8FD</scope><scope>JG9</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0003-4679-7919</orcidid><orcidid>https://orcid.org/0000-0002-2114-0120</orcidid><orcidid>https://orcid.org/0000-0002-9630-9031</orcidid><orcidid>https://orcid.org/0000-0002-5189-7562</orcidid></search><sort><creationdate>2021</creationdate><title>Automatic Chinese Meme Generation Using Deep Neural Networks</title><author>Wang, Lin ; Zhang, Qimeng ; Kim, Youngbin ; Wu, Ruizheng ; Jin, Hongyu ; Deng, Haoke ; Luo, Pengchu ; Kim, Chang-Hun</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c358t-99d907f1b21d8c418fb168fc29408eec4e65c47650262d81e9eb3689776815f23</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Artificial neural networks</topic><topic>Coders</topic><topic>Computer architecture</topic><topic>Computer Science</topic><topic>Computer Science, Information Systems</topic><topic>computer vision</topic><topic>Decoding</topic><topic>Deep learning</topic><topic>Engineering</topic><topic>Engineering, Electrical & Electronic</topic><topic>Feature extraction</topic><topic>image captioning</topic><topic>Internet</topic><topic>internet meme</topic><topic>meme generation</topic><topic>Science & Technology</topic><topic>Social networking (online)</topic><topic>Task analysis</topic><topic>Technology</topic><topic>Telecommunications</topic><topic>Texts</topic><topic>Transformers</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Wang, Lin</creatorcontrib><creatorcontrib>Zhang, Qimeng</creatorcontrib><creatorcontrib>Kim, Youngbin</creatorcontrib><creatorcontrib>Wu, Ruizheng</creatorcontrib><creatorcontrib>Jin, Hongyu</creatorcontrib><creatorcontrib>Deng, Haoke</creatorcontrib><creatorcontrib>Luo, Pengchu</creatorcontrib><creatorcontrib>Kim, Chang-Hun</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE Open Access Journals</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>Web of Science Core Collection</collection><collection>Science Citation Index Expanded</collection><collection>Web of Science - Science Citation Index Expanded - 2021</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Engineered Materials Abstracts</collection><collection>METADEX</collection><collection>Technology Research Database</collection><collection>Materials Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>IEEE access</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Wang, Lin</au><au>Zhang, Qimeng</au><au>Kim, Youngbin</au><au>Wu, Ruizheng</au><au>Jin, Hongyu</au><au>Deng, Haoke</au><au>Luo, Pengchu</au><au>Kim, Chang-Hun</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Automatic Chinese Meme Generation Using Deep Neural Networks</atitle><jtitle>IEEE access</jtitle><stitle>Access</stitle><stitle>IEEE ACCESS</stitle><date>2021</date><risdate>2021</risdate><volume>9</volume><spage>152657</spage><epage>152667</epage><pages>152657-152667</pages><issn>2169-3536</issn><eissn>2169-3536</eissn><coden>IAECCG</coden><abstract>Internet memes have become widely used by people for online communication and interaction, particularly through social media. Interest in meme-generation research has been increasing rapidly. In this study, we address the problem of meme generation as an image captioning task, which uses an encoder-decoder architecture to generate Chinese meme texts that match image content. First, to train the model on the characteristics of Chinese memes, we collected a dataset of 3,000 meme images with 30,000 corresponding humorous Chinese meme texts. Second, we introduced a Chinese meme generation system that can generate humorous and relevant texts from any given image. Our system used a pre-trained ResNet-50 for image feature extraction and a state-of-the-art transformer-based GPT-2 model to generate Chinese meme texts. Finally, we combined the generated text and images to form common image memes. We performed qualitative evaluations of the generated Chinese meme texts through different user studies. The evaluation results revealed that the Chinese memes generated by our model were indistinguishable from real ones.</abstract><cop>PISCATAWAY</cop><pub>IEEE</pub><doi>10.1109/ACCESS.2021.3127324</doi><tpages>11</tpages><orcidid>https://orcid.org/0000-0003-4679-7919</orcidid><orcidid>https://orcid.org/0000-0002-2114-0120</orcidid><orcidid>https://orcid.org/0000-0002-9630-9031</orcidid><orcidid>https://orcid.org/0000-0002-5189-7562</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 2169-3536 |
| ispartof | IEEE access, 2021, Vol.9, p.152657-152667 |
| issn | 2169-3536 2169-3536 |
| language | eng |
| recordid | cdi_proquest_journals_2599203649 |
| source | IEEE Open Access Journals; DOAJ Directory of Open Access Journals; Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals; Web of Science - Science Citation Index Expanded - 2021<img src="https://exlibris-pub.s3.amazonaws.com/fromwos-v2.jpg" /> |
| subjects | Artificial neural networks Coders Computer architecture Computer Science Computer Science, Information Systems computer vision Decoding Deep learning Engineering Engineering, Electrical & Electronic Feature extraction image captioning Internet internet meme meme generation Science & Technology Social networking (online) Task analysis Technology Telecommunications Texts Transformers |
| title | Automatic Chinese Meme Generation Using Deep Neural Networks |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-15T05%3A18%3A55IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Automatic%20Chinese%20Meme%20Generation%20Using%20Deep%20Neural%20Networks&rft.jtitle=IEEE%20access&rft.au=Wang,%20Lin&rft.date=2021&rft.volume=9&rft.spage=152657&rft.epage=152667&rft.pages=152657-152667&rft.issn=2169-3536&rft.eissn=2169-3536&rft.coden=IAECCG&rft_id=info:doi/10.1109/ACCESS.2021.3127324&rft_dat=%3Cproquest_cross%3E2599203649%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2599203649&rft_id=info:pmid/&rft_ieee_id=9611242&rft_doaj_id=oai_doaj_org_article_2d90e3c311a84b9f9e40afd641a880ae&rfr_iscdi=true |