NaRLE: Natural Language Models using Reinforcement Learning with Emotion Feedback

Current research in dialogue systems is focused on conversational assistants working on short conversations in either task-oriented or open domain settings. In this paper, we focus on improving task-based conversational assistants online, primarily those working on document-type conversations (e.g.,...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Zhou, Ruijie, Deshmukh, Soham, Greer, Jeremiah, Lee, Charles
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Zhou, Ruijie
Deshmukh, Soham
Greer, Jeremiah
Lee, Charles
description Current research in dialogue systems is focused on conversational assistants working on short conversations in either task-oriented or open domain settings. In this paper, we focus on improving task-based conversational assistants online, primarily those working on document-type conversations (e.g., emails) whose contents may or may not be completely related to the assistant's task. We propose "NARLE" a deep reinforcement learning (RL) framework for improving the natural language understanding (NLU) component of dialogue systems online without the need to collect human labels for customer data. The proposed solution associates user emotion with the assistant's action and uses that to improve NLU models using policy gradients. For two intent classification problems, we empirically show that using reinforcement learning to fine tune the pre-trained supervised learning models improves performance up to 43%. Furthermore, we demonstrate the robustness of the method to partial and noisy implicit feedback.
doi_str_mv 10.48550/arxiv.2110.02148
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2110_02148</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2110_02148</sourcerecordid><originalsourceid>FETCH-LOGICAL-a678-8a9e6fc246d9d1a9fc5f05021360507d646b744b7f3c18d9f2bc5122acf371573</originalsourceid><addsrcrecordid>eNotj7tOwzAYRr0woMIDMOEXSInvDhuqUkBKi6i6R398CRaJg5yEy9uTFqYjneHTdxC6IfmaayHyO0jf4XNNySJySri-RK97OFTlPd7DNCfocAWxnaF1eDdY1414HkNs8cGF6IdkXO_ihCsHKZ70V5jecNkPUxgi3jpnGzDvV-jCQze663-u0HFbHjdPWfXy-Lx5qDKQSmcaCie9oVzawhIovBE-F8spJhcoK7lsFOeN8swQbQtPGyMIpWA8U0QotkK3f7PnpvojhR7ST31qq89t7BcPLEj1</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>NaRLE: Natural Language Models using Reinforcement Learning with Emotion Feedback</title><source>arXiv.org</source><creator>Zhou, Ruijie ; Deshmukh, Soham ; Greer, Jeremiah ; Lee, Charles</creator><creatorcontrib>Zhou, Ruijie ; Deshmukh, Soham ; Greer, Jeremiah ; Lee, Charles</creatorcontrib><description>Current research in dialogue systems is focused on conversational assistants working on short conversations in either task-oriented or open domain settings. In this paper, we focus on improving task-based conversational assistants online, primarily those working on document-type conversations (e.g., emails) whose contents may or may not be completely related to the assistant's task. We propose "NARLE" a deep reinforcement learning (RL) framework for improving the natural language understanding (NLU) component of dialogue systems online without the need to collect human labels for customer data. The proposed solution associates user emotion with the assistant's action and uses that to improve NLU models using policy gradients. For two intent classification problems, we empirically show that using reinforcement learning to fine tune the pre-trained supervised learning models improves performance up to 43%. Furthermore, we demonstrate the robustness of the method to partial and noisy implicit feedback.</description><identifier>DOI: 10.48550/arxiv.2110.02148</identifier><language>eng</language><subject>Computer Science - Computation and Language ; Computer Science - Learning</subject><creationdate>2021-10</creationdate><rights>http://creativecommons.org/licenses/by/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2110.02148$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2110.02148$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Zhou, Ruijie</creatorcontrib><creatorcontrib>Deshmukh, Soham</creatorcontrib><creatorcontrib>Greer, Jeremiah</creatorcontrib><creatorcontrib>Lee, Charles</creatorcontrib><title>NaRLE: Natural Language Models using Reinforcement Learning with Emotion Feedback</title><description>Current research in dialogue systems is focused on conversational assistants working on short conversations in either task-oriented or open domain settings. In this paper, we focus on improving task-based conversational assistants online, primarily those working on document-type conversations (e.g., emails) whose contents may or may not be completely related to the assistant's task. We propose "NARLE" a deep reinforcement learning (RL) framework for improving the natural language understanding (NLU) component of dialogue systems online without the need to collect human labels for customer data. The proposed solution associates user emotion with the assistant's action and uses that to improve NLU models using policy gradients. For two intent classification problems, we empirically show that using reinforcement learning to fine tune the pre-trained supervised learning models improves performance up to 43%. Furthermore, we demonstrate the robustness of the method to partial and noisy implicit feedback.</description><subject>Computer Science - Computation and Language</subject><subject>Computer Science - Learning</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotj7tOwzAYRr0woMIDMOEXSInvDhuqUkBKi6i6R398CRaJg5yEy9uTFqYjneHTdxC6IfmaayHyO0jf4XNNySJySri-RK97OFTlPd7DNCfocAWxnaF1eDdY1414HkNs8cGF6IdkXO_ihCsHKZ70V5jecNkPUxgi3jpnGzDvV-jCQze663-u0HFbHjdPWfXy-Lx5qDKQSmcaCie9oVzawhIovBE-F8spJhcoK7lsFOeN8swQbQtPGyMIpWA8U0QotkK3f7PnpvojhR7ST31qq89t7BcPLEj1</recordid><startdate>20211005</startdate><enddate>20211005</enddate><creator>Zhou, Ruijie</creator><creator>Deshmukh, Soham</creator><creator>Greer, Jeremiah</creator><creator>Lee, Charles</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20211005</creationdate><title>NaRLE: Natural Language Models using Reinforcement Learning with Emotion Feedback</title><author>Zhou, Ruijie ; Deshmukh, Soham ; Greer, Jeremiah ; Lee, Charles</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a678-8a9e6fc246d9d1a9fc5f05021360507d646b744b7f3c18d9f2bc5122acf371573</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Computer Science - Computation and Language</topic><topic>Computer Science - Learning</topic><toplevel>online_resources</toplevel><creatorcontrib>Zhou, Ruijie</creatorcontrib><creatorcontrib>Deshmukh, Soham</creatorcontrib><creatorcontrib>Greer, Jeremiah</creatorcontrib><creatorcontrib>Lee, Charles</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Zhou, Ruijie</au><au>Deshmukh, Soham</au><au>Greer, Jeremiah</au><au>Lee, Charles</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>NaRLE: Natural Language Models using Reinforcement Learning with Emotion Feedback</atitle><date>2021-10-05</date><risdate>2021</risdate><abstract>Current research in dialogue systems is focused on conversational assistants working on short conversations in either task-oriented or open domain settings. In this paper, we focus on improving task-based conversational assistants online, primarily those working on document-type conversations (e.g., emails) whose contents may or may not be completely related to the assistant's task. We propose "NARLE" a deep reinforcement learning (RL) framework for improving the natural language understanding (NLU) component of dialogue systems online without the need to collect human labels for customer data. The proposed solution associates user emotion with the assistant's action and uses that to improve NLU models using policy gradients. For two intent classification problems, we empirically show that using reinforcement learning to fine tune the pre-trained supervised learning models improves performance up to 43%. Furthermore, we demonstrate the robustness of the method to partial and noisy implicit feedback.</abstract><doi>10.48550/arxiv.2110.02148</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2110.02148
ispartof
issn
language eng
recordid cdi_arxiv_primary_2110_02148
source arXiv.org
subjects Computer Science - Computation and Language
Computer Science - Learning
title NaRLE: Natural Language Models using Reinforcement Learning with Emotion Feedback
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-25T18%3A12%3A11IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=NaRLE:%20Natural%20Language%20Models%20using%20Reinforcement%20Learning%20with%20Emotion%20Feedback&rft.au=Zhou,%20Ruijie&rft.date=2021-10-05&rft_id=info:doi/10.48550/arxiv.2110.02148&rft_dat=%3Carxiv_GOX%3E2110_02148%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true