Towards Personalization of CTC Speech Recognition Models with Contextual Adapters and Adaptive Boosting
End-to-end speech recognition models trained using joint Connectionist Temporal Classification (CTC)-Attention loss have gained popularity recently. In these models, a non-autoregressive CTC decoder is often used at inference time due to its speed and simplicity. However, such models are hard to per...
Gespeichert in:
Veröffentlicht in: | arXiv.org 2022-11 |
---|---|
Hauptverfasser: | , , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | |
---|---|
container_issue | |
container_start_page | |
container_title | arXiv.org |
container_volume | |
creator | Dingliwal, Saket Sunkara, Monica Bodapati, Sravan Ronanki, Srikanth Farris, Jeff Kirchhoff, Katrin |
description | End-to-end speech recognition models trained using joint Connectionist Temporal Classification (CTC)-Attention loss have gained popularity recently. In these models, a non-autoregressive CTC decoder is often used at inference time due to its speed and simplicity. However, such models are hard to personalize because of their conditional independence assumption that prevents output tokens from previous time steps to influence future predictions. To tackle this, we propose a novel two-way approach that first biases the encoder with attention over a predefined list of rare long-tail and out-of-vocabulary (OOV) words and then uses dynamic boosting and phone alignment network during decoding to further bias the subword predictions. We evaluate our approach on open-source VoxPopuli and in-house medical datasets to showcase a 60% improvement in F1 score on domain-specific rare words over a strong CTC baseline. |
format | Article |
fullrecord | <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_2726181226</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2726181226</sourcerecordid><originalsourceid>FETCH-proquest_journals_27261812263</originalsourceid><addsrcrecordid>eNqNjdEKgkAQRZcgSMp_GOg50LHM15KilyDKd1ly1A3ZMWet6OuL6gN6ulzO5dyB8jCKwlkyRxwpX-QSBAHGS1wsIk9VGd91VwgcqBO2ujFP7Qxb4BLSLIVTS3Su4Uhnrqz5kD0X1AjcjashZevo4XrdwKrQrXtLQNviW8yNYM0szthqooalboT8X47VdLvJ0t2s7fjak7j8wn33_pcclxiHSYgYR_-tXhIUR4U</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2726181226</pqid></control><display><type>article</type><title>Towards Personalization of CTC Speech Recognition Models with Contextual Adapters and Adaptive Boosting</title><source>Freely Accessible Journals</source><creator>Dingliwal, Saket ; Sunkara, Monica ; Bodapati, Sravan ; Ronanki, Srikanth ; Farris, Jeff ; Kirchhoff, Katrin</creator><creatorcontrib>Dingliwal, Saket ; Sunkara, Monica ; Bodapati, Sravan ; Ronanki, Srikanth ; Farris, Jeff ; Kirchhoff, Katrin</creatorcontrib><description>End-to-end speech recognition models trained using joint Connectionist Temporal Classification (CTC)-Attention loss have gained popularity recently. In these models, a non-autoregressive CTC decoder is often used at inference time due to its speed and simplicity. However, such models are hard to personalize because of their conditional independence assumption that prevents output tokens from previous time steps to influence future predictions. To tackle this, we propose a novel two-way approach that first biases the encoder with attention over a predefined list of rare long-tail and out-of-vocabulary (OOV) words and then uses dynamic boosting and phone alignment network during decoding to further bias the subword predictions. We evaluate our approach on open-source VoxPopuli and in-house medical datasets to showcase a 60% improvement in F1 score on domain-specific rare words over a strong CTC baseline.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Autoregressive models ; Coders ; Decoding ; Speech recognition</subject><ispartof>arXiv.org, 2022-11</ispartof><rights>2022. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>780,784</link.rule.ids></links><search><creatorcontrib>Dingliwal, Saket</creatorcontrib><creatorcontrib>Sunkara, Monica</creatorcontrib><creatorcontrib>Bodapati, Sravan</creatorcontrib><creatorcontrib>Ronanki, Srikanth</creatorcontrib><creatorcontrib>Farris, Jeff</creatorcontrib><creatorcontrib>Kirchhoff, Katrin</creatorcontrib><title>Towards Personalization of CTC Speech Recognition Models with Contextual Adapters and Adaptive Boosting</title><title>arXiv.org</title><description>End-to-end speech recognition models trained using joint Connectionist Temporal Classification (CTC)-Attention loss have gained popularity recently. In these models, a non-autoregressive CTC decoder is often used at inference time due to its speed and simplicity. However, such models are hard to personalize because of their conditional independence assumption that prevents output tokens from previous time steps to influence future predictions. To tackle this, we propose a novel two-way approach that first biases the encoder with attention over a predefined list of rare long-tail and out-of-vocabulary (OOV) words and then uses dynamic boosting and phone alignment network during decoding to further bias the subword predictions. We evaluate our approach on open-source VoxPopuli and in-house medical datasets to showcase a 60% improvement in F1 score on domain-specific rare words over a strong CTC baseline.</description><subject>Autoregressive models</subject><subject>Coders</subject><subject>Decoding</subject><subject>Speech recognition</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><recordid>eNqNjdEKgkAQRZcgSMp_GOg50LHM15KilyDKd1ly1A3ZMWet6OuL6gN6ulzO5dyB8jCKwlkyRxwpX-QSBAHGS1wsIk9VGd91VwgcqBO2ujFP7Qxb4BLSLIVTS3Su4Uhnrqz5kD0X1AjcjashZevo4XrdwKrQrXtLQNviW8yNYM0szthqooalboT8X47VdLvJ0t2s7fjak7j8wn33_pcclxiHSYgYR_-tXhIUR4U</recordid><startdate>20221113</startdate><enddate>20221113</enddate><creator>Dingliwal, Saket</creator><creator>Sunkara, Monica</creator><creator>Bodapati, Sravan</creator><creator>Ronanki, Srikanth</creator><creator>Farris, Jeff</creator><creator>Kirchhoff, Katrin</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20221113</creationdate><title>Towards Personalization of CTC Speech Recognition Models with Contextual Adapters and Adaptive Boosting</title><author>Dingliwal, Saket ; Sunkara, Monica ; Bodapati, Sravan ; Ronanki, Srikanth ; Farris, Jeff ; Kirchhoff, Katrin</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_27261812263</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Autoregressive models</topic><topic>Coders</topic><topic>Decoding</topic><topic>Speech recognition</topic><toplevel>online_resources</toplevel><creatorcontrib>Dingliwal, Saket</creatorcontrib><creatorcontrib>Sunkara, Monica</creatorcontrib><creatorcontrib>Bodapati, Sravan</creatorcontrib><creatorcontrib>Ronanki, Srikanth</creatorcontrib><creatorcontrib>Farris, Jeff</creatorcontrib><creatorcontrib>Kirchhoff, Katrin</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Access via ProQuest (Open Access)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Dingliwal, Saket</au><au>Sunkara, Monica</au><au>Bodapati, Sravan</au><au>Ronanki, Srikanth</au><au>Farris, Jeff</au><au>Kirchhoff, Katrin</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>Towards Personalization of CTC Speech Recognition Models with Contextual Adapters and Adaptive Boosting</atitle><jtitle>arXiv.org</jtitle><date>2022-11-13</date><risdate>2022</risdate><eissn>2331-8422</eissn><abstract>End-to-end speech recognition models trained using joint Connectionist Temporal Classification (CTC)-Attention loss have gained popularity recently. In these models, a non-autoregressive CTC decoder is often used at inference time due to its speed and simplicity. However, such models are hard to personalize because of their conditional independence assumption that prevents output tokens from previous time steps to influence future predictions. To tackle this, we propose a novel two-way approach that first biases the encoder with attention over a predefined list of rare long-tail and out-of-vocabulary (OOV) words and then uses dynamic boosting and phone alignment network during decoding to further bias the subword predictions. We evaluate our approach on open-source VoxPopuli and in-house medical datasets to showcase a 60% improvement in F1 score on domain-specific rare words over a strong CTC baseline.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | EISSN: 2331-8422 |
ispartof | arXiv.org, 2022-11 |
issn | 2331-8422 |
language | eng |
recordid | cdi_proquest_journals_2726181226 |
source | Freely Accessible Journals |
subjects | Autoregressive models Coders Decoding Speech recognition |
title | Towards Personalization of CTC Speech Recognition Models with Contextual Adapters and Adaptive Boosting |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-25T20%3A35%3A06IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=Towards%20Personalization%20of%20CTC%20Speech%20Recognition%20Models%20with%20Contextual%20Adapters%20and%20Adaptive%20Boosting&rft.jtitle=arXiv.org&rft.au=Dingliwal,%20Saket&rft.date=2022-11-13&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E2726181226%3C/proquest%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2726181226&rft_id=info:pmid/&rfr_iscdi=true |