Small Footprint Convolutional Recurrent Networks for Streaming Wakeword Detection
In this work, we propose small footprint Convolutional Recurrent Neural Network models applied to the problem of wakeword detection and augment them with scaled dot product attention. We find that false accepts compared to Convolutional Neural Network models in a 250k parameter budget can be reduced...
Gespeichert in:
| Veröffentlicht in: | arXiv.org 2020-11 |
|---|---|
| Hauptverfasser: | , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | arXiv.org |
| container_volume | |
| creator | Mohammad Omar Khursheed Christin, Jose Kumar, Rajath Fu, Gengshen Kulis, Brian Cheekatmalla, Santosh Kumar |
| description | In this work, we propose small footprint Convolutional Recurrent Neural Network models applied to the problem of wakeword detection and augment them with scaled dot product attention. We find that false accepts compared to Convolutional Neural Network models in a 250k parameter budget can be reduced by 25% with a 10% reduction in parameter size by using CRNNs, and we can get up to 32% improvement at a 50k parameter budget with 75% reduction in parameter size compared to word-level Dense Neural Network models. We discuss solutions to the challenging problem of performing inference on streaming audio with CRNNs, as well as differences in start-end index errors and latency in comparison to CNN, DNN, and DNN-HMM models. |
| format | Article |
| fullrecord | <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_2464482620</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2464482620</sourcerecordid><originalsourceid>FETCH-proquest_journals_24644826203</originalsourceid><addsrcrecordid>eNqNjMsKwjAUBYMgWLT_EHBdiElau68WV4IPcFlCvZW2aW7NQ3_fCn6Aq4FhzpmRiAuxSXLJ-YLEznWMMZ5teZqKiJwug9Kaloh-tK3xtEDzQh18i0ZpeoY6WAuTP4J_o-0dbdDSi7eghtY86E31MPk73YGH-rtakXmjtIP4xyVZl_trcUhGi88AzlcdBjudu4rLTMqcZ5yJ_6oPI0FAOA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2464482620</pqid></control><display><type>article</type><title>Small Footprint Convolutional Recurrent Networks for Streaming Wakeword Detection</title><source>Free E- Journals</source><creator>Mohammad Omar Khursheed ; Christin, Jose ; Kumar, Rajath ; Fu, Gengshen ; Kulis, Brian ; Cheekatmalla, Santosh Kumar</creator><creatorcontrib>Mohammad Omar Khursheed ; Christin, Jose ; Kumar, Rajath ; Fu, Gengshen ; Kulis, Brian ; Cheekatmalla, Santosh Kumar</creatorcontrib><description>In this work, we propose small footprint Convolutional Recurrent Neural Network models applied to the problem of wakeword detection and augment them with scaled dot product attention. We find that false accepts compared to Convolutional Neural Network models in a 250k parameter budget can be reduced by 25% with a 10% reduction in parameter size by using CRNNs, and we can get up to 32% improvement at a 50k parameter budget with 75% reduction in parameter size compared to word-level Dense Neural Network models. We discuss solutions to the challenging problem of performing inference on streaming audio with CRNNs, as well as differences in start-end index errors and latency in comparison to CNN, DNN, and DNN-HMM models.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Artificial neural networks ; Budgets ; Mathematical models ; Network latency ; Neural networks ; Parameters ; Recurrent neural networks ; Reduction</subject><ispartof>arXiv.org, 2020-11</ispartof><rights>2020. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>780,784</link.rule.ids></links><search><creatorcontrib>Mohammad Omar Khursheed</creatorcontrib><creatorcontrib>Christin, Jose</creatorcontrib><creatorcontrib>Kumar, Rajath</creatorcontrib><creatorcontrib>Fu, Gengshen</creatorcontrib><creatorcontrib>Kulis, Brian</creatorcontrib><creatorcontrib>Cheekatmalla, Santosh Kumar</creatorcontrib><title>Small Footprint Convolutional Recurrent Networks for Streaming Wakeword Detection</title><title>arXiv.org</title><description>In this work, we propose small footprint Convolutional Recurrent Neural Network models applied to the problem of wakeword detection and augment them with scaled dot product attention. We find that false accepts compared to Convolutional Neural Network models in a 250k parameter budget can be reduced by 25% with a 10% reduction in parameter size by using CRNNs, and we can get up to 32% improvement at a 50k parameter budget with 75% reduction in parameter size compared to word-level Dense Neural Network models. We discuss solutions to the challenging problem of performing inference on streaming audio with CRNNs, as well as differences in start-end index errors and latency in comparison to CNN, DNN, and DNN-HMM models.</description><subject>Artificial neural networks</subject><subject>Budgets</subject><subject>Mathematical models</subject><subject>Network latency</subject><subject>Neural networks</subject><subject>Parameters</subject><subject>Recurrent neural networks</subject><subject>Reduction</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><recordid>eNqNjMsKwjAUBYMgWLT_EHBdiElau68WV4IPcFlCvZW2aW7NQ3_fCn6Aq4FhzpmRiAuxSXLJ-YLEznWMMZ5teZqKiJwug9Kaloh-tK3xtEDzQh18i0ZpeoY6WAuTP4J_o-0dbdDSi7eghtY86E31MPk73YGH-rtakXmjtIP4xyVZl_trcUhGi88AzlcdBjudu4rLTMqcZ5yJ_6oPI0FAOA</recordid><startdate>20201125</startdate><enddate>20201125</enddate><creator>Mohammad Omar Khursheed</creator><creator>Christin, Jose</creator><creator>Kumar, Rajath</creator><creator>Fu, Gengshen</creator><creator>Kulis, Brian</creator><creator>Cheekatmalla, Santosh Kumar</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20201125</creationdate><title>Small Footprint Convolutional Recurrent Networks for Streaming Wakeword Detection</title><author>Mohammad Omar Khursheed ; Christin, Jose ; Kumar, Rajath ; Fu, Gengshen ; Kulis, Brian ; Cheekatmalla, Santosh Kumar</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_24644826203</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Artificial neural networks</topic><topic>Budgets</topic><topic>Mathematical models</topic><topic>Network latency</topic><topic>Neural networks</topic><topic>Parameters</topic><topic>Recurrent neural networks</topic><topic>Reduction</topic><toplevel>online_resources</toplevel><creatorcontrib>Mohammad Omar Khursheed</creatorcontrib><creatorcontrib>Christin, Jose</creatorcontrib><creatorcontrib>Kumar, Rajath</creatorcontrib><creatorcontrib>Fu, Gengshen</creatorcontrib><creatorcontrib>Kulis, Brian</creatorcontrib><creatorcontrib>Cheekatmalla, Santosh Kumar</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Mohammad Omar Khursheed</au><au>Christin, Jose</au><au>Kumar, Rajath</au><au>Fu, Gengshen</au><au>Kulis, Brian</au><au>Cheekatmalla, Santosh Kumar</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>Small Footprint Convolutional Recurrent Networks for Streaming Wakeword Detection</atitle><jtitle>arXiv.org</jtitle><date>2020-11-25</date><risdate>2020</risdate><eissn>2331-8422</eissn><abstract>In this work, we propose small footprint Convolutional Recurrent Neural Network models applied to the problem of wakeword detection and augment them with scaled dot product attention. We find that false accepts compared to Convolutional Neural Network models in a 250k parameter budget can be reduced by 25% with a 10% reduction in parameter size by using CRNNs, and we can get up to 32% improvement at a 50k parameter budget with 75% reduction in parameter size compared to word-level Dense Neural Network models. We discuss solutions to the challenging problem of performing inference on streaming audio with CRNNs, as well as differences in start-end index errors and latency in comparison to CNN, DNN, and DNN-HMM models.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | EISSN: 2331-8422 |
| ispartof | arXiv.org, 2020-11 |
| issn | 2331-8422 |
| language | eng |
| recordid | cdi_proquest_journals_2464482620 |
| source | Free E- Journals |
| subjects | Artificial neural networks Budgets Mathematical models Network latency Neural networks Parameters Recurrent neural networks Reduction |
| title | Small Footprint Convolutional Recurrent Networks for Streaming Wakeword Detection |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-12T05%3A02%3A14IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=Small%20Footprint%20Convolutional%20Recurrent%20Networks%20for%20Streaming%20Wakeword%20Detection&rft.jtitle=arXiv.org&rft.au=Mohammad%20Omar%20Khursheed&rft.date=2020-11-25&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E2464482620%3C/proquest%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2464482620&rft_id=info:pmid/&rfr_iscdi=true |