FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses
This paper presents a novel multi-channel speech enhancement approach, FoVNet, that enables highly efficient speech enhancement within a configurable field of view (FoV) of a smart-glasses user without needing specific target-talker(s) directions. It advances over prior works by enhancing all speake...
Gespeichert in:
Veröffentlicht in: | arXiv.org 2024-08 |
---|---|
Hauptverfasser: | , , , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | |
---|---|
container_issue | |
container_start_page | |
container_title | arXiv.org |
container_volume | |
creator | Xu, Zhongweiyang Aroudi, Ali Tan, Ke Pandey, Ashutosh Jung-Suk, Lee Xu, Buye Nesta, Francesco |
description | This paper presents a novel multi-channel speech enhancement approach, FoVNet, that enables highly efficient speech enhancement within a configurable field of view (FoV) of a smart-glasses user without needing specific target-talker(s) directions. It advances over prior works by enhancing all speakers within any given FoV, with a hybrid signal processing and deep learning approach designed with high computational efficiency. The neural network component is designed with ultra-low computation (about 50 MMACS). A multi-channel Wiener filter and a post-processing module are further used to improve perceptual quality. We evaluate our algorithm with a microphone array on smart glasses, providing a configurable, efficient solution for augmented hearing on energy-constrained devices. FoVNet excels in both computational efficiency and speech quality across multiple scenarios, making it a promising solution for smart glasses applications. |
format | Article |
fullrecord | <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_3092951633</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3092951633</sourcerecordid><originalsourceid>FETCH-proquest_journals_30929516333</originalsourceid><addsrcrecordid>eNqNi80KgkAURocgKMp3uNBasJm0n21lLaKN4VYmveaUzrWZEV8_iR6g1cfhnG_EplyIpb9ZcT5hnrXPIAh4tOZhKKbsFVN6RbeDPelSPToj7zVCrLAufCr9VGEPSYuYV3DUldQ5Nqgd9MpVcKF-uDVt56RTpEHqAg7KOjJfLMlA0kjj4FRLa9HO2biUtUXvtzO2iI-3_dlvDb07tC57Umf0oDIRbPk2XEZCiP-qD9oxSG4</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3092951633</pqid></control><display><type>article</type><title>FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses</title><source>Free E- Journals</source><creator>Xu, Zhongweiyang ; Aroudi, Ali ; Tan, Ke ; Pandey, Ashutosh ; Jung-Suk, Lee ; Xu, Buye ; Nesta, Francesco</creator><creatorcontrib>Xu, Zhongweiyang ; Aroudi, Ali ; Tan, Ke ; Pandey, Ashutosh ; Jung-Suk, Lee ; Xu, Buye ; Nesta, Francesco</creatorcontrib><description>This paper presents a novel multi-channel speech enhancement approach, FoVNet, that enables highly efficient speech enhancement within a configurable field of view (FoV) of a smart-glasses user without needing specific target-talker(s) directions. It advances over prior works by enhancing all speakers within any given FoV, with a hybrid signal processing and deep learning approach designed with high computational efficiency. The neural network component is designed with ultra-low computation (about 50 MMACS). A multi-channel Wiener filter and a post-processing module are further used to improve perceptual quality. We evaluate our algorithm with a microphone array on smart glasses, providing a configurable, efficient solution for augmented hearing on energy-constrained devices. FoVNet excels in both computational efficiency and speech quality across multiple scenarios, making it a promising solution for smart glasses applications.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Algorithms ; Computational efficiency ; Computing time ; Machine learning ; Neural networks ; Speech processing ; Wiener filtering</subject><ispartof>arXiv.org, 2024-08</ispartof><rights>2024. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>780,784</link.rule.ids></links><search><creatorcontrib>Xu, Zhongweiyang</creatorcontrib><creatorcontrib>Aroudi, Ali</creatorcontrib><creatorcontrib>Tan, Ke</creatorcontrib><creatorcontrib>Pandey, Ashutosh</creatorcontrib><creatorcontrib>Jung-Suk, Lee</creatorcontrib><creatorcontrib>Xu, Buye</creatorcontrib><creatorcontrib>Nesta, Francesco</creatorcontrib><title>FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses</title><title>arXiv.org</title><description>This paper presents a novel multi-channel speech enhancement approach, FoVNet, that enables highly efficient speech enhancement within a configurable field of view (FoV) of a smart-glasses user without needing specific target-talker(s) directions. It advances over prior works by enhancing all speakers within any given FoV, with a hybrid signal processing and deep learning approach designed with high computational efficiency. The neural network component is designed with ultra-low computation (about 50 MMACS). A multi-channel Wiener filter and a post-processing module are further used to improve perceptual quality. We evaluate our algorithm with a microphone array on smart glasses, providing a configurable, efficient solution for augmented hearing on energy-constrained devices. FoVNet excels in both computational efficiency and speech quality across multiple scenarios, making it a promising solution for smart glasses applications.</description><subject>Algorithms</subject><subject>Computational efficiency</subject><subject>Computing time</subject><subject>Machine learning</subject><subject>Neural networks</subject><subject>Speech processing</subject><subject>Wiener filtering</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><recordid>eNqNi80KgkAURocgKMp3uNBasJm0n21lLaKN4VYmveaUzrWZEV8_iR6g1cfhnG_EplyIpb9ZcT5hnrXPIAh4tOZhKKbsFVN6RbeDPelSPToj7zVCrLAufCr9VGEPSYuYV3DUldQ5Nqgd9MpVcKF-uDVt56RTpEHqAg7KOjJfLMlA0kjj4FRLa9HO2biUtUXvtzO2iI-3_dlvDb07tC57Umf0oDIRbPk2XEZCiP-qD9oxSG4</recordid><startdate>20240812</startdate><enddate>20240812</enddate><creator>Xu, Zhongweiyang</creator><creator>Aroudi, Ali</creator><creator>Tan, Ke</creator><creator>Pandey, Ashutosh</creator><creator>Jung-Suk, Lee</creator><creator>Xu, Buye</creator><creator>Nesta, Francesco</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20240812</creationdate><title>FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses</title><author>Xu, Zhongweiyang ; Aroudi, Ali ; Tan, Ke ; Pandey, Ashutosh ; Jung-Suk, Lee ; Xu, Buye ; Nesta, Francesco</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_30929516333</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Algorithms</topic><topic>Computational efficiency</topic><topic>Computing time</topic><topic>Machine learning</topic><topic>Neural networks</topic><topic>Speech processing</topic><topic>Wiener filtering</topic><toplevel>online_resources</toplevel><creatorcontrib>Xu, Zhongweiyang</creatorcontrib><creatorcontrib>Aroudi, Ali</creatorcontrib><creatorcontrib>Tan, Ke</creatorcontrib><creatorcontrib>Pandey, Ashutosh</creatorcontrib><creatorcontrib>Jung-Suk, Lee</creatorcontrib><creatorcontrib>Xu, Buye</creatorcontrib><creatorcontrib>Nesta, Francesco</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Xu, Zhongweiyang</au><au>Aroudi, Ali</au><au>Tan, Ke</au><au>Pandey, Ashutosh</au><au>Jung-Suk, Lee</au><au>Xu, Buye</au><au>Nesta, Francesco</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses</atitle><jtitle>arXiv.org</jtitle><date>2024-08-12</date><risdate>2024</risdate><eissn>2331-8422</eissn><abstract>This paper presents a novel multi-channel speech enhancement approach, FoVNet, that enables highly efficient speech enhancement within a configurable field of view (FoV) of a smart-glasses user without needing specific target-talker(s) directions. It advances over prior works by enhancing all speakers within any given FoV, with a hybrid signal processing and deep learning approach designed with high computational efficiency. The neural network component is designed with ultra-low computation (about 50 MMACS). A multi-channel Wiener filter and a post-processing module are further used to improve perceptual quality. We evaluate our algorithm with a microphone array on smart glasses, providing a configurable, efficient solution for augmented hearing on energy-constrained devices. FoVNet excels in both computational efficiency and speech quality across multiple scenarios, making it a promising solution for smart glasses applications.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | EISSN: 2331-8422 |
ispartof | arXiv.org, 2024-08 |
issn | 2331-8422 |
language | eng |
recordid | cdi_proquest_journals_3092951633 |
source | Free E- Journals |
subjects | Algorithms Computational efficiency Computing time Machine learning Neural networks Speech processing Wiener filtering |
title | FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-07T20%3A44%3A49IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=FoVNet:%20Configurable%20Field-of-View%20Speech%20Enhancement%20with%20Low%20Computation%20and%20Distortion%20for%20Smart%20Glasses&rft.jtitle=arXiv.org&rft.au=Xu,%20Zhongweiyang&rft.date=2024-08-12&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E3092951633%3C/proquest%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3092951633&rft_id=info:pmid/&rfr_iscdi=true |