A video indexing and retrieval computational prototype based on transcribed speech
Using the voice to interact with systems is attractive in medicine and other areas due to its friendliness and flexibility. Video indexing and retrieval have benefited from this resource. However, few initiatives use speech recognition to support both tasks. This work aims to develop and evaluate a...
Gespeichert in:
| Veröffentlicht in: | Multimedia tools and applications 2021-10, Vol.80 (25), p.33971-34017 |
|---|---|
| Hauptverfasser: | , , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 34017 |
|---|---|
| container_issue | 25 |
| container_start_page | 33971 |
| container_title | Multimedia tools and applications |
| container_volume | 80 |
| creator | Spolaôr, Newton Lee, Huei Diana Takaki, Weber Shoity Resende Ensina, Leandro Augusto Parmezan, Antonio Rafael Sabino Oliva, Jefferson Tales Coy, Claudio Saddy Rodrigues Wu, Feng Chung |
| description | Using the voice to interact with systems is attractive in medicine and other areas due to its friendliness and flexibility. Video indexing and retrieval have benefited from this resource. However, few initiatives use speech recognition to support both tasks. This work aims to develop and evaluate a prototype system to index and retrieve videos from speech transcription. In particular, the user can narrate each video’s content, generating the utterance that is captured, transformed into text and timestamped by the computational system. Simple text processing techniques are then applied to the obtained transcript before indexing. Afterward, the user can also query by speech or text to find relevant videos previously indexed. We conducted an experimental evaluation of the prototype in sets of 50 and 10 public videos. As part of this process, one collaborator manually narrated the 50 videos, while four others narrated a subset of 13 videos. An automatic narration scheme was also applied to this subset and the set of 10 videos. The evaluation showed promising results regarding Brazilian Portuguese speech recognition and retrieval performance. For example, the average word error rate reached down to 0.03 and the mean average precision achieved up to 1.00. Besides performing well, the computational tool is flexible since few changes are required to support other languages. |
| doi_str_mv | 10.1007/s11042-021-11401-1 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2584133032</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2584133032</sourcerecordid><originalsourceid>FETCH-LOGICAL-c319t-4a92d7113bcf6aad02062a4b8eacc37c72067d0482c34b162e0ea39001a8bd553</originalsourceid><addsrcrecordid>eNp9kE9LAzEQxYMoWKtfwFPAc3QmyW62x1L8BwVB9ByySVq3tJs1SYt-e6MrePMyMw_eGx4_Qi4RrhFA3SREkJwBR4YoocwjMsFKCaYUx-NyiwaYqgBPyVlKGwCsKy4n5HlOD53zgXa98x9dv6amdzT6HDt_MFtqw27YZ5O70Bc1xJBD_hw8bU3yjoae5mj6ZGPXFpkG7-3bOTlZmW3yF797Sl7vbl8WD2z5dP-4mC-ZFTjLTJoZdwpRtHZVG-OAQ82NbBtvrBXKqqKVA9lwK2SLNffgjZiV5qZpXVWJKbka_5ZW73ufst6EfSw1k-ZVI1EIELy4-OiyMaQU_UoPsduZ-KkR9Dc7PbLThZ3-YaexhMQYSsXcr338e_1P6gtjp3Gu</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2584133032</pqid></control><display><type>article</type><title>A video indexing and retrieval computational prototype based on transcribed speech</title><source>Springer Nature - Complete Springer Journals</source><creator>Spolaôr, Newton ; Lee, Huei Diana ; Takaki, Weber Shoity Resende ; Ensina, Leandro Augusto ; Parmezan, Antonio Rafael Sabino ; Oliva, Jefferson Tales ; Coy, Claudio Saddy Rodrigues ; Wu, Feng Chung</creator><creatorcontrib>Spolaôr, Newton ; Lee, Huei Diana ; Takaki, Weber Shoity Resende ; Ensina, Leandro Augusto ; Parmezan, Antonio Rafael Sabino ; Oliva, Jefferson Tales ; Coy, Claudio Saddy Rodrigues ; Wu, Feng Chung</creatorcontrib><description>Using the voice to interact with systems is attractive in medicine and other areas due to its friendliness and flexibility. Video indexing and retrieval have benefited from this resource. However, few initiatives use speech recognition to support both tasks. This work aims to develop and evaluate a prototype system to index and retrieve videos from speech transcription. In particular, the user can narrate each video’s content, generating the utterance that is captured, transformed into text and timestamped by the computational system. Simple text processing techniques are then applied to the obtained transcript before indexing. Afterward, the user can also query by speech or text to find relevant videos previously indexed. We conducted an experimental evaluation of the prototype in sets of 50 and 10 public videos. As part of this process, one collaborator manually narrated the 50 videos, while four others narrated a subset of 13 videos. An automatic narration scheme was also applied to this subset and the set of 10 videos. The evaluation showed promising results regarding Brazilian Portuguese speech recognition and retrieval performance. For example, the average word error rate reached down to 0.03 and the mean average precision achieved up to 1.00. Besides performing well, the computational tool is flexible since few changes are required to support other languages.</description><identifier>ISSN: 1380-7501</identifier><identifier>EISSN: 1573-7721</identifier><identifier>DOI: 10.1007/s11042-021-11401-1</identifier><language>eng</language><publisher>New York: Springer US</publisher><subject>Computer Communication Networks ; Computer Science ; Data Structures and Information Theory ; Indexing ; Multimedia Information Systems ; Prototypes ; Retrieval ; Software ; Special Purpose and Application-Based Systems ; Speech recognition ; Video ; Voice recognition</subject><ispartof>Multimedia tools and applications, 2021-10, Vol.80 (25), p.33971-34017</ispartof><rights>The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2021</rights><rights>The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2021.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c319t-4a92d7113bcf6aad02062a4b8eacc37c72067d0482c34b162e0ea39001a8bd553</citedby><cites>FETCH-LOGICAL-c319t-4a92d7113bcf6aad02062a4b8eacc37c72067d0482c34b162e0ea39001a8bd553</cites><orcidid>0000-0003-0748-3693</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s11042-021-11401-1$$EPDF$$P50$$Gspringer$$H</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s11042-021-11401-1$$EHTML$$P50$$Gspringer$$H</linktohtml><link.rule.ids>314,776,780,27901,27902,41464,42533,51294</link.rule.ids></links><search><creatorcontrib>Spolaôr, Newton</creatorcontrib><creatorcontrib>Lee, Huei Diana</creatorcontrib><creatorcontrib>Takaki, Weber Shoity Resende</creatorcontrib><creatorcontrib>Ensina, Leandro Augusto</creatorcontrib><creatorcontrib>Parmezan, Antonio Rafael Sabino</creatorcontrib><creatorcontrib>Oliva, Jefferson Tales</creatorcontrib><creatorcontrib>Coy, Claudio Saddy Rodrigues</creatorcontrib><creatorcontrib>Wu, Feng Chung</creatorcontrib><title>A video indexing and retrieval computational prototype based on transcribed speech</title><title>Multimedia tools and applications</title><addtitle>Multimed Tools Appl</addtitle><description>Using the voice to interact with systems is attractive in medicine and other areas due to its friendliness and flexibility. Video indexing and retrieval have benefited from this resource. However, few initiatives use speech recognition to support both tasks. This work aims to develop and evaluate a prototype system to index and retrieve videos from speech transcription. In particular, the user can narrate each video’s content, generating the utterance that is captured, transformed into text and timestamped by the computational system. Simple text processing techniques are then applied to the obtained transcript before indexing. Afterward, the user can also query by speech or text to find relevant videos previously indexed. We conducted an experimental evaluation of the prototype in sets of 50 and 10 public videos. As part of this process, one collaborator manually narrated the 50 videos, while four others narrated a subset of 13 videos. An automatic narration scheme was also applied to this subset and the set of 10 videos. The evaluation showed promising results regarding Brazilian Portuguese speech recognition and retrieval performance. For example, the average word error rate reached down to 0.03 and the mean average precision achieved up to 1.00. Besides performing well, the computational tool is flexible since few changes are required to support other languages.</description><subject>Computer Communication Networks</subject><subject>Computer Science</subject><subject>Data Structures and Information Theory</subject><subject>Indexing</subject><subject>Multimedia Information Systems</subject><subject>Prototypes</subject><subject>Retrieval</subject><subject>Software</subject><subject>Special Purpose and Application-Based Systems</subject><subject>Speech recognition</subject><subject>Video</subject><subject>Voice recognition</subject><issn>1380-7501</issn><issn>1573-7721</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>8G5</sourceid><sourceid>BENPR</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><recordid>eNp9kE9LAzEQxYMoWKtfwFPAc3QmyW62x1L8BwVB9ByySVq3tJs1SYt-e6MrePMyMw_eGx4_Qi4RrhFA3SREkJwBR4YoocwjMsFKCaYUx-NyiwaYqgBPyVlKGwCsKy4n5HlOD53zgXa98x9dv6amdzT6HDt_MFtqw27YZ5O70Bc1xJBD_hw8bU3yjoae5mj6ZGPXFpkG7-3bOTlZmW3yF797Sl7vbl8WD2z5dP-4mC-ZFTjLTJoZdwpRtHZVG-OAQ82NbBtvrBXKqqKVA9lwK2SLNffgjZiV5qZpXVWJKbka_5ZW73ufst6EfSw1k-ZVI1EIELy4-OiyMaQU_UoPsduZ-KkR9Dc7PbLThZ3-YaexhMQYSsXcr338e_1P6gtjp3Gu</recordid><startdate>20211001</startdate><enddate>20211001</enddate><creator>Spolaôr, Newton</creator><creator>Lee, Huei Diana</creator><creator>Takaki, Weber Shoity Resende</creator><creator>Ensina, Leandro Augusto</creator><creator>Parmezan, Antonio Rafael Sabino</creator><creator>Oliva, Jefferson Tales</creator><creator>Coy, Claudio Saddy Rodrigues</creator><creator>Wu, Feng Chung</creator><general>Springer US</general><general>Springer Nature B.V</general><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7SC</scope><scope>7WY</scope><scope>7WZ</scope><scope>7XB</scope><scope>87Z</scope><scope>8AL</scope><scope>8AO</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>8FK</scope><scope>8FL</scope><scope>8G5</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BEZIV</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FRNLG</scope><scope>F~G</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K60</scope><scope>K6~</scope><scope>K7-</scope><scope>L.-</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>M0C</scope><scope>M0N</scope><scope>M2O</scope><scope>MBDVC</scope><scope>P5Z</scope><scope>P62</scope><scope>PQBIZ</scope><scope>PQBZA</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>Q9U</scope><orcidid>https://orcid.org/0000-0003-0748-3693</orcidid></search><sort><creationdate>20211001</creationdate><title>A video indexing and retrieval computational prototype based on transcribed speech</title><author>Spolaôr, Newton ; Lee, Huei Diana ; Takaki, Weber Shoity Resende ; Ensina, Leandro Augusto ; Parmezan, Antonio Rafael Sabino ; Oliva, Jefferson Tales ; Coy, Claudio Saddy Rodrigues ; Wu, Feng Chung</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c319t-4a92d7113bcf6aad02062a4b8eacc37c72067d0482c34b162e0ea39001a8bd553</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Computer Communication Networks</topic><topic>Computer Science</topic><topic>Data Structures and Information Theory</topic><topic>Indexing</topic><topic>Multimedia Information Systems</topic><topic>Prototypes</topic><topic>Retrieval</topic><topic>Software</topic><topic>Special Purpose and Application-Based Systems</topic><topic>Speech recognition</topic><topic>Video</topic><topic>Voice recognition</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Spolaôr, Newton</creatorcontrib><creatorcontrib>Lee, Huei Diana</creatorcontrib><creatorcontrib>Takaki, Weber Shoity Resende</creatorcontrib><creatorcontrib>Ensina, Leandro Augusto</creatorcontrib><creatorcontrib>Parmezan, Antonio Rafael Sabino</creatorcontrib><creatorcontrib>Oliva, Jefferson Tales</creatorcontrib><creatorcontrib>Coy, Claudio Saddy Rodrigues</creatorcontrib><creatorcontrib>Wu, Feng Chung</creatorcontrib><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Computer and Information Systems Abstracts</collection><collection>ABI/INFORM Collection</collection><collection>ABI/INFORM Global (PDF only)</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>ABI/INFORM Global (Alumni Edition)</collection><collection>Computing Database (Alumni Edition)</collection><collection>ProQuest Pharma Collection</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ABI/INFORM Collection (Alumni Edition)</collection><collection>Research Library (Alumni Edition)</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies & Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Business Premium Collection</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>Business Premium Collection (Alumni)</collection><collection>ABI/INFORM Global (Corporate)</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>ProQuest Business Collection (Alumni Edition)</collection><collection>ProQuest Business Collection</collection><collection>Computer Science Database</collection><collection>ABI/INFORM Professional Advanced</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>ABI/INFORM Global</collection><collection>Computing Database</collection><collection>Research Library</collection><collection>Research Library (Corporate)</collection><collection>Advanced Technologies & Aerospace Database</collection><collection>ProQuest Advanced Technologies & Aerospace Collection</collection><collection>One Business (ProQuest)</collection><collection>ProQuest One Business (Alumni)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>ProQuest Central Basic</collection><jtitle>Multimedia tools and applications</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Spolaôr, Newton</au><au>Lee, Huei Diana</au><au>Takaki, Weber Shoity Resende</au><au>Ensina, Leandro Augusto</au><au>Parmezan, Antonio Rafael Sabino</au><au>Oliva, Jefferson Tales</au><au>Coy, Claudio Saddy Rodrigues</au><au>Wu, Feng Chung</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A video indexing and retrieval computational prototype based on transcribed speech</atitle><jtitle>Multimedia tools and applications</jtitle><stitle>Multimed Tools Appl</stitle><date>2021-10-01</date><risdate>2021</risdate><volume>80</volume><issue>25</issue><spage>33971</spage><epage>34017</epage><pages>33971-34017</pages><issn>1380-7501</issn><eissn>1573-7721</eissn><abstract>Using the voice to interact with systems is attractive in medicine and other areas due to its friendliness and flexibility. Video indexing and retrieval have benefited from this resource. However, few initiatives use speech recognition to support both tasks. This work aims to develop and evaluate a prototype system to index and retrieve videos from speech transcription. In particular, the user can narrate each video’s content, generating the utterance that is captured, transformed into text and timestamped by the computational system. Simple text processing techniques are then applied to the obtained transcript before indexing. Afterward, the user can also query by speech or text to find relevant videos previously indexed. We conducted an experimental evaluation of the prototype in sets of 50 and 10 public videos. As part of this process, one collaborator manually narrated the 50 videos, while four others narrated a subset of 13 videos. An automatic narration scheme was also applied to this subset and the set of 10 videos. The evaluation showed promising results regarding Brazilian Portuguese speech recognition and retrieval performance. For example, the average word error rate reached down to 0.03 and the mean average precision achieved up to 1.00. Besides performing well, the computational tool is flexible since few changes are required to support other languages.</abstract><cop>New York</cop><pub>Springer US</pub><doi>10.1007/s11042-021-11401-1</doi><tpages>47</tpages><orcidid>https://orcid.org/0000-0003-0748-3693</orcidid></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 1380-7501 |
| ispartof | Multimedia tools and applications, 2021-10, Vol.80 (25), p.33971-34017 |
| issn | 1380-7501 1573-7721 |
| language | eng |
| recordid | cdi_proquest_journals_2584133032 |
| source | Springer Nature - Complete Springer Journals |
| subjects | Computer Communication Networks Computer Science Data Structures and Information Theory Indexing Multimedia Information Systems Prototypes Retrieval Software Special Purpose and Application-Based Systems Speech recognition Video Voice recognition |
| title | A video indexing and retrieval computational prototype based on transcribed speech |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-29T21%3A01%3A40IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20video%20indexing%20and%20retrieval%20computational%20prototype%20based%20on%20transcribed%20speech&rft.jtitle=Multimedia%20tools%20and%20applications&rft.au=Spola%C3%B4r,%20Newton&rft.date=2021-10-01&rft.volume=80&rft.issue=25&rft.spage=33971&rft.epage=34017&rft.pages=33971-34017&rft.issn=1380-7501&rft.eissn=1573-7721&rft_id=info:doi/10.1007/s11042-021-11401-1&rft_dat=%3Cproquest_cross%3E2584133032%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2584133032&rft_id=info:pmid/&rfr_iscdi=true |