Reconstructing speech from human auditory cortex
How the human auditory system extracts perceptually relevant acoustic features of speech is unknown. To address this question, we used intracranial recordings from nonprimary auditory cortex in the human superior temporal gyrus to determine what acoustic information in speech sounds can be reconstru...
Gespeichert in:
Veröffentlicht in: | PLoS biology 2012-01, Vol.10 (1), p.e1001251-e1001251 |
---|---|
Hauptverfasser: | , , , , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | e1001251 |
---|---|
container_issue | 1 |
container_start_page | e1001251 |
container_title | PLoS biology |
container_volume | 10 |
creator | Pasley, Brian N David, Stephen V Mesgarani, Nima Flinker, Adeen Shamma, Shihab A Crone, Nathan E Knight, Robert T Chang, Edward F |
description | How the human auditory system extracts perceptually relevant acoustic features of speech is unknown. To address this question, we used intracranial recordings from nonprimary auditory cortex in the human superior temporal gyrus to determine what acoustic information in speech sounds can be reconstructed from population neural activity. We found that slow and intermediate temporal fluctuations, such as those corresponding to syllable rate, were accurately reconstructed using a linear model based on the auditory spectrogram. However, reconstruction of fast temporal fluctuations, such as syllable onsets and offsets, required a nonlinear sound representation based on temporal modulation energy. Reconstruction accuracy was highest within the range of spectro-temporal fluctuations that have been found to be critical for speech intelligibility. The decoded speech representations allowed readout and identification of individual words directly from brain activity during single trial sound presentations. These findings reveal neural encoding mechanisms of speech acoustic parameters in higher order human auditory cortex. |
doi_str_mv | 10.1371/journal.pbio.1001251 |
format | Article |
fullrecord | <record><control><sourceid>gale_plos_</sourceid><recordid>TN_cdi_plos_journals_1303361270</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><galeid>A279722281</galeid><doaj_id>oai_doaj_org_article_825de0b5af424181b4ccf7bb930283b0</doaj_id><sourcerecordid>A279722281</sourcerecordid><originalsourceid>FETCH-LOGICAL-c760t-7c7f752c2e634c9ebb76c510395e12fb2019cfa3ff481ed834b647c1d85054293</originalsourceid><addsrcrecordid>eNqVkltrFDEUx4Motq5-A9EFH8SHXXOdJC9CKV4WioV6eQ1JJpnNMjNZkxmx396sOy0d6YOSh4ST3_mfKwDPEVwjwtHbXRxTr9v13oS4RhAizNADcIoYZSsuBHt4530CnuS8gxBjicVjcIIxgQQLdArglbOxz0Ma7RD6Zpn3ztnt0qfYLbdjp_ulHuswxHS9tDEN7tdT8MjrNrtn070A3z68_3r-aXVx-XFzfnaxsryCw4pb7jnDFruKUCudMbyyDEEimUPYGwyRtF4T76lArhaEmopyi2rBIKNYkgV4edTdtzGrqdisUEmcVAhzWIjNkaij3ql9Cp1O1yrqoP4YYmqUTkOwrVMCs9pBw7SnmCKBDLXWc2MkgVgQc9B6N0UbTedq6_oh6XYmOv_pw1Y18aciuJK0tHMBXk8CKf4YXR5UF7J1bat7F8esJJKSMQFZIV_9Rd5f3EQ1uuQfeh9LWHvQVGeYS45xGV-h1vdQ5dSuC2Wuzodinzm8mTkUpsx0aPSYs9p8ufoP9vO_s5ff5yw9sjbFnJPzt21GUB1W-6Yh6rDaalrt4vbi7ohunW52mfwG0ufxnw</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1303361270</pqid></control><display><type>article</type><title>Reconstructing speech from human auditory cortex</title><source>MEDLINE</source><source>DOAJ Directory of Open Access Journals</source><source>Public Library of Science (PLoS)</source><source>EZB-FREE-00999 freely available EZB journals</source><source>PubMed Central</source><creator>Pasley, Brian N ; David, Stephen V ; Mesgarani, Nima ; Flinker, Adeen ; Shamma, Shihab A ; Crone, Nathan E ; Knight, Robert T ; Chang, Edward F</creator><contributor>Zatorre, Robert</contributor><creatorcontrib>Pasley, Brian N ; David, Stephen V ; Mesgarani, Nima ; Flinker, Adeen ; Shamma, Shihab A ; Crone, Nathan E ; Knight, Robert T ; Chang, Edward F ; Zatorre, Robert</creatorcontrib><description>How the human auditory system extracts perceptually relevant acoustic features of speech is unknown. To address this question, we used intracranial recordings from nonprimary auditory cortex in the human superior temporal gyrus to determine what acoustic information in speech sounds can be reconstructed from population neural activity. We found that slow and intermediate temporal fluctuations, such as those corresponding to syllable rate, were accurately reconstructed using a linear model based on the auditory spectrogram. However, reconstruction of fast temporal fluctuations, such as syllable onsets and offsets, required a nonlinear sound representation based on temporal modulation energy. Reconstruction accuracy was highest within the range of spectro-temporal fluctuations that have been found to be critical for speech intelligibility. The decoded speech representations allowed readout and identification of individual words directly from brain activity during single trial sound presentations. These findings reveal neural encoding mechanisms of speech acoustic parameters in higher order human auditory cortex.</description><identifier>ISSN: 1545-7885</identifier><identifier>ISSN: 1544-9173</identifier><identifier>EISSN: 1545-7885</identifier><identifier>DOI: 10.1371/journal.pbio.1001251</identifier><identifier>PMID: 22303281</identifier><language>eng</language><publisher>United States: Public Library of Science</publisher><subject>Algorithms ; Auditory Cortex - physiology ; Biology ; Brain ; Brain Mapping ; Computer Simulation ; Ears & hearing ; Electrodes, Implanted ; Electroencephalography ; Engineering ; Female ; Health aspects ; Humans ; Linear Models ; Male ; Models, Biological ; Neurology ; Phonetics ; Population ; Regression analysis ; Social and Behavioral Sciences ; Sound ; Speech ; Speech Acoustics ; Speech perception ; Studies ; Surgery</subject><ispartof>PLoS biology, 2012-01, Vol.10 (1), p.e1001251-e1001251</ispartof><rights>COPYRIGHT 2012 Public Library of Science</rights><rights>2012 Pasley et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited: Pasley BN, David SV, Mesgarani N, Flinker A, Shamma SA, et al. (2012) Reconstructing Speech from Human Auditory Cortex. PLoS Biol 10(1): e1001251. doi:10.1371/journal.pbio.1001251</rights><rights>Pasley et al. 2012</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c760t-7c7f752c2e634c9ebb76c510395e12fb2019cfa3ff481ed834b647c1d85054293</citedby><cites>FETCH-LOGICAL-c760t-7c7f752c2e634c9ebb76c510395e12fb2019cfa3ff481ed834b647c1d85054293</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC3269422/pdf/$$EPDF$$P50$$Gpubmedcentral$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC3269422/$$EHTML$$P50$$Gpubmedcentral$$Hfree_for_read</linktohtml><link.rule.ids>230,314,724,777,781,861,882,2096,2915,23847,27905,27906,53772,53774,79349,79350</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/22303281$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><contributor>Zatorre, Robert</contributor><creatorcontrib>Pasley, Brian N</creatorcontrib><creatorcontrib>David, Stephen V</creatorcontrib><creatorcontrib>Mesgarani, Nima</creatorcontrib><creatorcontrib>Flinker, Adeen</creatorcontrib><creatorcontrib>Shamma, Shihab A</creatorcontrib><creatorcontrib>Crone, Nathan E</creatorcontrib><creatorcontrib>Knight, Robert T</creatorcontrib><creatorcontrib>Chang, Edward F</creatorcontrib><title>Reconstructing speech from human auditory cortex</title><title>PLoS biology</title><addtitle>PLoS Biol</addtitle><description>How the human auditory system extracts perceptually relevant acoustic features of speech is unknown. To address this question, we used intracranial recordings from nonprimary auditory cortex in the human superior temporal gyrus to determine what acoustic information in speech sounds can be reconstructed from population neural activity. We found that slow and intermediate temporal fluctuations, such as those corresponding to syllable rate, were accurately reconstructed using a linear model based on the auditory spectrogram. However, reconstruction of fast temporal fluctuations, such as syllable onsets and offsets, required a nonlinear sound representation based on temporal modulation energy. Reconstruction accuracy was highest within the range of spectro-temporal fluctuations that have been found to be critical for speech intelligibility. The decoded speech representations allowed readout and identification of individual words directly from brain activity during single trial sound presentations. These findings reveal neural encoding mechanisms of speech acoustic parameters in higher order human auditory cortex.</description><subject>Algorithms</subject><subject>Auditory Cortex - physiology</subject><subject>Biology</subject><subject>Brain</subject><subject>Brain Mapping</subject><subject>Computer Simulation</subject><subject>Ears & hearing</subject><subject>Electrodes, Implanted</subject><subject>Electroencephalography</subject><subject>Engineering</subject><subject>Female</subject><subject>Health aspects</subject><subject>Humans</subject><subject>Linear Models</subject><subject>Male</subject><subject>Models, Biological</subject><subject>Neurology</subject><subject>Phonetics</subject><subject>Population</subject><subject>Regression analysis</subject><subject>Social and Behavioral Sciences</subject><subject>Sound</subject><subject>Speech</subject><subject>Speech Acoustics</subject><subject>Speech perception</subject><subject>Studies</subject><subject>Surgery</subject><issn>1545-7885</issn><issn>1544-9173</issn><issn>1545-7885</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2012</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><sourceid>DOA</sourceid><recordid>eNqVkltrFDEUx4Motq5-A9EFH8SHXXOdJC9CKV4WioV6eQ1JJpnNMjNZkxmx396sOy0d6YOSh4ST3_mfKwDPEVwjwtHbXRxTr9v13oS4RhAizNADcIoYZSsuBHt4530CnuS8gxBjicVjcIIxgQQLdArglbOxz0Ma7RD6Zpn3ztnt0qfYLbdjp_ulHuswxHS9tDEN7tdT8MjrNrtn070A3z68_3r-aXVx-XFzfnaxsryCw4pb7jnDFruKUCudMbyyDEEimUPYGwyRtF4T76lArhaEmopyi2rBIKNYkgV4edTdtzGrqdisUEmcVAhzWIjNkaij3ql9Cp1O1yrqoP4YYmqUTkOwrVMCs9pBw7SnmCKBDLXWc2MkgVgQc9B6N0UbTedq6_oh6XYmOv_pw1Y18aciuJK0tHMBXk8CKf4YXR5UF7J1bat7F8esJJKSMQFZIV_9Rd5f3EQ1uuQfeh9LWHvQVGeYS45xGV-h1vdQ5dSuC2Wuzodinzm8mTkUpsx0aPSYs9p8ufoP9vO_s5ff5yw9sjbFnJPzt21GUB1W-6Yh6rDaalrt4vbi7ohunW52mfwG0ufxnw</recordid><startdate>20120101</startdate><enddate>20120101</enddate><creator>Pasley, Brian N</creator><creator>David, Stephen V</creator><creator>Mesgarani, Nima</creator><creator>Flinker, Adeen</creator><creator>Shamma, Shihab A</creator><creator>Crone, Nathan E</creator><creator>Knight, Robert T</creator><creator>Chang, Edward F</creator><general>Public Library of Science</general><general>Public Library of Science (PLoS)</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>IOV</scope><scope>ISN</scope><scope>ISR</scope><scope>3V.</scope><scope>7QG</scope><scope>7QL</scope><scope>7SN</scope><scope>7SS</scope><scope>7T5</scope><scope>7TK</scope><scope>7TM</scope><scope>7X7</scope><scope>7XB</scope><scope>88E</scope><scope>8FD</scope><scope>8FE</scope><scope>8FH</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>ABUWG</scope><scope>AEUYN</scope><scope>AFKRA</scope><scope>ATCPS</scope><scope>AZQEC</scope><scope>BBNVY</scope><scope>BENPR</scope><scope>BHPHI</scope><scope>C1K</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FR3</scope><scope>FYUFA</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>H94</scope><scope>HCIFZ</scope><scope>K9.</scope><scope>LK8</scope><scope>M0S</scope><scope>M1P</scope><scope>M7N</scope><scope>M7P</scope><scope>P64</scope><scope>PATMY</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PYCSY</scope><scope>RC3</scope><scope>7X8</scope><scope>5PM</scope><scope>DOA</scope><scope>CZG</scope></search><sort><creationdate>20120101</creationdate><title>Reconstructing speech from human auditory cortex</title><author>Pasley, Brian N ; David, Stephen V ; Mesgarani, Nima ; Flinker, Adeen ; Shamma, Shihab A ; Crone, Nathan E ; Knight, Robert T ; Chang, Edward F</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c760t-7c7f752c2e634c9ebb76c510395e12fb2019cfa3ff481ed834b647c1d85054293</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2012</creationdate><topic>Algorithms</topic><topic>Auditory Cortex - physiology</topic><topic>Biology</topic><topic>Brain</topic><topic>Brain Mapping</topic><topic>Computer Simulation</topic><topic>Ears & hearing</topic><topic>Electrodes, Implanted</topic><topic>Electroencephalography</topic><topic>Engineering</topic><topic>Female</topic><topic>Health aspects</topic><topic>Humans</topic><topic>Linear Models</topic><topic>Male</topic><topic>Models, Biological</topic><topic>Neurology</topic><topic>Phonetics</topic><topic>Population</topic><topic>Regression analysis</topic><topic>Social and Behavioral Sciences</topic><topic>Sound</topic><topic>Speech</topic><topic>Speech Acoustics</topic><topic>Speech perception</topic><topic>Studies</topic><topic>Surgery</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Pasley, Brian N</creatorcontrib><creatorcontrib>David, Stephen V</creatorcontrib><creatorcontrib>Mesgarani, Nima</creatorcontrib><creatorcontrib>Flinker, Adeen</creatorcontrib><creatorcontrib>Shamma, Shihab A</creatorcontrib><creatorcontrib>Crone, Nathan E</creatorcontrib><creatorcontrib>Knight, Robert T</creatorcontrib><creatorcontrib>Chang, Edward F</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Gale In Context: Opposing Viewpoints</collection><collection>Gale In Context: Canada</collection><collection>Gale In Context: Science</collection><collection>ProQuest Central (Corporate)</collection><collection>Animal Behavior Abstracts</collection><collection>Bacteriology Abstracts (Microbiology B)</collection><collection>Ecology Abstracts</collection><collection>Entomology Abstracts (Full archive)</collection><collection>Immunology Abstracts</collection><collection>Neurosciences Abstracts</collection><collection>Nucleic Acids Abstracts</collection><collection>Health & Medical Collection</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Medical Database (Alumni Edition)</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Natural Science Collection</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest One Sustainability</collection><collection>ProQuest Central UK/Ireland</collection><collection>Agricultural & Environmental Science Collection</collection><collection>ProQuest Central Essentials</collection><collection>Biological Science Collection</collection><collection>ProQuest Central</collection><collection>Natural Science Collection</collection><collection>Environmental Sciences and Pollution Management</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>Engineering Research Database</collection><collection>Health Research Premium Collection</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>AIDS and Cancer Research Abstracts</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Health & Medical Complete (Alumni)</collection><collection>ProQuest Biological Science Collection</collection><collection>Health & Medical Collection (Alumni Edition)</collection><collection>Medical Database</collection><collection>Algology Mycology and Protozoology Abstracts (Microbiology C)</collection><collection>Biological Science Database</collection><collection>Biotechnology and BioEngineering Abstracts</collection><collection>Environmental Science Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Environmental Science Collection</collection><collection>Genetics Abstracts</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><collection>DOAJ Directory of Open Access Journals</collection><collection>PLoS Biology</collection><jtitle>PLoS biology</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Pasley, Brian N</au><au>David, Stephen V</au><au>Mesgarani, Nima</au><au>Flinker, Adeen</au><au>Shamma, Shihab A</au><au>Crone, Nathan E</au><au>Knight, Robert T</au><au>Chang, Edward F</au><au>Zatorre, Robert</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Reconstructing speech from human auditory cortex</atitle><jtitle>PLoS biology</jtitle><addtitle>PLoS Biol</addtitle><date>2012-01-01</date><risdate>2012</risdate><volume>10</volume><issue>1</issue><spage>e1001251</spage><epage>e1001251</epage><pages>e1001251-e1001251</pages><issn>1545-7885</issn><issn>1544-9173</issn><eissn>1545-7885</eissn><abstract>How the human auditory system extracts perceptually relevant acoustic features of speech is unknown. To address this question, we used intracranial recordings from nonprimary auditory cortex in the human superior temporal gyrus to determine what acoustic information in speech sounds can be reconstructed from population neural activity. We found that slow and intermediate temporal fluctuations, such as those corresponding to syllable rate, were accurately reconstructed using a linear model based on the auditory spectrogram. However, reconstruction of fast temporal fluctuations, such as syllable onsets and offsets, required a nonlinear sound representation based on temporal modulation energy. Reconstruction accuracy was highest within the range of spectro-temporal fluctuations that have been found to be critical for speech intelligibility. The decoded speech representations allowed readout and identification of individual words directly from brain activity during single trial sound presentations. These findings reveal neural encoding mechanisms of speech acoustic parameters in higher order human auditory cortex.</abstract><cop>United States</cop><pub>Public Library of Science</pub><pmid>22303281</pmid><doi>10.1371/journal.pbio.1001251</doi><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 1545-7885 |
ispartof | PLoS biology, 2012-01, Vol.10 (1), p.e1001251-e1001251 |
issn | 1545-7885 1544-9173 1545-7885 |
language | eng |
recordid | cdi_plos_journals_1303361270 |
source | MEDLINE; DOAJ Directory of Open Access Journals; Public Library of Science (PLoS); EZB-FREE-00999 freely available EZB journals; PubMed Central |
subjects | Algorithms Auditory Cortex - physiology Biology Brain Brain Mapping Computer Simulation Ears & hearing Electrodes, Implanted Electroencephalography Engineering Female Health aspects Humans Linear Models Male Models, Biological Neurology Phonetics Population Regression analysis Social and Behavioral Sciences Sound Speech Speech Acoustics Speech perception Studies Surgery |
title | Reconstructing speech from human auditory cortex |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-19T20%3A54%3A13IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-gale_plos_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Reconstructing%20speech%20from%20human%20auditory%20cortex&rft.jtitle=PLoS%20biology&rft.au=Pasley,%20Brian%20N&rft.date=2012-01-01&rft.volume=10&rft.issue=1&rft.spage=e1001251&rft.epage=e1001251&rft.pages=e1001251-e1001251&rft.issn=1545-7885&rft.eissn=1545-7885&rft_id=info:doi/10.1371/journal.pbio.1001251&rft_dat=%3Cgale_plos_%3EA279722281%3C/gale_plos_%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1303361270&rft_id=info:pmid/22303281&rft_galeid=A279722281&rft_doaj_id=oai_doaj_org_article_825de0b5af424181b4ccf7bb930283b0&rfr_iscdi=true |