Audiovisual Affect Recognition for Autonomous Vehicles: Applications and Future Agendas
Emotion and a broader range of affective and cognitive states play an important role on the road. While this has been predominantly investigated in terms of driver safety, the approaching advent of autonomous vehicles (AVs) is expected to bring a fundamental shift in focus for emotion recognition in...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on intelligent transportation systems 2024-06, Vol.25 (6), p.4918-4932 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 4932 |
|---|---|
| container_issue | 6 |
| container_start_page | 4918 |
| container_title | IEEE transactions on intelligent transportation systems |
| container_volume | 25 |
| creator | Karas, Vincent Schuller, Dagmar M. Schuller, Bjorn W. |
| description | Emotion and a broader range of affective and cognitive states play an important role on the road. While this has been predominantly investigated in terms of driver safety, the approaching advent of autonomous vehicles (AVs) is expected to bring a fundamental shift in focus for emotion recognition in the car, from the driver to the passengers. This work presents a number of affect-enabled applications, including adapting the driving style for an emotional experience or tailoring the infotainment to personal preferences. It attempts to foresee upcoming challenges and provides suggestions for multimodal affect modelling, with a focus on the audio and visual modalities. In particular, this includes context awareness, reliable diarisation of multiple passengers, group affect, and personalisation. Finally, we provide some recommendations on future research directions, including explainability, privacy, and holistic modelling. |
| doi_str_mv | 10.1109/TITS.2023.3333749 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_ieee_primary_10337780</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10337780</ieee_id><sourcerecordid>3062735966</sourcerecordid><originalsourceid>FETCH-LOGICAL-c246t-c28c76352cf54f86693a555ae17738a1480744aa1a7b607d2829f6a6b7bc13063</originalsourceid><addsrcrecordid>eNpNkE1LAzEQhoMoWKs_QPAQ8Nya7-x6W4rVgiBo1WNIs0lN2W7WZCP4792lPTiHmWHmfWfgAeAaoznGqLxbr9Zvc4IIndMhJCtPwARzXswQwuJ07AmblYijc3CR0m6YMo7xBHxWufbhx6esG1g5Z00PX60J29b3PrTQhQir3Ic27ENO8MN-edPYdA-rrmu80aMoQd3WcJn7HC2stratdboEZ043yV4d6xS8Lx_Wi6fZ88vjalE9zwxhoh9yYaSgnBjHmSuEKKnmnGuLpaSFxqxAkjGtsZYbgWRNClI6ocVGbgymSNApuD3c7WL4zjb1ahdybIeXalgTSXkpRhU-qEwMKUXrVBf9XsdfhZEa-amRnxr5qSO_wXNz8Hhr7T_9sJUFon_D42uZ</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3062735966</pqid></control><display><type>article</type><title>Audiovisual Affect Recognition for Autonomous Vehicles: Applications and Future Agendas</title><source>IEEE Electronic Library (IEL)</source><creator>Karas, Vincent ; Schuller, Dagmar M. ; Schuller, Bjorn W.</creator><creatorcontrib>Karas, Vincent ; Schuller, Dagmar M. ; Schuller, Bjorn W.</creatorcontrib><description>Emotion and a broader range of affective and cognitive states play an important role on the road. While this has been predominantly investigated in terms of driver safety, the approaching advent of autonomous vehicles (AVs) is expected to bring a fundamental shift in focus for emotion recognition in the car, from the driver to the passengers. This work presents a number of affect-enabled applications, including adapting the driving style for an emotional experience or tailoring the infotainment to personal preferences. It attempts to foresee upcoming challenges and provides suggestions for multimodal affect modelling, with a focus on the audio and visual modalities. In particular, this includes context awareness, reliable diarisation of multiple passengers, group affect, and personalisation. Finally, we provide some recommendations on future research directions, including explainability, privacy, and holistic modelling.</description><identifier>ISSN: 1524-9050</identifier><identifier>EISSN: 1558-0016</identifier><identifier>DOI: 10.1109/TITS.2023.3333749</identifier><identifier>CODEN: ITISFG</identifier><language>eng</language><publisher>New York: IEEE</publisher><subject>Automobiles ; Autonomous vehicles ; Cameras ; Emotion recognition ; Emotions ; human-machine interaction ; Intelligent sensors ; interior sensing ; Modelling ; Monitoring ; Sensors ; Vehicle safety ; Vehicles</subject><ispartof>IEEE transactions on intelligent transportation systems, 2024-06, Vol.25 (6), p.4918-4932</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c246t-c28c76352cf54f86693a555ae17738a1480744aa1a7b607d2829f6a6b7bc13063</cites><orcidid>0000-0001-8364-8301</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10337780$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,796,27923,27924,54757</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/10337780$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Karas, Vincent</creatorcontrib><creatorcontrib>Schuller, Dagmar M.</creatorcontrib><creatorcontrib>Schuller, Bjorn W.</creatorcontrib><title>Audiovisual Affect Recognition for Autonomous Vehicles: Applications and Future Agendas</title><title>IEEE transactions on intelligent transportation systems</title><addtitle>TITS</addtitle><description>Emotion and a broader range of affective and cognitive states play an important role on the road. While this has been predominantly investigated in terms of driver safety, the approaching advent of autonomous vehicles (AVs) is expected to bring a fundamental shift in focus for emotion recognition in the car, from the driver to the passengers. This work presents a number of affect-enabled applications, including adapting the driving style for an emotional experience or tailoring the infotainment to personal preferences. It attempts to foresee upcoming challenges and provides suggestions for multimodal affect modelling, with a focus on the audio and visual modalities. In particular, this includes context awareness, reliable diarisation of multiple passengers, group affect, and personalisation. Finally, we provide some recommendations on future research directions, including explainability, privacy, and holistic modelling.</description><subject>Automobiles</subject><subject>Autonomous vehicles</subject><subject>Cameras</subject><subject>Emotion recognition</subject><subject>Emotions</subject><subject>human-machine interaction</subject><subject>Intelligent sensors</subject><subject>interior sensing</subject><subject>Modelling</subject><subject>Monitoring</subject><subject>Sensors</subject><subject>Vehicle safety</subject><subject>Vehicles</subject><issn>1524-9050</issn><issn>1558-0016</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNpNkE1LAzEQhoMoWKs_QPAQ8Nya7-x6W4rVgiBo1WNIs0lN2W7WZCP4792lPTiHmWHmfWfgAeAaoznGqLxbr9Zvc4IIndMhJCtPwARzXswQwuJ07AmblYijc3CR0m6YMo7xBHxWufbhx6esG1g5Z00PX60J29b3PrTQhQir3Ic27ENO8MN-edPYdA-rrmu80aMoQd3WcJn7HC2stratdboEZ043yV4d6xS8Lx_Wi6fZ88vjalE9zwxhoh9yYaSgnBjHmSuEKKnmnGuLpaSFxqxAkjGtsZYbgWRNClI6ocVGbgymSNApuD3c7WL4zjb1ahdybIeXalgTSXkpRhU-qEwMKUXrVBf9XsdfhZEa-amRnxr5qSO_wXNz8Hhr7T_9sJUFon_D42uZ</recordid><startdate>20240601</startdate><enddate>20240601</enddate><creator>Karas, Vincent</creator><creator>Schuller, Dagmar M.</creator><creator>Schuller, Bjorn W.</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>FR3</scope><scope>JQ2</scope><scope>KR7</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0001-8364-8301</orcidid></search><sort><creationdate>20240601</creationdate><title>Audiovisual Affect Recognition for Autonomous Vehicles: Applications and Future Agendas</title><author>Karas, Vincent ; Schuller, Dagmar M. ; Schuller, Bjorn W.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c246t-c28c76352cf54f86693a555ae17738a1480744aa1a7b607d2829f6a6b7bc13063</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Automobiles</topic><topic>Autonomous vehicles</topic><topic>Cameras</topic><topic>Emotion recognition</topic><topic>Emotions</topic><topic>human-machine interaction</topic><topic>Intelligent sensors</topic><topic>interior sensing</topic><topic>Modelling</topic><topic>Monitoring</topic><topic>Sensors</topic><topic>Vehicle safety</topic><topic>Vehicles</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Karas, Vincent</creatorcontrib><creatorcontrib>Schuller, Dagmar M.</creatorcontrib><creatorcontrib>Schuller, Bjorn W.</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>Engineering Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Civil Engineering Abstracts</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE transactions on intelligent transportation systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Karas, Vincent</au><au>Schuller, Dagmar M.</au><au>Schuller, Bjorn W.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Audiovisual Affect Recognition for Autonomous Vehicles: Applications and Future Agendas</atitle><jtitle>IEEE transactions on intelligent transportation systems</jtitle><stitle>TITS</stitle><date>2024-06-01</date><risdate>2024</risdate><volume>25</volume><issue>6</issue><spage>4918</spage><epage>4932</epage><pages>4918-4932</pages><issn>1524-9050</issn><eissn>1558-0016</eissn><coden>ITISFG</coden><abstract>Emotion and a broader range of affective and cognitive states play an important role on the road. While this has been predominantly investigated in terms of driver safety, the approaching advent of autonomous vehicles (AVs) is expected to bring a fundamental shift in focus for emotion recognition in the car, from the driver to the passengers. This work presents a number of affect-enabled applications, including adapting the driving style for an emotional experience or tailoring the infotainment to personal preferences. It attempts to foresee upcoming challenges and provides suggestions for multimodal affect modelling, with a focus on the audio and visual modalities. In particular, this includes context awareness, reliable diarisation of multiple passengers, group affect, and personalisation. Finally, we provide some recommendations on future research directions, including explainability, privacy, and holistic modelling.</abstract><cop>New York</cop><pub>IEEE</pub><doi>10.1109/TITS.2023.3333749</doi><tpages>15</tpages><orcidid>https://orcid.org/0000-0001-8364-8301</orcidid></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | ISSN: 1524-9050 |
| ispartof | IEEE transactions on intelligent transportation systems, 2024-06, Vol.25 (6), p.4918-4932 |
| issn | 1524-9050 1558-0016 |
| language | eng |
| recordid | cdi_ieee_primary_10337780 |
| source | IEEE Electronic Library (IEL) |
| subjects | Automobiles Autonomous vehicles Cameras Emotion recognition Emotions human-machine interaction Intelligent sensors interior sensing Modelling Monitoring Sensors Vehicle safety Vehicles |
| title | Audiovisual Affect Recognition for Autonomous Vehicles: Applications and Future Agendas |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-12T03%3A09%3A51IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Audiovisual%20Affect%20Recognition%20for%20Autonomous%20Vehicles:%20Applications%20and%20Future%20Agendas&rft.jtitle=IEEE%20transactions%20on%20intelligent%20transportation%20systems&rft.au=Karas,%20Vincent&rft.date=2024-06-01&rft.volume=25&rft.issue=6&rft.spage=4918&rft.epage=4932&rft.pages=4918-4932&rft.issn=1524-9050&rft.eissn=1558-0016&rft.coden=ITISFG&rft_id=info:doi/10.1109/TITS.2023.3333749&rft_dat=%3Cproquest_RIE%3E3062735966%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3062735966&rft_id=info:pmid/&rft_ieee_id=10337780&rfr_iscdi=true |