Let me join you! Real-time F-formation recognition by a socially aware robot

This paper presents a novel architecture to detect social groups in real-time from a continuous image stream of an ego-vision camera. F-formation defines social orientations in space where two or more person tends to communicate in a social place. Thus, essentially, we detect F-formations in social...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Barua, Hrishav Bakul, Pramanick, Pradip, Sarkar, Chayan, Mg, Theint Haythi
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Barua, Hrishav Bakul
Pramanick, Pradip
Sarkar, Chayan
Mg, Theint Haythi
description This paper presents a novel architecture to detect social groups in real-time from a continuous image stream of an ego-vision camera. F-formation defines social orientations in space where two or more person tends to communicate in a social place. Thus, essentially, we detect F-formations in social gatherings such as meetings, discussions, etc. and predict the robot's approach angle if it wants to join the social group. Additionally, we also detect outliers, i.e., the persons who are not part of the group under consideration. Our proposed pipeline consists of -- a) a skeletal key points estimator (a total of 17) for the detected human in the scene, b) a learning model (using a feature vector based on the skeletal points) using CRF to detect groups of people and outlier person in a scene, and c) a separate learning model using a multi-class Support Vector Machine (SVM) to predict the exact F-formation of the group of people in the current scene and the angle of approach for the viewing robot. The system is evaluated using two data-sets. The results show that the group and outlier detection in a scene using our method establishes an accuracy of 91%. We have made rigorous comparisons of our systems with a state-of-the-art F-formation detection system and found that it outperforms the state-of-the-art by 29% for formation detection and 55% for combined detection of the formation and approach angle.
doi_str_mv 10.48550/arxiv.2008.10078
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2008_10078</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2008_10078</sourcerecordid><originalsourceid>FETCH-LOGICAL-a678-9331bba1f7d03dff11d7922526e4ca2b16e7e6a99cfde801b52b80e180dc1dd73</originalsourceid><addsrcrecordid>eNotj81KxDAURrNxIaMP4Mr4AKk3ybRJlzI4KhQEmX25SW4k0jaSqT99e8fq6jt8iwOHsSsJ1dbWNdxi-U6flQKwlQQw9px1Hc18JP6W08SX_HHDXwgHMafTtxcxlxHnlCdeyOfXKa3sFo78mH3CYTjhFxbiJbs8X7CziMORLv93ww77-8PuUXTPD0-7u05gY6xotZbOoYwmgA4xShlMq1StGtp6VE42ZKjBtvUxkAXpauUskLQQvAzB6A27_tOuOf17SSOWpf_N6tcs_QN610fp</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Let me join you! Real-time F-formation recognition by a socially aware robot</title><source>arXiv.org</source><creator>Barua, Hrishav Bakul ; Pramanick, Pradip ; Sarkar, Chayan ; Mg, Theint Haythi</creator><creatorcontrib>Barua, Hrishav Bakul ; Pramanick, Pradip ; Sarkar, Chayan ; Mg, Theint Haythi</creatorcontrib><description>This paper presents a novel architecture to detect social groups in real-time from a continuous image stream of an ego-vision camera. F-formation defines social orientations in space where two or more person tends to communicate in a social place. Thus, essentially, we detect F-formations in social gatherings such as meetings, discussions, etc. and predict the robot's approach angle if it wants to join the social group. Additionally, we also detect outliers, i.e., the persons who are not part of the group under consideration. Our proposed pipeline consists of -- a) a skeletal key points estimator (a total of 17) for the detected human in the scene, b) a learning model (using a feature vector based on the skeletal points) using CRF to detect groups of people and outlier person in a scene, and c) a separate learning model using a multi-class Support Vector Machine (SVM) to predict the exact F-formation of the group of people in the current scene and the angle of approach for the viewing robot. The system is evaluated using two data-sets. The results show that the group and outlier detection in a scene using our method establishes an accuracy of 91%. We have made rigorous comparisons of our systems with a state-of-the-art F-formation detection system and found that it outperforms the state-of-the-art by 29% for formation detection and 55% for combined detection of the formation and approach angle.</description><identifier>DOI: 10.48550/arxiv.2008.10078</identifier><language>eng</language><subject>Computer Science - Artificial Intelligence ; Computer Science - Computer Vision and Pattern Recognition ; Computer Science - Robotics</subject><creationdate>2020-08</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,781,886</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2008.10078$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2008.10078$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Barua, Hrishav Bakul</creatorcontrib><creatorcontrib>Pramanick, Pradip</creatorcontrib><creatorcontrib>Sarkar, Chayan</creatorcontrib><creatorcontrib>Mg, Theint Haythi</creatorcontrib><title>Let me join you! Real-time F-formation recognition by a socially aware robot</title><description>This paper presents a novel architecture to detect social groups in real-time from a continuous image stream of an ego-vision camera. F-formation defines social orientations in space where two or more person tends to communicate in a social place. Thus, essentially, we detect F-formations in social gatherings such as meetings, discussions, etc. and predict the robot's approach angle if it wants to join the social group. Additionally, we also detect outliers, i.e., the persons who are not part of the group under consideration. Our proposed pipeline consists of -- a) a skeletal key points estimator (a total of 17) for the detected human in the scene, b) a learning model (using a feature vector based on the skeletal points) using CRF to detect groups of people and outlier person in a scene, and c) a separate learning model using a multi-class Support Vector Machine (SVM) to predict the exact F-formation of the group of people in the current scene and the angle of approach for the viewing robot. The system is evaluated using two data-sets. The results show that the group and outlier detection in a scene using our method establishes an accuracy of 91%. We have made rigorous comparisons of our systems with a state-of-the-art F-formation detection system and found that it outperforms the state-of-the-art by 29% for formation detection and 55% for combined detection of the formation and approach angle.</description><subject>Computer Science - Artificial Intelligence</subject><subject>Computer Science - Computer Vision and Pattern Recognition</subject><subject>Computer Science - Robotics</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotj81KxDAURrNxIaMP4Mr4AKk3ybRJlzI4KhQEmX25SW4k0jaSqT99e8fq6jt8iwOHsSsJ1dbWNdxi-U6flQKwlQQw9px1Hc18JP6W08SX_HHDXwgHMafTtxcxlxHnlCdeyOfXKa3sFo78mH3CYTjhFxbiJbs8X7CziMORLv93ww77-8PuUXTPD0-7u05gY6xotZbOoYwmgA4xShlMq1StGtp6VE42ZKjBtvUxkAXpauUskLQQvAzB6A27_tOuOf17SSOWpf_N6tcs_QN610fp</recordid><startdate>20200823</startdate><enddate>20200823</enddate><creator>Barua, Hrishav Bakul</creator><creator>Pramanick, Pradip</creator><creator>Sarkar, Chayan</creator><creator>Mg, Theint Haythi</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20200823</creationdate><title>Let me join you! Real-time F-formation recognition by a socially aware robot</title><author>Barua, Hrishav Bakul ; Pramanick, Pradip ; Sarkar, Chayan ; Mg, Theint Haythi</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a678-9331bba1f7d03dff11d7922526e4ca2b16e7e6a99cfde801b52b80e180dc1dd73</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Computer Science - Artificial Intelligence</topic><topic>Computer Science - Computer Vision and Pattern Recognition</topic><topic>Computer Science - Robotics</topic><toplevel>online_resources</toplevel><creatorcontrib>Barua, Hrishav Bakul</creatorcontrib><creatorcontrib>Pramanick, Pradip</creatorcontrib><creatorcontrib>Sarkar, Chayan</creatorcontrib><creatorcontrib>Mg, Theint Haythi</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Barua, Hrishav Bakul</au><au>Pramanick, Pradip</au><au>Sarkar, Chayan</au><au>Mg, Theint Haythi</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Let me join you! Real-time F-formation recognition by a socially aware robot</atitle><date>2020-08-23</date><risdate>2020</risdate><abstract>This paper presents a novel architecture to detect social groups in real-time from a continuous image stream of an ego-vision camera. F-formation defines social orientations in space where two or more person tends to communicate in a social place. Thus, essentially, we detect F-formations in social gatherings such as meetings, discussions, etc. and predict the robot's approach angle if it wants to join the social group. Additionally, we also detect outliers, i.e., the persons who are not part of the group under consideration. Our proposed pipeline consists of -- a) a skeletal key points estimator (a total of 17) for the detected human in the scene, b) a learning model (using a feature vector based on the skeletal points) using CRF to detect groups of people and outlier person in a scene, and c) a separate learning model using a multi-class Support Vector Machine (SVM) to predict the exact F-formation of the group of people in the current scene and the angle of approach for the viewing robot. The system is evaluated using two data-sets. The results show that the group and outlier detection in a scene using our method establishes an accuracy of 91%. We have made rigorous comparisons of our systems with a state-of-the-art F-formation detection system and found that it outperforms the state-of-the-art by 29% for formation detection and 55% for combined detection of the formation and approach angle.</abstract><doi>10.48550/arxiv.2008.10078</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2008.10078
ispartof
issn
language eng
recordid cdi_arxiv_primary_2008_10078
source arXiv.org
subjects Computer Science - Artificial Intelligence
Computer Science - Computer Vision and Pattern Recognition
Computer Science - Robotics
title Let me join you! Real-time F-formation recognition by a socially aware robot
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-16T20%3A02%3A28IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Let%20me%20join%20you!%20Real-time%20F-formation%20recognition%20by%20a%20socially%20aware%20robot&rft.au=Barua,%20Hrishav%20Bakul&rft.date=2020-08-23&rft_id=info:doi/10.48550/arxiv.2008.10078&rft_dat=%3Carxiv_GOX%3E2008_10078%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true