Faces of Emotion: Investigating Emotional Facial Expressions Towards a Robot

Emotions have always been an intriguing topic in everyday life as well as in science. As robots are starting to move from industry halls to our private homes, emotions have become a vital theme for the field of human–robot interaction. Since Darwin, research suggests facial expressions are associated with emotions. Facial expressions could provide an ideal tool for a natural, social human–robot interaction. Despite a growing body of research on the implementation of emotions in robots (mostly based on facial expressions), systematic research on users’ emotions and facial expressions towards robots remains largely neglected (cf. Arkin and Moshkina in Calvo R, D’Mello S, Gratch J, Kappas A (eds) The Oxford handbook of affective computing. Oxford University Press, New York, pp 483–493, 2015 on challenges in effective testing in affective human–robot interaction). We experimentally investigated the multilevel phenomenon of emotions by using a multi-method approach. Since self-reports of emotions are prone to biases such as social desirability, we supplemented it by an objective behavioral measurement. By using the Facial Action Coding System we analyzed the facial expressions of 62 participants who watched the entertainment robot dinosaur Pleo either in a friendly interaction or being tortured. Participants differed in the type and frequency of Action Units displayed as well as in their self-reported feelings depending on the type of treatment they had watched (friendly or torture). In line with a previous study by Rosenthal-von der Pütten et al. (Int J Soc Robot 5(1):17–34, 2013 . https://doi.org/10.1007/s12369-012-0173-8 ), participants reported feeling more positive after the friendly video and more negative after the torture video. In the torture condition, participants furthermore showed a wide range of different Action Units primarily associated with negative emotions. For example, the Action Unit 4 (“Brow Lowerer”) that is common in negative emotions such as anger and sadness was displayed more frequently in the torture condition than in the friendly condition. The Action Unit 12 (“Lip Corner Puller”) however, an Action Unit commonly associated with joy, was present in both conditions and thus not necessarily predictive of positive emotions. The findings indicate the importance for a thorough investigation of the variables of emotional facial expressions. In investigating the Action Units participants display due to an emotional situation, we aim to pro