Investigating emotional responses to self-selected sad music via self-report and automated facial analysis

People often listen to sad music in spite of its seemingly negative qualities. Sad music, and especially sad music with a personal significance, has been shown to evoke a wide span of emotions with both positive and negative qualities. We compared emotional responses to familiar self-selected sad music (SSSM) with both unfamiliar sad and unfamiliar happy music. Alongside self-reports, a commercial, continuous measure of discrete facial expressions was applied, promising an in-depth assessment of both the quality and strength of experienced affective states at any given point in time. Results of the facial analysis showed that SSSM evoked more mixed affective states than unfamiliar sad music. Also, listeners reacted with consistent facial expressions to distinct musical events, e.g. the introduction of a lead voice. SSSM evoked more self-reported feelings of nostalgia, reminiscence, being moved, and chills and tears than unfamiliar sad and happy music. Furthermore, SSSM resulted in more self-reported happiness and a similar trend with happy facial expressions compared to unfamiliar sad music. These results point to the emotional diversity and the strong involvement of positive affective states elicited by SSSM, even when compared with music of similar quality, such as unfamiliar sad music. Automated facial analysis allows us to observe emotions on a more detailed level in terms of time resolution, onset, intensity and concurrence of discrete affective states. This technique is promising for future research, particularly when investigating mixed emotions and the social aspect of emotions in response to music.