Expressivity, Affective Nuance, and Presentation Modality in a Performer-to-Audience Communication

It is a given that a live concert is fundamentally different from a recorded live performance, but we wanted to investigate in a controlled setting how different renditions of the same piece of music affected an audience. In our previous case study using Rachmaninoff's pieces, a pianist expressed richer affective nuances when he performed his artistic rendition of the piece than his contrived deadpan (i.e., mechanical) and exaggerated renditions (Shoda & Adachi, 2012). The primary goal of the present study was to show the role of presentation modality (i.e., sound-only, video-only, both) in the audience's perception of the multifaceted affective nuances projected in his artistic rendition as compared with those in the deadpan and the exaggerated renditions. Each of 106 participants experienced each rendition of the performances in sound-only, video-only, and sound and video modalities, and rated the perceived degree of its expressivity and affective nuances. We identified a 2-dimensional map for performer-to-audience communication of the complex affective nuances of music with multiple adjectives by means of exploratory positioning analysis, a multidimensional statistical technique for Likert-type ratings in within-participant design. As expected, visual clues indeed assisted an audience to decode a pianist's intended level of expressivity, but with a surprising twist: The affective nuances portrayed by the pianist in artistic rendition were communicated more successfully via auditory information alone. These results suggest that auditory and visual cues serve different roles in pianist-to-audience communications.