Evaluating Creativity in Automatic Reactive Accompaniment of Jazz Improvisation

Music generating computer programs can support jazz musicians and students during performance and practice, for instance by providing accompaniment for solo improvisation. However, such software typically plays sequences of static precomposed snippets and does not react to the user. In that context, it is hardly possible to determine whether such a system has any of its own creative powers. Within the scope of a user study with 20 participants, we evaluate and compare the mobile application iReal Pro to our own system, the evolutionary automatic and reactive system called 'EAR Drummer' that generates drum patterns as accompaniment to jazz solo improvisation. It adapts its behaviour in real-time by heuristic rules based on music properties derived from the user's melodies. The user-based evaluation is performed by following the standardised procedure for evaluating creative systems (SPECS). The analysis of the results is based on a Linear Mixed Effects Model to consider fixed and random effects on the survey data. The model reveals that our system outperforms iReal Pro in all of SPECS's partial components of creativity and significantly outperforms it for 7 of those 14 components including variety, originality, emotional involvement, and social interaction. Further, it is characterised as "better" and "more interesting" in the user survey. A conflicting observation is that while 70% of the study participants tend to prefer our more "creative" system as support for stage performances, only 40% find it more suitable for practice. Further analysis addresses differences between user groups defined by their played instrument, age, and musical experience. Keywords: Evolutionary music accompaniment generation, drum patterns, evaluation of creative music systems, Linear Mixed Effects Model, SPECS user study, learning jazz music