Chaos based non-linear cognitive study of different stimulus in the cross-modal perspective

The relationship between color and music as part of the complex system consisting of visual and auditory domain has been investigated in this study. As both the stimulus forms are processed in the same part of the human body, i.e., the brain, it will be really interesting to examine whether they share a similarity in perception. Needless to say that color and music both have strong impact on emotion and feelings & also a few studies have been reported in literature to explore causal relationship between color and emotion. This work reports a neuro-cognitive study on response of brain to two different stimulus and their cross-modal associations. In this study the correlation between emotional arousal and the effect of audio and visual stimuli has been studied from a new perspective. 93 participants were asked to hear 6 different music pieces (each of 30 s duration). The type of emotion elicited by different music pieces were identified by the participants from a given collection of possible emotional responses. Then they are asked to assign a color associating the emotion from a given color wheel (structured according to Munsell color system/RGB color space). Each color, associated with a particular music piece, is a mixture of specific Red, Green and Blue values (RGB triplet) and has a specific HEX number (hexadecimal representation), which is recorded for each response. Then, the musical pieces used were further zoomed with the help of fractal technique to identify different emotions related to music in a quantitative approach. Here, to analyze the complexity of the sound signal (which are non-stationary and scale varying in nature), we have used Multifractal detrended fluctuation analysis (MFDFA), which is capable of determining multifractal scaling behavior of non-stationary time series. From the experimental data, it is seen that the visual and emotional response to the auditory stimulus follows a specific trend which is directly related to the stimulus complexity. •It is a novel neuro-cognitive study on response of brain to two different stimuli.•93 participants were asked to listen to 6 different music pieces.•Then they assigned emotions and colors to each of the six pieces.•MFDFA analysis has been used to measure the complexity of the auditory stimulus.•Correlation between signal complexity, emotional response & RGB analysis is studied.•Visual and emotional response exhibits trends directly related to signal complexity.