When sound and picture do not fit: Mismatch negativity and sensory interaction

The Mismatch Negativity (MMN), an index of auditory stimulus representation, was studied in a design of simultaneous auditory and visual stimulus presentation. A counting task drew the attention of participants to picture presentation and was unrelated to the occurring sounds. With the picture of a hammer hitting a nail a standard sound “bang” came along. Rarely, a deviant picture showed the hammer hitting the finger combined with a corresponding deviant sound “ouch.” Sometimes this deviant picture was combined with the standard sound. In spite of the absence of a sound deviancy to the standard sounds, also in the latter case a significant MMN was detected, indicating that the actual visual context affected the incoming auditory information leading to a mismatch with the prevailing auditory memory trace. Herewith, MMN gains importance as an index of auditory stimulus representation also in studies of sensory interaction.