When spoken and written words meet in the brain: A developmental ERP study

Integrating visual and auditory language information is critical for reading. Suppression and congruency effects in audiovisual paradigms with letters and speech sounds provided information about low-level mechanisms of grapheme-phoneme integration during reading. However, the central question about how such processes relate to reading entire words remains unexplored. Using ERPs, we investigate (1) when audiovisual integration occurs for entire words and pseudowords, (2) whether this integration is reflected by differences in map strength or differences in map topography, and (3) whether this integration is influenced by reading fluency. A 128-channel EEG was recorded while 69 monolingual (Swiss)-German speaking first-graders performed an oddball detection task. Word and pseudoword stimuli were presented in blocks either auditorily (A), visually (V) or audiovisually (matching: AVM; nonmatching: AVN). Corresponding ERPs were computed, and unimodal ERPs were summated (sumAV). We applied TANOVAs to identify time windows with significant integration effects: suppression (sumAV-AVM) and congruency (AVN-AVM). These integration effects were further characterized using GFP and 3D-centroid analyses. If significant, these analyses were recomputed adding reading fluency as covariate. Significant audiovisual suppression effects were observed for words and pseudowords in similar time windows, in agreement with previous studies on letters and speech sounds. Significant congruency effects were found only for words, but not for pseudowords, suggesting early integration processes specific for words. Suppression effects tended to be characterized by differences in map strength, whereas congruency effects tended to be characterized by differences in map topography. Among all audiovisual integration effects, reading fluency modulated the word suppression effect around 300ms.