The processing of Dutch prosody with cochlear implants and vocoder simulations

This doctoral dissertation reports five studies investigating the processing of prosody and music with cochlear implants or simulations thereof (vocoders) in Dutch. Cochlear implants are implanted hearing prostheses that partly restore hearing for profoundly deaf individuals by presenting an electrical reconstruction of sound to the hearing nerve. Prosody is the melody and rhythm of speech and is crucial in spoken communication. Important functions of prosody include the conveying of emotions (emotional prosody) and the marking of new or old information in utterances (linguistic prosody). These functions are realized by speakers, among other ways, by means of variation in intonation and the duration of parts of an utterance. Of these forms, the perception of intonation variations, but not particularly of duration variations, is notoriously difficult for cochlear implant users. This difficulty is caused by limitations of the device and the interface between cochlear implants and the hearing nerve. Possible limitations in perception and production of linguistic and emotional prosody by actual and simulated cochlear implant hearing had never been systematically compared. This thesis shows that cochlear implant users might have more difficulty discriminating emotional than linguistic prosody and that they rely relatively much on intonation cues for emotional prosody but on duration cues for linguistic prosody. Tests with vocoders showed that sharpening the slopes of spectral filters (simulating reduced spectral smearing) improves prosody perception up to values much extremer than tested before. Taken together, this set of experiments discusses issues to take into account when studying the perception and production of prosody by cochlear implant users and with vocoder simulations.