Eye-movement patterns of hearing-impaired listeners measure comprehension of a multitalker conversation

The ability to understand speech in complex listening environments reflects an interaction of cognitive and sensory capacities that are difficult to capture with behavioural tests. The study of natural listening behaviours may lead to the development of new metrics that better reflect real-life communication abilities. To this end, we investigated the relationship between speech comprehension and eye-movements among hearing-impaired people in a challenging listening situation. While previous research has investigated the effect of background noise on listeners’ gaze patterns with single talkers, the effect of noise in multitalker conversations remains unknown. We tracked eye-movements of seven aided hearing-impaired adults while they viewed video recordings of two life-sized talkers engaged in an unscripted dialogue. Hearing loss ranged from moderate to severe. We used multiple-choice questions to measure participants’ comprehension of the conversation in multitalker babble noise at three different signal-to-noise ratios. All participants made saccades between the two talkers more frequently than the talkers’ conversational turns. This measure tended to correlate positively with participants’ comprehension scores, but the effect was significant in only one signal-to-noise ratio condition. Post-hoc investigation suggests that intertalker saccade rate is driven by an interaction of hearing ability and conversational turn-taking events, which will be further discussed.