Effects of age and multiple talking faces on the visual speech advantage in noise

Understanding speech in noise is challenging, especially for older adults. Seeing a talker’s face helps speech recognition in noise compared to an auditory only baseline (i.e., a visual speech advantage); however, previous work on the visual speech advantage is limited in that it has only presented listeners with a single talking face. In a cocktail party environment, more than one talking face is likely in an individual’s visual field. This study aimed to evaluate how multiple sources of visual speech effect speech perception in noise for younger and older adults.

Younger and older adults completed the exact same version of the experiment (i.e., volume and SNR were consistent across groups). For each trial, participants listened to a sentence spoken in speech shaped noise (SNR=-3dB). Concurrently, 1, 2, 4, or 6 talking faces were presented. One talking face always matched the auditory signal; the other face(s) did not. Twenty percent of trials were auditory only, that is, the visual stimulus consisted of a static picture of a face or faces. Participants typed what they heard and were scored on key words correct.

For younger adults, all visual speech conditions provided a visual speech advantage. The advantage was greatest for the single face condition and declined as more talking faces were added. There was no difference between the 4 and 6 talking face conditions.

Preliminary results from a group of older adults also revealed a significant visual speech advantage for the single face condition. When any additional talking faces were added, however, older adults’ scores were no better than in the auditory only condition.

These results suggest that processing a talker’s visual speech requires attention. In the conditions with multiple talking faces, participants likely had to locate the matching talking face to gain a visual speech advantage. Older adults seem to be particularly distracted by multiple sources of visual speech. Susceptibility to distraction from visual stimuli could contribute to difficulties understanding speech in cocktail party listening environments.