Context shapes neural speech coding: Effects of visual speech, prior auditory speech and listener’s goals on cortical and subcortical speech processing

Natural speech perception occurs in a complex context of co-occurring visual speech information, prior auditory speech information and the goals of the listener. I will present results from several experiments, which investigated the effect of different contextual factors on neural speech coding. First, results from an electrocorticography (ECoG) study on audiovisual sentence processing will be presented. All subjects showed electrodes that entrained either to the auditory sentence envelope or the visual lip movements. Several electrodes over posterior superior temporal gyrus (pSTG) showed entrainment to both modalities and enhanced high-gamma band responses to audiovisual compared to auditory-only speech. These results provide evidence that posterior superior temporal gyrus is important for multisensory integration of audiovisual speech. Next, results from a functional magnetic resonance imaging (fMRI) study will be presented, which show that a distributed network is activated when prior auditory speech information is integrated to understand degraded sublexical speech and that successful speech comprehension correlates with greater activation in early auditory cortex. Finally, I will highlight the important role of subcortical structures such as the thalamic centromedian-parafascicular complex (CM-Pf) in goal-oriented behavior selection, including the selection of a speaker-stream. Electrodes in the CM-Pf exhibited transient neural responses in the alpha/beta range to speech and non-speech cues that required a goal-oriented behavior. Taken together, these findings show which cortical and subcortical regions play a role in the integration of different contextual information during speech perception.