The Neural Substrates of Multisensory Speech Perception

Comprehending speech is one of the most important human behaviors, but we are only beginning to understand how the brain accomplishes this difficult task. One key to speech perception seems to be that the brain integrates the independent sources of information available in the auditory and visual modalities in a process known as multisensory integration. This allows speech perception to be accurate, even in environments in which one modality or the other is ambiguous in the context of noise. Previous electrophysiological and functional magnetic resonance imaging (fMRI) experiments have implicated the posterior superior temporal sulcus (STS) in auditory-visual integration of both speech and non-speech stimuli. While evidence from prior imaging studies have found increases in STS activity for audiovisual speech compared with unisensory auditory or visual speech, these studies do not provide a clear mechanism as to how the STS communicates with early sensory areas to integrate the two streams of information into a coherent audiovisual percept. Furthermore, it is currently unknown if the activity within the STS is directly correlated with strength of audiovisual perception. In order to better understand the cortical mechanisms that underlie audiovisual speech perception, we first studied the STS activity and connectivity during the perception of speech with auditory and visual components of varying intelligibility. By studying fMRI activity during these noisy audiovisual speech stimuli, we found that STS connectivity with auditory and visual cortical areas mirrored perception; when the information from one modality is unreliable and noisy, the STS interacts less with the cortex processing that modality and more with the cortex processing the reliable information. We next characterized the role of STS activity during a striking audiovisual speech illusion, the McGurk effect, to determine if activity within the STS predicts how strongly a person integrates auditory and visual speech information. Subjects with greater susceptibility to the McGurk effect exhibited stronger fMRI activation of the STS during perception of McGurk syllables, implying a direct correlation between strength of audiovisual integration of speech and activity within an the multisensory STS.

VISUAL MEMORY PROCESSING BY PRIMATES AND AVIANS IN A RELATIONAL TASK

Three rhesus monkeys (Macaca mulatta) and four pigeons (Columba livia) were trained in a visual serial probe recognition (SPR) task. A list of visual stimuli (slides) was presented sequentially to the subjects. Following the list and after a delay interval, a probe stimulus was presented that could be either from the list (Same) or not from the list (Different). The monkeys readily acquired a variable list length SPR task, while pigeons showed acquisition only under constant list length condition. However, monkeys memorized the responses to the probes (absolute strategy) when overtrained with the same lists and probes, while pigeons compared the probe to the list in memory (relational strategy). Performance of the pigeon on 4-items constant list length was disrupted when blocks of trials of different list lengths were imbedded between the 4-items blocks. Serial position curves for recognition at variable probe delays showed better relative performance on the last items of the list at short delays (0-0.5 seconds) and better relative performance on the initial items of the list at long delays (6-10 seconds for the pigeons and 20-30 seconds for the monkeys and a human adolescent). The serial position curves also showed reliable primacy and recency effects at intermediate probe delays. The monkeys showed evidence of using a relational strategy in the variable probe delay task. The results are the first demonstration of relational serial probe recognition performance in an avian and suggest similar underlying dynamic recognition memory mechanisms in primates and avians. ^

The contributions of the hippocampal formation, perirhinal cortex and areas TH/TF to object, spatial and contextual recognition memory in primates

Adult monkeys (Macaca mulatta) with lesions of the hippocampal formation, perirhinal cortex, areas TH/TF, as well as controls were tested on tasks of object, spatial and contextual recognition memory. ^ Using a visual paired-comparison (VPC) task, all experimental groups showed a lack of object recognition relative to controls, although this impairment emerged at 10 sec with perirhinal lesions, 30 sec with areas TH/TF lesions and 60 sec with hippocampal lesions. In contrast, only perirhinal lesions impaired performance on delayed nonmatching-to-sample (DNMS), another task of object recognition memory. All groups were tested on DNMS with distraction (dDNMS) to examine whether the use of active cognitive strategies during the delay period could enable good performance on DNMS in spite of impaired recognition memory (revealed by the VPC task). Distractors affected performance of animals with perirhinal lesions at the 10-sec delay (the only delay in which their DNMS performance was above chance). They did not affect performance of animals with areas TH/TF lesions. Hippocampectomized animals were impaired at the 600-sec delay (the only delay at which prevention of active strategies would likely affect their behavior). ^ While lesions of areas TH/TF impaired spatial location memory and object-in-place memory, hippocampal lesions impaired only object-in-place memory. The pattern of results for perirhinal cortex lesions on the different task conditions indicated that this cortical area is not critical for spatial memory. ^ Finally, all three lesions impaired contextual recognition memory processes. The pattern of impairment appeared to result from the formation of only a global representation of the object and background, and suggests that all three areas are recruited for associating information across sources. ^ These results support the view that (1) the perirhinal cortex maintains storage of information about object and the context in which it is learned for a brief period of time, (2) areas TH/TF maintain information about spatial location and form associations between objects and their spatial relationship (a process that likely requires additional time) and (3) the hippocampal formation mediates associations between objects, their spatial relationship and the general context in which these associations are formed (an integrative function that requires additional time). ^

Maintenance of phosphatidylserine asymmetry in erythroid cells during differentiation, aging, and its significance for macrophage recognition

Phosphatidylserine (PS) is distributed almost entirely in the inner leaflet of the erythrocyte membrane bilayer, and appears to be maintained by a 32 kDa integral membrane protein (PS translocase). The expression of PS on the outer leaflet may serve as a recognition signal for macrophages, since insertion of PS into erythrocytes enhances their adherence to macrophages and clearance from the circulation. Therefore I have hypothesized that erythroid cells display PS on their outer leaflet early in differentiation and upon aging. Analysis of murine erythroleukemia cells (MELC, undifferentiated erythroid progenitor cells) showed high levels of PS on the outer leaflet that decreased during differentiation, correlating with the pattern of macrophage adherence. The activity of the PS translocase during differentiation appears to be unchanged although the equilibrium distribution of PS differs. This difference may be due to qualitative changes in the PS translocase. $\sp{125}$I-Bolton/Hunter-labeled-pyridyldithioethylamine ($\sp{125}$I-B/H-PDA), a radiolabeled probe for the PS translocase, labeled a 32 kDa protein in mature erythrocytes whereas in MELC a 45 kDa protein as well as a 32 kDa protein was identified. The abundance of the 45 kDa protein in relation to the 32 kDa protein declined during differentiation, possibly indicating this protein was a precursor of the 32 kDa protein. Analysis of the 45 kDa protein by N-glycosidase F and endoproteinase cleavage suggested this protein was not a glycosylated form of the 32 kDa protein but appeared to share some structural homology. Aged murine erythrocytes had elevated levels of PS on their outer leaflet, as well as decreased PS translocase activity. $\sp{125}$I-B/H-PDA labeled a 32 kDa protein in both normal and aged erythrocytes. However, the latter cells also contained a 28 kDa protein. Experimental evidence suggests that the appearance of the 28 kDa protein may be due to increased oxidation of aged erythrocytes. Examination of PS distribution showed that the levels of PS on the outer leaflet were elevated early in differentiation, decreased during the mature state, and returned to high levels as the erythrocyte aged. In conclusion,the levels of outer leaflet PS correlated with the differentiation status and macrophage recognition of erythroid cells. ^