The Neural Substrates of Multisensory Speech Perception

Comprehending speech is one of the most important human behaviors, but we are only beginning to understand how the brain accomplishes this difficult task. One key to speech perception seems to be that the brain integrates the independent sources of information available in the auditory and visual modalities in a process known as multisensory integration. This allows speech perception to be accurate, even in environments in which one modality or the other is ambiguous in the context of noise. Previous electrophysiological and functional magnetic resonance imaging (fMRI) experiments have implicated the posterior superior temporal sulcus (STS) in auditory-visual integration of both speech and non-speech stimuli. While evidence from prior imaging studies have found increases in STS activity for audiovisual speech compared with unisensory auditory or visual speech, these studies do not provide a clear mechanism as to how the STS communicates with early sensory areas to integrate the two streams of information into a coherent audiovisual percept. Furthermore, it is currently unknown if the activity within the STS is directly correlated with strength of audiovisual perception. In order to better understand the cortical mechanisms that underlie audiovisual speech perception, we first studied the STS activity and connectivity during the perception of speech with auditory and visual components of varying intelligibility. By studying fMRI activity during these noisy audiovisual speech stimuli, we found that STS connectivity with auditory and visual cortical areas mirrored perception; when the information from one modality is unreliable and noisy, the STS interacts less with the cortex processing that modality and more with the cortex processing the reliable information. We next characterized the role of STS activity during a striking audiovisual speech illusion, the McGurk effect, to determine if activity within the STS predicts how strongly a person integrates auditory and visual speech information. Subjects with greater susceptibility to the McGurk effect exhibited stronger fMRI activation of the STS during perception of McGurk syllables, implying a direct correlation between strength of audiovisual integration of speech and activity within an the multisensory STS.

Generalized fuzzy clustering for segmentation of multi-spectral magnetic resonance images.

An integrated approach for multi-spectral segmentation of MR images is presented. This method is based on the fuzzy c-means (FCM) and includes bias field correction and contextual constraints over spatial intensity distribution and accounts for the non-spherical cluster's shape in the feature space. The bias field is modeled as a linear combination of smooth polynomial basis functions for fast computation in the clustering iterations. Regularization terms for the neighborhood continuity of intensity are added into the FCM cost functions. To reduce the computational complexity, the contextual regularizations are separated from the clustering iterations. Since the feature space is not isotropic, distance measure adopted in Gustafson-Kessel (G-K) algorithm is used instead of the Euclidean distance, to account for the non-spherical shape of the clusters in the feature space. These algorithms are quantitatively evaluated on MR brain images using the similarity measures.

Volume and shape in feature space on adaptive FCM in MRI segmentation.

Intensity non-uniformity (bias field) correction, contextual constraints over spatial intensity distribution and non-spherical cluster's shape in the feature space are incorporated into the fuzzy c-means (FCM) for segmentation of three-dimensional multi-spectral MR images. The bias field is modeled by a linear combination of smooth polynomial basis functions for fast computation in the clustering iterations. Regularization terms for the neighborhood continuity of either intensity or membership are added into the FCM cost functions. Since the feature space is not isotropic, distance measures, other than the Euclidean distance, are used to account for the shape and volumetric effects of clusters in the feature space. The performance of segmentation is improved by combining the adaptive FCM scheme with the criteria used in Gustafson-Kessel (G-K) and Gath-Geva (G-G) algorithms through the inclusion of the cluster scatter measure. The performance of this integrated approach is quantitatively evaluated on normal MR brain images using the similarity measures. The improvement in the quality of segmentation obtained with our method is also demonstrated by comparing our results with those produced by FSL (FMRIB Software Library), a software package that is commonly used for tissue classification.

Molecular and biochemical analysis of the {\it Drosophila\/} segmentation gene {\it runt\/}

Genetic evidence has indicated that the segmentation gene runt plays a key role in regulating gene expression of the pair-rule genes hairy, even-skipped, and fushi tarazu. In contrast to other pair-rule genes, sequence data of the runt open reading frame did not reveal homologies to DNA-binding motifs of known transcriptional regulatory proteins. This thesis project examined several properties of the runt gene based on the sequence of the transcription unit, including the subcellular localization of the protein in vivo, its ability to bind DNA, and the functionality of a putative nucleotide binding domain.^ A runt-specific antibody was generated and used to demonstrate that runt is localized in the nucleus. Since the precise overlap of the pair-rule stripes is thought to be critical for the determination of cellular identity along the anterior-posterior axis, phasing of early runt expression in the blastoderm was examined with regard to the segmentation genes hairy, even-skipped, and fushi tarazu. runt was also expressed at later stages of embryogenesis, including expression in neuroblasts, and ganglion mother cells of the developing nervous system. Expression at this stage was required for the subsequent formation of specific neurons and runt was extensively expressed in the central and peripheral nervous systems.^ Several experiments were done to address the biochemical function of the runt protein. A direct interaction of runt with DNA was first examined. Although bacterial expressed runt was found to bind dsDNA-cellulose, subsequent experiments failed to detect sequence-specific interactions with DNA. Inter-species conservation of the putative nucleotide binding domain suggested that this region was functionally important, and runt protein bound a labeled ATP analog with high affinity in vitro. Finally, the effect of substitution of a critical residue of the nucleotide binding domain on runt activity was examined in vivo. Ectopic expression of the mutant protein indicated that this conserved substitution altered, but did not eliminate, runt activity as evaluated by segmentation phenotype and viability. ^