The Neural Substrates of Multisensory Speech Perception

Comprehending speech is one of the most important human behaviors, but we are only beginning to understand how the brain accomplishes this difficult task. One key to speech perception seems to be that the brain integrates the independent sources of information available in the auditory and visual modalities in a process known as multisensory integration. This allows speech perception to be accurate, even in environments in which one modality or the other is ambiguous in the context of noise. Previous electrophysiological and functional magnetic resonance imaging (fMRI) experiments have implicated the posterior superior temporal sulcus (STS) in auditory-visual integration of both speech and non-speech stimuli. While evidence from prior imaging studies have found increases in STS activity for audiovisual speech compared with unisensory auditory or visual speech, these studies do not provide a clear mechanism as to how the STS communicates with early sensory areas to integrate the two streams of information into a coherent audiovisual percept. Furthermore, it is currently unknown if the activity within the STS is directly correlated with strength of audiovisual perception. In order to better understand the cortical mechanisms that underlie audiovisual speech perception, we first studied the STS activity and connectivity during the perception of speech with auditory and visual components of varying intelligibility. By studying fMRI activity during these noisy audiovisual speech stimuli, we found that STS connectivity with auditory and visual cortical areas mirrored perception; when the information from one modality is unreliable and noisy, the STS interacts less with the cortex processing that modality and more with the cortex processing the reliable information. We next characterized the role of STS activity during a striking audiovisual speech illusion, the McGurk effect, to determine if activity within the STS predicts how strongly a person integrates auditory and visual speech information. Subjects with greater susceptibility to the McGurk effect exhibited stronger fMRI activation of the STS during perception of McGurk syllables, implying a direct correlation between strength of audiovisual integration of speech and activity within an the multisensory STS.

Multifactor effects and evidence of potential interaction between complement factor H Y402H and LOC387715 A69S in age-related macular degeneration.

BACKGROUND: Variants in the complement cascade genes and the LOC387715/HTRA1, have been widely reported to associate with age-related macular degeneration (AMD), the most common cause of visual impairment in industrialized countries. METHODS/PRINCIPAL FINDINGS: We investigated the association between the LOC387715 A69S and complement component C3 R102G risk alleles in the Finnish case-control material and found a significant association with both variants (OR 2.98, p = 3.75 x 10(-9); non-AMD controls and OR 2.79, p = 2.78 x 10(-19), blood donor controls and OR 1.83, p = 0.008; non-AMD controls and OR 1.39, p = 0.039; blood donor controls), respectively. Previously, we have shown a strong association between complement factor H (CFH) Y402H and AMD in the Finnish population. A carrier of at least one risk allele in each of the three susceptibility loci (LOC387715, C3, CFH) had an 18-fold risk of AMD when compared to a non-carrier homozygote in all three loci. A tentative gene-gene interaction between the two major AMD-associated loci, LOC387715 and CFH, was found in this study using a multiplicative (logistic regression) model, a synergy index (departure-from-additivity model) and the mutual information method (MI), suggesting that a common causative pathway may exist for these genes. Smoking (ever vs. never) exerted an extra risk for AMD, but somewhat surprisingly, only in connection with other factors such as sex and the C3 genotype. Population attributable risks (PAR) for the CFH, LOC387715 and C3 variants were 58.2%, 51.4% and 5.8%, respectively, the summary PAR for the three variants being 65.4%. CONCLUSIONS/SIGNIFICANCE: Evidence for gene-gene interaction between two major AMD associated loci CFH and LOC387715 was obtained using three methods, logistic regression, a synergy index and the mutual information (MI) index.

Dynamic population coding in primary visual cortex

More than a century ago Ramon y Cajal pioneered the description of neural circuits. Currently, new techniques are being developed to streamline the characterization of entire neural circuits. Even if this 'connectome' approach is successful, it will represent only a static description of neural circuits. Thus, a fundamental question in neuroscience is to understand how information is dynamically represented by neural populations. In this thesis, I studied two main aspects of dynamical population codes. ^ First, I studied how the exposure or adaptation, for a fraction of a second to oriented gratings dynamically changes the population response of primary visual cortex neurons. The effects of adaptation to oriented gratings have been extensively explored in psychophysical and electrophysiological experiments. However, whether rapid adaptation might induce a change in the primary visual cortex's functional connectivity to dynamically impact the population coding accuracy is currently unknown. To address this issue, we performed multi-electrode recordings in primary visual cortex, where adaptation has been previously shown to induce changes in the selectivity and response amplitude of individual neurons. We found that adaptation improves the population coding accuracy. The improvement was more prominent for iso- and orthogonal orientation adaptation, consistent with previously reported psychophysical experiments. We propose that selective decorrelation is a metabolically inexpensive mechanism that the visual system employs to dynamically adapt the neural responses to the statistics of the input stimuli to improve coding efficiency. ^ Second, I investigated how ongoing activity modulates orientation coding in single neurons, neural populations and behavior. Cortical networks are never silent even in the absence of external stimulation. The ongoing activity can account for up to 80% of the metabolic energy consumed by the brain. Thus, a fundamental question is to understand the functional role of ongoing activity and its impact on neural computations. I studied how the orientation coding by individual neurons and cell populations in primary visual cortex depend on the spontaneous activity before stimulus presentation. We hypothesized that since the ongoing activity of nearby neurons is strongly correlated, it would influence the ability of the entire population of orientation-selective cells to process orientation depending on the prestimulus spontaneous state. Our findings demonstrate that ongoing activity dynamically filters incoming stimuli to shape the accuracy of orientation coding by individual neurons and cell populations and this interaction affects behavioral performance. In summary, this thesis is a contribution to the study of how dynamic internal states such as rapid adaptation and ongoing activity modulate the population code accuracy. ^

The SanaViz: Human Centered Geovisualization to facilitate visual exploration of Public Health data

These three manuscripts are presented as a PhD dissertation for the study of using GeoVis application to evaluate telehealth programs. The primary reason of this research was to understand how the GeoVis applications can be designed and developed using combined approaches of HC approach and cognitive fit theory and in terms utilized to evaluate telehealth program in Brazil. First manuscript The first manuscript in this dissertation presented a background about the use of GeoVisualization to facilitate visual exploration of public health data. The manuscript covered the existing challenges that were associated with an adoption of existing GeoVis applications. The manuscript combines the principles of Human Centered approach and Cognitive Fit Theory and a framework using a combination of these approaches is developed that lays the foundation of this research. The framework is then utilized to propose the design, development and evaluation of “the SanaViz” to evaluate telehealth data in Brazil, as a proof of concept. Second manuscript The second manuscript is a methods paper that describes the approaches that can be employed to design and develop “the SanaViz” based on the proposed framework. By defining the various elements of the HC approach and CFT, a mixed methods approach is utilized for the card sorting and sketching techniques. A representative sample of 20 study participants currently involved in the telehealth program at the NUTES telehealth center at UFPE, Recife, Brazil was enrolled. The findings of this manuscript helped us understand the needs of the diverse group of telehealth users, the tasks that they perform and helped us determine the essential features that might be necessary to be included in the proposed GeoVis application “the SanaViz”. Third manuscript The third manuscript involved mix- methods approach to compare the effectiveness and usefulness of the HC GeoVis application “the SanaViz” against a conventional GeoVis application “Instant Atlas”. The same group of 20 study participants who had earlier participated during Aim 2 was enrolled and a combination of quantitative and qualitative assessments was done. Effectiveness was gauged by the time that the participants took to complete the tasks using both the GeoVis applications, the ease with which they completed the tasks and the number of attempts that were taken to complete each task. Usefulness was assessed by System Usability Scale (SUS), a validated questionnaire tested in prior studies. In-depth interviews were conducted to gather opinions about both the GeoVis applications. This manuscript helped us in the demonstration of the usefulness and effectiveness of HC GeoVis applications to facilitate visual exploration of telehealth data, as a proof of concept. Together, these three manuscripts represent challenges of combining principles of Human Centered approach, Cognitive Fit Theory to design and develop GeoVis applications as a method to evaluate Telehealth data. To our knowledge, this is the first study to explore the usefulness and effectiveness of GeoVis to facilitate visual exploration of telehealth data. The results of the research enabled us to develop a framework for the design and development of GeoVis applications related to the areas of public health and especially telehealth. The results of our study showed that the varied users were involved with the telehealth program and the tasks that they performed. Further it enabled us to identify the components that might be essential to be included in these GeoVis applications. The results of our research answered the following questions; (a) Telehealth users vary in their level of understanding about GeoVis (b) Interaction features such as zooming, sorting, and linking and multiple views and representation features such as bar chart and choropleth maps were considered the most essential features of the GeoVis applications. (c) Comparing and sorting were two important tasks that the telehealth users would perform for exploratory data analysis. (d) A HC GeoVis prototype application is more effective and useful for exploration of telehealth data than a conventional GeoVis application. Future studies should be done to incorporate the proposed HC GeoVis framework to enable comprehensive assessment of the users and the tasks they perform to identify the features that might be necessary to be a part of the GeoVis applications. The results of this study demonstrate a novel approach to comprehensively and systematically enhance the evaluation of telehealth programs using the proposed GeoVis Framework.