The Neural Substrates of Multisensory Speech Perception

Comprehending speech is one of the most important human behaviors, but we are only beginning to understand how the brain accomplishes this difficult task. One key to speech perception seems to be that the brain integrates the independent sources of information available in the auditory and visual modalities in a process known as multisensory integration. This allows speech perception to be accurate, even in environments in which one modality or the other is ambiguous in the context of noise. Previous electrophysiological and functional magnetic resonance imaging (fMRI) experiments have implicated the posterior superior temporal sulcus (STS) in auditory-visual integration of both speech and non-speech stimuli. While evidence from prior imaging studies have found increases in STS activity for audiovisual speech compared with unisensory auditory or visual speech, these studies do not provide a clear mechanism as to how the STS communicates with early sensory areas to integrate the two streams of information into a coherent audiovisual percept. Furthermore, it is currently unknown if the activity within the STS is directly correlated with strength of audiovisual perception. In order to better understand the cortical mechanisms that underlie audiovisual speech perception, we first studied the STS activity and connectivity during the perception of speech with auditory and visual components of varying intelligibility. By studying fMRI activity during these noisy audiovisual speech stimuli, we found that STS connectivity with auditory and visual cortical areas mirrored perception; when the information from one modality is unreliable and noisy, the STS interacts less with the cortex processing that modality and more with the cortex processing the reliable information. We next characterized the role of STS activity during a striking audiovisual speech illusion, the McGurk effect, to determine if activity within the STS predicts how strongly a person integrates auditory and visual speech information. Subjects with greater susceptibility to the McGurk effect exhibited stronger fMRI activation of the STS during perception of McGurk syllables, implying a direct correlation between strength of audiovisual integration of speech and activity within an the multisensory STS.

Temporal lobe white matter asymmetry and language laterality in epilepsy patients.

Recent studies using diffusion tensor imaging (DTI) have advanced our knowledge of the organization of white matter subserving language function. It remains unclear, however, how DTI may be used to predict accurately a key feature of language organization: its asymmetric representation in one cerebral hemisphere. In this study of epilepsy patients with unambiguous lateralization on Wada testing (19 left and 4 right lateralized subjects; no bilateral subjects), the predictive value of DTI for classifying the dominant hemisphere for language was assessed relative to the existing standard-the intra-carotid Amytal (Wada) procedure. Our specific hypothesis is that language laterality in both unilateral left- and right-hemisphere language dominant subjects may be predicted by hemispheric asymmetry in the relative density of three white matter pathways terminating in the temporal lobe implicated in different aspects of language function: the arcuate (AF), uncinate (UF), and inferior longitudinal fasciculi (ILF). Laterality indices computed from asymmetry of high anisotropy AF pathways, but not the other pathways, classified the majority (19 of 23) of patients using the Wada results as the standard. A logistic regression model incorporating information from DTI of the AF, fMRI activity in Broca's area, and handedness was able to classify 22 of 23 (95.6%) patients correctly according to their Wada score. We conclude that evaluation of highly anisotropic components of the AF alone has significant predictive power for determining language laterality, and that this markedly asymmetric distribution in the dominant hemisphere may reflect enhanced connectivity between frontal and temporal sites to support fluent language processes. Given the small sample reported in this preliminary study, future research should assess this method on a larger group of patients, including subjects with bi-hemispheric dominance.

Neuroimaging of epilepsy: lesions, networks, oscillations

While analysis and interpretation of structural epileptogenic lesion is an essential task for the neuroradiologist in clinical practice, a substantial body of epilepsy research has shown that focal lesions influence brain areas beyond the epileptogenic lesion, across ensembles of functionally and anatomically connected brain areas. In this review article, we aim to provide an overview about altered network compositions in epilepsy, as measured with current advanced neuroimaging techniques to characterize the initiation and spread of epileptic activity in the brain with multimodal noninvasive imaging techniques. We focus on resting-state functional magnetic resonance imaging (MRI) and simultaneous electroencephalography/fMRI, and oppose the findings in idiopathic generalized versus focal epilepsies. These data indicate that circumscribed epileptogenic lesions can have extended effects on many brain systems. Although epileptic seizures may involve various brain areas, seizure activity does not spread diffusely throughout the brain but propagates along specific anatomic pathways that characterize the underlying epilepsy syndrome. Such a functionally oriented approach may help to better understand a range of clinical phenomena such as the type of cognitive impairment, the development of pharmacoresistance, the propagation pathways of seizures, or the success of epilepsy surgery.

Brain fiber tract plasticity in experimental spinal cord injury: diffusion tensor imaging.

Diffusion tensor imaging (DTI) and immunohistochemistry were performed in spinal cord injured rats to understand the basis for activation of multiple regions in the brain observed in functional magnetic resonance imaging (fMRI) studies. The measured fractional anisotropy (FA), a scalar measure of diffusion anisotropy, along the region encompassing corticospinal tracts (CST) indicates significant differences between control and injured groups in the 3 to 4 mm area posterior to bregma that correspond to internal capsule and cerebral peduncle. Additionally, DTI-based tractography in injured animals showed increased number of fibers that extend towards the cortex terminating in the regions that were activated in fMRI. Both the internal capsule and cerebral peduncle demonstrated an increase in GFAP-immunoreactivity compared to control animals. GAP-43 expression also indicates plasticity in the internal capsule. These studies suggest that the previously observed multiple regions of activation in spinal cord injury are, at least in part, due to the formation of new fibers.

Jazz drummers recruit language-specific areas for the processing of rhythmic structure

Rhythm is a central characteristic of music and speech, the most important domains of human communication using acoustic signals. Here, we investigated how rhythmical patterns in music are processed in the human brain, and, in addition, evaluated the impact of musical training on rhythm processing. Using fMRI, we found that deviations from a rule-based regular rhythmic structure activated the left planum temporale together with Broca's area and its right-hemispheric homolog across subjects, that is, a network also crucially involved in the processing of harmonic structure in music and the syntactic analysis of language. Comparing the BOLD responses to rhythmic variations between professional jazz drummers and musical laypersons, we found that only highly trained rhythmic experts show additional activity in left-hemispheric supramarginal gyrus, a higher-order region involved in processing of linguistic syntax. This suggests an additional functional recruitment of brain areas usually dedicated to complex linguistic syntax processing for the analysis of rhythmical patterns only in professional jazz drummers, who are especially trained to use rhythmical cues for communication.

Visuospatial working memory in very preterm and term born children - Impact of age and performance

Working memory is crucial for meeting the challenges of daily life and performing academic tasks, such as reading or arithmetic. Very preterm born children are at risk of low working memory capacity. The aim of this study was to examine the visuospatial working memory network of school-aged preterm children and to determine the effect of age and performance on the neural working memory network. Working memory was assessed in 41 very preterm born children and 36 term born controls (aged 7–12 years) using functional magnetic resonance imaging (fMRI) and neuropsychological assessment. While preterm children and controls showed equal working memory performance, preterm children showed less involvement of the right middle frontal gyrus, but higher fMRI activation in superior frontal regions than controls. The younger and low-performing preterm children presented an atypical working memory network whereas the older high-performing preterm children recruited a working memory network similar to the controls. Results suggest that younger and low-performing preterm children show signs of less neural efficiency in frontal brain areas. With increasing age and performance, compensational mechanisms seem to occur, so that in preterm children, the typical visuospatial working memory network is established by the age of 12 years.

A new methodical approach in neuroscience: assessing inter-personal brain coupling using functional near-infrared imaging (fNIRI) hyperscanning

Since the first demonstration of how to simultaneously measure brain activity using functional magnetic resonance imaging (fMRI) on two subjects about 10 years ago, a new paradigm in neuroscience is emerging: measuring brain activity from two or more people simultaneously, termed "hyperscanning". The hyperscanning approach has the potential to reveal inter-personal brain mechanisms underlying interaction-mediated brain-to-brain coupling. These mechanisms are engaged during real social interactions, and cannot be captured using single-subject recordings. In particular, functional near-infrared imaging (fNIRI) hyperscanning is a promising new method, offering a cost-effective, easy to apply and reliable technology to measure inter-personal interactions in a natural context. In this short review we report on fNIRI hyperscanning studies published so far and summarize opportunities and challenges for future studies.

The Frontal Eye Field Is Involved in Visual Vector Inversion in Humans – A Theta Burst Stimulation Study

In the antisaccade task, subjects are requested to suppress a reflexive saccade towards a visual target and to perform a saccade towards the opposite side. In addition, in order to reproduce an accurate saccadic amplitude, the visual saccade vector (i.e., the distance between a central fixation point and the peripheral target) must be exactly inverted from one visual hemifield to the other. Results from recent studies using a correlational approach (i.e., fMRI, MEG) suggest that not only the posterior parietal cortex (PPC) but also the frontal eye field (FEF) might play an important role in such a visual vector inversion process. In order to assess whether the FEF contributes to visual vector inversion, we applied an interference approach with continuous theta burst stimulation (cTBS) during a memory-guided antisaccade task. In 10 healthy subjects, one train of cTBS was applied over the right FEF prior to a memory-guided antisaccade task. In comparison to the performance without stimulation or with sham stimulation, cTBS over the right FEF induced a hypometric gain for rightward but not leftward antisaccades. These results obtained with an interference approach confirm that the FEF is also involved in the process of visual vector inversion.

The mentalizing network orchestrates the impact of parochial altruism on social norm enforcement.

Parochial altruism - a preference for altruistic behavior towards ingroup members and mistrust or hostility towards outgroup members--is a pervasive feature in human society and strongly shapes the enforcement of social norms. Since the uniqueness of human society critically depends on the enforcement of norms, the understanding of the neural circuitry of the impact of parochial altruism on social norm enforcement is key, but unexplored. To fill this gap, we measured brain activity with functional magnetic resonance imaging (fMRI) while subjects had the opportunity to punish ingroup members and outgroup members for violating social norms. Findings revealed that subjects' strong punishment of defecting outgroup members is associated with increased activity in a functionally connected network involved in sanction-related decisions (right orbitofrontal gyrus, right lateral prefrontal cortex, right dorsal caudatus). Moreover, the stronger the connectivity in this network, the more outgroup members are punished. In contrast, the much weaker punishment of ingroup members who committed the very same norm violation is associated with increased activity and connectivity in the mentalizing-network (dorsomedial prefrontal cortex, bilateral temporo-parietal junction), as if subjects tried to understand or justify ingroup members' behavior. Finally, connectivity analyses between the two networks suggest that the mentalizing-network modulates punishment by affecting the activity in the right orbitofrontal gyrus and right lateral prefrontal cortex, notably in the same areas showing enhanced activity and connectivity whenever third-parties strongly punished defecting outgroup members.

Oxytocin shapes the neural circuitry of trust and trust adaptation in humans.

Trust and betrayal of trust are ubiquitous in human societies. Recent behavioral evidence shows that the neuropeptide oxytocin increases trust among humans, thus offering a unique chance of gaining a deeper understanding of the neural mechanisms underlying trust and the adaptation to breach of trust. We examined the neural circuitry of trusting behavior by combining the intranasal, double-blind, administration of oxytocin with fMRI. We find that subjects in the oxytocin group show no change in their trusting behavior after they learned that their trust had been breached several times while subjects receiving placebo decrease their trust. This difference in trust adaptation is associated with a specific reduction in activation in the amygdala, the midbrain regions, and the dorsal striatum in subjects receiving oxytocin, suggesting that neural systems mediating fear processing (amygdala and midbrain regions) and behavioral adaptations to feedback information (dorsal striatum) modulate oxytocin's effect on trust. These findings may help to develop deeper insights into mental disorders such as social phobia and autism, which are characterized by persistent fear or avoidance of social interactions.

Feeling present in arousing virtual reality worlds: prefrontal brain regions differentially orchestrate presence experience in adults and children

Virtual reality (VR) is a powerful tool for simulating aspects of the real world. The success of VR is thought to depend on its ability to evoke a sense of "being there", that is, the feeling of "Presence". In view of the rapid progress in the development of increasingly more sophisticated virtual environments (VE), the importance of understanding the neural underpinnings of presence is growing. To date however, the neural correlates of this phenomenon have received very scant attention. An fMRI-based study with 52 adults and 25 children was therefore conducted using a highly immersive VE. The experience of presence in adult subjects was found to be modulated by two major strategies involving two homologous prefrontal brain structures. Whereas the right DLPFC controlled the sense of presence by down-regulating the activation in the egocentric dorsal visual processing stream, the left DLPFC up-regulated widespread areas of the medial prefrontal cortex known to be involved in self-reflective and stimulus-independent thoughts. In contrast, there was no evidence of these two strategies in children. In fact, anatomical analyses showed that these two prefrontal areas have not yet reached full maturity in children. Taken together, this study presents the first findings that show activation of a highly specific neural network orchestrating the experience of presence in adult subjects, and that the absence of activity in this neural network might contribute to the generally increased susceptibility of children for the experience of presence in VEs.

When the systems fail to communicate. Cerebral resting state networks in schizophrenia spectrum disorders.

There have been numerous attempts to reveal the neurobiological basis of schizophrenia spectrum disorders. Results however, remain as heterogeneous as the schizophrenia spectrum disorders itself. Therefore, one aim of this thesis was to divide patients affected by this disorder into subgroups in order to homogenize the results of future studies. In a first study it is suggested that psychopathological rating scales should focus on symptoms-clusters that may have a common neurophysiological background. The here presented Bern Psychopathology Scale (BPS) proposes that alterations in three wellknown brain systems (motor, language, and affective) are largely leading to the communication failures observable on a behavioral level, but also - as repeatedly hypothesized - to dysconnectivity within and between brain systems in schizophrenia spectrum disorders. The external validity of the motor domain in the BPS was tested against the objective measure of 24 hours wrist actigraphy, in a second study. The subjective, the quantitative, as well as the global rating of the degree of motor disorders in this patient group showed significant correlations to the acquired motor activity. This result confirmed in a first step the practicability of the motor domain of the BPS, but needs further validation regarding pathological brain alterations. Finally, in a third study (independent from the two other studies), two cerebral Resting State Networks frequently altered in schizophrenia were investigated for the first time using simultaneous EEG/fMRI: The well-known default mode network and the left working memory network. Besides the changes in these fMRI-based networks, there are well-documented findings that patients exhibit alterations in EEG spectra compared to healthy controls. However, only through the multimodal approach it was possible to discover that patients with schizophrenia spectrum disorders have a slower driving frequency of the Resting State Networks compared to the matched healthy controls. Such a dysfunctional coupling between neuronal frequency and functional brain organization could explain in a uni- or multifactorial way (dysfunctional cross-frequency coupling, maturational effects, vigilance fluctuations, task-related suppression), how the typical psychotic symptoms might occur. To conclude, the major contributions presented in this thesis were on one hand the development of a psychopathology rating scale that is based on the assumption of dysfunctional brain networks, as well as the new evidence of a dysfunctional triggering frequency of Resting State Networks from the simultaneous EEG/fMRI study in patients affected by a schizophrenia spectrum disorder.

Individual differences in working memory capacity explain the relationship between general discrimination ability and psychometric intelligence

The close association between psychometric intelligence and general discrimination ability (GDA), conceptualized as latent variable derived from performance on different sensory discrimination tasks, is empirically well-established but theoretically widely unclear. The present study contrasted two alternative explanations for this association. The first explanation is based on what Spearman (1904) referred to as a central function underlying this relationship in the sense of the g factor of intelligence and becoming most evident in GDA. In this case, correlations between different aspects of cognitive abilities, such as working memory (WM) capacity, and psychometric intelligence should be mediated by GDA if their correlation is caused by g. Alternatively, the second explanation for the relationship between psychometric intelligence and GDA proceeds from fMRI studies which emphasize the role of WM functioning for sensory discrimination. Given the well-known relationship between WM and psychometric intelligence, the relationship between GDA and psychometric intelligence might be attributed to WM. The present study investigated these two alternative explanations at the level of latent variables. In 197 young adults, a model in which WM mediated the relationship between GDA and psychometric intelligence described the data better than a model in which GDA mediated the relationship between WM and psychometric intelligence. Moreover, GDA failed to explain portions of variance of psychometric intelligence above and beyond WM. These findings clearly support the view that the association between psychometric intelligence and GDA must be understood in terms of WM functioning.

Functional connectivity in BOLD and CBF data: similarity and reliability of resting brain networks

Resting-state functional connectivity (FC) fMRI (rs-fcMRI) offers an appealing approach to mapping the brain's intrinsic functional organization. Blood oxygen level dependent (BOLD) and arterial spin labeling (ASL) are the two main rs-fcMRI approaches to assess alterations in brain networks associated with individual differences, behavior and psychopathology. While the BOLD signal is stronger with a higher temporal resolution, ASL provides quantitative, direct measures of the physiology and metabolism of specific networks. This study systematically investigated the similarity and reliability of resting brain networks (RBNs) in BOLD and ASL. A 2×2×2 factorial design was employed where each subject underwent repeated BOLD and ASL rs-fcMRI scans on two occasions on two MRI scanners respectively. Both independent and joint FC analyses revealed common RBNs in ASL and BOLD rs-fcMRI with a moderate to high level of spatial overlap, verified by Dice Similarity Coefficients. Test-retest analyses indicated more reliable spatial network patterns in BOLD (average modal Intraclass Correlation Coefficients: 0.905±0.033 between-sessions; 0.885±0.052 between-scanners) than ASL (0.545±0.048; 0.575±0.059). Nevertheless, ASL provided highly reproducible (0.955±0.021; 0.970±0.011) network-specific CBF measurements. Moreover, we observed positive correlations between regional CBF and FC in core areas of all RBNs indicating a relationship between network connectivity and its baseline metabolism. Taken together, the combination of ASL and BOLD rs-fcMRI provides a powerful tool for characterizing the spatiotemporal and quantitative properties of RBNs. These findings pave the way for future BOLD and ASL rs-fcMRI studies in clinical populations that are carried out across time and scanners.

The brain’s pre-stimulus default state interacts with working memory in a load-dependent manner

Recently, multiple studies showed that spatial and temporal features of a task-negative default mode network (DMN) (Greicius et al., 2003) are important markers for psychiatric diseases (Balsters et al., 2013). Another prominent indicator of cognitive functioning, yielding information about the mental condition in health and disease, is working memory (WM) processing. In EEG and MEG studies, frontal-midline theta power has been shown to increase with load during WM retention in healthy subjects (Brookes et al., 2011). Negative correlations between DMN activity and theta amplitude have been found during resting state (Jann et al., 2010) as well as during WM (Michels et al., 2010). Likewise, WM training resulted in higher resting state theta power as well as increased small-worldness of the resting brain (Langer et al., 2013). Further, increased fMRI connectivity between nodes of the DMN correlated with better WM performance (Hampson et al., 2006). Hence, the brain’s default state might influence it’s functioning during task. We therefore hypothesized correlations between pre-stimulus DMN activity and EEG-theta power during WM maintenance, depending on the WM load. 17 healthy subjects performed a Sternberg WM task while being measured simultaneously with EEG and fMRI. Data was recorded within a multicenter-study: 12 subjects were measured in Zurich with a 64-channels MR-compatible system (Brain Products) in a 3T Philips scanner, 5 subjects with a 96-channel MR-compatible system (Brain Products) in a 3T Siemens Scanner in Bern. The DMN components was obtained by a group BOLD-ICA approach over the full task duration (figure 1). The subject-wise dynamics were obtained by back-reconstructed onto each subject’s fMRI data and normalized to percent signal change values. The single trial pre-stimulus-DMN activation was then temporally correlated with the single trial EEG-theta (3-8 Hz) spectral power during retention intervals. This so-called covariance mapping (Jann et al., 2010) yielded the spatial distribution of the theta EEG fluctuations during retention associated with the dynamics of the pre-stimulus DMN. In line with previous findings, theta power was increased at frontal-midline electrodes in high- versus low-load conditions during early WM retention (figure 2). However, correlations of DMN with theta power resulted in primarily positive correlations in low-load conditions, while during high-load conditions negative correlations of DMN activity and theta power were observed at frontal-midline electrodes. This DMN-dependent load effect reached significance in the middle of the retention period (TANOVA, p<0.05) (figure 3). Our results show a complex and load-dependent interaction of pre-stimulus DMN activity and theta power during retention, varying over time. While at a more global, load-independent view pre-stimulus DMN activity correlated positively with theta power during retention, the correlation was inversed during certain time windows in high-load trials, meaning that in trials with enhanced pre-stimulus DMN activity theta power decreases during retention. Since both WM performance and DMN activity are markers of mental health our results could be important for further investigations of psychiatric populations.