Emotional attention as a modulatory system of perception

Report for the scientific sojourn carried out at the University Medical Center, Swiss, from 2010 to 2012. Abundant evidence suggests that negative emotional stimuli are prioritized in the perceptual systems, eliciting enhanced neural responses in early sensory regions as compared with neutral information. This facilitated detection is generally paralleled by larger neural responses in early sensory areas, relative to the processing of neutral information. In this sense, the amygdala and other limbic regions, such as the orbitofrontal cortex, may play a critical role by sending modulatory projections onto the sensory cortices via direct or indirect feedback.The present project aimed at investigating two important issues regarding these mechanisms of emotional attention, by means of functional magnetic resonance imaging. In Study I, we examined the modulatory effects of visual emotion signals on the processing of task-irrelevant visual, auditory, and somatosensory input, that is, the intramodal and crossmodal effects of emotional attention. We observed that brain responses to auditory and tactile stimulation were enhanced during the processing of visual emotional stimuli, as compared to neutral, in bilateral primary auditory and somatosensory cortices, respectively. However, brain responses to visual task-irrelevant stimulation were diminished in left primary and secondary visual cortices in the same conditions. The results also suggested the existence of a multimodal network associated with emotional attention, presumably involving mediofrontal, temporal and orbitofrontal regions Finally, Study II examined the different brain responses along the low-level visual pathways and limbic regions, as a function of the number of retinal spikes during visual emotional processing. The experiment used stimuli resulting from an algorithm that simulates how the visual system perceives a visual input after a given number of retinal spikes. The results validated the visual model in human subjects and suggested differential emotional responses in the amygdala and visual regions as a function of spike-levels. A list of publications resulting from work in the host laboratory is included in the report.

Temporo-Parietal Junction encodes the Embodied Self: Neuro-Robotics, fMRI, and Lesion mapping.

Introduction: Neuroimaging of the self focused on high-level mechanisms such as language, memory or imagery of the self. Recent evidence suggests that low-level mechanisms of multisensory and sensorimotor integration may play a fundamental role in encoding self-location and the first-person perspective (Blanke and Metzinger, 2009). Neurological patients with out-of body experiences (OBE) suffer from abnormal self-location and the first-person perspective due to a damage in the temporo-parietal junction (Blanke et al., 2004). Although self-location and the first-person perspective can be studied experimentally (Lenggenhager et al., 2009), the neural underpinnings of self-location have yet to be investigated. To investigate the brain network involved in self-location and first-person perspective we used visuo-tactile multisensory conflict, magnetic resonance (MR)-compatible robotics, and fMRI in study 1, and lesion analysis in a sample of 9 patients with OBE due to focal brain damage in study 2. Methods: Twenty-two participants saw a video showing either a person's back or an empty room being stroked (visual stimuli) while the MR-compatible robotic device stroked their back (tactile stimulation). Direction and speed of the seen stroking could either correspond (synchronous) or not (asynchronous) to those of the seen stroking. Each run comprised the four conditions according to a 2x2 factorial design with Object (Body, No-Body) and Synchrony (Synchronous, Asynchronous) as main factors. Self-location was estimated using the mental ball dropping (MBD; Lenggenhager et al., 2009). After the fMRI session participants completed a 6-item adapted from the original questionnaire created by Botvinick and Cohen (1998) and based on questions and data obtained by Lenggenhager et al. (2007, 2009). They were also asked to complete a questionnaire to disclose the perspective they adopted during the illusion. Response times (RTs) for the MBD and fMRI data were analyzed with a 3-way mixed model ANOVA with the in-between factor Perspective (up, down) and the two with-in factors Object (body, no-body) and Stroking (synchronous, asynchronous). Quantitative lesion analysis was performed using MRIcron (Rorden et al., 2007). We compared the distributions of brain lesions confirmed by multimodality imaging (Knowlton, 2004) in patients with OBE with those showing complex visual hallucinations involving people or faces, but without any disturbance of self-location and first person perspective. Nine patients with OBE were investigated. The control group comprised 8 patients. Structural imaging data were available for normalization and co-registration in all the patients. Normalization of each patient's lesion into the common MNI (Montreal Neurological Institute) reference space permitted simple, voxel-wise, algebraic comparisons to be made. Results: Even if in the scanner all participants were lying on their back and were facing upwards, analysis of perspective showed that half of the participants had the impression to be looking down at the virtual human body below them, despite any cues about their body position (Down-group). The other participants had the impression to be looking up at the virtual body above them (Up-group). Analysis of Q3 ("How strong was the feeling that the body you saw was you?") indicated stronger self-identification with the virtual body during the synchronous stroking. RTs in the MBD task confirmed these subjective data (significant 3-way interaction between perspective, object and stroking). fMRI results showed eight cortical regions where the BOLD signal was significantly different during at least one of the conditions resulting from the combination of Object and Stroking, relative to baseline: right and left temporo-parietal junction, right EBA, left middle occipito-temporal gyrus, left postcentral gyrus, right medial parietal lobe, bilateral medial occipital lobe (Fig 1). The activation patterns in right and left temporo-parietal junction and right EBA reflected changes in self-location and perspective as revealed by statistical analysis that was performed on the percentage of BOLD change with respect to the baseline. Statistical lesion overlap comparison (using nonparametric voxel based lesion symptom mapping) with respect to the control group revealed the right temporo-parietal junction, centered at the angular gyrus (Talairach coordinates x = 54, y =-52, z = 26; p>0.05, FDR corrected). Conclusions: The present questionnaire and behavioural results show that - despite the noisy and constraining MR environment) our participants had predictable changes in self-location, self-identification, and first-person perspective when robotic tactile stroking was applied synchronously with the robotic visual stroking. fMRI data in healthy participants and lesion data in patients with abnormal self-location and first-person perspective jointly revealed that the temporo-parietal cortex especially in the right hemisphere encodes these conscious experiences. We argue that temporo-parietal activity reflects the experience of the conscious "I" as embodied and localized within bodily space.

Hedgehog signaling governs the development of otic sensory epithelium and its associated innervation in zebrafish

The inner ear is responsible for the perception of motion and sound in vertebrates. Its functional unit, the sensory patch, contains mechanosensory hair cells innervated by sensory neurons from the statoacoustic ganglion (SAG) that project to the corresponding nuclei in the brainstem. How hair cells develop at specific positions, and how otic neurons are sorted to specifically innervate each endorgan and to convey the extracted information to the hindbrain is not completely understood. In this work, we study the generation of macular sensory patches and investigate the role of Hedgehog (Hh) signaling in the production of their neurosensory elements. Using zebrafish transgenic lines to visualize the dynamics of hair cell and neuron production, we show that the development of the anterior and posterior maculae is asynchronic, suggesting they are independently regulated. Tracing experiments demonstrate the SAG is topologically organized in two different neuronal subpopulations, which are spatially segregated and innervate specifically each macula. Functional experiments identify the Hh pathway as crucial in coordinating the production of hair cells in the posterior macula, and the formation of its specific innervation. Finally, gene expression analyses suggest that Hh influences the balance between different SAG neuronal subpopulations. These results lead to a model in which Hh orients functionally the development of inner ear towards an auditory fate in all vertebrate species.

Predicting clinical scores from magnetic resonance scans in Alzheimer's disease.

Machine learning and pattern recognition methods have been used to diagnose Alzheimer's disease (AD) and mild cognitive impairment (MCI) from individual MRI scans. Another application of such methods is to predict clinical scores from individual scans. Using relevance vector regression (RVR), we predicted individuals' performances on established tests from their MRI T1 weighted image in two independent data sets. From Mayo Clinic, 73 probable AD patients and 91 cognitively normal (CN) controls completed the Mini-Mental State Examination (MMSE), Dementia Rating Scale (DRS), and Auditory Verbal Learning Test (AVLT) within 3months of their scan. Baseline MRI's from the Alzheimer's disease Neuroimaging Initiative (ADNI) comprised the other data set; 113 AD, 351 MCI, and 122 CN subjects completed the MMSE and Alzheimer's Disease Assessment Scale-Cognitive subtest (ADAS-cog) and 39 AD, 92 MCI, and 32 CN ADNI subjects completed MMSE, ADAS-cog, and AVLT. Predicted and actual clinical scores were highly correlated for the MMSE, DRS, and ADAS-cog tests (P<0.0001). Training with one data set and testing with another demonstrated stability between data sets. DRS, MMSE, and ADAS-Cog correlated better than AVLT with whole brain grey matter changes associated with AD. This result underscores their utility for screening and tracking disease. RVR offers a novel way to measure interactions between structural changes and neuropsychological tests beyond that of univariate methods. In clinical practice, we envision using RVR to aid in diagnosis and predict clinical outcome.

Multisensory facilitation of behavior in monkeys: effects of stimulus intensity.

Multisensory stimuli can improve performance, facilitating RTs on sensorimotor tasks. This benefit is referred to as the redundant signals effect (RSE) and can exceed predictions on the basis of probability summation, indicative of integrative processes. Although an RSE exceeding probability summation has been repeatedly observed in humans and nonprimate animals, there are scant and inconsistent data from nonhuman primates performing similar protocols. Rather, existing paradigms have instead focused on saccadic eye movements. Moreover, the extant results in monkeys leave unresolved how stimulus synchronicity and intensity impact performance. Two trained monkeys performed a simple detection task involving arm movements to auditory, visual, or synchronous auditory-visual multisensory pairs. RSEs in excess of predictions on the basis of probability summation were observed and thus forcibly follow from neural response interactions. Parametric variation of auditory stimulus intensity revealed that in both animals, RT facilitation was limited to situations where the auditory stimulus intensity was below or up to 20 dB above perceptual threshold, despite the visual stimulus always being suprathreshold. No RT facilitation or even behavioral costs were obtained with auditory intensities 30-40 dB above threshold. The present study demonstrates the feasibility and the suitability of behaving monkeys for investigating links between psychophysical and neurophysiologic instantiations of multisensory interactions.

Organizing principles across the senses

ABSTRACT This thesis is composed of two main parts. The first addressed the question of whether the auditory and somatosensory systems, like their visual counterpart, comprise parallel functional pathways for processing identity and spatial attributes (so-called `what' and `where' pathways, respectively). The second part examined the independence of control processes mediating task switching across 'what' and `where' pathways in the auditory and visual modalities. Concerning the first part, electrical neuroimaging of event-related potentials identified the spatio-temporal mechanisms subserving auditory (see Appendix, Study n°1) and vibrotactile (see Appendix, Study n°2) processing during two types of blocks of trials. `What' blocks varied stimuli in their frequency independently of their location.. `Where' blocks varied the same stimuli in their location independently of their frequency. Concerning the second part (see Appendix, Study n°3), a psychophysical task-switching paradigm was used to investigate the hypothesis that the efficacy of control processes depends on the extent of overlap between the neural circuitry mediating the different tasks at hand, such that more effective task preparation (and by extension smaller switch costs) is achieved when the anatomical/functional overlap of this circuitry is small. Performance costs associated with switching tasks and/or switching sensory modalities were measured. Tasks required the analysis of either the identity or spatial location of environmental objects (`what' and `where' tasks, respectively) that were presented either visually or acoustically on any given trial. Pretrial cues informed participants of the upcoming task, but not of the sensory modality. - In the audio-visual domain, the results showed that switch costs between tasks were significantly smaller when the sensory modality of the task switched versus when it repeated. In addition, switch costs between the senses were correlated only when the sensory modality of the task repeated across trials and not when it switched. The collective evidence not only supports the independence of control processes mediating task switching and modality switching, but also the hypothesis that switch costs reflect competitive interterence between neural circuits that in turn can be diminished when these neural circuits are distinct. - In the auditory and somatosensory domains, the findings show that a segregation of location vs. recognition information is observed across sensory systems and that these happen around 100ms for both sensory modalities. - Also, our results show that functionally specialized pathways for audition and somatosensation involve largely overlapping brain regions, i.e. posterior superior and middle temporal cortices and inferior parietal areas. Both these properties (synchrony of differential processing and overlapping brain regions) probably optimize the relationships across sensory modalities. - Therefore, these results may be indicative of a computationally advantageous organization for processing spatial anal identity information.

Spontaneous pre-stimulus fluctuations in the activity of right fronto-parietal areas influence inhibitory control performance.

Inhibitory control refers to the ability to suppress planned or ongoing cognitive or motor processes. Electrophysiological indices of inhibitory control failure have been found to manifest even before the presentation of the stimuli triggering the inhibition, suggesting that pre-stimulus brain-states modulate inhibition performance. However, previous electrophysiological investigations on the state-dependency of inhibitory control were based on averaged event-related potentials (ERPs), a method eliminating the variability in the ongoing brain activity not time-locked to the event of interest. These studies thus left unresolved whether spontaneous variations in the brain-state immediately preceding unpredictable inhibition-triggering stimuli also influence inhibitory control performance. To address this question, we applied single-trial EEG topographic analyses on the time interval immediately preceding NoGo stimuli in conditions where the responses to NoGo trials were correctly inhibited [correct rejection (CR)] vs. committed [false alarms (FAs)] during an auditory spatial Go/NoGo task. We found a specific configuration of the EEG voltage field manifesting more frequently before correctly inhibited responses to NoGo stimuli than before FAs. There was no evidence for an EEG topography occurring more frequently before FAs than before CR. The visualization of distributed electrical source estimations of the EEG topography preceding successful response inhibition suggested that it resulted from the activity of a right fronto-parietal brain network. Our results suggest that the fluctuations in the ongoing brain activity immediately preceding stimulus presentation contribute to the behavioral outcomes during an inhibitory control task. Our results further suggest that the state-dependency of sensory-cognitive processing might not only concern perceptual processes, but also high-order, top-down inhibitory control mechanisms.

Repetition-induced plasticity of motor representations of action sounds.

Action-related sounds are known to increase the excitability of motoneurones within the primary motor cortex (M1), but the role of this auditory input remains unclear. We investigated repetition priming-induced plasticity, which is characteristic of semantic representations, in M1 by applying transcranial magnetic stimulation pulses to the hand area. Motor evoked potentials (MEPs) were larger while subjects were listening to sounds related versus unrelated to manual actions. Repeated exposure to the same manual-action-related sound yielded a significant decrease in MEPs when right, hand area was stimulated; no repetition effect was observed for manual-action-unrelated sounds. The shared repetition priming characteristics suggest that auditory input to the right primary motor cortex is part of auditory semantic representations.

Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds.

Evidence of multisensory interactions within low-level cortices and at early post-stimulus latencies has prompted a paradigm shift in conceptualizations of sensory organization. However, the mechanisms of these interactions and their link to behavior remain largely unknown. One behaviorally salient stimulus is a rapidly approaching (looming) object, which can indicate potential threats. Based on findings from humans and nonhuman primates suggesting there to be selective multisensory (auditory-visual) integration of looming signals, we tested whether looming sounds would selectively modulate the excitability of visual cortex. We combined transcranial magnetic stimulation (TMS) over the occipital pole and psychophysics for "neurometric" and psychometric assays of changes in low-level visual cortex excitability (i.e., phosphene induction) and perception, respectively. Across three experiments we show that structured looming sounds considerably enhance visual cortex excitability relative to other sound categories and white-noise controls. The time course of this effect showed that modulation of visual cortex excitability started to differ between looming and stationary sounds for sound portions of very short duration (80 ms) that were significantly below (by 35 ms) perceptual discrimination threshold. Visual perceptions are thus rapidly and efficiently boosted by sounds through early, preperceptual and stimulus-selective modulation of neuronal excitability within low-level visual cortex.

Bilateral dorsal and ventral fiber pathways for the processing of affective prosody identified by probabilistic fiber tracking.

Dorsal and ventral pathways for syntacto-semantic speech processing in the left hemisphere are represented in the dual-stream model of auditory processing. Here we report new findings for the right dorsal and ventral temporo-frontal pathway during processing of affectively intonated speech (i.e. affective prosody) in humans, together with several left hemispheric structural connections, partly resembling those for syntacto-semantic speech processing. We investigated white matter fiber connectivity between regions responding to affective prosody in several subregions of the bilateral superior temporal cortex (secondary and higher-level auditory cortex) and of the inferior frontal cortex (anterior and posterior inferior frontal gyrus). The fiber connectivity was investigated by using probabilistic diffusion tensor based tractography. The results underscore several so far underestimated auditory pathway connections, especially for the processing of affective prosody, such as a right ventral auditory pathway. The results also suggest the existence of a dual-stream processing in the right hemisphere, and a general predominance of the dorsal pathways in both hemispheres underlying the neural processing of affective prosody in an extended temporo-frontal network.

The brain uses single-trial multisensory memories to discriminate without awareness.

Multisensory experiences enhance perceptions and facilitate memory retrieval processes, even when only unisensory information is available for accessing such memories. Using fMRI, we identified human brain regions involved in discriminating visual stimuli according to past multisensory vs. unisensory experiences. Subjects performed a completely orthogonal task, discriminating repeated from initial image presentations intermixed within a continuous recognition task. Half of initial presentations were multisensory, and all repetitions were exclusively visual. Despite only single-trial exposures to initial image presentations, accuracy in indicating image repetitions was significantly improved by past auditory-visual multisensory experiences over images only encountered visually. Similarly, regions within the lateral-occipital complex-areas typically associated with visual object recognition processes-were more active to visual stimuli with multisensory than unisensory pasts. Additional differential responses were observed in the anterior cingulate and frontal cortices. Multisensory experiences are registered by the brain even when of no immediate behavioral relevance and can be used to categorize memories. These data reveal the functional efficacy of multisensory processing.

The structural nature of perceptual experiences

Intuitively, we think of perception as providing us with direct cognitive access to physical objects and their properties. But this common sense picture of perception becomes problematic when we notice that perception is not always veridical. In fact, reflection on illusions and hallucinations seems to indicate that perception cannot be what it intuitively appears to be. This clash between intuition and reflection is what generates the puzzle of perception. The task and enterprise of unravelling this puzzle took, and still takes, centre stage in the philosophy of perception. The goal of my dissertation is to make a contribution to this enterprise by formulating and defending a new structural approach to perception and perceptual consciousness. The argument for my structural approach is developed in several steps. Firstly, I develop an empirically inspired causal argument against naïve and direct realist conceptions of perceptual consciousness. Basically, the argument says that perception and hallucination can have the same proximal causes and must thus belong to the same mental kind. I emphasise that this insight gives us good reasons to abandon what we are instinctively driven to believe - namely that perception is directly about the outside physical world. The causal argument essentially highlights that the information that the subject acquires in perceiving a worldly object is always indirect. To put it another way, the argument shows that what we, as perceivers, are immediately aware of, is not an aspect of the world but an aspect of our sensory response to it. A view like this is traditionally known as a Representative Theory of Perception. As a second step, emphasis is put on the task of defending and promoting a new structural version of the Representative Theory of Perception; one that is immune to some major objections that have been standardly levelled at other Representative Theories of Perception. As part of this defence and promotion, I argue that it is only the structural features of perceptual experiences that are fit to represent the empirical world. This line of thought is backed up by a detailed study of the intriguing phenomenon of synaesthesia. More precisely, I concentrate on empirical cases of synaesthetic experiences and argue that some of them provide support for a structural approach to perception. The general picture that emerges in this dissertation is a new perspective on perceptual consciousness that is structural through and through.

Noise in Brain Activity Engenders Perception and Influences Discrimination Sensitivity.

Behavioral and brain responses to identical stimuli can vary with experimental and task parameters, including the context of stimulus presentation or attention. More surprisingly, computational models suggest that noise-related random fluctuations in brain responses to stimuli would alone be sufficient to engender perceptual differences between physically identical stimuli. In two experiments combining psychophysics and EEG in healthy humans, we investigated brain mechanisms whereby identical stimuli are (erroneously) perceived as different (higher vs lower in pitch or longer vs shorter in duration) in the absence of any change in the experimental context. Even though, as expected, participants' percepts to identical stimuli varied randomly, a classification algorithm based on a mixture of Gaussians model (GMM) showed that there was sufficient information in single-trial EEG to reliably predict participants' judgments of the stimulus dimension. By contrasting electrical neuroimaging analyses of auditory evoked potentials (AEPs) to the identical stimuli as a function of participants' percepts, we identified the precise timing and neural correlates (strength vs topographic modulations) as well as intracranial sources of these erroneous perceptions. In both experiments, AEP differences first occurred ∼100 ms after stimulus onset and were the result of topographic modulations following from changes in the configuration of active brain networks. Source estimations localized the origin of variations in perceived pitch of identical stimuli within right temporal and left frontal areas and of variations in perceived duration within right temporoparietal areas. We discuss our results in terms of providing neurophysiologic evidence for the contribution of random fluctuations in brain activity to conscious perception.

French version validation of the psychotic symptom rating scales (PSYRATS) for outpatients with persistent psychotic symptoms.

ABSTRACT: BACKGROUND: Most scales that assess the presence and severity of psychotic symptoms often measure a broad range of experiences and behaviours, something that restricts the detailed measurement of specific symptoms such as delusions or hallucinations. The Psychotic Symptom Rating Scales (PSYRATS) is a clinical assessment tool that focuses on the detailed measurement of these core symptoms. The goal of this study was to examine the psychometric properties of the French version of the PSYRATS. METHODS: A sample of 103 outpatients suffering from schizophrenia or schizoaffective disorders and presenting persistent psychotic symptoms over the previous three months was assessed using the PSYRATS. Seventy-five sample participants were also assessed with the Positive And Negative Syndrome Scale (PANSS). RESULTS: ICCs were superior to .90 for all items of the PSYRATS. Factor analysis replicated the factorial structure of the original version of the delusions scale. Similar to previous replications, the factor structure of the hallucinations scale was partially replicated. Convergent validity indicated that some specific PSYRATS items do not correlate with the PANSS delusions or hallucinations. The distress items of the PSYRATS are negatively correlated with the grandiosity scale of the PANSS. CONCLUSIONS: The results of this study are limited by the relatively small sample size as well as the selection of participants with persistent symptoms. The French version of the PSYRATS partially replicates previously published results. Differences in factor structure of the hallucinations scale might be explained by greater variability of its elements. The future development of the scale should take into account the presence of grandiosity in order to better capture details of the psychotic experience.

GPS analysis of human locomotion: further evidence for long-range correlations in stride-to-stride fluctuations of gait parameters.

During free walking, gait is automatically adjusted to provide optimal mechanical output and minimal energy expenditure; gait parameters, such as cadence, fluctuate from one stride to the next around average values. It was described that this fluctuation exhibited long-range correlations and fractal-like patterns. In addition, it was suggested that these long-range correlations disappeared if the participant followed the beep of metronome to regulate his or her pace. Until now, these fractal fluctuations were only observed for stride interval, because no technique existed to adequately analyze an extended time of free walking. The aim of the present study was to measure walking speed (WS), step frequency (SF) and step length (SL) with high accuracy (<1 cm) satellite positioning method (global positioning system or GPS) in order to detect long-range correlations in the stride-to-stride fluctuations. Eight participants walked 30 min under free and constrained (metronome) conditions. Under free walking conditions, DFA (detrended fluctuation analysis) and surrogate data tests showed that the fluctuation of WS, SL and SF exhibited a fractal pattern (i.e., scaling exponent alpha: 0.5 < alpha < 1) in a large majority of participants (7/8). Under constrained conditions (metronome), SF fluctuations became significantly anti-correlated (alpha < 0.5) in all participants. However, the scaling exponent of SL and WS was not modified. We conclude that, when the walking pace is controlled by an auditory signal, the feedback loop between the planned movement (at supraspinal level) and the sensory inputs induces a continual shifting of SF around the mean (persistent anti-correlation), but with no effect on the fluctuation dynamics of the other parameters (SL, WS).