Single-trial multisensory memories affect later auditory and visual object discrimination.

Multisensory memory traces established via single-trial exposures can impact subsequent visual object recognition. This impact appears to depend on the meaningfulness of the initial multisensory pairing, implying that multisensory exposures establish distinct object representations that are accessible during later unisensory processing. Multisensory contexts may be particularly effective in influencing auditory discrimination, given the purportedly inferior recognition memory in this sensory modality. The possibility of this generalization and the equivalence of effects when memory discrimination was being performed in the visual vs. auditory modality were at the focus of this study. First, we demonstrate that visual object discrimination is affected by the context of prior multisensory encounters, replicating and extending previous findings by controlling for the probability of multisensory contexts during initial as well as repeated object presentations. Second, we provide the first evidence that single-trial multisensory memories impact subsequent auditory object discrimination. Auditory object discrimination was enhanced when initial presentations entailed semantically congruent multisensory pairs and was impaired after semantically incongruent multisensory encounters, compared to sounds that had been encountered only in a unisensory manner. Third, the impact of single-trial multisensory memories upon unisensory object discrimination was greater when the task was performed in the auditory vs. visual modality. Fourth, there was no evidence for correlation between effects of past multisensory experiences on visual and auditory processing, suggestive of largely independent object processing mechanisms between modalities. We discuss these findings in terms of the conceptual short term memory (CSTM) model and predictive coding. Our results suggest differential recruitment and modulation of conceptual memory networks according to the sensory task at hand.

The role of auditory cortices in the retrieval of single-trial auditory-visual object memories.

Single-trial encounters with multisensory stimuli affect both memory performance and early-latency brain responses to visual stimuli. Whether and how auditory cortices support memory processes based on single-trial multisensory learning is unknown and may differ qualitatively and quantitatively from comparable processes within visual cortices due to purported differences in memory capacities across the senses. We recorded event-related potentials (ERPs) as healthy adults (n = 18) performed a continuous recognition task in the auditory modality, discriminating initial (new) from repeated (old) sounds of environmental objects. Initial presentations were either unisensory or multisensory; the latter entailed synchronous presentation of a semantically congruent or a meaningless image. Repeated presentations were exclusively auditory, thus differing only according to the context in which the sound was initially encountered. Discrimination abilities (indexed by d') were increased for repeated sounds that were initially encountered with a semantically congruent image versus sounds initially encountered with either a meaningless or no image. Analyses of ERPs within an electrical neuroimaging framework revealed that early stages of auditory processing of repeated sounds were affected by prior single-trial multisensory contexts. These effects followed from significantly reduced activity within a distributed network, including the right superior temporal cortex, suggesting an inverse relationship between brain activity and behavioural outcome on this task. The present findings demonstrate how auditory cortices contribute to long-term effects of multisensory experiences on auditory object discrimination. We propose a new framework for the efficacy of multisensory processes to impact both current multisensory stimulus processing and unisensory discrimination abilities later in time.

The brain uses single-trial multisensory memories to discriminate without awareness.

Multisensory experiences enhance perceptions and facilitate memory retrieval processes, even when only unisensory information is available for accessing such memories. Using fMRI, we identified human brain regions involved in discriminating visual stimuli according to past multisensory vs. unisensory experiences. Subjects performed a completely orthogonal task, discriminating repeated from initial image presentations intermixed within a continuous recognition task. Half of initial presentations were multisensory, and all repetitions were exclusively visual. Despite only single-trial exposures to initial image presentations, accuracy in indicating image repetitions was significantly improved by past auditory-visual multisensory experiences over images only encountered visually. Similarly, regions within the lateral-occipital complex-areas typically associated with visual object recognition processes-were more active to visual stimuli with multisensory than unisensory pasts. Additional differential responses were observed in the anterior cingulate and frontal cortices. Multisensory experiences are registered by the brain even when of no immediate behavioral relevance and can be used to categorize memories. These data reveal the functional efficacy of multisensory processing.

Multisensory Integration and Perceptual Enhancement

We perceive our environment through multiple sensory channels. Nonetheless, research has traditionally focused on the investigation of sensory processing within single modalities. Thus, investigating how our brain integrates multisensory information is of crucial importance for understanding how organisms cope with a constantly changing and dynamic environment. During my thesis I have investigated how multisensory events impact our perception and brain responses, either when auditory-visual stimuli were presented simultaneously or how multisensory events at one point in time impact later unisensory processing. In "Looming signals reveal synergistic principles of multisensory integration" (Cappe, Thelen et al., 2012) we investigated the neuronal substrates involved in motion detection in depth under multisensory vs. unisensory conditions. We have shown that congruent auditory-visual looming (i.e. approaching) signals are preferentially integrated by the brain. Further, we show that early effects under these conditions are relevant for behavior, effectively speeding up responses to these combined stimulus presentations. In "Electrical neuroimaging of memory discrimination based on single-trial multisensory learning" (Thelen et al., 2012), we investigated the behavioral impact of single encounters with meaningless auditory-visual object parings upon subsequent visual object recognition. In addition to showing that these encounters lead to impaired recognition accuracy upon repeated visual presentations, we have shown that the brain discriminates images as soon as ~100ms post-stimulus onset according to the initial encounter context. In "Single-trial multisensory memories affect later visual and auditory object recognition" (Thelen et al., in review) we have addressed whether auditory object recognition is affected by single-trial multisensory memories, and whether recognition accuracy of sounds was similarly affected by the initial encounter context as visual objects. We found that this is in fact the case. We propose that a common underlying brain network is differentially involved during encoding and retrieval of images and sounds based on our behavioral findings. - Nous percevons l'environnement qui nous entoure à l'aide de plusieurs organes sensoriels. Antérieurement, la recherche sur la perception s'est focalisée sur l'étude des systèmes sensoriels indépendamment les uns des autres. Cependant, l'étude des processus cérébraux qui soutiennent l'intégration de l'information multisensorielle est d'une importance cruciale pour comprendre comment notre cerveau travail en réponse à un monde dynamique en perpétuel changement. Pendant ma thèse, j'ai ainsi étudié comment des événements multisensoriels impactent notre perception immédiate et/ou ultérieure et comment ils sont traités par notre cerveau. Dans l'étude " Looming signals reveal synergistic principles of multisensory integration" (Cappe, Thelen et al., 2012), nous nous sommes intéressés aux processus neuronaux impliqués dans la détection de mouvements à l'aide de l'utilisation de stimuli audio-visuels seuls ou combinés. Nos résultats ont montré que notre cerveau intègre de manière préférentielle des stimuli audio-visuels combinés s'approchant de l'observateur. De plus, nous avons montré que des effets précoces, observés au niveau de la réponse cérébrale, influencent notre comportement, en accélérant la détection de ces stimuli. Dans l'étude "Electrical neuroimaging of memory discrimination based on single-trial multisensory learning" (Thelen et al., 2012), nous nous sommes intéressés à l'impact qu'a la présentation d'un stimulus audio-visuel sur l'exactitude de reconnaissance d'une image. Nous avons étudié comment la présentation d'une combinaison audio-visuelle sans signification, impacte, au niveau comportementale et cérébral, sur la reconnaissance ultérieure de l'image. Les résultats ont montré que l'exactitude de la reconnaissance d'images, présentées dans le passé, avec un son sans signification, est inférieure à celle obtenue dans le cas d'images présentées seules. De plus, notre cerveau différencie ces deux types de stimuli très tôt dans le traitement d'images. Dans l'étude "Single-trial multisensory memories affect later visual and auditory object recognition" (Thelen et al., in review), nous nous sommes posés la question si l'exactitude de ia reconnaissance de sons était affectée de manière semblable par la présentation d'événements multisensoriels passés. Ceci a été vérifié par nos résultats. Nous avons proposé que cette similitude puisse être expliquée par le recrutement différentiel d'un réseau neuronal commun.

The role of multisensory memories in unisensory object discrimination.

Past multisensory experiences can influence current unisensory processing and memory performance. Repeated images are better discriminated if initially presented as auditory-visual pairs, rather than only visually. An experience's context thus plays a role in how well repetitions of certain aspects are later recognized. Here, we investigated factors during the initial multisensory experience that are essential for generating improved memory performance. Subjects discriminated repeated versus initial image presentations intermixed within a continuous recognition task. Half of initial presentations were multisensory, and all repetitions were only visual. Experiment 1 examined whether purely episodic multisensory information suffices for enhancing later discrimination performance by pairing visual objects with either tones or vibrations. We could therefore also assess whether effects can be elicited with different sensory pairings. Experiment 2 examined semantic context by manipulating the congruence between auditory and visual object stimuli within blocks of trials. Relative to images only encountered visually, accuracy in discriminating image repetitions was significantly impaired by auditory-visual, yet unaffected by somatosensory-visual multisensory memory traces. By contrast, this accuracy was selectively enhanced for visual stimuli with semantically congruent multisensory pasts and unchanged for those with semantically incongruent multisensory pasts. The collective results reveal opposing effects of purely episodic versus semantic information from auditory-visual multisensory events. Nonetheless, both types of multisensory memory traces are accessible for processing incoming stimuli and indeed result in distinct visual object processing, leading to either impaired or enhanced performance relative to unisensory memory traces. We discuss these results as supporting a model of object-based multisensory interactions.

The role of actions in auditory object discrimination.

Action representations can interact with object recognition processes. For example, so-called mirror neurons respond both when performing an action and when seeing or hearing such actions. Investigations of auditory object processing have largely focused on categorical discrimination, which begins within the initial 100 ms post-stimulus onset and subsequently engages distinct cortical networks. Whether action representations themselves contribute to auditory object recognition and the precise kinds of actions recruiting the auditory-visual mirror neuron system remain poorly understood. We applied electrical neuroimaging analyses to auditory evoked potentials (AEPs) in response to sounds of man-made objects that were further subdivided between sounds conveying a socio-functional context and typically cuing a responsive action by the listener (e.g. a ringing telephone) and those that are not linked to such a context and do not typically elicit responsive actions (e.g. notes on a piano). This distinction was validated psychophysically by a separate cohort of listeners. Beginning approximately 300 ms, responses to such context-related sounds significantly differed from context-free sounds both in the strength and topography of the electric field. This latency is >200 ms subsequent to general categorical discrimination. Additionally, such topographic differences indicate that sounds of different action sub-types engage distinct configurations of intracranial generators. Statistical analysis of source estimations identified differential activity within premotor and inferior (pre)frontal regions (Brodmann's areas (BA) 6, BA8, and BA45/46/47) in response to sounds of actions typically cuing a responsive action. We discuss our results in terms of a spatio-temporal model of auditory object processing and the interplay between semantic and action representations.

Multisensory context portends object memory.

Multisensory processes facilitate perception of currently-presented stimuli and can likewise enhance later object recognition. Memories for objects originally encountered in a multisensory context can be more robust than those for objects encountered in an exclusively visual or auditory context [1], upturning the assumption that memory performance is best when encoding and recognition contexts remain constant [2]. Here, we used event-related potentials (ERPs) to provide the first evidence for direct links between multisensory brain activity at one point in time and subsequent object discrimination abilities. Across two experiments we found that individuals showing a benefit and those impaired during later object discrimination could be predicted by their brain responses to multisensory stimuli upon their initial encounter. These effects were observed despite the multisensory information being meaningless, task-irrelevant, and presented only once. We provide critical insights into the advantages associated with multisensory interactions; they are not limited to the processing of current stimuli, but likewise encompass the ability to determine the benefit of one's memories for object recognition in later, unisensory contexts.

Setting boundaries: brain dynamics of modal and amodal illusory shape completion in humans.

Normal visual perception requires differentiating foreground from background objects. Differences in physical attributes sometimes determine this relationship. Often such differences must instead be inferred, as when two objects or their parts have the same luminance. Modal completion refers to such perceptual "filling-in" of object borders that are accompanied by concurrent brightness enhancement, in turn termed illusory contours (ICs). Amodal completion is filling-in without concurrent brightness enhancement. Presently there are controversies regarding whether both completion processes use a common neural mechanism and whether perceptual filling-in is a bottom-up, feedforward process initiating at the lowest levels of the cortical visual pathway or commences at higher-tier regions. We previously examined modal completion (Murray et al., 2002) and provided evidence that the earliest modal IC sensitivity occurs within higher-tier object recognition areas of the lateral occipital complex (LOC). We further proposed that previous observations of IC sensitivity in lower-tier regions likely reflect feedback modulation from the LOC. The present study tested these proposals, examining the commonality between modal and amodal completion mechanisms with high-density electrical mapping, spatiotemporal topographic analyses, and the local autoregressive average distributed linear inverse source estimation. A common initial mechanism for both types of completion processes (140 msec) that manifested as a modulation in response strength within higher-tier visual areas, including the LOC and parietal structures, is demonstrated, whereas differential mechanisms were evident only at a subsequent time period (240 msec), with amodal completion relying on continued strong responses in these structures.

The efficacy of single-trial multisensory memories.

This review article summarizes evidence that multisensory experiences at one point in time have long-lasting effects on subsequent unisensory visual and auditory object recognition. The efficacy of single-trial exposure to task-irrelevant multisensory events is its ability to modulate memory performance and brain activity to unisensory components of these events presented later in time. Object recognition (either visual or auditory) is enhanced if the initial multisensory experience had been semantically congruent and can be impaired if this multisensory pairing was either semantically incongruent or entailed meaningless information in the task-irrelevant modality, when compared to objects encountered exclusively in a unisensory context. Processes active during encoding cannot straightforwardly explain these effects; performance on all initial presentations was indistinguishable despite leading to opposing effects with stimulus repetitions. Brain responses to unisensory stimulus repetitions differ during early processing stages (-100 ms post-stimulus onset) according to whether or not they had been initially paired in a multisensory context. Plus, the network exhibiting differential responses varies according to whether or not memory performance is enhanced or impaired. The collective findings we review indicate that multisensory associations formed via single-trial learning exert influences on later unisensory processing to promote distinct object representations that manifest as differentiable brain networks whose activity is correlated with memory performance. These influences occur incidentally, despite many intervening stimuli, and are distinguishable from the encoding/learning processes during the formation of the multisensory associations. The consequences of multisensory interactions that persist over time to impact memory retrieval and object discrimination.

Electrical neuroimaging of memory discrimination based on single-trial multisensory learning.

Multisensory experiences influence subsequent memory performance and brain responses. Studies have thus far concentrated on semantically congruent pairings, leaving unresolved the influence of stimulus pairing and memory sub-types. Here, we paired images with unique, meaningless sounds during a continuous recognition task to determine if purely episodic, single-trial multisensory experiences can incidentally impact subsequent visual object discrimination. Psychophysics and electrical neuroimaging analyses of visual evoked potentials (VEPs) compared responses to repeated images either paired or not with a meaningless sound during initial encounters. Recognition accuracy was significantly impaired for images initially presented as multisensory pairs and could not be explained in terms of differential attention or transfer of effects from encoding to retrieval. VEP modulations occurred at 100-130ms and 270-310ms and stemmed from topographic differences indicative of network configuration changes within the brain. Distributed source estimations localized the earlier effect to regions of the right posterior temporal gyrus (STG) and the later effect to regions of the middle temporal gyrus (MTG). Responses in these regions were stronger for images previously encountered as multisensory pairs. Only the later effect correlated with performance such that greater MTG activity in response to repeated visual stimuli was linked with greater performance decrements. The present findings suggest that brain networks involved in this discrimination may critically depend on whether multisensory events facilitate or impair later visual memory performance. More generally, the data support models whereby effects of multisensory interactions persist to incidentally affect subsequent behavior as well as visual processing during its initial stages.

Developmental critical period in gentic redox dysregulation: animal and human studies in schizophrenia

Converging evidence favors an abnormal susceptibility to oxidative stress in schizophrenia. Decreased levels of glutathione (GSH), the major cellular antioxidant and redox regulator, was observed in cerebrospinal-fluid and prefrontal cortex of patients. Importantly, abnormal GSH synthesis of genetic origin was observed: Two case-control studies showed an association with a GAG trinucleotide repeat (TNR) polymorphism in the GSH key synthesizing enzyme glutamate-cysteine-ligase (GCL) catalytic subunit (GCLC) gene. The most common TNR genotype 7/7 was more frequent in controls, whereas the rarest TNR genotype 8/8 was three times more frequent in patients. The disease associated genotypes (35% of patients) correlated with decreased GCLC protein, GCL activity and GSH content. Similar GSH system anomalies were observed in early psychosis patients. Such redox dysregulation combined with environmental stressors at specific developmental stages could underlie structural and functional connectivity anomalies. In pharmacological and knock-out (KO) models, GSH deficit induces anomalies analogous to those reported in patients. (a) morphology: spine density and GABA-parvalbumine immunoreactivity (PV-I) were decreased in anterior cingulate cortex. KO mice showed delayed cortical PV-I at PD10. This effect is exacerbated in mice with increased DA from PD5-10. KO mice exhibit cortical impairment in myelin and perineuronal net known to modulate PV connectivity. (b) physiology: In cultured neurons, NMDA response are depressed by D2 activation. In hippocampus, NMDA-dependent synaptic plasticity is impaired and kainate induced g-oscillations are reduced in parallel to PV-I. (c) cognition: low GSH models show increased sensitivity to stress, hyperactivity, abnormal object recognition, olfactory integration and social behavior. In a clinical study, GSH precursor N-acetyl cysteine (NAC) as add on therapy, improves the negative symptoms and decreases the side effects of antipsychotics. In an auditory oddball paradigm, NAC improves the mismatched negativity, an evoked potential related to pre-attention and to NMDA receptors function. In summary, clinical and experimental evidence converge to demonstrate that a genetically induced dysregulation of GSH synthesis combined with environmental insults in early development represent a major risk factor contributing to the development of schizophrenia Conclusion Based on these data, we proposed a model for PSIP1 promoter activity involving a complex interplay between yet undefined regulatory elements to modulate gene expression.

D-JNKi, a peptide inhibitor of c-Jun N-terminal kinase, promotes functional recovery after transient focal cerebral ischemia in rats

The c-Jun-N-terminal kinase (JNK) pathway has been shown to play an important role in excitotoxic neuronal death and several studies have demonstrated a neuroprotective effect of D-JNKi, a peptide inhibitor of JNK, in various models of cerebral ischemia. We have now investigated the effect of D-JNKi in a model of transient focal cerebral ischemia (90 min) induced by middle cerebral artery occlusion (MCAo) in adult male rats. D-JNKi (0.1 mg/kg), significantly decreased the volume of infarct, 3 days after cerebral ischemia. Sensorimotor and cognitive deficits were then evaluated over a period of 6 or 10 days after ischemia and infarct volumes were measured after behavioral testing. In behavioral studies, D-JNKi improved the general state of the animals as demonstrated by the attenuation of body weight loss and improvement in neurological score, as compared with animals receiving the vehicle. Moreover, D-JNKi decreased sensorimotor deficits in the adhesive removal test and improved cognitive function in the object recognition test. In contrast, D-JNKi did not significantly affect the infarct volume at day 6 and at day 10. This study shows that D-JNKi can improve functional recovery after transient focal cerebral ischemia in the rat and therefore supports the use of this molecule as a potential therapy for stroke.

Musical training intensity yields opposite effects on grey matter density in cognitive versus sensorimotor networks.

Using optimized voxel-based morphometry, we performed grey matter density analyses on 59 age-, sex- and intelligence-matched young adults with three distinct, progressive levels of musical training intensity or expertise. Structural brain adaptations in musicians have been repeatedly demonstrated in areas involved in auditory perception and motor skills. However, musical activities are not confined to auditory perception and motor performance, but are entangled with higher-order cognitive processes. In consequence, neuronal systems involved in such higher-order processing may also be shaped by experience-driven plasticity. We modelled expertise as a three-level regressor to study possible linear relationships of expertise with grey matter density. The key finding of this study resides in a functional dissimilarity between areas exhibiting increase versus decrease of grey matter as a function of musical expertise. Grey matter density increased with expertise in areas known for their involvement in higher-order cognitive processing: right fusiform gyrus (visual pattern recognition), right mid orbital gyrus (tonal sensitivity), left inferior frontal gyrus (syntactic processing, executive function, working memory), left intraparietal sulcus (visuo-motor coordination) and bilateral posterior cerebellar Crus II (executive function, working memory) and in auditory processing: left Heschl's gyrus. Conversely, grey matter density decreased with expertise in bilateral perirolandic and striatal areas that are related to sensorimotor function, possibly reflecting high automation of motor skills. Moreover, a multiple regression analysis evidenced that grey matter density in the right mid orbital area and the inferior frontal gyrus predicted accuracy in detecting fine-grained incongruities in tonal music.

Tau hyperphosphorylation and increased BACE1 and RAGE levels in the cortex of PPARβ/δ-null mice.

The role of peroxisome proliferator activator receptor (PPAR)β/δ in the pathogenesis of Alzheimer's disease has only recently been explored through the use of PPARβ/δ agonists. Here we evaluated the effects of PPARβ/δ deficiency on the amyloidogenic pathway and tau hyperphosphorylation. PPARβ/δ-null mice showed cognitive impairment in the object recognition task, accompanied by enhanced DNA-binding activity of NF-κB in the cortex and increased expression of IL-6. In addition, two NF-κB-target genes involved in β-amyloid (Aβ) synthesis and deposition, the β site APP cleaving enzyme 1 (Bace1) and the receptor for advanced glycation endproducts (Rage), respectively, increased in PPARβ/δ-null mice compared to wild type animals. The protein levels of glial fibrillary acidic protein (GFAP) increased in the cortex of PPARβ/δ-null mice, which would suggest the presence of astrogliosis. Finally, tau hyperphosphorylation at Ser199 and enhanced levels of PHF-tau were associated with increased levels of the tau kinases CDK5 and phospho-ERK1/2 in the cortex of PPARβ/δ(-/-) mice. Collectively, our findings indicate that PPARβ/δ deficiency results in cognitive impairment associated with enhanced inflammation, astrogliosis and tau hyperphosphorylation in the cortex.

Genetic redox dysregulation induces behavioural deficits in the adult mouse: an animal model for schizophrenia

Background: Glutathione (GSH), a major cellular redox regulator and antioxidant, is decreased in cerebrospinal fluid and prefrontal cortex of schizophrenia patients. The gene of the key GSH-synthesizing enzyme, glutamate-cysteine ligase, modifier (GCLM) subunit, is associated with schizophrenia, suggesting that the deficit in the GSH system is of genetic origin. Using the GCLM knock-out (KO) mouse as model system with 60% decreased brain GSH levels and, thus, strong vulnerability to oxidative stress, we have shown that GSH dysregulation results in abnormal mouse brain morphology (e.g., reduced parvalbumin, PV, immuno-reactivity in frontal areas) and function. Additional oxidative stress, induced by GBR12909 (a dopamine re-uptake inhibitor), enhances morphological changes even further. Aim: In the present study we use the GCLM KO mouse model system, asking now, whether GSH dysregulation also compromises mouse behaviour and cognition. Methods: Male and female wildtype (WT) and GCLM-KO mice are treated with GBR12909 or phosphate buffered saline (PBS) from postnatal day (P) 5 to 10, and are behaviourally tested at P 60 and older. Results: In comparison to WT, KO animals of both sexes are hyperactive in the open field, display more frequent open arm entries on the elevated plus maze, longer float latencies in the Porsolt swim test, and more frequent contacts of novel and familiar objects. Contrary to other reports of animal models with reduced PV immuno-reactivity, GCLM-KO mice display normal rule learning capacity and perform normally on a spatial recognition task. GCLM-KO mice do, however, show a strong deficit in object-recognition after a 15 minutes retention delay. GBR12909 treatment exerts no additional effect. Conclusions: The results suggest that animals with impaired regulation of brain oxidative stress are impulsive and have reduced behavioural control in novel, unpredictable contexts. Moreover, GSH dysregulation seems to induce a selective attentional or stimulus-encoding deficit: despite intensive object exploration, GCLM-KO mice cannot discriminate between novel and familiar objects. In conclusion, the present data indicate that GSH dysregulation may contribute to the manifestation of behavioural and cognitive anomalies that are associated with schizophrenia.