The role of multisensory memories in unisensory object discrimination.

Past multisensory experiences can influence current unisensory processing and memory performance. Repeated images are better discriminated if initially presented as auditory-visual pairs, rather than only visually. An experience's context thus plays a role in how well repetitions of certain aspects are later recognized. Here, we investigated factors during the initial multisensory experience that are essential for generating improved memory performance. Subjects discriminated repeated versus initial image presentations intermixed within a continuous recognition task. Half of initial presentations were multisensory, and all repetitions were only visual. Experiment 1 examined whether purely episodic multisensory information suffices for enhancing later discrimination performance by pairing visual objects with either tones or vibrations. We could therefore also assess whether effects can be elicited with different sensory pairings. Experiment 2 examined semantic context by manipulating the congruence between auditory and visual object stimuli within blocks of trials. Relative to images only encountered visually, accuracy in discriminating image repetitions was significantly impaired by auditory-visual, yet unaffected by somatosensory-visual multisensory memory traces. By contrast, this accuracy was selectively enhanced for visual stimuli with semantically congruent multisensory pasts and unchanged for those with semantically incongruent multisensory pasts. The collective results reveal opposing effects of purely episodic versus semantic information from auditory-visual multisensory events. Nonetheless, both types of multisensory memory traces are accessible for processing incoming stimuli and indeed result in distinct visual object processing, leading to either impaired or enhanced performance relative to unisensory memory traces. We discuss these results as supporting a model of object-based multisensory interactions.

La reconnaissance des objets sonores chez l'humain

Abstract : This thesis investigated the spatio-temporal brain mechanisms of three processes involved in recognizing environmental sounds produced by living (animal vocalisations) and man-made (manufactured) objects: their discrimination, their plasticity, and the involvement of action representations. Results showed rapid brain discrimination between these categories beginning at ~70ms. Then, beginning at ~150ms, effects of plasticity are observed, without any influence of the categories of sounds. Both of these processes of discrimination and repetition priming involved brain structures located in temporal and frontal lobes. Activation of brain areas BA21 and BA22 suggest an access to semantic representations and/or linked to object manipulation. To investigate the involvement of action representations in sound recognition, analyses were restricted to sounds produced by man-made objects. Results suggest an access to representations linked to action functionally related to sound rather than to representations linked to action that produced sound. These effects occurred at ~300ms post-stimulus onset and involved differential activity brain regions attributed to the mirror neuron system. These data are discussed in regard to motor preparation of actions functionally linked to sounds. Collectively these data showed a sequential progression of cerebral activity underlying the recognizing of environmental sounds. The processes occurred firstly in a shared network of brain areas before propagating elsewhere and/or leading to differential activity in these structures. Cerebral responses observed in this work allowed establishing a dynamic model of discrimination of sounds produced by living and man-made objects.