The role of attention in multisensory integration processes

Report for the scientific sojourn at the University of Queensland, Australia, between June to October 2007. The integration of sensory information coming from different modalities is a subjectively effortless and automatic process. However, recent findings converge on the idea that there might be some limitations on how much multisensory information can our brains integrate at a given moment. The overall objective of this research visit was to expand and reinforce our current line of research addressing the role that attention plays on the mechanisms responsible for multisensory integration. There is also a critical review of the literature as well as a list of research questions of potential interest in the area of the neuropsychology of multisensory attention.

Multisensory stimulation can induce an illusion of larger belly size in immersive virtual reality

Body change illusions have been of great interest in recent years for the understanding of how the brain represents the body. Appropriate multisensory stimulation can induce an illusion of ownership over a rubber or virtual arm, simple types of out-of-the-body experiences, and even ownership with respect to an alternate whole body. Here we use immersive virtual reality to investigate whether the illusion of a dramatic increase in belly size can be induced in males through (a) first person perspective position (b) synchronous visual-motor correlation between real and virtual arm movements, and (c) self-induced synchronous visual-tactile stimulation in the stomach area.

The building blocks of the full body ownership illusion

Previous work has reported that it is not difficult to give people the illusion of ownership over an artificial body, providing a powerful tool for the investigation of the neural and cognitive mechanisms underlying body perception and self consciousness. We present an experimental study that uses immersive virtual reality (IVR) focused on identifying the perceptual building blocks of this illusion. We systematically manipulated visuotactile and visual sensorimotor contingencies, visual perspective, and the appearance of the virtual body in order to assess their relative role and mutual interaction. Consistent results from subjective reports and physiological measures showed that a first person perspective over a fake humanoid body is essential for eliciting a body ownership illusion. We found that the illusion of ownership can be generated when the virtual body has a realistic skin tone and spatially substitutes the real body seen from a first person perspective. In this case there is no need for an additional contribution of congruent visuotactile or sensorimotor cues. Additionally, we found that the processing of incongruent perceptual cues can be modulated by the level of the illusion: when the illusion is strong, incongruent cues are not experienced as incorrect. Participants exposed to asynchronous visuotactile stimulation can experience the ownership illusion and perceive touch as originating from an object seen to contact the virtual body. Analogously, when the level of realism of the virtual body is not high enough and/or when there is no spatial overlap between the two bodies, then the contribution of congruent multisensory and/or sensorimotor cues is required for evoking the illusion. On the basis of these results and inspired by findings from neurophysiological recordings in the monkey, we propose a model that accounts for many of the results reported in the literature.

Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion

Previous studies have examined the experience of owning a virtual surrogate body or body part through specific combinations of cross-modal multisensory stimulation. Both visuomotor (VM) and visuotactile (VT) synchronous stimulation have been shown to be important for inducing a body ownership illusion, each tested separately or both in combination. In this study we compared the relative importance of these two cross-modal correlations, when both are provided in the same immersive virtual reality setup and the same experiment. We systematically manipulated VT and VM contingencies in order to assess their relative role and mutual interaction. Moreover, we present a new method for measuring the induced body ownership illusion through time, by recording reports of breaks in the illusion of ownership ("breaks") throughout the experimental phase. The balance of the evidence, from both questionnaires and analysis of the breaks, suggests that while VM synchronous stimulation contributes the greatest to the attainment of the illusion, a disruption of either (through asynchronous stimulation) contributes equally to the probability of a break in the illusion.

Extending body space in immersive virtual reality: a very long arm illusion

Recent studies have shown that a fake body part can be incorporated into human body representation through synchronous multisensory stimulation on the fake and corresponding real body part- the most famous example being the Rubber Hand Illusion. However, the extent to which gross asymmetries in the fake body can be assimilated remains unknown. Participants experienced, through a head-tracked stereo head-mounted display a virtual body coincident with their real body. There were 5 conditions in a between-groups experiment, with 10 participants per condition. In all conditions there was visuo-motor congruence between the real and virtual dominant arm. In an Incongruent condition (I), where the virtual arm length was equal to the real length, there was visuo-tactile incongruence. In four Congruent conditions there was visuo-tactile congruence, but the virtual arm lengths were either equal to (C1), double (C2), triple (C3) or quadruple (C4) the real ones. Questionnaire scores and defensive withdrawal movements in response to a threat showed that the overall level of ownership was high in both C1 and I, and there was no significant difference between these conditions. Additionally, participants experienced ownership over the virtual arm up to three times the length of the real one, and less strongly at four times the length. The illusion did decline, however, with the length of the virtual arm. In the C2-C4 conditions although a measure of proprioceptive drift positively correlated with virtual arm length, there was no correlation between the drift and ownership of the virtual arm, suggesting different underlying mechanisms between ownership and drift. Overall, these findings extend and enrich previous results that multisensory and sensorimotor information can reconstruct our perception of the body shape, size and symmetry even when this is not consistent with normal body proportions.

The contribution of real-time mirror reflections of motor actions on virtual body ownership in an immersive virtual environment

This paper reports an experiment that investigated people"s body ownership of an avatar that was observed in a virtual mirror. Twenty subjects were recruited in a within-groups study where 10 first experienced a virtual character that synchronously reflected their upper-body movements as seen in a virtual mirror, and then an asynchronous condition where the mirror avatar displayed prerecorded actions, unrelated to those of the participant. The other 10 subjects experienced the conditions in the opposite order. In both conditions the participant could carry out actions that led to elevation above ground level, as seen from their first person perspective and correspondingly in the mirror. A rotating virtual fan eventually descended to 2m above the ground. The hypothesis was that synchronous mirror reflection would result in higher subjective sense of ownership. A questionnaire analysis showed that the body ownership illusion was significantly greater for thesynchronous than asynchronous condition. Additionally participants in the synchronous condition avoided collision with the descending fan significantly more often than those in the asynchronous condition. The results of this experiment are put into context within similar experiments on multisensory correlation and body ownership within cognitive neuroscience.

Towards a digital body: the virtual arm illusion

The integration of the human brain with computers is an interesting new area of applied neuroscience, where one application is replacement of a person"s real body by a virtual representation. Here we demonstrate that a virtual limb can be made to feel part of your body if appropriate multisensory correlations are provided. We report an illusion that is invoked through tactile stimulation on a person"s hidden real right hand with synchronous virtual visual stimulation on an aligned 3D stereo virtual arm projecting horizontally out of their shoulder. An experiment with 21 male participants showed displacement of ownership towards the virtual hand, as illustrated by questionnaire responses and proprioceptive drift. A control experiment with asynchronous tapping was carried out with a different set of 20 male participants who did not experience the illusion. After 5 min of stimulation the virtual arm rotated. Evidence suggests that the extent of the illusion was also correlated with the degree of muscle activity onset in the right arm as measured by EMG during this period that the arm was rotating, for the synchronous but not the asynchronous condition. A completely virtual object can therefore be experienced as part of one"s self, which opens up the possibility that an entire virtual body could be felt as one"s own in future virtual reality applications or online games, and be an invaluable tool for the understanding of the brain mechanisms underlying body ownership.

Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion

Previous studies have examined the experience of owning a virtual surrogate body or body part through specific combinations of cross-modal multisensory stimulation. Both visuomotor (VM) and visuotactile (VT) synchronous stimulation have been shown to be important for inducing a body ownership illusion, each tested separately or both in combination. In this study we compared the relative importance of these two cross-modal correlations, when both are provided in the same immersive virtual reality setup and the same experiment. We systematically manipulated VT and VM contingencies in order to assess their relative role and mutual interaction. Moreover, we present a new method for measuring the induced body ownership illusion through time, by recording reports of breaks in the illusion of ownership ("breaks") throughout the experimental phase. The balance of the evidence, from both questionnaires and analysis of the breaks, suggests that while VM synchronous stimulation contributes the greatest to the attainment of the illusion, a disruption of either (through asynchronous stimulation) contributes equally to the probability of a break in the illusion.

Transcending the self in immersive virtual reality

Cognitive neuroscientists have discovered various experimental setups that suggest that our body representation is surprisingly flexible, where the brain can easily be tricked into the illusion that a rubber hand is your hand or that a manikin body is your body. These multisensory illusions work well in immersive virtual reality (IVR). What is even more surprising is that such embodiment induces perceptual, attitudinal and behavioural changes that are concomitant with the displayed body type. Here we outline some recent findings in this field, and suggest that this offers a powerful tool for neuroscience, psychology and a new path for IVR.