Visuotactile learning and body representation: an ERP study with rubber hands and rubber objects

We studied how the integration of seen and felt tactile stimulation modulates somatosensory processing, and investigated whether visuotactile integration depends on temporal contiguity of stimulation, and its coherence with a pre-existing body representation. During training, participants viewed a rubber hand or a rubber object that was tapped either synchronously with stimulation of their own hand, or in an uncorrelated fashion. In a subsequent test phase, somatosensory event-related potentials (ERPs) were recorded to tactile stimulation of the left or right hand, to assess how tactile processing was affected by previous visuotactile experience during training. An enhanced somatosensory N140 component was elicited after synchronous, compared with uncorrelated, visuotactile training, irrespective of whether participants viewed a rubber hand or rubber object. This early effect of visuotactile integration on somatosensory processing is interpreted as a candidate electrophysiological correlate of the rubber hand illusion that is determined by temporal contiguity, but not by pre-existing body representations. ERPmodulations were observed beyond 200msec post-stimulus, suggesting an attentional bias induced by visuotactile training. These late modulations were absent when the stimulation of a rubber hand and the participant’s own hand was uncorrelated during training, suggesting that pre-existing body representations may affect later stages of tactile processing.

Manual response preparation and saccade programming are linked to attention shifts: ERP evidence for covert attentional orienting and spatially specific modulations of visual processing

The premotor theory of attention claims that attentional shifts are triggered during response programming, regardless of which response modality is involved. To investigate this claim, event-related brain potentials (ERPs) were recorded while participants covertly prepared a left or right response, as indicated by a precue presented at the beginning of each trial. Cues signalled a left or right eye movement in the saccade task, and a left or right manual response in the manual task. The cued response had to be executed or withheld following the presentation of a Go/Nogo stimulus. Although there were systematic differences between ERPs triggered during covert manual and saccade preparation, lateralised ERP components sensitive to the direction of a cued response were very similar for both tasks, and also similar to the components previously found during cued shifts of endogenous spatial attention. This is consistent with the claim that the control of attention and of covert response preparation are closely linked. N1 components triggered by task-irrelevant visual probes presented during the covert response preparation interval were enhanced when these probes were presented close to cued response hand in the manual task, and at the saccade target location in the saccade task. This demonstrates that both manual and saccade preparation result in spatially specific modulations of visual processing, in line with the predictions of the premotor theory.

EEG activation differences in the pre-motor cortex and supplementary motor area between normal individuals with high and low traits of autism

The human mirror neuron system (hMNS) is believed to provide a basic mechanism for social cognition. Event-related desynchronization (ERD) in alpha (8–12 Hz) and low beta band (12–20 Hz) over sensori-motor cortex has been suggested to index mirror neurons' activity. We tested whether autistic traits revealed by high and low scores on the Autistic Quotient (AQ) in the normal population are linked to variations in the electroencephalogram (EEG) over motor, pre-motor cortex and supplementary motor area (SMA) during action observation. Results revealed that in the low AQ group, the pre-motor cortex and SMA were more active during hand action than static hand observation whereas in the high AQ group the same areas were active both during static and hand action observation. In fact participants with high traits of autism showed greater low beta ERD while observing the static hand than those with low traits and this low beta ERD was not significantly different when they watched hand actions. Over primary motor cortex, the classical alpha and low beta ERD during hand actions relative to static hand observation was found across all participants. These findings suggest that the observation–execution matching system works differently according to the degree of autism traits in the normal population and that this is differentiated in terms of the EEG according to scalp site and bandwidth.

Neural correlates of true and false memory in mild cognitive impairment

The goal of this research was to investigate the changes in neural processing in mild cognitive impairment. We measured phase synchrony, amplitudes, and event-related potentials in veridical and false memory to determine whether these differed in participants with mild cognitive impairment compared with typical, age-matched controls. Empirical mode decomposition phase locking analysis was used to assess synchrony, which is the first time this analysis technique has been applied in a complex cognitive task such as memory processing. The technique allowed assessment of changes in frontal and parietal cortex connectivity over time during a memory task, without a priori selection of frequency ranges, which has been shown previously to influence synchrony detection. Phase synchrony differed significantly in its timing and degree between participant groups in the theta and alpha frequency ranges. Timing differences suggested greater dependence on gist memory in the presence of mild cognitive impairment. The group with mild cognitive impairment had significantly more frontal theta phase locking than the controls in the absence of a significant behavioural difference in the task, providing new evidence for compensatory processing in the former group. Both groups showed greater frontal phase locking during false than true memory, suggesting increased searching when no actual memory trace was found. Significant inter-group differences in frontal alpha phase locking provided support for a role for lower and upper alpha oscillations in memory processing. Finally, fronto-parietal interaction was significantly reduced in the group with mild cognitive impairment, supporting the notion that mild cognitive impairment could represent an early stage in Alzheimer’s disease, which has been described as a ‘disconnection syndrome’.

Epoch-specific functional networks involved in working memory

Working memory (WM) is not a unitary construct. There are distinct processes involved in encoding information, maintaining it on-line, and using it to guide responses. The anatomical configurations of these processes are more accurately analyzed as functionally connected networks than collections of individual regions. In the current study we analyzed event-related functional magnetic resonance imaging (fMRI) data from a Sternberg Item Recognition Paradigm WM task using a multivariate analysis method that allowed the linking of functional networks to temporally-separated WM epochs. The length of the delay epochs was varied to optimize isolation of the hemodynamic response (HDR) for each task epoch. All extracted functional networks displayed statistically significant sensitivity to delay length. Novel information extracted from these networks that was not apparent in the univariate analysis of these data included involvement of the hippocampus in encoding/probe, and decreases in BOLD signal in the superior temporal gyrus (STG), along with default-mode regions, during encoding/delay. The bilateral hippocampal activity during encoding/delay fits with theoretical models of WM in which memoranda held across the short term are activated long-term memory representations. The BOLD signal decreases in the STG were unexpected, and may reflect repetition suppression effects invoked by internal repetition of letter stimuli. Thus, analysis methods focusing on how network dynamics relate to experimental conditions allowed extraction of novel information not apparent in univariate analyses, and are particularly recommended for WM experiments for which task epochs cannot be randomized.

Unconscious effects of grammatical gender during object categorisation

Does language modulate perception and categorisation of everyday objects? Here, we approach this question from the perspective of grammatical gender in bilinguals. We tested Spanish–English bilinguals and control native speakers of English in a semantic categorisation task on triplets of pictures in an all-in-English context while measuring event-related brain potentials (ERPs). Participants were asked to press a button when the third picture of a triplet belonged to the same semantic category as the first two, and another button when it belonged to a different category. Unbeknownst to them, in half of the trials, the gender of the third picture name in Spanish had the same gender as that of the first two, and the opposite gender in the other half. We found no priming in behavioural results of either semantic relatedness or gender consistency. In contrast, ERPs revealed not only the expected semantic priming effect in both groups, but also a negative modulation by gender inconsistency in Spanish–English bilinguals, exclusively. These results provide evidence for spontaneous and unconscious access to grammatical gender in participants functioning in a context requiring no access to such information, thereby providing support for linguistic relativity effects in the grammatical domain.

Individual differences and the role of the L1 in L2 processing: an ERP investigation

This study used ERP (event-related potentials) to examine both the role of the L1 and the role of individual differences in the processing of agreement violations. Theories of L2 acquisition differ with regard to whether or not native-like acquisition of L2 features is possible (Schwartz and Sprouse, 1994, 1996; Tsimpli and Mastropavlou, 2007), and the results of previous ERP studies are inconsistent when it comes to whether or not native-like processing is observed in response to L2 agreement violations (e.g., Sabourin, 2003; Tokowicz and MacWhinney, 2005). Furthermore, studies of learners in early stages of L2 acquisition have found variability in the emergence of native-like responses (e.g., McLaughlin et al., 2010; Tanner et al., 2009), but sources of variability have not been investigated. The current study examines responses to gender and number agreement violations in English-speaking learners of Spanish (n=24). Stimuli targeted agreement in three conditions: subject-verb agreement (el barco flota/*flotan), which is similar in Spanish and English; number agreement on adjectival predicates (la isla rocosa/*rocosas), a context in which agreement is not instantiated in English; and gender agreement on adjectival predicates (la isla rocosa/*rocoso), which is unique to Spanish. Grammaticality judgments and ERP responses were also tested for correlations with aptitude scores on the Modern Languages Aptitude Test (MLAT; Carroll and Sapon, 1959) and the Raven Advanced Progressive Matrices (Raven, 1965). Results are in line with theories that claim native-like processing is acquirable, since learners demonstrated similar ERP responses to a control group of native Spanish-speakers (n=8) with regard to all three agreement types. Additionally, the MLAT (but not the Raven) was significantly correlated with sensitivity to number violations, both in terms of grammaticality judgments and ERP amplitudes, indicating a role for verbal but not nonverbal aptitude in L2 processing.

N1 enhancement in synesthesia during visual and audio-visual perception in semantic cross-modal conflict situations: an ERP study

Synesthesia entails a special kind of sensory perception, where stimulation in one sensory modality leads to an internally generated perceptual experience of another, not stimulated sensory modality. This phenomenon can be viewed as an abnormal multisensory integration process as here the synesthetic percept is aberrantly fused with the stimulated modality. Indeed, recent synesthesia research has focused on multimodal processing even outside of the specific synesthesia-inducing context and has revealed changed multimodal integration, thus suggesting perceptual alterations at a global level. Here, we focused on audio-visual processing in synesthesia using a semantic classification task in combination with visually or auditory-visually presented animated and in animated objects in an audio-visual congruent and incongruent manner. Fourteen subjects with auditory-visual and/or grapheme-color synesthesia and 14 control subjects participated in the experiment. During presentation of the stimuli, event-related potentials were recorded from 32 electrodes. The analysis of reaction times and error rates revealed no group differences with best performance for audio-visually congruent stimulation indicating the well-known multimodal facilitation effect. We found enhanced amplitude of the N1 component over occipital electrode sites for synesthetes compared to controls. The differences occurred irrespective of the experimental condition and therefore suggest a global influence on early sensory processing in synesthetes.

The organization of emotionally autobiographical memories: How emotional knowledge for personal events is represented in autobiographical knowledge base?

The present study aimed to investigate the organization of autobiographical memory and to reveal how emotional knowledge for personal events is represented in autobiographical knowledge base. For these purposes, the event-cueing technique was employed (Brown & Schopflocher, 1998). Forty-six participants were provided eight retrieval cues and asked to generate a personal event related to each of them (i.e., cueing events). Following this, they responded to each cueing event by retrieving two personal episodes (i.e., cued events). The results indicated that cued events shared the life themes with cueing events, suggesting the thematic organization of autobiographical memory. We also found that the life themes of each personal episode determined types of emotional states with which they were associated. The implications for the affect and memory literature and the emotion regulation literature were discussed.

Effect of instructive visual stimuli on neurofeedback training for motor imagery-based braincomputer interface

Event-related desynchronization (ERD) of the electroencephalogram (EEG) from the motor cortex is associated with execution, observation, and mental imagery of motor tasks. Generation of ERD by motor imagery (MI) has been widely used for brain-computer interfaces (BCIs) linked to neuroprosthetics and other motor assistance devices. Control of MI-based BCIs can be acquired by neurofeedback training to reliably induce MI-associated ERD. To develop more effective training conditions, we investigated the effect of static and dynamic visual representations of target movements (a picture of forearms or a video clip of hand grasping movements) during the BCI training. After 4 consecutive training days, the group that performed MI while viewing the video showed significant improvement in generating MI-associated ERD compared with the group that viewed the static image. This result suggests that passively observing the target movement during MI would improve the associated mental imagery and enhance MI-based BCIs skills.

The plasticity of the mirror system: how reward learning modulates cortical motor simulation of others

Cortical motor simulation supports the understanding of others' actions and intentions. This mechanism is thought to rely on the mirror neuron system (MNS), a brain network that is active both during action execution and observation. Indirect evidence suggests that alpha/beta suppression, an electroencephalographic (EEG) index of MNS activity, is modulated by reward. In this study we aimed to test the plasticity of the MNS by directly investigating the link between alpha/beta suppression and reward. 40 individuals from a general population sample took part in an evaluative conditioning experiment, where different neutral faces were associated with high or low reward values. In the test phase, EEG was recorded while participants viewed videoclips of happy expressions made by the conditioned faces. Alpha/beta suppression (identified using event-related desynchronisation of specific independent components) in response to rewarding faces was found to be greater than for non-rewarding faces. This result provides a mechanistic insight into the plasticity of the MNS and, more generally, into the role of reward in modulating physiological responses linked to empathy.

Exploration of the neural correlates of cerebral palsy for sensorimotor BCI control

Cerebral palsy (CP) includes a broad range of disorders, which can result in impairment of posture and movement control. Brain-computer interfaces (BCIs) have been proposed as assistive devices for individuals with CP. Better understanding of the neural processing underlying motor control in affected individuals could lead to more targeted BCI rehabilitation and treatment options. We have explored well-known neural correlates of movement, including event-related desynchronization (ERD), phase synchrony, and a recently-introduced measure of phase dynamics, in participants with CP and healthy control participants. Although present, significantly less ERD and phase locking were found in the group with CP. Additionally, inter-group differences in phase dynamics were also significant. Taken together these findings suggest that users with CP exhibit lower levels of motor cortex activation during motor imagery, as reflected in lower levels of ongoing mu suppression and less functional connectivity. These differences indicate that development of BCIs for individuals with CP may pose additional challenges beyond those faced in providing BCIs to healthy individuals.

Automated identification of neural correlates of continuous variables

Background: The electroencephalogram (EEG) may be described by a large number of different feature types and automated feature selection methods are needed in order to reliably identify features which correlate with continuous independent variables. New method: A method is presented for the automated identification of features that differentiate two or more groups inneurologicaldatasets basedupona spectraldecompositionofthe feature set. Furthermore, the method is able to identify features that relate to continuous independent variables. Results: The proposed method is first evaluated on synthetic EEG datasets and observed to reliably identify the correct features. The method is then applied to EEG recorded during a music listening task and is observed to automatically identify neural correlates of music tempo changes similar to neural correlates identified in a previous study. Finally,the method is applied to identify neural correlates of music-induced affective states. The identified neural correlates reside primarily over the frontal cortex and are consistent with widely reported neural correlates of emotions. Comparison with existing methods: The proposed method is compared to the state-of-the-art methods of canonical correlation analysis and common spatial patterns, in order to identify features differentiating synthetic event-related potentials of different amplitudes and is observed to exhibit greater performance as the number of unique groups in the dataset increases. Conclusions: The proposed method is able to identify neural correlates of continuous variables in EEG datasets and is shown to outperform canonical correlation analysis and common spatial patterns.

An optimized ERP brain–computer interface based on facial expression changes

Objective. Interferences from spatially adjacent non-target stimuli are known to evoke event-related potentials (ERPs) during non-target flashes and, therefore, lead to false positives. This phenomenon was commonly seen in visual attention-based brain–computer interfaces (BCIs) using conspicuous stimuli and is known to adversely affect the performance of BCI systems. Although users try to focus on the target stimulus, they cannot help but be affected by conspicuous changes of the stimuli (such as flashes or presenting images) which were adjacent to the target stimulus. Furthermore, subjects have reported that conspicuous stimuli made them tired and annoyed. In view of this, the aim of this study was to reduce adjacent interference, annoyance and fatigue using a new stimulus presentation pattern based upon facial expression changes. Our goal was not to design a new pattern which could evoke larger ERPs than the face pattern, but to design a new pattern which could reduce adjacent interference, annoyance and fatigue, and evoke ERPs as good as those observed during the face pattern. Approach. Positive facial expressions could be changed to negative facial expressions by minor changes to the original facial image. Although the changes are minor, the contrast is big enough to evoke strong ERPs. In this paper, a facial expression change pattern between positive and negative facial expressions was used to attempt to minimize interference effects. This was compared against two different conditions, a shuffled pattern containing the same shapes and colours as the facial expression change pattern, but without the semantic content associated with a change in expression, and a face versus no face pattern. Comparisons were made in terms of classification accuracy and information transfer rate as well as user supplied subjective measures. Main results. The results showed that interferences from adjacent stimuli, annoyance and the fatigue experienced by the subjects could be reduced significantly (p < 0.05) by using the facial expression change patterns in comparison with the face pattern. The offline results show that the classification accuracy of the facial expression change pattern was significantly better than that of the shuffled pattern (p < 0.05) and the face pattern (p < 0.05). Significance. The facial expression change pattern presented in this paper reduced interference from adjacent stimuli and decreased the fatigue and annoyance experienced by BCI users significantly (p < 0.05) compared to the face pattern.

An optimized ERP Brain-computer interface based on facial expression changes

OBJECTIVE: Interferences from spatially adjacent non-target stimuli are known to evoke event-related potentials (ERPs) during non-target flashes and, therefore, lead to false positives. This phenomenon was commonly seen in visual attention-based brain-computer interfaces (BCIs) using conspicuous stimuli and is known to adversely affect the performance of BCI systems. Although users try to focus on the target stimulus, they cannot help but be affected by conspicuous changes of the stimuli (such as flashes or presenting images) which were adjacent to the target stimulus. Furthermore, subjects have reported that conspicuous stimuli made them tired and annoyed. In view of this, the aim of this study was to reduce adjacent interference, annoyance and fatigue using a new stimulus presentation pattern based upon facial expression changes. Our goal was not to design a new pattern which could evoke larger ERPs than the face pattern, but to design a new pattern which could reduce adjacent interference, annoyance and fatigue, and evoke ERPs as good as those observed during the face pattern. APPROACH: Positive facial expressions could be changed to negative facial expressions by minor changes to the original facial image. Although the changes are minor, the contrast is big enough to evoke strong ERPs. In this paper, a facial expression change pattern between positive and negative facial expressions was used to attempt to minimize interference effects. This was compared against two different conditions, a shuffled pattern containing the same shapes and colours as the facial expression change pattern, but without the semantic content associated with a change in expression, and a face versus no face pattern. Comparisons were made in terms of classification accuracy and information transfer rate as well as user supplied subjective measures. MAIN RESULTS: The results showed that interferences from adjacent stimuli, annoyance and the fatigue experienced by the subjects could be reduced significantly (p < 0.05) by using the facial expression change patterns in comparison with the face pattern. The offline results show that the classification accuracy of the facial expression change pattern was significantly better than that of the shuffled pattern (p < 0.05) and the face pattern (p < 0.05). SIGNIFICANCE: The facial expression change pattern presented in this paper reduced interference from adjacent stimuli and decreased the fatigue and annoyance experienced by BCI users significantly (p < 0.05) compared to the face pattern.