An Integrated Neural Network-Event-Related Potentials Model of Temporal and Probability Context Effects on Event Categorization

We present a neural network that adapts and integrates several preexisting or new modules to categorize events in short term memory (STM), encode temporal order in working memory, evaluate timing and probability context in medium and long term memory. The model shows how processed contextual information modulates event recognition and categorization, focal attention and incentive motivation. The model is based on a compendium of Event Related Potentials (ERPs) and behavioral results either collected by the authors or compiled from the classical ERP literature. Its hallmark is, at the functional level, the interplay of memory registers endowed with widely different dynamical ranges, and at the structural level, the attempt to relate the different modules to known anatomical structures.

Improvement in visual search with practice: mapping learning-related changes in neurocognitive stages of processing.

Practice can improve performance on visual search tasks; the neural mechanisms underlying such improvements, however, are not clear. Response time typically shortens with practice, but which components of the stimulus-response processing chain facilitate this behavioral change? Improved search performance could result from enhancements in various cognitive processing stages, including (1) sensory processing, (2) attentional allocation, (3) target discrimination, (4) motor-response preparation, and/or (5) response execution. We measured event-related potentials (ERPs) as human participants completed a five-day visual-search protocol in which they reported the orientation of a color popout target within an array of ellipses. We assessed changes in behavioral performance and in ERP components associated with various stages of processing. After practice, response time decreased in all participants (while accuracy remained consistent), and electrophysiological measures revealed modulation of several ERP components. First, amplitudes of the early sensory-evoked N1 component at 150 ms increased bilaterally, indicating enhanced visual sensory processing of the array. Second, the negative-polarity posterior-contralateral component (N2pc, 170-250 ms) was earlier and larger, demonstrating enhanced attentional orienting. Third, the amplitude of the sustained posterior contralateral negativity component (SPCN, 300-400 ms) decreased, indicating facilitated target discrimination. Finally, faster motor-response preparation and execution were observed after practice, as indicated by latency changes in both the stimulus-locked and response-locked lateralized readiness potentials (LRPs). These electrophysiological results delineate the functional plasticity in key mechanisms underlying visual search with high temporal resolution and illustrate how practice influences various cognitive and neural processing stages leading to enhanced behavioral performance.

P300 investigation of phoneme change detection in dyslexic adults

A specific impairment in phoneme awareness has been hypothesized as one of the current explanations for dyslexia. We examined attentional shifts towards phonological information as indexed by event-related potentials (ERPs) in normal readers and dyslexic adults. Participants performed a lexical decision task on spoken stimuli of which 80% started with a standard phoneme and 20% with a deviant phoneme. A P300 modulation was expected for deviants in control adults, indicating that the phonological change had been detected. A mild and right-lateralized P300 was observed for deviant stimuli in controls, but was absent in dyslexic adults. This result suggests that dyslexic adults fail to make shifts of attention to phonological cues in the same way that normal adult readers do. (C) 2003 Elsevier Ireland Ltd. All rights reserved.

Brain responses to audiovisual speech mismatch in infants are associated with individual differences in looking behaviour

Research on audiovisual speech integration has reported high levels of individual variability, especially among young infants. In the present study we tested the hypothesis that this variability results from individual differences in the maturation of audiovisual speech processing during infancy. A developmental shift in selective attention to audiovisual speech has been demonstrated between 6 and 9 months with an increase in the time spent looking to articulating mouths as compared to eyes (Lewkowicz & Hansen-Tift. (2012) Proc. Natl Acad. Sci. USA, 109, 1431–1436; Tomalski et al. (2012) Eur. J. Dev. Psychol., 1–14). In the present study we tested whether these changes in behavioural maturational level are associated with differences in brain responses to audiovisual speech across this age range. We measured high-density event-related potentials (ERPs) in response to videos of audiovisually matching and mismatched syllables /ba/ and /ga/, and subsequently examined visual scanning of the same stimuli with eye-tracking. There were no clear age-specific changes in ERPs, but the amplitude of audiovisual mismatch response (AVMMR) to the combination of visual /ba/ and auditory /ga/ was strongly negatively associated with looking time to the mouth in the same condition. These results have significant implications for our understanding of individual differences in neural signatures of audiovisual speech processing in infants, suggesting that they are not strictly related to chronological age but instead associated with the maturation of looking behaviour, and develop at individual rates in the second half of the first year of life.

ERP and cardiovascular correlates of source memory discrimination in older and younger adults /

There is much evidence to support an age-related decline in source memory ability. However, the underlying mechanisms responsible for this decline are not well understood. The current study was carried out to determine the electrophysiological correlates of source memory discrimination in younger and older adults. Event-related potentials (ERPs) and continuous electrocardiographic (ECG) data were collected from younger (M= 21 years) and older (M= 71 years) adults during a source memory task. Older adults were more likely to make source memory errors for recently repeated, non-target words than were younger adults. Moreover, their ERP records for correct trials showed an increased amplitude in the late positive (LP) component (400-800 msec) for the most recently presented, non-target stimuli relative to the LP noted for target items. Younger adults showed an opposite pattern, with a large LP component for target items, and a much smaller LP component for the recently repeated non-target items. Computation of parasympathetic activity in the vagus nerve was performed on the ECG data (Porges, 1985). The resulting measure, vagal tone, was used as an index of physiological responsivity. The vagal tone index of physiological responsivity was negatively related to the LP amplitude for the most recently repeated, non-target words in both groups, after accounting for age effects. The ERP data support the hypothesis that the tendency to make source memory errors on the part of older adults is related to the ability to selectively control attentional processes during task performance. Furthermore, the relationship between vagal tone and ERP reactivity suggests that there is a physiological basis to the heightened reactivity measured in the LP response to recently repeated non-target items such that, under decreased physiological resources, there is an impairment in the ability to selectively inhibit bottom-up, stimulus based properties in favour of task-related goals in older adults. The inconsistency of these results with other explanatory models of source memory deficits is discussed. It is concluded that the data are consistent with a physiological reactivity model requiring inhibition of reactivity to irrelevant, but perceptually-fluent, stimuli.

Electrophysiological investigations of the timing of face processing

As important social stimuli, faces playa critical role in our lives. Much of our interaction with other people depends on our ability to recognize faces accurately. It has been proposed that face processing consists of different stages and interacts with other systems (Bruce & Young, 1986). At a perceptual level, the initial two stages, namely structural encoding and face recognition, are particularly relevant and are the focus of this dissertation. Event-related potentials (ERPs) are averaged EEG signals time-locked to a particular event (such as the presentation of a face). With their excellent temporal resolution, ERPs can provide important timing information about neural processes. Previous research has identified several ERP components that are especially related to face processing, including the N 170, the P2 and the N250. Their nature with respect to the stages of face processing is still unclear, and is examined in Studies 1 and 2. In Study 1, participants made gender decisions on a large set of female faces interspersed with a few male faces. The ERP responses to facial characteristics of the female faces indicated that the N 170 amplitude from each side of the head was affected by information from eye region and by facial layout: the right N 170 was affected by eye color and by face width, while the left N 170 was affected by eye size and by the relation between the sizes of the top and bottom parts of a face. In contrast, the P100 and the N250 components were largely unaffected by facial characteristics. These results thus provided direct evidence for the link between the N 170 and structural encoding of faces. In Study 2, focusing on the face recognition stage, we manipulated face identity strength by morphing individual faces to an "average" face. Participants performed a face identification task. The effect of face identity strength was found on the late P2 and the N250 components: as identity strength decreased from an individual face to the "average" face, the late P2 increased and the N250 decreased. In contrast, the P100, the N170 and the early P2 components were not affected by face identity strength. These results suggest that face recognition occurs after 200 ms, but not earlier. Finally, because faces are often associated with social information, we investigated in Study 3 how group membership might affect ERP responses to faces. After participants learned in- and out-group memberships of the face stimuli based on arbitrarily assigned nationality and university affiliation, we found that the N170 latency differentiated in-group and out-group faces, taking longer to process the latter. In comparison, without group memberships, there was no difference in N170 latency among the faces. This dissertation provides evidence that at a neural level, structural encoding of faces, indexed by the N170, occurs within 200 ms. Face recognition, indexed by the late P2 and the N250, occurs shortly afterwards between 200 and 300 ms. Social cognitive factors can also influence face processing. The effect is already evident as early as 130-200 ms at the structural encoding stage.

Genetic and Electrophysiological Correlates of Self-Regulation in Adolescence

Self-regulation is considered a powerful predictor of behavioral and mental health outcomes during adolescence and emerging adulthood. In this dissertation I address some electrophysiological and genetic correlates of this important skill set in a series of four studies. Across all studies event-related potentials (ERPs) were recorded as participants responded to tones presented in attended and unattended channels in an auditory selective attention task. In Study 1, examining these ERPs in relation to parental reports on the Behavior Rating Inventory of Executive Function (BRIEF) revealed that an early frontal positivity (EFP) elicited by to-be-ignored/unattended tones was larger in those with poorer self-regulation. As is traditionally found, N1 amplitudes were more negative for the to-be-attended rather than unattended tones. Additionally, N1 latencies to unattended tones correlated with parent-ratings on the BRIEF, where shorter latencies predicted better self-regulation. In Study 2 I tested a model of the associations between selfregulation scores and allelic variations in monoamine neurotransmitter genes, and their concurrent links to ERP markers of attentional control. Allelic variations in dopaminerelated genes predicted both my ERP markers and self-regulatory variables, and played a moderating role in the association between the two. In Study 3 I examined whether training in Integra Mindfulness Martial Arts, an intervention program which trains elements of self-regulation, would lead to improvement in ERP markers of attentional control and parent-report BRIEF scores in a group of adolescents with self-regulatory difficulties. I found that those in the treatment group amplified their processing of attended relative to unattended stimuli over time, and reduced their levels of problematic behaviour whereas those in the waitlist control group showed little to no change on both of these metrics. In Study 4 I examined potential associations between self-regulation and attentional control in a group of emerging adults. Both event-related spectral perturbations (ERSPs) and intertrial coherence (ITC) in the alpha and theta range predicted individual differences in self-regulation. Across the four studies I was able to conclude that real-world self-regulation is indeed associated with the neural markers of attentional control. Targeted interventions focusing on attentional control may improve self-regulation in those experiencing difficulties in this regard.

La saturation de la mémoire auditive à court terme, une étude en électrophysiologie

La présente étude vise à approfondir les connaissances relatives aux mécanismes neuronaux qui sous-tendent la maintenance de sons variant en hauteur dans la mémoire auditive à court terme (MACT), plus précisément lors de sa saturation. À cet effet, la technique des potentiels reliés aux évènements (PRE) en électrophysiologie a été utilisée. La sélection des participants s’est déroulée par l’entremise de deux expériences comportementales : l’une était une tâche de discrimination et l’autre, une tâche qui évaluait l’habileté générale des participants à réussir une tâche similaire à celle de l’expérience principale en électroencéphalographie (EEG). Les résultats comportementaux de notre tâche en EEG ont montré que la performance diminuait de façon significative plus la charge en mémoire augmentait (séquences de 2, 4, 6 et 8 sons) et que l’estimation de la capacité de la MACT mesurée par K augmentait entre 2 et 4 sons pour atteindre un plafond à 4 sons (effet plafond). Le K maximum étant de 2.84 sons, l’empan mnésique (EM) auditif semble être près de 3 sons. Les résultats électrophysiologiques ont montré que la composante électrophysiologique reliée à la maintenance de sons en MACT, la Sustained Anterior Negativity (SAN), était modulée par le nombre de sons à maintenir : son amplitude augmentait de 2 à 4 sons et ce, jusqu’à l’atteinte d’un plafond à 4 sons. Ces résultats suggèrent que la maintenance de sons additionnels dans la MACT n’est plus possible après sa saturation. Nous soutenons donc que la SAN est un index électrophysiologique de l’activité neuronale associée à la maintenance d’items auditifs dans la MACT et que son amplitude est un bon indicateur de la capacité individuelle de la MACT, estimée par K. Des résultats post-hoc ont démontré que les musiciens et les non-musiciens tendent à avoir des différences au niveau de la SAN, sans pour autant modifier l’effet de charge en mémoire. Une analyse qualitative et quantitative de l’utilisation des stratégies mnésiques ont permis de clarifier leur implication et leur nature au sein d’une tâche cognitive de mémoire, plus précisément en audition. Pour conclure, l’ensemble de ces résultats suggère également que la SAN est reliée à la maintenance de sons dans la MACT et ainsi, un bon indicateur de sa capacité.

Effets du vieillissement sur les index électrophysiologiques de l’évaluation des relations spatiales en mémoire de travail visuelle

Le processus de vieillissement humain est un processus complexe qui varie grandement d’une personne à l’autre. Malgré l’ampleur des recherches faites sur le sujet, il reste encore beaucoup à explorer et à comprendre. Cette thèse propose trois expériences qui nous permettent d’améliorer notre compréhension des changements qui s’opèrent dans la mémoire de travail visuelle et l’attention visuospatiale avec la prise en âge. La première expérience propose d’examiner les changements dans les capacités de mémoire de travail visuelle entre des jeunes adultes, des adultes âgés sains et des personnes atteintes de trouble cognitif léger (TCL). De plus, grâce à un suivi fait avec les personnes ayant un TCL, nous avons pu examiner si des différences existaient au niveau comportemental entre les âgés qui ont déclinés vers un type de démence et ceux dont l’état est resté stable. Plusieurs techniques peuvent être utilisées pour étudier les effets du vieillissement sur le cerveau. Les tests neuropsychologiques et les tâches comportementales présentées dans la première expérience en sont un exemple. La neuroimagerie peut aussi s’avérer particulièrement utile. En effet, certaines mesures électrophysiologiques, aussi appelées potentiels reliés aux évènements (PRE), sont associées à des fonctions cognitives précises. Ces composantes nous permettent de suivre ces processus et d’observer les modulations causées par les caractéristiques des stimuli ou l’âge par exemple. C’est le cas de la N2pc (négativité 2 postérieure controlatérale) et de la SPCN (sustained posterior contralateral negativity), des composantes électrophysiologiques liées respectivement à l’attention visuospatiale et la mémoire de travail visuelle. On connait bien ces deux composantes ainsi que les facteurs qui les modulent, or elles sont peu utilisées pour les changements qui occurrent dans l’attention et la mémoire de travail visuelle dans le cadre du processus de vieillissement. Les deuxième et troisième expériences proposent d’utiliser une tâche de recherche visuelle (nombre d’items de couleur et identification d’une relation spatiale entre deux items de couleur) afin d’explorer les changements observables sur ces composantes électrophysiologiques. La deuxième expérience examine l’efficacité d’un paradigme à présentations multiples (‘multiple frame’) afin de mesurer la N2pc et la SPCN chez de jeunes adultes. La troisième expérience a pour but d’examiner les effets du vieillissement normal sur l’amplitude et la latence de la N2pc et la SPCN en utilisant le même type de tâche de recherche visuelle.

Watching the infant brain learn words: Effects of vocabulary size and experience

Previous investigations comparing auditory event-related potentials (ERPs) to words whose meanings infants did or did not comprehend, found bilateral differences in brain activity to known versus unknown words in 13-month-old infants, in contrast with unilateral, left hemisphere, differences in activity in 20-month-old infants. We explore two alternative explanations for these findings. Changes in hemispheric specialization may result from a qualitative shift in the way infants process known words between 13 and 20 months. Alternatively, hemispheric specialization may arise from increased familiarity with the individual words tested. We contrasted these two explanations by measuring ERPs from 20-month-old infants with high and low production scores, for novel words they had just learned. A bilateral distribution of ERP differences was observed in both groups of infants, though the difference was larger in the left hemisphere for the high producers. These findings suggest that word familiarity is an important factor in determining the distribution of brain regions involved in word learning. An emerging left hemispheric specialization may reflect increased efficiency in the manner in which infants process familiar and novel words. (c) 2004 Elsevier Inc. All rights reserved.

Visuotactile learning and body representation: an ERP study with rubber hands and rubber objects

We studied how the integration of seen and felt tactile stimulation modulates somatosensory processing, and investigated whether visuotactile integration depends on temporal contiguity of stimulation, and its coherence with a pre-existing body representation. During training, participants viewed a rubber hand or a rubber object that was tapped either synchronously with stimulation of their own hand, or in an uncorrelated fashion. In a subsequent test phase, somatosensory event-related potentials (ERPs) were recorded to tactile stimulation of the left or right hand, to assess how tactile processing was affected by previous visuotactile experience during training. An enhanced somatosensory N140 component was elicited after synchronous, compared with uncorrelated, visuotactile training, irrespective of whether participants viewed a rubber hand or rubber object. This early effect of visuotactile integration on somatosensory processing is interpreted as a candidate electrophysiological correlate of the rubber hand illusion that is determined by temporal contiguity, but not by pre-existing body representations. ERPmodulations were observed beyond 200msec post-stimulus, suggesting an attentional bias induced by visuotactile training. These late modulations were absent when the stimulation of a rubber hand and the participant’s own hand was uncorrelated during training, suggesting that pre-existing body representations may affect later stages of tactile processing.

An optimized ERP brain–computer interface based on facial expression changes

Objective. Interferences from spatially adjacent non-target stimuli are known to evoke event-related potentials (ERPs) during non-target flashes and, therefore, lead to false positives. This phenomenon was commonly seen in visual attention-based brain–computer interfaces (BCIs) using conspicuous stimuli and is known to adversely affect the performance of BCI systems. Although users try to focus on the target stimulus, they cannot help but be affected by conspicuous changes of the stimuli (such as flashes or presenting images) which were adjacent to the target stimulus. Furthermore, subjects have reported that conspicuous stimuli made them tired and annoyed. In view of this, the aim of this study was to reduce adjacent interference, annoyance and fatigue using a new stimulus presentation pattern based upon facial expression changes. Our goal was not to design a new pattern which could evoke larger ERPs than the face pattern, but to design a new pattern which could reduce adjacent interference, annoyance and fatigue, and evoke ERPs as good as those observed during the face pattern. Approach. Positive facial expressions could be changed to negative facial expressions by minor changes to the original facial image. Although the changes are minor, the contrast is big enough to evoke strong ERPs. In this paper, a facial expression change pattern between positive and negative facial expressions was used to attempt to minimize interference effects. This was compared against two different conditions, a shuffled pattern containing the same shapes and colours as the facial expression change pattern, but without the semantic content associated with a change in expression, and a face versus no face pattern. Comparisons were made in terms of classification accuracy and information transfer rate as well as user supplied subjective measures. Main results. The results showed that interferences from adjacent stimuli, annoyance and the fatigue experienced by the subjects could be reduced significantly (p < 0.05) by using the facial expression change patterns in comparison with the face pattern. The offline results show that the classification accuracy of the facial expression change pattern was significantly better than that of the shuffled pattern (p < 0.05) and the face pattern (p < 0.05). Significance. The facial expression change pattern presented in this paper reduced interference from adjacent stimuli and decreased the fatigue and annoyance experienced by BCI users significantly (p < 0.05) compared to the face pattern.

An optimized ERP Brain-computer interface based on facial expression changes

OBJECTIVE: Interferences from spatially adjacent non-target stimuli are known to evoke event-related potentials (ERPs) during non-target flashes and, therefore, lead to false positives. This phenomenon was commonly seen in visual attention-based brain-computer interfaces (BCIs) using conspicuous stimuli and is known to adversely affect the performance of BCI systems. Although users try to focus on the target stimulus, they cannot help but be affected by conspicuous changes of the stimuli (such as flashes or presenting images) which were adjacent to the target stimulus. Furthermore, subjects have reported that conspicuous stimuli made them tired and annoyed. In view of this, the aim of this study was to reduce adjacent interference, annoyance and fatigue using a new stimulus presentation pattern based upon facial expression changes. Our goal was not to design a new pattern which could evoke larger ERPs than the face pattern, but to design a new pattern which could reduce adjacent interference, annoyance and fatigue, and evoke ERPs as good as those observed during the face pattern. APPROACH: Positive facial expressions could be changed to negative facial expressions by minor changes to the original facial image. Although the changes are minor, the contrast is big enough to evoke strong ERPs. In this paper, a facial expression change pattern between positive and negative facial expressions was used to attempt to minimize interference effects. This was compared against two different conditions, a shuffled pattern containing the same shapes and colours as the facial expression change pattern, but without the semantic content associated with a change in expression, and a face versus no face pattern. Comparisons were made in terms of classification accuracy and information transfer rate as well as user supplied subjective measures. MAIN RESULTS: The results showed that interferences from adjacent stimuli, annoyance and the fatigue experienced by the subjects could be reduced significantly (p < 0.05) by using the facial expression change patterns in comparison with the face pattern. The offline results show that the classification accuracy of the facial expression change pattern was significantly better than that of the shuffled pattern (p < 0.05) and the face pattern (p < 0.05). SIGNIFICANCE: The facial expression change pattern presented in this paper reduced interference from adjacent stimuli and decreased the fatigue and annoyance experienced by BCI users significantly (p < 0.05) compared to the face pattern.

Mental states monitoring through passive Brain-Computer Interface systems

The monitoring of cognitive functions aims at gaining information about the current cognitive state of the user by decoding brain signals. In recent years, this approach allowed to acquire valuable information about the cognitive aspects regarding the interaction of humans with external world. From this consideration, researchers started to consider passive application of brain–computer interface (BCI) in order to provide a novel input modality for technical systems solely based on brain activity. The objective of this thesis is to demonstrate how the passive Brain Computer Interfaces (BCIs) applications can be used to assess the mental states of the users, in order to improve the human machine interaction. Two main studies has been proposed. The first one allows to investigate whatever the Event Related Potentials (ERPs) morphological variations can be used to predict the users’ mental states (e.g. attentional resources, mental workload) during different reactive BCI tasks (e.g. P300-based BCIs), and if these information can predict the subjects’ performance in performing the tasks. In the second study, a passive BCI system able to online estimate the mental workload of the user by relying on the combination of the EEG and the ECG biosignals has been proposed. The latter study has been performed by simulating an operative scenario, in which the occurrence of errors or lack of performance could have significant consequences. The results showed that the proposed system is able to estimate online the mental workload of the subjects discriminating three different difficulty level of the tasks ensuring a high reliability.

Neurophysiological correlates of delinquent behaviour in adult subjects with ADHD

The attention deficit/hyperactivity disorder (ADHD) shows an increased prevalence in arrested offenders compared to the normal population. The aim of the present study was to investigate whether ADHD symptoms are a major risk factor for criminal behaviour, or whether further deficits, mainly abnormalities in emotion-processing, have to be considered as important additional factors that promote delinquency in the presence of ADHD symptomatology. Event related potentials (ERPs) of 13 non-delinquent and 13 delinquent subjects with ADHD and 13 controls were compared using a modified visual Go/Nogo continuous performance task (VCPT) and a newly developed version of the visual CPT that additionally requires emotional evaluation (ECPT). ERPs were analyzed regarding their topographies and Global Field Power (GFP). Offenders with ADHD differed from non-delinquent subjects with ADHD in the ERPs representing higher-order visual processing of objects and faces (N170) and facial affect (P200), and in late monitoring and evaluative functions (LPC) of behavioural response inhibition. Concerning neural activity thought to reflect the allocation of neural resources and cognitive processing capability (P300 Go), response inhibition (P300 Nogo), and attention/expectancy (CNV), deviances were observable in both ADHD groups and may thus be attributed to ADHD rather than to delinquency. In conclusion, ADHD symptomatology may be a risk factor for delinquency, since some neural information processing deficits found in ADHD seemed to be even more pronounced in offenders with ADHD. However, our results suggest additional risk factors consisting of deviant higher-order visual processing, especially of facial affect, as well as abnormalities in monitoring and evaluative functions of response inhibition.