An optimized ERP brain–computer interface based on facial expression changes

Objective. Interferences from spatially adjacent non-target stimuli are known to evoke event-related potentials (ERPs) during non-target flashes and, therefore, lead to false positives. This phenomenon was commonly seen in visual attention-based brain–computer interfaces (BCIs) using conspicuous stimuli and is known to adversely affect the performance of BCI systems. Although users try to focus on the target stimulus, they cannot help but be affected by conspicuous changes of the stimuli (such as flashes or presenting images) which were adjacent to the target stimulus. Furthermore, subjects have reported that conspicuous stimuli made them tired and annoyed. In view of this, the aim of this study was to reduce adjacent interference, annoyance and fatigue using a new stimulus presentation pattern based upon facial expression changes. Our goal was not to design a new pattern which could evoke larger ERPs than the face pattern, but to design a new pattern which could reduce adjacent interference, annoyance and fatigue, and evoke ERPs as good as those observed during the face pattern. Approach. Positive facial expressions could be changed to negative facial expressions by minor changes to the original facial image. Although the changes are minor, the contrast is big enough to evoke strong ERPs. In this paper, a facial expression change pattern between positive and negative facial expressions was used to attempt to minimize interference effects. This was compared against two different conditions, a shuffled pattern containing the same shapes and colours as the facial expression change pattern, but without the semantic content associated with a change in expression, and a face versus no face pattern. Comparisons were made in terms of classification accuracy and information transfer rate as well as user supplied subjective measures. Main results. The results showed that interferences from adjacent stimuli, annoyance and the fatigue experienced by the subjects could be reduced significantly (p < 0.05) by using the facial expression change patterns in comparison with the face pattern. The offline results show that the classification accuracy of the facial expression change pattern was significantly better than that of the shuffled pattern (p < 0.05) and the face pattern (p < 0.05). Significance. The facial expression change pattern presented in this paper reduced interference from adjacent stimuli and decreased the fatigue and annoyance experienced by BCI users significantly (p < 0.05) compared to the face pattern.

An optimized ERP Brain-computer interface based on facial expression changes

OBJECTIVE: Interferences from spatially adjacent non-target stimuli are known to evoke event-related potentials (ERPs) during non-target flashes and, therefore, lead to false positives. This phenomenon was commonly seen in visual attention-based brain-computer interfaces (BCIs) using conspicuous stimuli and is known to adversely affect the performance of BCI systems. Although users try to focus on the target stimulus, they cannot help but be affected by conspicuous changes of the stimuli (such as flashes or presenting images) which were adjacent to the target stimulus. Furthermore, subjects have reported that conspicuous stimuli made them tired and annoyed. In view of this, the aim of this study was to reduce adjacent interference, annoyance and fatigue using a new stimulus presentation pattern based upon facial expression changes. Our goal was not to design a new pattern which could evoke larger ERPs than the face pattern, but to design a new pattern which could reduce adjacent interference, annoyance and fatigue, and evoke ERPs as good as those observed during the face pattern. APPROACH: Positive facial expressions could be changed to negative facial expressions by minor changes to the original facial image. Although the changes are minor, the contrast is big enough to evoke strong ERPs. In this paper, a facial expression change pattern between positive and negative facial expressions was used to attempt to minimize interference effects. This was compared against two different conditions, a shuffled pattern containing the same shapes and colours as the facial expression change pattern, but without the semantic content associated with a change in expression, and a face versus no face pattern. Comparisons were made in terms of classification accuracy and information transfer rate as well as user supplied subjective measures. MAIN RESULTS: The results showed that interferences from adjacent stimuli, annoyance and the fatigue experienced by the subjects could be reduced significantly (p < 0.05) by using the facial expression change patterns in comparison with the face pattern. The offline results show that the classification accuracy of the facial expression change pattern was significantly better than that of the shuffled pattern (p < 0.05) and the face pattern (p < 0.05). SIGNIFICANCE: The facial expression change pattern presented in this paper reduced interference from adjacent stimuli and decreased the fatigue and annoyance experienced by BCI users significantly (p < 0.05) compared to the face pattern.

Selecting salient objects in real scenes: An oscillatory correlation model

Attention is a critical mechanism for visual scene analysis. By means of attention, it is possible to break down the analysis of a complex scene to the analysis of its parts through a selection process. Empirical studies demonstrate that attentional selection is conducted on visual objects as a whole. We present a neurocomputational model of object-based selection in the framework of oscillatory correlation. By segmenting an input scene and integrating the segments with their conspicuity obtained from a saliency map, the model selects salient objects rather than salient locations. The proposed system is composed of three modules: a saliency map providing saliency values of image locations, image segmentation for breaking the input scene into a set of objects, and object selection which allows one of the objects of the scene to be selected at a time. This object selection system has been applied to real gray-level and color images and the simulation results show the effectiveness of the system. (C) 2010 Elsevier Ltd. All rights reserved.

Chaotic phase synchronization and desynchronization in an oscillator network for object selection

Object selection refers to the mechanism of extracting objects of interest while ignoring other objects and background in a given visual scene. It is a fundamental issue for many computer vision and image analysis techniques and it is still a challenging task to artificial Visual systems. Chaotic phase synchronization takes place in cases involving almost identical dynamical systems and it means that the phase difference between the systems is kept bounded over the time, while their amplitudes remain chaotic and may be uncorrelated. Instead of complete synchronization, phase synchronization is believed to be a mechanism for neural integration in brain. In this paper, an object selection model is proposed. Oscillators in the network representing the salient object in a given scene are phase synchronized, while no phase synchronization occurs for background objects. In this way, the salient object can be extracted. In this model, a shift mechanism is also introduced to change attention from one object to another. Computer simulations show that the model produces some results similar to those observed in natural vision systems.

Dano de frio em limas-ácidas Tahiti, colhidas em diferentes épocas e submetidas a tratamentos térmicos e bioquímicos

O estudo objetivou o estabelecimento de um método efetivo e satisfatório do controle do dano de frio em limas-ácidas. Os frutos colhidos no município de Boa Vista-RR, 140 e 150 dias após a floração, apresentaram valores médios de 7,9 e 8,2 ºBrix; 6,3 e 6,0 mL de ácido cítrico.100mL de polpa-1 e pH de 2,8 e 3,0, respectivamente, nas duas colheitas. Após cada colheita, os frutos foram levados ao laboratório de Fitotecnia/UFRR, onde foram selecionados, limpos e submetidos aos tratamentos: T1 - controle; T2, T3 e T4 - condicionamento a 35ºC, por 6, 12 e 24 horas, respectivamente; T5 - aquecimento intermitente a 20ºC, por 8 horas, após 5 e 10 dias a 1ºC; T6 - aquecimento intermitente a 20ºC, por 8 horas, após 10 e 20 dias a 1ºC; T7 - ethephon a 1.500 mg.L-1; T8 - ethephon a 3.000 mg.L-1. Os tratamentos T9 ao T16, diferenciaram-se dos tratamentos T1 a T8, apenas, na data da colheita (10 dias após a primeira). O experimento foi avaliado a cada 15 dias, durante 75 dias, a 1 ± 0,5 ºC e 92 ± 5 % de UR, quanto ao dano de frio, aspecto visual, perda de massa fresca, sólidos solúveis (SS), acidez titulável (AT), SS/AT (ratio - RT), clorofila total e ácido ascórbico. O atraso na colheita não proporcionou efeito significativo algum. Todos os tratamentos, à exceção do controle e do aquecimento intermitente aos 10 e 20 dias, foram eficientes no controle do dano de frio. No entanto, o tratamento químico e o condicionamento térmico aceleraram precocemente o metabolismo dos frutos, principalmente no que concerne à perda de massa fresca e ao aspecto visual. O maior teor de clorofila total e de ácido ascórbico, bem como o melhor aspecto visual, a não-incidência de podridões e a menor perda de massa fresca foram detectadas nos frutos submetidos ao aquecimento intermitente aos 5 e 10 dias. Os SS, AT e RT estavam dentro dos padrões de qualidade e não variaram estatisticamente entre os tratamentos.

Componentes sensoriais e atencionais do tempo de reação: efeitos do tamanho, excentricidade e previsibilidade de estímulos visuais

A percepção visual depende do arcabouço sensorial e do processamento atencional. Este trabalho estudou o efeito, sobre o tempo de reação manual (TR), do tamanho, excentricidade e previsibilidade de estímulos visuais. No experimento 1 (n=8), um alvo foi apresentado aleatoriamente em uma de quatro excentricidades diferentes, possuindo três possíveis tamanhos. O experimento 2 (n=12) apresentava configuração similar, porém uma pista indicava o quadrante de maior probabilidade (70%) de apresentação do alvo. Os resultados mostraram um aumento do TR em função da excentricidade do alvo, além de uma diminuição do TR com o aumento do tamanho do alvo e indicação correta da pista. Uma análise das interações sugere uma superposição de mecanismos atencionais e puramente sensoriais compartilhando um estágio comum do processamento visual.

Bilateral impairment of concurrent saccade programming in hemispatial neglect

When healthy observers make a saccade that is erroneously directed toward a distracter stimulus, they often produce a corrective saccade within 100ms after the end of the primary saccade. Such short inter-saccadic intervals indicate that programming of the secondary saccade has been initiated prior to the execution of the primary saccade and hence that the two saccades have been programmed concurrently. Here we show that concurrent saccade programming is bilaterally impaired in left spatial neglect, a strongly lateralized disorder of visual attention resulting from extensive right cerebral damage. Neglect patients were asked to make saccades to targets presented left or right of fixation while disregarding a distracter presented in the opposite hemifield. We examined those experimental trials on which participants first made a saccade to the distracter, followed by a secondary (corrective) saccade to the target. Compared to healthy and right-hemisphere damaged control participants the proportion of secondary saccades directing gaze to the target instead of bringing it even closer to the distracter was bilaterally reduced in neglect patients. In addition, the characteristic reduction of secondary saccade latency observed in both control groups was absent in neglect patients, whether the secondary saccade was directed to the left or right hemifield. This pattern is consistent with a severe, bilateral impairment of concurrent saccade programming in left spatial neglect.

The influence of colour on oculomotor behaviour during image perception

The aim of this study was to investigate how oculomotor behaviour depends on the availability of colour information in pictorial stimuli. Forty study participants viewed complex images in colour or grey-scale, while their eye movements were recorded. We found two major effects of colour. First, although colour increases the complexity of an image, fixations on colour images were shorter than on their grey-scale versions. This suggests that colour enhances discriminability and thus affects low-level perceptual processing. Second, colour decreases the similarity of spatial fixation patterns between participants. The role of colour on visual attention seems to be more important than previously assumed, in theoretical as well as methodological terms.

Evaluation of Binocular Eye Trackers and Algorithms for 3D Gaze Interaction in Virtual Reality Environments

Tracking user’s visual attention is a fundamental aspect in novel human-computer interaction paradigms found in Virtual Reality. For example, multimodal interfaces or dialogue-based communications with virtual and real agents greatly benefit from the analysis of the user’s visual attention as a vital source for deictic references or turn-taking signals. Current approaches to determine visual attention rely primarily on monocular eye trackers. Hence they are restricted to the interpretation of two-dimensional fixations relative to a defined area of projection. The study presented in this article compares precision, accuracy and application performance of two binocular eye tracking devices. Two algorithms are compared which derive depth information as required for visual attention-based 3D interfaces. This information is further applied to an improved VR selection task in which a binocular eye tracker and an adaptive neural network algorithm is used during the disambiguation of partly occluded objects.

Pupillometry: the eyes shed fresh light on the mind

Recent studies provide promising methodological advances in the use of pupillometry as on-line measurement of cognitive processes and show that visual attention allocation, mind-wandering, mental imagery, and even rhyme expectations can influence the size of the human pupil.

The functional significance of EEG microstates-Associations with modalities of thinking

The momentary, global functional state of the brain is reflected by its electric field configuration. Cluster analytical approaches consistently extracted four head-surface brain electric field configurations that optimally explain the variance of their changes across time in spontaneous EEG recordings. These four configurations are referred to as EEG microstate classes A, B, C, and D and have been associated with verbal/phonological, visual, attention reorientation, and subjective interoceptive-autonomic processing, respectively. The present study tested these associations via an intra-individual and inter-individual analysis approach. The intra-individual approach tested the effect of task-induced increased modality-specific processing on EEG microstate parameters. The inter-individual approach tested the effect of personal modality-specific parameters on EEG microstate parameters. We obtained multichannel EEG from 61 healthy, right-handed, male students during four eyes-closed conditions: object-visualization, spatial-visualization, verbalization (6 runs each), and resting (7 runs). After each run, we assessed participants' degrees of object-visual, spatial-visual, and verbal thinking using subjective reports. Before and after the recording, we assessed modality-specific cognitive abilities and styles using nine cognitive tests and two questionnaires. The EEG of all participants, conditions, and runs was clustered into four classes of EEG microstates (A, B, C, and D). RMANOVAs, ANOVAs and post-hoc paired t-tests compared microstate parameters between conditions. TANOVAs compared microstate class topographies between conditions. Differences were localized using eLORETA. Pearson correlations assessed interrelationships between personal modality-specific parameters and EEG microstate parameters during no-task resting. As hypothesized, verbal as opposed to visual conditions consistently affected the duration, occurrence, and coverage of microstate classes A and B. Contrary to associations suggested by previous reports, parameters were increased for class A during visualization, and class B during verbalization. In line with previous reports, microstate D parameters were increased during no-task resting compared to the three internal, goal-directed tasks. Topographic differences between conditions concerned particular sub-regions of components of the metabolic default mode network. Modality-specific personal parameters did not consistently correlate with microstate parameters except verbal cognitive style which correlated negatively with microstate class A duration and positively with class C occurrence. This is the first study that aimed to induce EEG microstate class parameter changes based on their hypothesized functional significance. Beyond, the associations of microstate classes A and B with visual and verbal processing, respectively and microstate class D with interoceptive-autonomic processing, our results suggest that a finely-tuned interplay between all four EEG microstate classes is necessary for the continuous formation of visual and verbal thoughts, as well as interoceptive-autonomic processing. Our results point to the possibility that the EEG microstate classes may represent the head-surface measured activity of intra-cortical sources primarily exhibiting inhibitory functions. However, additional studies are needed to verify and elaborate on this hypothesis.

Structural Organization of the Corpus Callosum Predicts Attentional Shifts after Continuous Theta Burst Stimulation

Repetitive transcranial magnetic stimulation (rTMS) applied over the right posterior parietal cortex (PPC) in healthy participants has been shown to trigger a significant rightward shift in the spatial allocation of visual attention, temporarily mimicking spatial deficits observed in neglect. In contrast, rTMS applied over the left PPC triggers a weaker or null attentional shift. However, large interindividual differences in responses to rTMS have been reported. Studies measuring changes in brain activation suggest that the effects of rTMS may depend on both interhemispheric and intrahemispheric interactions between cortical loci controlling visual attention. Here, we investigated whether variability in the structural organization of human white matter pathways subserving visual attention, as assessed by diffusion magnetic resonance imaging and tractography, could explain interindividual differences in the effects of rTMS. Most participants showed a rightward shift in the allocation of spatial attention after rTMS over the right intraparietal sulcus (IPS), but the size of this effect varied largely across participants. Conversely, rTMS over the left IPS resulted in strikingly opposed individual responses, with some participants responding with rightward and some with leftward attentional shifts. We demonstrate that microstructural and macrostructural variability within the corpus callosum, consistent with differential effects on cross-hemispheric interactions, predicts both the extent and the direction of the response to rTMS. Together, our findings suggest that the corpus callosum may have a dual inhibitory and excitatory function in maintaining the interhemispheric dynamics that underlie the allocation of spatial attention. SIGNIFICANCE STATEMENT: The posterior parietal cortex (PPC) controls allocation of attention across left versus right visual fields. Damage to this area results in neglect, characterized by a lack of spatial awareness of the side of space contralateral to the brain injury. Transcranial magnetic stimulation over the PPC is used to study cognitive mechanisms of spatial attention and to examine the potential of this technique to treat neglect. However, large individual differences in behavioral responses to stimulation have been reported. We demonstrate that the variability in the structural organization of the corpus callosum accounts for these differences. Our findings suggest novel dual mechanism of the corpus callosum function in spatial attention and have broader implications for the use of stimulation in neglect rehabilitation.

The asymmetrical influence of increasing time-on-task on attentional disengagement

Increasing time-on-task leads to fatigue and, as shown by previous research, differentially affects the deployment of visual attention towards the left and the right visual space. In healthy participants, an increasing rightward bias is commonly observed with increasing time-on-task. Yet, it is unclear whether specific mechanisms involved in the spatial deployment of visual attention are differentially affected by increasing time-on-task. The aim of the present study was to investigate whether prolonged time-on-task would affect a specific mechanism of visuo-spatial attentional deployment, namely attentional disengagement, in an asymmetrical fashion. For this purpose, we administered to healthy participants a prolonged gap/overlap saccadic paradigm, with left- and right-sided target stimuli. This oculomotor paradigm allowed to quantify disengagement costs according to the direction of the subsequent attentional shifts, and to evaluate the temporal development of disengagement costs with increasing time-on-task. Our results show that, with increasing time-on-task, participants demonstrated significantly lower disengagement costs for rightward compared to leftward saccades. These effects were specific, since concurring side differences of saccadic latencies were found for overlap trials (requiring attentional disengagement), but not for gap trials (requiring no or less attentional disengagement). Moreover, the results were paralleled by a non-lateralised decrease in saccadic peak velocity with increasing time-on-task, a common finding indicating an increasing level of fatigue. Our findings support the idea that non-spatial attentional aspects, such as fatigue due to increasing time-on-task, can have a substantial influence on the spatial deployment of visual attention, in particular on its disengagement, depending on the direction of the subsequent attentional shift.

Organic geochemistry and sedimentology of lower to mid-Cretaceous at DSDP Leg 77 Holes

Analyses of 40 carbonate core samples - 27 from Site 535, 12 from Site 540, and 1 from Site 538A - have confirmed many of the findings of the Shipboard Scientific Party. The samples, all but one Early to mid-Cretaceous in age (Berriasian to Cenomanian), reflect sequences of cyclically anoxic and oxic depositional environments. They are moderately to very dark colored, dominantly planar-parallel, laminated lime mudstones. Most show the effects of intense mechanical compaction. Visual kerogen characteristics and conventional Rock-Eval parameters indicate that these deep basinal carbonates contain varying mixtures of thermally immature kerogen derived from both marine and terrigenous precursors. However, variations in kerogen chemistry are evident upon analysis of the pyrolysis mass spectral data in conjunction with the other geochemical analyses. Particularly diagnostic is the reduction index, Rl, a measure of H2S produced during pyrolysis. Total organic carbon, TOC, ranges from 0.6 to 6.6%, with an overall average of 2.4%. Average TOCs for these fine-grained mudstones are: late Eocene 2.5% (1 sample), Cenomanian 2.2% (6), Albian 2.0% (10), Aptian 1.3% (1), Barremian-Hauterivian 2.8% (11), late Valanginian 4.8% (3), Berriasian-early Valanginian 1.6% (7). Most of the carbonates have source-potential ratings of fair to very good of predominantly oil-prone to mixed kerogen, with only a few gas-prone samples. The ratings correlate well with the inferred depositional environments, i.e., whether oxic or anoxic. Several new organic-geochemical parameters, especially Rl, based on pyrolysis mass spectrometry of powdered whole-rock samples, support this view. Tar from fractures in laminated to bioturbated limestones of Unit IV (late Valanginian) at 535-58-4, 19-20 cm (530 m sub-bottom) appears to be mature, biodegraded, and of migrated rather than on site indigenous origin.

Depth perceptual video coding for free viewpoint video based on H.264/AVC

A novel scheme for depth sequences compression, based on a perceptual coding algorithm, is proposed. A depth sequence describes the object position in the 3D scene, and is used, in Free Viewpoint Video, for the generation of synthetic video sequences. In perceptual video coding the human visual system characteristics are exploited to improve the compression efficiency. As depth sequences are never shown, the perceptual video coding, assessed over them, is not effective. The proposed algorithm is based on a novel perceptual rate distortion optimization process, assessed over the perceptual distortion of the rendered views generated through the encoded depth sequences. The experimental results show the effectiveness of the proposed method, able to obtain a very considerable improvement of the rendered view perceptual quality.