Non-visual and multisensory BCI systems: present and future

During the past decade, brain–computer interfaces (BCIs) have rapidly developed, both in technological and application domains. However, most of these interfaces rely on the visual modality. Only some research groups have been studying non-visual BCIs, primarily based on auditory and, sometimes, on somatosensory signals. These non-visual BCI approaches are especially useful for severely disabled patients with poor vision. From a broader perspective, multisensory BCIs may offer more versatile and user-friendly paradigms for control and feedback. This chapter describes current systems that are used within auditory and somatosensory BCI research. Four categories of noninvasive BCI paradigms are employed: (1) P300 evoked potentials, (2) steady-state evoked potentials, (3) slow cortical potentials, and (4) mental tasks. Comparing visual and non-visual BCIs, we propose and discuss different possible multisensory combinations, as well as their pros and cons. We conclude by discussing potential future research directions of multisensory BCIs and related research questions

Is it significant? Guidelines for reporting BCI performance

Recent growth in brain-computer interface (BCI) research has increased pressure to report improved performance. However, different research groups report performance in different ways. Hence, it is essential that evaluation procedures are valid and reported in sufficient detail. In this chapter we give an overview of available performance measures such as classification accuracy, cohen’s kappa, information transfer rate (ITR), and written symbol rate. We show how to distinguish results from chance level using confidence intervals for accuracy or kappa. Furthermore, we point out common pitfalls when moving from offline to online analysis and provide a guide on how to conduct statistical tests on (BCI) results.

Evaluating perceptual separation in a pilot system for affective composition

Research evaluating perceptual responses to music has identified many structural features as correlates that might be incorporated in computer music systems for affectively charged algorithmic composition and/or expressive music performance. In order to investigate the possible integration of isolated musical features to such a system, a discrete feature known to correlate some with emotional responses – rhythmic density – was selected from a literature review and incorporated into a prototype system. This system produces variation in rhythm density via a transformative process. A stimulus set created using this system was then subjected to a perceptual evaluation. Pairwise comparisons were used to scale differences between 48 stimuli. Listener responses were analysed with Multidimensional scaling (MDS). The 2-Dimensional solution was then rotated to place the stimuli with the largest range of variation across the horizontal plane. Stimuli with variation in rhythmic density were placed further from the source material than stimuli that were generated by random permutation. This, combined with the striking similarity between the MDS scaling and that of the 2-dimensional emotional model used by some affective algorithmic composition systems, suggests that isolated musical feature manipulation can now be used to parametrically control affectively charged automated composition in a larger system.

An optimized auditory P300 BCI based on spatially distributed sound in different voices

In this paper, a new paradigm is presented, to improve the performance of audio-based P300 Brain-computer interfaces (BCIs), by using spatially distributed natural sound stimuli. The new paradigm was compared to a conventional paradigm using spatially distributed sound to demonstrate the performance of this new paradigm. The results show that the new paradigm enlarged the N200 and P300 components, and yielded significantly better BCI performance than the conventional paradigm.

Brain-computer music interfacing for continuous control of musical tempo

A Brain-computer music interface (BCMI) is developed to allow for continuous modification of the tempo of dynamically generated music. Six out of seven participants are able to control the BCMI at significant accuracies and their performance is observed to increase over time.

Dynamic game soundtrack generation in response to a continously varying emotional trajectory

Dynamic soundtracking presents various practical and aesthetic challenges to composers working with games. This paper presents an implementation of a system addressing some of these challenges with an affectively-driven music generation algorithm based on a second order Markov-model. The system can respond in real-time to emotional trajectories derived from 2-dimensions of affect on the circumplex model (arousal and valence), which are mapped to five musical parameters. A transition matrix is employed to vary the generated output in continuous response to the affective state intended by the gameplay.

Automated artifact removal from the electroencephalogram; a comparative study

Contamination of the electroencephalogram (EEG) by artifacts greatly reduces the quality of the recorded signals. There is a need for automated artifact removal methods. However, such methods are rarely evaluated against one another via rigorous criteria, with results often presented based upon visual inspection alone. This work presents a comparative study of automatic methods for removing blink, electrocardiographic, and electromyographic artifacts from the EEG. Three methods are considered; wavelet, blind source separation (BSS), and multivariate singular spectrum analysis (MSSA)-based correction. These are applied to data sets containing mixtures of artifacts. Metrics are devised to measure the performance of each method. The BSS method is seen to be the best approach for artifacts of high signal to noise ratio (SNR). By contrast, MSSA performs well at low SNRs but at the expense of a large number of false positive corrections.

Motor imagery induced EEG patterns in spinal cord injury patients and their impact on Brain-Computer Interface accuracy

OBJECTIVE: Assimilating the diagnosis complete spinal cord injury (SCI) takes time and is not easy, as patients know that there is no 'cure' at the present time. Brain-computer interfaces (BCIs) can facilitate daily living. However, inter-subject variability demands measurements with potential user groups and an understanding of how they differ to healthy users BCIs are more commonly tested with. Thus, a three-class motor imagery (MI) screening (left hand, right hand, feet) was performed with a group of 10 able-bodied and 16 complete spinal-cord-injured people (paraplegics, tetraplegics) with the objective of determining what differences were present between the user groups and how they would impact upon the ability of these user groups to interact with a BCI. APPROACH: Electrophysiological differences between patient groups and healthy users are measured in terms of sensorimotor rhythm deflections from baseline during MI, electroencephalogram microstate scalp maps and strengths of inter-channel phase synchronization. Additionally, using a common spatial pattern algorithm and a linear discriminant analysis classifier, the classification accuracy was calculated and compared between groups. MAIN RESULTS: It is seen that both patient groups (tetraplegic and paraplegic) have some significant differences in event-related desynchronization strengths, exhibit significant increases in synchronization and reach significantly lower accuracies (mean (M) = 66.1%) than the group of healthy subjects (M = 85.1%). SIGNIFICANCE: The results demonstrate significant differences in electrophysiological correlates of motor control between healthy individuals and those individuals who stand to benefit most from BCI technology (individuals with SCI). They highlight the difficulty in directly translating results from healthy subjects to participants with SCI and the challenges that, therefore, arise in providing BCIs to such individuals.

An optimized ERP Brain-computer interface based on facial expression changes

OBJECTIVE: Interferences from spatially adjacent non-target stimuli are known to evoke event-related potentials (ERPs) during non-target flashes and, therefore, lead to false positives. This phenomenon was commonly seen in visual attention-based brain-computer interfaces (BCIs) using conspicuous stimuli and is known to adversely affect the performance of BCI systems. Although users try to focus on the target stimulus, they cannot help but be affected by conspicuous changes of the stimuli (such as flashes or presenting images) which were adjacent to the target stimulus. Furthermore, subjects have reported that conspicuous stimuli made them tired and annoyed. In view of this, the aim of this study was to reduce adjacent interference, annoyance and fatigue using a new stimulus presentation pattern based upon facial expression changes. Our goal was not to design a new pattern which could evoke larger ERPs than the face pattern, but to design a new pattern which could reduce adjacent interference, annoyance and fatigue, and evoke ERPs as good as those observed during the face pattern. APPROACH: Positive facial expressions could be changed to negative facial expressions by minor changes to the original facial image. Although the changes are minor, the contrast is big enough to evoke strong ERPs. In this paper, a facial expression change pattern between positive and negative facial expressions was used to attempt to minimize interference effects. This was compared against two different conditions, a shuffled pattern containing the same shapes and colours as the facial expression change pattern, but without the semantic content associated with a change in expression, and a face versus no face pattern. Comparisons were made in terms of classification accuracy and information transfer rate as well as user supplied subjective measures. MAIN RESULTS: The results showed that interferences from adjacent stimuli, annoyance and the fatigue experienced by the subjects could be reduced significantly (p < 0.05) by using the facial expression change patterns in comparison with the face pattern. The offline results show that the classification accuracy of the facial expression change pattern was significantly better than that of the shuffled pattern (p < 0.05) and the face pattern (p < 0.05). SIGNIFICANCE: The facial expression change pattern presented in this paper reduced interference from adjacent stimuli and decreased the fatigue and annoyance experienced by BCI users significantly (p < 0.05) compared to the face pattern.

Separating heart and brain: on the reduction of physiological noise from multichannel functional near-infrared spectroscopy (fNIRS) signals

Objective. Functional near-infrared spectroscopy (fNIRS) is an emerging technique for the in vivo assessment of functional activity of the cerebral cortex as well as in the field of brain–computer interface (BCI) research. A common challenge for the utilization of fNIRS in these areas is a stable and reliable investigation of the spatio-temporal hemodynamic patterns. However, the recorded patterns may be influenced and superimposed by signals generated from physiological processes, resulting in an inaccurate estimation of the cortical activity. Up to now only a few studies have investigated these influences, and still less has been attempted to remove/reduce these influences. The present study aims to gain insights into the reduction of physiological rhythms in hemodynamic signals (oxygenated hemoglobin (oxy-Hb), deoxygenated hemoglobin (deoxy-Hb)). Approach. We introduce the use of three different signal processing approaches (spatial filtering, a common average reference (CAR) method; independent component analysis (ICA); and transfer function (TF) models) to reduce the influence of respiratory and blood pressure (BP) rhythms on the hemodynamic responses. Main results. All approaches produce large reductions in BP and respiration influences on the oxy-Hb signals and, therefore, improve the contrast-to-noise ratio (CNR). In contrast, for deoxy-Hb signals CAR and ICA did not improve the CNR. However, for the TF approach, a CNR-improvement in deoxy-Hb can also be found. Significance. The present study investigates the application of different signal processing approaches to reduce the influences of physiological rhythms on the hemodynamic responses. In addition to the identification of the best signal processing method, we also show the importance of noise reduction in fNIRS data.

FORCe: fully online and automated artifact removal for brain-computer interfacing

A fully automated and online artifact removal method for the electroencephalogram (EEG) is developed for use in brain-computer interfacing. The method (FORCe) is based upon a novel combination of wavelet decomposition, independent component analysis, and thresholding. FORCe is able to operate on a small channel set during online EEG acquisition and does not require additional signals (e.g. electrooculogram signals). Evaluation of FORCe is performed offline on EEG recorded from 13 BCI particpants with cerebral palsy (CP) and online with three healthy participants. The method outperforms the state-of the-art automated artifact removal methods Lagged auto-mutual information clustering (LAMIC) and Fully automated statistical thresholding (FASTER), and is able to remove a wide range of artifact types including blink, electromyogram (EMG), and electrooculogram (EOG) artifacts.

Sensory enhancement, a pilot perceptual study of subdermal magnetic implants

Subdermal magnetic implants originated as an art form in the world of body modification. To date an in depth scientific analysis of the benefits of this implant has yet to be established. This research explores the concept of sensory extension of the tactile sense utilising this form of implantation. This relatively simple procedure enables the tactile sense to respond to static and alternating magnetic fields. This is not to say that the underlying biology of the system has changed; i.e. the concept does not increase our tactile frequency response range or sensitivity to pressure, but now does invoke a perceptual response to a stimulus that is not innately available to humans. Within this research two social surveys have been conducted in order to ascertain one, the social acceptance of the general notion of human enhancement, and two the perceptual experiences of individuals with the magnetic implants themselves. In terms of acceptance to the notion of sensory improvement (via implantation) ~39% of the general population questioned responded positively with a further ~25% of the respondents answering with the indecisive response. Thus with careful dissemination a large proportion of individuals may adopt this technology much like this if it were to become available for consumers. Interestingly of the responses collected from the magnetic implants survey ~60% of the respondents actually underwent the implant for magnetic vision purposes. The main contribution of this research however comes from a series of psychophysical testing. In which 7 subjects with subdermal magnetic implants, were cross compared with 7 subjects that had similar magnets superficially attached to their dermis. The experimentation examined multiple psychometric thresholds of the candidates including intensity, frequency and temporal. Whilst relatively simple, the experimental setup for the perceptual experimentation conducted was novel in that custom hardware and protocols were created in order to determine the subjective thresholds of the individuals. Abstract iv The overall purpose of this research is to utilise this concept in high stress scenarios, such as driving or piloting; whereby alerts and warnings could be relayed to an operator without intruding upon their other (typically overloaded) exterior senses (i.e. the auditory and visual senses). Hence each of the thresholding experiments were designed with the intention of utilising the results in the design of signals for information transfer. The findings from the study show that the implanted group of subjects significantly outperformed the superficial group in the absolute intensity threshold experiment, i.e. the implanted group required significantly less force than the superficial group in order to perceive the stimulus. The results for the frequency difference threshold showed no significant difference in the two groups tested. Interestingly however at low frequencies, i.e. 20 and 50 Hz, the ability of the subjects tested to discriminate frequencies significantly increased with more complex waveforms i.e. square and sawtooth, when compared against the typically used sinewave. Furthermore a novel protocol for establishing the temporal gap detection threshold during a temporal numerosity study has been established in this thesis. This experiment measured the subjects’ capability to correctly determine the number of concatenated signals presented to them whilst the time between the signals, referred to as pulses, tended to zero. A significant finding was that when altering the length of, the frequency of, and the number of cycles of the pulses, the time between pulses for correct recognition altered. This finding will ultimately aid in the design of the tactile alerts for this method of information transfer. Preliminary development work for the use of this method of input to the body, in an automotive scenario, is also presented within this thesis in the form of a driving simulation. The overall goal of which is to present warning alerts to a driver, such as rear-to-end collision, or excessive speeds on roads, in order to prevent incidents and penalties from occurring. Discussion on the broader utility of this implant has been presented, reflecting on its potential use as a basis for vibrotactile, and sensory substitution, devices. This discussion furthers with postulations on its use as a human machine interface, as well as how a similar implant could be used within the ear as a hearing aid device.

The impact of date palm fruits and their component polyphenols, on gut microbial ecology, bacterial metabolites and colon cancer cell proliferation

The fruit of the date palm (Phoenix dactylifera L.) is a rich source of dietary fibre and polyphenols. We have investigated gut bacterial changes induced by the whole date fruit extract (digested date extract; DDE) and its polyphenol-rich extract (date polyphenol extract; DPE) using faecal, pH-controlled, mixed batch cultures mimicking the distal part of the human large intestine, and utilising an array of microbial group-specific 16S rRNA oligonucleotide probes. Fluorescence microscopic enumeration indicated that there was a significant increase in the growth of bifidobacteria in response to both treatments, whilst whole dates also increased bacteroides at 24 h and the total bacterial counts at later fermentation time points when compared with DPE alone. Bacterial metabolism of whole date fruit led to the production of SCFA, with acetate significantly increasing following bacterial incubation with DDE. In addition, the production of flavonoid aglycones (myricetin, luteolin, quercetin and apigenin) and the anthocyanidin petunidin in less than 1 h was also observed. Lastly, the potential of DDE, DPE and metabolites to inhibit Caco-2 cell growth was investigated, indicating that both were capable of potentially acting as antiproliferative agents in vitro, following a 48 h exposure. This potential to inhibit growth was reduced following fermentation. Together these data suggest that consumption of date fruits may enhance colon health by increasing beneficial bacterial growth and inhibiting the proliferation of colon cancer cells. This is an early suggestion that date intake by humans may aid in the maintenance of bowel health and even the reduction of colorectal cancer development.

Quantifying particle size and turbulent scale dependence of dust uplift in the Sahara using aircraft measurements

The first size-resolved airborne measurements of dust fluxes and the first dust flux measurements from the central Sahara are presented and compared with a parameterization by Kok (2011a). High-frequency measurements of dust size distribution were obtained from 0.16 to 300 µm diameter, and eddy covariance fluxes were derived. This is more than an order of magnitude larger size range than previous flux estimates. Links to surface emission are provided by analysis of particle drift velocities. Number flux is described by a −2 power law between 1 and 144 µm diameter, significantly larger than the 12 µm upper limit suggested by Kok (2011a). For small particles, the deviation from a power law varies with terrain type and the large size cutoff is correlated with atmospheric vertical turbulent kinetic energy, suggesting control by vertical transport rather than emission processes. The measured mass flux mode is in the range 30–100 µm. The turbulent scales important for dust flux are from 0.1 km to 1–10 km. The upper scale increases during the morning as boundary layer depth and eddy size increase. All locations where large dust fluxes were measured had large topographical variations. These features are often linked with highly erodible surface features, such as wadis or dunes. We also hypothesize that upslope flow and flow separation over such features enhance the dust flux by transporting large particles out of the saltation layer. The tendency to locate surface flux measurements in open, flat terrain means these favored dust sources have been neglected in previous studies.