Non-visual and multisensory BCI systems: present and future

During the past decade, brain–computer interfaces (BCIs) have rapidly developed, both in technological and application domains. However, most of these interfaces rely on the visual modality. Only some research groups have been studying non-visual BCIs, primarily based on auditory and, sometimes, on somatosensory signals. These non-visual BCI approaches are especially useful for severely disabled patients with poor vision. From a broader perspective, multisensory BCIs may offer more versatile and user-friendly paradigms for control and feedback. This chapter describes current systems that are used within auditory and somatosensory BCI research. Four categories of noninvasive BCI paradigms are employed: (1) P300 evoked potentials, (2) steady-state evoked potentials, (3) slow cortical potentials, and (4) mental tasks. Comparing visual and non-visual BCIs, we propose and discuss different possible multisensory combinations, as well as their pros and cons. We conclude by discussing potential future research directions of multisensory BCIs and related research questions

Neurocognitive control in dance perception and performance

Dance is a rich source of material for researchers interested in the integration of movement and cognition. The multiple aspects of embodied cognition involved in performing and perceiving dance have inspired scientists to use dance as a means for studying motor control, expertise, and action-perception links. The aim of this review is to present basic research on cognitive and neural processes implicated in the execution, expression, and observation of dance, and to bring into relief contemporary issues and open research questions. The review addresses six topics: 1) dancers’ exemplary motor control, in terms of postural control, equilibrium maintenance, and stabilization; 2) how dancers’ timing and on-line synchronization are influenced by attention demands and motor experience; 3) the critical roles played by sequence learning and memory; 4) how dancers make strategic use of visual and motor imagery; 5) the insights into the neural coupling between action and perception yielded through exploration of the brain architecture mediating dance observation; and 6) a neuroaesthetics perspective that sheds new light on the way audiences perceive and evaluate dance expression. Current and emerging issues are presented regarding future directions that will facilitate the ongoing dialogue between science and dance.

'You show me yours, I'll show you mine': the negotiation of shifts from textual to visual modes in computer-mediated interaction among gay men

This article explores the way users of an online gay chat room negotiate the exchange of photographs and the conduct of video conferencing sessions and how this negotiation changes the way participants manage their interactions and claim and impute social identities. Different modes of communication provide users with different resources for the control of information, affecting not just what users are able to reveal, but also what they are able to conceal. Thus, the shift from a purely textual mode for interacting to one involving visual images fundamentally changes the kinds of identities and relationships available to users. At the same time, the strategies users employ to negotiate these shifts of mode can alter the resources available in different modes. The kinds of social actions made possible through different modes, it is argued, are not just a matter of the modes themselves but also of how modes are introduced into the ongoing flow of interaction.

Thermal and visual performance of real and theoretical thermochromic glazing solutions for office buildings

Thermochromic windows are able to modulate their transmittance in both the visible and the near-infrared field as a function of their temperature. As a consequence, they allow to control the solar gains in summer, thus reducing the energy needs for space cooling. However, they may also yield a reduction in the daylight availability, which results in the energy consumption for indoor artificial lighting being increased. This paper investigates, by means of dynamic simulations, the application of thermochromic windows to an existing office building in terms of energy savings on an annual basis, while also focusing on the effects in terms of daylighting and thermal comfort. In particular, due attention is paid to daylight availability, described through illuminance maps and by the calculation of the daylight factor, which in several countries is subject thresholds. The study considers both a commercially available thermochromic pane and a series of theoretical thermochromic glazing. The expected performance is compared to static clear and reflective insulating glass units. The simulations are repeated in different climatic conditions, showing that the overall energy savings compared to clear glazing can range from around 5% for cold climates to around 20% in warm climates, while not compromising daylight availability. Moreover the role played by the transition temperature of the pane is examined, pointing out an optimal transition temperatures that is irrespective of the climatic conditions.

Are visual threats prioritized without awareness? A critical review and meta analysis involving 3 behavioral paradigms and 2696 observers

Given capacity limits, only a subset of stimuli 1 give rise to a conscious percept. Neurocognitive models suggest that humans have evolved mechanisms that operate without awareness and prioritize threatening stimuli over neutral stimuli in subsequent perception. In this meta analysis, we review evidence for this ‘standard hypothesis’ emanating from three widely used, but rather different experimental paradigms that have been used to manipulate awareness. We found a small pooled threat-bias effect in the masked visual probe paradigm, a medium effect in the binocular rivalry paradigm and highly inconsistent effects in the breaking continuous flash suppression paradigm. Substantial heterogeneity was explained by the stimulus type: the only threat stimuli that were robustly prioritized across all three paradigms were fearful faces. Meta regression revealed that anxiety may modulate threat biases, but only under specific presentation conditions. We also found that insufficiently rigorous awareness measures, inadequate control of response biases and low level confounds may undermine claims of genuine unconscious threat processing. Considering the data together, we suggest that uncritical acceptance of the standard hypothesis is premature: current behavioral evidence for threat-sensitive visual processing that operates without awareness is weak.

Optimising mirid control on cocoa farms through complementary monitoring systems

Mirids (Sahlbergella singularis and Distantiella theobroma) are the most important insect pests affecting cocoa production across West Africa. Understanding the population dynamics of mirids is key to their management, however, the current recommended hand-height assessment method is labour intensive. The objective of the study was to compare recently developed mirid sex pheromone trapping and visual hand-height assessment methods as monitoring tools on cocoa farms and to consider implications for a decision support system. Ten farms from the Eastern and Ashanti regions of Ghana were used for the study. Mirid numbers and damage were assessed fortnightly on twenty trees per farm, using both methods, from January 2012 to April 2013. The mirid population increased rapidly in June, reached a peak in September and began to decline in October. There was a significant linear relationship between numbers of mirids sampled to hand-height and mirid damage. High numbers of male mirids were recorded in pheromone traps between January and April 2012 after which there was a gradual decline. There was a significant inverse relationship between numbers of trapped adult mirids and mirids sampled to hand-height (predominantly nymphs). Higher temperatures and lower relative humidities in the first half of the year were associated with fewer mirids at hand-height but larger numbers of adult males were caught in pheromone traps. The study showed that relying solely on one method is not sufficient to provide accurate information on mirid population dynamics and a combination of the two methods is necessary.

Automatic camera control in virtual environments augmented using multiple sparse videos

Automated virtual camera control has been widely used in animation and interactive virtual environments. We have developed a multiple sparse camera based free view video system prototype that allows users to control the position and orientation of a virtual camera, enabling the observation of a real scene in three dimensions (3D) from any desired viewpoint. Automatic camera control can be activated to follow selected objects by the user. Our method combines a simple geometric model of the scene composed of planes (virtual environment), augmented with visual information from the cameras and pre-computed tracking information of moving targets to generate novel perspective corrected 3D views of the virtual camera and moving objects. To achieve real-time rendering performance, view-dependent textured mapped billboards are used to render the moving objects at their correct locations and foreground masks are used to remove the moving objects from the projected video streams. The current prototype runs on a PC with a common graphics card and can generate virtual 2D views from three cameras of resolution 768 x 576 with several moving objects at about 11 fps. (C)2011 Elsevier Ltd. All rights reserved.

PAIN IN FIBROMYALGIA AND DISCRIMINATIVE POWER OF THE INSTRUMENTS: VISUAL ANALOG SCALE, DOLORIMETRY AND THE MCGILL PAIN QUESTIONNAIRE

Objective: The aim of this study was to verify the discriminative power of the most widely used pain assessment instruments. Methods: The sample consisted of 279 subjects divided into Fibromyalgia Group (FM- 205 patients with fibromyalgia) and Control Group (CG-74 healthy subjects), mean age 49.29 +/- 10.76 years. Only 9 subjects were male, 6 in FM and 3 in CG. FM were outpatients from the Rheumatology Clinic of the University of Sao Paulo - Hospital das Clinicas (HCFMUSP); the CG included people accompanying patients and hospital staff with similar socio-demographic characteristics. Three instruments were used to assess pain: the McGill Pain Questionnaire (MPQ), the Visual Analog Scale (VAS), and the Dolorimetry, to measure pain threshold on tender points (generating the TP index). In order to assess the discriminative power of the instruments, the measurements obtained were submitted to descriptive analysis and inferential analysis using ROC Curve - sensibility (S), specificity (S I) and area under the curve (AUC) - and Contingence tables with Chi-square Test and odds ratio. Significance level was 0.05. Results: Higher sensibility, specificity and area under the curve was obtained by VAS (80%, 80% and 0.864, respectively), followed by Dolorimetry (S 77%, S177% and AUC 0.851), McGill Sensory (S 72%, S167% and AUC 0.765) and McGill Affective (S 69%, S1 67% and AUC 0.753). Conclusions: VAS presented the higher sensibility, specificity and AUC, showing the greatest discriminative power among the instruments. However, these values are considerably similar to those of Dolorimetry.

Whole-body cortisol increases after direct and visual contact with a predator in zebrafish, Danio rerio

Ornamental fish may be severely affected by a stressful environment. Stressors impair the immune response, reproduction and growth rate; thus, the identification of possible stressors will aid to improve the overall quality of ornamental fish. The aim of this study was to determine whole-body cortisol of adult zebrafish, Danio rerio, following visual or direct contact with a predator species. Zebrafish were distributed in three groups: the first group, which consisted of zebrafish reared completely isolated of the predator, was considered the negative control; the second group, in which the predator, Parachromis managuensis was stocked together with zebrafish, was considered the positive control; the third group consisted of zebrafish stocked in a glass aquarium, with direct visual contact with the predator. The mean whole-body cortisol concentration in zebrafish from the negative control was 6.78 +/- 1.12 ng g(-1), a concentration statistically lower than that found in zebrafish having visual contact with the predator (9.26 +/- 0.88 ng g(-1)) which, in turn, was statistically lower than the mean whole-body cortisol of the positive control group (12.35 +/- 1.59 ng g(-1)). The higher whole-body cortisol concentration found in fish from the positive control can be attributed to the detection, by the zebrafish, of relevant risk situations that may involve a combination of chemical, olfactory and visual cues. One of the functions of elevated cortisol is to mobilize energy from body resources to cope with stress. The elevation of whole-body cortisol in fish subjected to visual contact with the predator involves only the visual cue in the recognition of predation risk. We hypothesized that the zebrafish could recognize predator characteristics in P managuensis, such as length, shape, color and behavior. Nonetheless, the elevation of whole-body cortisol in zebrafish suggested that the visual contact of the predator may elicit a stress response in prey fish. This assertion has a strong practical application concerning the species distribution in ornamental fish markets in which prey species should not be allowed to see predator species. Minimizing visual contact between prey and predator fish may improve the quality, viability and welfare of small fish in ornamental fish markets. (c) 2007 Elsevier B.V. All rights reserved.

Role of midbrain in the control of breathing in anuran amphibians

The present study was designed to explore systematically the midbrain of unanesthetized, decerebrate anuran amphibians (bullfrogs), using chemical and electrical stimulation and midbrain transections to identify sites capable of exciting and inhibiting breathing. Ventilation was measured as fictive motor output from the mandibular branch of the trigeminal nerve and the laryngeal branch of the vagus nerve. The results of our transection studies suggest that, under resting conditions, the net effect of inputs from sites within the rostral half of the midbrain is to increase fictive breathing frequency, whereas inputs from sites within the caudal half of the midbrain have no net effect on fictive breathing frequency but appear to act on the medullary central rhythm generator to produce episodic breathing. The results of our stimulation experiments indicate that the principal sites in the midbrain that are capable of exciting or inhibiting the fictive frequency of lung ventilation, and potentially clustering breaths into episodes, appear to be those primarily involved in visual and auditory integration, motor functions, and attentional state.

Navegação cooperativa de um robô humanóide e um robô com rodas usando informação visual

This work presents a cooperative navigation systemof a humanoid robot and a wheeled robot using visual information, aiming to navigate the non-instrumented humanoid robot using information obtained from the instrumented wheeled robot. Despite the humanoid not having sensors to its navigation, it can be remotely controlled by infra-red signals. Thus, the wheeled robot can control the humanoid positioning itself behind him and, through visual information, find it and navigate it. The location of the wheeled robot is obtained merging information from odometers and from landmarks detection, using the Extended Kalman Filter. The marks are visually detected, and their features are extracted by image processing. Parameters obtained by image processing are directly used in the Extended Kalman Filter. Thus, while the wheeled robot locates and navigates the humanoid, it also simultaneously calculates its own location and maps the environment (SLAM). The navigation is done through heuristic algorithms based on errors between the actual and desired pose for each robot. The main contribution of this work was the implementation of a cooperative navigation system for two robots based on visual information, which can be extended to other robotic applications, as the ability to control robots without interfering on its hardware, or attaching communication devices

Sistema de supervisão aérea baseado em navegação visual para detecção de anomalias em instalações de petróleo e gás

This work deals with the development of a prototype of a helicopter quadrotor for monitoring applications in oil facilities. Anomaly detection problems can be resolved through monitoringmissions performed by a suitably instrumented quadrotor, i.e. infrared thermosensors should be embedded. The proposed monitoring system aims to reduce accidents as well as to make possible the use of non-destructive techniques for detection and location of leaks caused by corrosion. To this end, the implementation of a prototype, its stabilization and a navigation strategy have been proposed. The control strategy is based on dividing the problem into two control hierarchical levels: the lower level stabilizes the angles and the altitude of the vehicle at the desired values, while the higher one provide appropriate references signals to the lower level in order the quadrotor performs the desired movements. The navigation strategy for helicopter quadrotor is made using information provided by a acquisition image system (monocular camera) embedded onto the helicopter. Considering that the low-level control has been solved, the proposed vision-based navigation technique treats the problem as high level control strategies, such as, relative position control, trajectory generation and trajectory tracking. For the position control we use a control technique for visual servoing based on image features. The trajectory generation is done in a offline step, which is a visual trajectory composed of a sequence of images. For the trajectory tracking problem is proposed a control strategy by continuous servovision, thus enabling a navigation strategy without metric maps. Simulation and experimental results are presented to validate the proposal

Desenvolvimento de esquema de controle com realimentação visual para um robô manipulador

This work proposes a kinematic control scheme, using visual feedback for a robot arm with five degrees of freedom. Using computational vision techniques, a method was developed to determine the cartesian 3d position and orientation of the robot arm (pose) using a robot image obtained through a camera. A colored triangular label is disposed on the robot manipulator tool and efficient heuristic rules are used to obtain the vertexes of that label in the image. The tool pose is obtained from those vertexes through numerical methods. A color calibration scheme based in the K-means algorithm was implemented to guarantee the robustness of the vision system in the presence of light variations. The extrinsic camera parameters are computed from the image of four coplanar points whose cartesian 3d coordinates, related to a fixed frame, are known. Two distinct poses of the tool, initial and final, obtained from image, are interpolated to generate a desired trajectory in cartesian space. The error signal in the proposed control scheme consists in the difference between the desired tool pose and the actual tool pose. Gains are applied at the error signal and the signal resulting is mapped in joint incrementals using the pseudoinverse of the manipulator jacobian matrix. These incrementals are applied to the manipulator joints moving the tool to the desired pose

Controle de Manipulador Redundante Utilizando Realimentação Visual

In conventional robot manipulator control, the desired path is specified in cartesian space and converted to joint space through inverse kinematics mapping. The joint references generated by this mapping are utilized for dynamic control in joint space. Thus, the end-effector position is, in fact, controlled indirectly, in open-loop, and the accuracy of grip position control directly depends on the accuracy of the available kinematic model. In this report, a new scheme for redundant manipulator kinematic control, based on visual servoing is proposed. In the proposed system, a robot image acquired through a CCD camera is processed in order to compute the position and orientation of each link of the robot arm. The robot task is specified as a temporal sequence of reference images of the robot arm. Thus, both the measured pose and the reference pose are specified in the same image space, and its difference is utilized to generate a cartesian space error for kinematic control purposes. The proposed control scheme was applied in a four degree-of-freedom planar redundant robot arm, experimental results are shown

Dynamics of inter-modality re-weighting during human postural control

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)