Real Time Eye Tracking and Blink Detection with USB Cameras

A human-computer interface (HCI) system designed for use by people with severe disabilities is presented. People that are severely paralyzed or afflicted with diseases such as ALS (Lou Gehrig's disease) or multiple sclerosis are unable to move or control any parts of their bodies except for their eyes. The system presented here detects the user's eye blinks and analyzes the pattern and duration of the blinks, using them to provide input to the computer in the form of a mouse click. After the automatic initialization of the system occurs from the processing of the user's involuntary eye blinks in the first few seconds of use, the eye is tracked in real time using correlation with an online template. If the user's depth changes significantly or rapid head movement occurs, the system is automatically reinitialized. There are no lighting requirements nor offline templates needed for the proper functioning of the system. The system works with inexpensive USB cameras and runs at a frame rate of 30 frames per second. Extensive experiments were conducted to determine both the system's accuracy in classifying voluntary and involuntary blinks, as well as the system's fitness in varying environment conditions, such as alternative camera placements and different lighting conditions. These experiments on eight test subjects yielded an overall detection accuracy of 95.3%.

Elicitation techniques for the evaluation of speech under stress: Towards a framework for use in a controlled environment

Existing work in Computer Science and Electronic Engineering demonstrates that Digital Signal Processing techniques can effectively identify the presence of stress in the speech signal. These techniques use datasets containing real or actual stress samples i.e. real-life stress such as 911 calls and so on. Studies that use simulated or laboratory-induced stress have been less successful and inconsistent. Pervasive, ubiquitous computing is increasingly moving towards voice-activated and voice-controlled systems and devices. Speech recognition and speaker identification algorithms will have to improve and take emotional speech into account. Modelling the influence of stress on speech and voice is of interest to researchers from many different disciplines including security, telecommunications, psychology, speech science, forensics and Human Computer Interaction (HCI). The aim of this work is to assess the impact of moderate stress on the speech signal. In order to do this, a dataset of laboratory-induced stress is required. While attempting to build this dataset it became apparent that reliably inducing measurable stress in a controlled environment, when speech is a requirement, is a challenging task. This work focuses on the use of a variety of stressors to elicit a stress response during tasks that involve speech content. Biosignal analysis (commercial Brain Computer Interfaces, eye tracking and skin resistance) is used to verify and quantify the stress response, if any. This thesis explains the basis of the author’s hypotheses on the elicitation of affectively-toned speech and presents the results of several studies carried out throughout the PhD research period. These results show that the elicitation of stress, particularly the induction of affectively-toned speech, is not a simple matter and that many modulating factors influence the stress response process. A model is proposed to reflect the author’s hypothesis on the emotional response pathways relating to the elicitation of stress with a required speech content. Finally the author provides guidelines and recommendations for future research on speech under stress. Further research paths are identified and a roadmap for future research in this area is defined.

Context-aware real-time assistant architecture for pervasive healthcare

The aging population in many countries brings into focus rising healthcare costs and pressure on conventional healthcare services. Pervasive healthcare has emerged as a viable solution capable of providing a technology-driven approach to alleviate such problems by allowing healthcare to move from the hospital-centred care to self-care, mobile care, and at-home care. The state-of-the-art studies in this field, however, lack a systematic approach for providing comprehensive pervasive healthcare solutions from data collection to data interpretation and from data analysis to data delivery. In this thesis we introduce a Context-aware Real-time Assistant (CARA) architecture that integrates novel approaches with state-of-the-art technology solutions to provide a full-scale pervasive healthcare solution with the emphasis on context awareness to help maintaining the well-being of elderly people. CARA collects information about and around the individual in a home environment, and enables accurately recognition and continuously monitoring activities of daily living. It employs an innovative reasoning engine to provide accurate real-time interpretation of the context and current situation assessment. Being mindful of the use of the system for sensitive personal applications, CARA includes several mechanisms to make the sophisticated intelligent components as transparent and accountable as possible, it also includes a novel cloud-based component for more effective data analysis. To deliver the automated real-time services, CARA supports interactive video and medical sensor based remote consultation. Our proposal has been validated in three application domains that are rich in pervasive contexts and real-time scenarios: (i) Mobile-based Activity Recognition, (ii) Intelligent Healthcare Decision Support Systems and (iii) Home-based Remote Monitoring Systems.

Temporal variations of IEEE 802.11b signal strengths in an in-building environment

Recent popularity of the IEEE 802.11b Wireless Local Area Networks (WLANs) in a host of current-day applications has instigated a suite of research challenges. The 802.11b WLANs are highly reliable and wide spread. In this work, we study the temporal characteristics of RSSI in the real-working environment by conducting a controlled set of experiments. Our results indicate that a significant variability in the RSSI can occur over time. Some of this variability in the RSSI may be due to systematic causes while the other component can be expressed as stochastic noise. We present an analysis of both these aspects of RSSI. We treat the moving average of the RSSI as the systematic causes and the noise as the stochastic causes. We give a reasonable estimate for the moving average to compute the noise accurately. We attribute the changes in the environment such as the movement of people and the noise associated with the NIC circuitry and the network access point as causes for this variability. We find that the results of our analysis are of primary importance to active research areas such as location determination of users in a WLAN. The techniques used in some of the RF-based WLAN location determination systems, exploit the characteristics of the RSSI presented in this work to infer the location of a wireless client in a WLAN. Thus our results form the building blocks for other users of the exact characteristics of the RSSI.

Bound on quantum computation time: Quantum error correction in a critical environment

We obtain an upper bound on the time available for quantum computation for a given quantum computer and decohering environment with quantum error correction implemented. First, we derive an explicit quantum evolution operator for the logical qubits and show that it has the same form as that for the physical qubits but with a reduced coupling strength to the environment. Using this evolution operator, we find the trace distance between the real and ideal states of the logical qubits in two cases. For a super-Ohmic bath, the trace distance saturates, while for Ohmic or sub-Ohmic baths, there is a finite time before the trace distance exceeds a value set by the user. © 2010 The American Physical Society.

An environment for OpenMP code parallelization

This chapter discusses the code parallelization environment, where a number of tools that address the main tasks, such as code parallelization, debugging, and optimization are available. The parallelization tools include ParaWise and CAPO, which enable the near automatic parallelization of real world scientific application codes for shared and distributed memory-based parallel systems. The chapter discusses the use of ParaWise and CAPO to transform the original serial code into an equivalent parallel code that contains appropriate OpenMP directives. Additionally, as user involvement can introduce errors, a relative debugging tool (P2d2) is also available and can be used to perform near automatic relative debugging of an OpenMP program that has been parallelized either using the tools or manually. In order for these tools to be effective in parallelizing a range of applications, a high quality fully inter-procedural dependence analysis, as well as user interaction is vital to the generation of efficient parallel code and in the optimization of the backtracking and speculation process used in relative debugging. Results of parallelized NASA codes are discussed and show the benefits of using the environment.

Real-time sea surface air temperature retrieval over Southern Ocean using AMSR-E data

The AMSR-E satellite data and in-situ data were applied to retrieve sea surface air temperature (Ta) over the Southern Ocean. The in-situ data were obtained from the 24~(th) -26~(th) Chinese Antarctic Expeditions during 2008-2010. First, Ta was used to analyze the relativity with the bright temperature (Tb) from the twelve channels of AMSR-E, and no high relativity was found between Ta and Tb from any of the channels. The highest relativity was 0.38 (with 23.8 GHz). The dataset for the modeling was obtained by using in-situ data to match up with Tb, and two methods were applied to build the retrieval model. In multi-parameters regression method, the Tbs from 12 channels were used to the model and the region was divided into two parts according to the latitude of 50°S. The retrieval results were compared with the in-situ data. The Root Mean Square Error (RMS) and relativity of high latitude zone were 0.96℃and 0.93, respectively. And those of low latitude zone were 1.29 ℃ and 0.96, respectively. Artificial neural network (ANN) method was applied to retrieve Ta.The RMS and relativity were 1.26 ℃ and 0.98, respectively.

Can benthic community structure be used to predict the process of bioturbation in real ecosystems

Disentangling the roles of environmental change and natural environmental variability on biologically mediated ecosystem processes is paramount to predict future marine ecosystem functioning. Bioturbation, the biogenic mixing of sediments, has a regulating role in marine biogeochemical processes. However, our understanding of bioturbation as a community level process and of its environmental drivers is still limited by loose use of terminology, and a lack of consensus about what bioturbation is. To help resolve these challenges, this empirical study investigated the links between four different attributes of bioturbation (bioturbation depth, activity and distance, and biodiffusive transport); the ability of an index of bioturbation (BPc) to predict each of them; and their relation to seasonality, in a shallow coastal system – the Western Channel Observatory, UK. Bioturbation distance depended on changes in benthic community structure, while the other three attributes were more directly influenced by seasonality in food availability. In parallel, BPc successfully predicted bioturbation distance but not the other attributes of bioturbation. This study therefore highlights that community bioturbation results from this combination of processes responding to environmental variability at different time-scales. However, community level measurements of bioturbation across environmental variability are still scarce, and BPc is calculated using commonly available data on benthic community structure and the functional classification of invertebrates. Therefore, BPc could be used to support the growth of landscape scale bioturbation research, but future uses of the index need to consider which bioturbation attributes the index actually predicts. As BPc predicts bioturbation distance, estimated here using a random-walk model applicable to community settings, studies using either of the metrics should be directly comparable and contribute to a more integrated future for bioturbation research.

Real-time error detection but not error correction drives automatic visuomotor adaptation

We investigated the role of visual feedback of task performance in visuomotor adaptation. Participants produced novel two degrees of freedom movements (elbow flexion-extension, forearm pronation-supination) to move a cursor towards visual targets. Following trials with no rotation, participants were exposed to a 60A degrees visuomotor rotation, before returning to the non-rotated condition. A colour cue on each trial permitted identification of the rotated/non-rotated contexts. Participants could not see their arm but received continuous and concurrent visual feedback (CF) of a cursor representing limb position or post-trial visual feedback (PF) representing the movement trajectory. Separate groups of participants who received CF were instructed that online modifications of their movements either were, or were not, permissible as a means of improving performance. Feedforward-mediated performance improvements occurred for both CF and PF groups in the rotated environment. Furthermore, for CF participants this adaptation occurred regardless of whether feedback modifications of motor commands were permissible. Upon re-exposure to the non-rotated environment participants in the CF, but not PF, groups exhibited post-training aftereffects, manifested as greater angular deviations from a straight initial trajectory, with respect to the pre-rotation trials. Accordingly, the nature of the performance improvements that occurred was dependent upon the timing of the visual feedback of task performance. Continuous visual feedback of task performance during task execution appears critical in realising automatic visuomotor adaptation through a recalibration of the visuomotor mapping that transforms visual inputs into appropriate motor commands.

Comparison of passive millimeter-wave and IR imagery in a nautical environment

Results are presented from a trial in which a real-time passive millimetre-wave camera was mounted on a landing craft. The vessel was operated on rivers in the UK, and imagery of surrounding terrain, structures, obstacles and other vessels was obtained. An IR camera was also used, and the differences in signatures of various features are discussed. Opportunities for image fusion are highlighted.

Keeping it real! Enhancing realism in standardised patient OSCE stations

Background: Objective structured clinical examinations (OSCEs) are a
commonly used method of assessing clinical competency in healthcare education. They can providean opportunity to observe candidates interacting with patients.
There are many challenges in using real patients in OSCEs, and increasingly standardised patients are being used as a preference. However, by using standardised patients there is a risk of making the encounter arti?cial and removed from actual clinical practice.
Context: Efforts made in terms of cognitive, auditory, visual, tactile, psychological and emotional cues can minimise the differences between a simulated
and real clinical scenario. However, a number of factors, including feasibility, cost and usability, need to be considered if such techniques are to be practicable
within an OSCE framework.
Innovation: This article describes a series of techniques that have been used in our institution to enhance the realism of a standardised patient encounter in an
OSCE. Efforts in preparing standardised patient roles, and how they portray these roles, will be considered. A wide variety of equipment can also be used in
combination with a patient and the surrounding environment, which can further enhance the authenticity of the simulated scenario.
Implications: By enhancing the realism in simulated patient OSCE encounters, there is potential to trigger more authentic conscious responses from candidates and implicit reactions that the candidates themselves may be less
aware of. Furthermore, using such techniques may allow faculty members to select scenarios that were previously not thought possible in an OSCE

Real-time GPU color-based segmentation of football players

In this paper, we propose a multi-camera application capable of processing high resolution images and extracting features based on colors patterns over graphic processing units (GPU). The goal is to work in real time under the uncontrolled environment of a sport event like a football match. Since football players are composed for diverse and complex color patterns, a Gaussian Mixture Models (GMM) is applied as segmentation paradigm, in order to analyze sport live images and video. Optimization techniques have also been applied over the C++ implementation using profiling tools focused on high performance. Time consuming tasks were implemented over NVIDIA's CUDA platform, and later restructured and enhanced, speeding up the whole process significantly. Our resulting code is around 4-11 times faster on a low cost GPU than a highly optimized C++ version on a central processing unit (CPU) over the same data. Real time has been obtained processing until 64 frames per second. An important conclusion derived from our study is the scalability of the application to the number of cores on the GPU. © 2011 Springer-Verlag.

An integrated approach for real-time model-based state-of-charge estimation of lithium-ion batteries

Lithium-ion batteries have been widely adopted in electric vehicles (EVs), and accurate state of charge (SOC) estimation is of paramount importance for the EV battery management system. Though a number of methods have been proposed, the SOC estimation for Lithium-ion batteries, such as LiFePo4 battery, however, faces two key challenges: the flat open circuit voltage (OCV) vs SOC relationship for some SOC ranges and the hysteresis effect. To address these problems, an integrated approach for real-time model-based SOC estimation of Lithium-ion batteries is proposed in this paper. Firstly, an auto-regression model is adopted to reproduce the battery terminal behaviour, combined with a non-linear complementary model to capture the hysteresis effect. The model parameters, including linear parameters and non-linear parameters, are optimized off-line using a hybrid optimization method that combines a meta-heuristic method (i.e., the teaching learning based optimization method) and the least square method. Secondly, using the trained model, two real-time model-based SOC estimation methods are presented, one based on the real-time battery OCV regression model achieved through weighted recursive least square method, and the other based on the state estimation using the extended Kalman filter method (EKF). To tackle the problem caused by the flat OCV-vs-SOC segments when the OCV-based SOC estimation method is adopted, a method combining the coulombic counting and the OCV-based method is proposed. Finally, modelling results and SOC estimation results are presented and analysed using the data collected from LiFePo4 battery cell. The results confirmed the effectiveness of the proposed approach, in particular the joint-EKF method.

Virtual Thrower Versus Real Goalkeeper: The Influence of Different Visual Conditions on Performance

In order to use virtual reality as a sport analysis tool, we need to be sure that an immersed athlete reacts realistically in a virtual environment. This has been validated for a real handball goalkeeper facing a virtual thrower. However, we currently ignore which visual variables induce a realistic motor behavior of the immersed handball goalkeeper. In this study, we used virtual reality to dissociate the visual information related to the movements of the player from the visual information related to the trajectory of the ball. Thus, the aim is to evaluate the relative influence of these different visual information sources on the goalkeeper's motor behavior. We tested 10 handball goalkeepers who had to predict the final position of the virtual ball in the goal when facing the following: only the throwing action of the attacking player (TA condition), only the resulting ball trajectory (BA condition), and both the throwing action of the attacking player and the resulting ball trajectory (TB condition). Here we show that performance was better in the BA and TB conditions, but contrary to expectations, performance was substantially worse in the TA condition. A significant effect of ball landing zone does, however, suggest that the relative importance between visual information from the player and the ball depends on the targeted zone in the goal. In some cases, body-based cues embedded in the throwing actions may have a minor influence on the ball trajectory and vice versa. Kinematics analysis was then combined with these results to determine why such differences occur depending on the ball landing zone and consequently how it can clarify the role of different sources of visual information on the motor behavior of an athlete immersed in a virtual environment.