Enhancing Activity Recognition by Fusing Inertial and Biometric Information

Activity recognition is an active research field nowadays, as it enables the development of highly adaptive applications, e.g. in the field of personal health. In this paper, a light high-level fusion algorithm to detect the activity that an individual is performing is presented. The algorithm relies on data gathered from accelerometers placed on different parts of the body, and on biometric sensors. Inertial sensors allow detecting activity by analyzing signal features such as amplitude or peaks. In addition, there is a relationship between the activity intensity and biometric response, which can be considered together with acceleration data to improve the accuracy of activity detection. The proposed algorithm is designed to work with minimum computational cost, being ready to run in a mobile device as part of a context-aware application. In order to enable different user scenarios, the algorithm offers best-effort activity estimation: its quality of estimation depends on the position and number of the available inertial sensors, and also on the presence of biometric information.

Analysis of pattern recognition techniques for in-air signature biometrics

As a result of advances in mobile technology, new services which benefit from the ubiquity of these devices are appearing. Some of these services require the identification of the subject since they may access private user information. In this paper, we propose to identify each user by drawing his/her handwritten signature in the air (in-airsignature). In order to assess the feasibility of an in-airsignature as a biometric feature, we have analysed the performance of several well-known patternrecognitiontechniques—Hidden Markov Models, Bayes classifiers and dynamic time warping—to cope with this problem. Each technique has been tested in the identification of the signatures of 96 individuals. Furthermore, the robustness of each method against spoofing attacks has also been analysed using six impostors who attempted to emulate every signature. The best results in both experiments have been reached by using a technique based on dynamic time warping which carries out the recognition by calculating distances to an average template extracted from several training instances. Finally, a permanence analysis has been carried out in order to assess the stability of in-airsignature over time.

Dysfunctional and compensatory duality in mild cognitive impairment during a continuous recognition memory task

One of the current issues of debate in the study of mild cognitive impairment (MCI) is deviations of oscillatory brain responses from normal brain states and its dynamics. This work aims to characterize the differences of power in brain oscillations during the execution of a recognition memory task in MCI subjects in comparison with elderly controls. Magnetoencephalographic (MEG) signals were recorded during a continuous recognition memory task performance. Oscillatory brain activity during the recognition phase of the task was analyzed by wavelet transform in the source space by means of minimum norm algorithm. Both groups obtained a 77% hit ratio. In comparison with healthy controls, MCI subjects showed increased theta (p < 0.001), lower beta reduction (p < 0.001) and decreased alpha and gamma power (p < 0.002 and p < 0.001 respectively) in frontal, temporal and parietal areas during early and late latencies. Our results point towards a dual pattern of activity (increase and decrease) which is indicative of MCI and specific to certain time windows, frequency bands and brain regions. These results could represent two neurophysiological sides of MCI. Characterizing these opposing processes may contribute to the understanding of the disorder.

A Satisfaction-based Model for Affect Recognition from Conversational Features in Spoken Dialog Systems

Detecting user affect automatically during real-time conversation is the main challenge towards our greater aim of infusing social intelligence into a natural-language mixed-initiative High-Fidelity (Hi-Fi) audio control spoken dialog agent. In recent years, studies on affect detection from voice have moved on to using realistic, non-acted data, which is subtler. However, it is more challenging to perceive subtler emotions and this is demonstrated in tasks such as labelling and machine prediction. This paper attempts to address part of this challenge by considering the role of user satisfaction ratings and also conversational/dialog features in discriminating contentment and frustration, two types of emotions that are known to be prevalent within spoken human-computer interaction. However, given the laboratory constraints, users might be positively biased when rating the system, indirectly making the reliability of the satisfaction data questionable. Machine learning experiments were conducted on two datasets, users and annotators, which were then compared in order to assess the reliability of these datasets. Our results indicated that standard classifiers were significantly more successful in discriminating the abovementioned emotions and their intensities (reflected by user satisfaction ratings) from annotator data than from user data. These results corroborated that: first, satisfaction data could be used directly as an alternative target variable to model affect, and that they could be predicted exclusively by dialog features. Second, these were only true when trying to predict the abovementioned emotions using annotator?s data, suggesting that user bias does exist in a laboratory-led evaluation.

User-independent accelerometer-based gesture recognition for mobile devices

Many mobile devices embed nowadays inertial sensors. This enables new forms of human-computer interaction through the use of gestures (movements performed with the mobile device) as a way of communication. This paper presents an accelerometer-based gesture recognition system for mobile devices which is able to recognize a collection of 10 different hand gestures. The system was conceived to be light and to operate in a user -independent manner in real time. The recognition system was implemented in a smart phone and evaluated through a collection of user tests, which showed a recognition accuracy similar to other state-of-the art techniques and a lower computational complexity. The system was also used to build a human -robot interface that enables controlling a wheeled robot with the gestures made with the mobile phone.

Towards a fuzzy-based multi-classifier selection module for activity recognition applications

Performing activity recognition using the information provided by the different sensors embedded in a smartphone face limitations due to the capabilities of those devices when the computations are carried out in the terminal. In this work a fuzzy inference module is implemented in order to decide which classifier is the most appropriate to be used at a specific moment regarding the application requirements and the device context characterized by its battery level, available memory and CPU load. The set of classifiers that is considered is composed of Decision Tables and Trees that have been trained using different number of sensors and features. In addition, some classifiers perform activity recognition regardless of the on-body device position and others rely on the previous recognition of that position to use a classifier that is trained with measurements gathered with the mobile placed on that specific position. The modules implemented show that an evaluation of the classifiers allows sorting them so the fuzzy inference module can choose periodically the one that best suits the device context and application requirements.

Authentication in mobile devices through hand gesture recognition

This article proposes an innovative biometric technique based on the idea of authenticating a person on a mobile device by gesture recognition. To accomplish this aim, a user is prompted to be recognized by a gesture he/she performs moving his/her hand while holding a mobile device with an accelerometer embedded. As users are not able to repeat a gesture exactly in the air, an algorithm based on sequence alignment is developed to correct slight differences between repetitions of the same gesture. The robustness of this biometric technique has been studied within 2 different tests analyzing a database of 100 users with real falsifications. Equal Error Rates of 2.01 and 4.82% have been obtained in a zero-effort and an active impostor attack, respectively. A permanence evaluation is also presented from the analysis of the repetition of the gestures of 25 users in 10 sessions over a month. Furthermore, two different gesture databases have been developed: one made up of 100 genuine identifying 3-D hand gestures and 3 impostors trying to falsify each of them and another with 25 volunteers repeating their identifying 3- D hand gesture in 10 sessions over a month. These databases are the most extensive in published studies, to the best of our knowledge.

Age-Related Effects in Working Memory Recognition Modulated by Retroactive Interference

One of the main causes for age-related declines in working memory is a higher vulnerability to retroactive interference due to a reduced ability to suppress irrelevant information. However, the underlying neural correlates remain to be established. Magnetoencephalography was used to investigate differential neural patterns in young and older adults performing an interference-based memory task with two experimental conditions, interrupting and distracting, during successful recognition. Behaviorally, both types of retroactive interference significantly impaired accuracy at recognition more in older adults than in young adults with the latter exhibiting greater disruptions by interrupters. Magnetoencephalography revealed the presence of differential age-related neural patterns. Specifically, time-modulated activations in temporo-occipital and superior parietal regions were higher in young adults compared with older adults for the interrupting condition. These results suggest that age-related deficits in inhibitory mechanisms that increase vulnerability to retroactive interference may be associated with neural under-recruitments in a high-interference task.

Phonotactic language recognition using i-vectors and phoneme posteriogram counts

This paper describes a novel approach to phonotactic LID, where instead of using soft-counts based on phoneme lattices, we use posteriogram to obtain n-gram counts. The high-dimensional vectors of counts are reduced to low-dimensional units for which we adapted the commonly used term i-vectors. The reduction is based on multinomial subspace modeling and is designed to work in the total-variability space. The proposed technique was tested on the NIST 2009 LRE set with better results to a system based on using soft-counts (Cavg on 30s: 3.15% vs 3.43%), and with very good results when fused with an acoustic i-vector LID system (Cavg on 30s acoustic 2.4% vs 1.25%). The proposed technique is also compared with another low dimensional projection system based on PCA. In comparison with the original soft-counts, the proposed technique provides better results, reduces the problems due to sparse counts, and avoids the process of using pruning techniques when creating the lattices.

Architecture for Text Sign Localization and Recognition

This paper describes a low complexity strategy for detecting and recognizing text signs automatically. Traditional approaches use large image algorithms for detecting the text sign, followed by the application of an Optical Character Recognition (OCR) algorithm in the previously identified areas. This paper proposes a new architecture that applies the OCR to a whole lightly treated image and then carries out the text detection process of the OCR output. The strategy presented in this paper significantly reduces the processing time required for text localization in an image, while guaranteeing a high recognition rate. This strategy will facilitate the incorporation of video processing-based applications into the automatic detection of text sign similar to that of a smartphone. These applications will increase the autonomy of visually impaired people in their daily life.

Gesture recognition using mobile phone's inertial sensors

The availability of inertial sensors embedded in mobile devices has enabled a new type of interaction based on the movements or “gestures” made by the users when holding the device. In this paper we propose a gesture recognition system for mobile devices based on accelerometer and gyroscope measurements. The system is capable of recognizing a set of predefined gestures in a user-independent way, without the need of a training phase. Furthermore, it was designed to be executed in real-time in resource-constrained devices, and therefore has a low computational complexity. The performance of the system is evaluated offline using a dataset of gestures, and also online, through some user tests with the system running in a smart phone.

Mars: a personalised mobile activity recognition system

Mobile activity recognition focuses on inferring the current activities of a mobile user by leveraging the sensory data that is available on today’s smart phones. The state of the art in mobile activity recognition uses traditional classification learning techniques. Thus, the learning process typically involves: i) collection of labelled sensory data that is transferred and collated in a centralised repository; ii) model building where the classification model is trained and tested using the collected data; iii) a model deployment stage where the learnt model is deployed on-board a mobile device for identifying activities based on new sensory data. In this paper, we demonstrate the Mobile Activity Recognition System (MARS) where for the first time the model is built and continuously updated on-board the mobile device itself using data stream mining. The advantages of the on-board approach are that it allows model personalisation and increased privacy as the data is not sent to any external site. Furthermore, when the user or its activity profile changes MARS enables promptly adaptation. MARS has been implemented on the Android platform to demonstrate that it can achieve accurate mobile activity recognition. Moreover, we can show in practise that MARS quickly adapts to user profile changes while at the same time being scalable and efficient in terms of consumption of the device resources.

Fusion of probabilistic knowledge-based classification rules and learning automata for automatic recognition of digital images

In this paper, the fusion of probabilistic knowledge-based classification rules and learning automata theory is proposed and as a result we present a set of probabilistic classification rules with self-learning capability. The probabilities of the classification rules change dynamically guided by a supervised reinforcement process aimed at obtaining an optimum classification accuracy. This novel classifier is applied to the automatic recognition of digital images corresponding to visual landmarks for the autonomous navigation of an unmanned aerial vehicle (UAV) developed by the authors. The classification accuracy of the proposed classifier and its comparison with well-established pattern recognition methods is finally reported.

Minimal template size for Iris-Recognition

A method to achieve improvement in template size for an iris-recognition system is reported. To achieve this result, the biological characteristics of the human iris have been studied. Processing has been performed by image processing techniques, isolating the iris and enhancing the area of study, after which multi resolution analysis is made. Reduction of the pattern obtained has been obtained via statistical study.