Towards Emotional Interaction: Using Movies to Automatically Learn Users' Emotional States

The HCI community is actively seeking novel methodologies to gain insight into the user’s experience during interaction with both the application and the content. We propose an emotional recognition engine capable of automatically recognizing a set of human emotional states using psychophysiological measures of the autonomous nervous system, including galvanic skin response, respiration, and heart rate. A novel pattern recognition system, based on discriminant analysis and support vector machine classifiers is trained using movies’ scenes selected to induce emotions ranging from the positive to the negative valence dimension, including happiness, anger, disgust, sadness, and fear. In this paper we introduce an emotion recognition system and evaluate its accuracy by presenting the results of an experiment conducted with three physiologic sensors.

Inperlys – independente personal location system

O desenvolvimento de sistemas de localização pedestre com recurso a técnicas de dead reckoning tem mostrado ser uma área em expansão no mundo académico e não só. Existem algumas soluções criadas, no entanto, nem todas as soluções serão facilmente implementadas no mercado, quer seja pelo hardware caro, ou pelo sistema em si, que é desenvolvido tendo em conta um cenário em particular. INPERLYS é um sistema que visa apresentar uma solução de localização pedestre, independentemente do cenário, utilizando recursos que poderão ser facilmente usados. Trata-se de um sistema que utiliza uma técnica de dead reckonig para dar a localização do utilizador. Em cenários outdoor, um receptor GPS fornece a posição do utilizador, fornecendo uma posição absoluta ao sistema. Quando não é possível utilizar o GPS, recorre-se a um sensor MEMS e a uma bússola para se obter posições relativas à última posição válida do GPS. Para interligar todos os sensores foi utilizado o protocolo de comunicações sem fios ZigBee™. A escolha recaiu neste protocolo devido a factores como os seus baixos consumos e o seu baixo custo. Assim o sistema torna-se de uso fácil e confortável para o utilizador, ao contrário de sistemas similares desenvolvidos, que utilizam cabos para interligarem os diferentes componentes do sistema. O sensor MEMS do tipo acelerómetro tem a função de ler a aceleração horizontal, ao nível do pé. Esta aceleração será usada por um algoritmo de reconhecimento do padrão das acelerações para se detectar os passos dados. Após a detecção do passo, a aceleração máxima registada nesse passo é fornecida ao coordenador, para se obter o deslocamento efectuado. Foram efectuados alguns testes para se perceber a eficiência do INPERLYS. Os testes decorreram num percurso plano, efectuados a uma velocidade normal e com passadas normais. Verificou-se que, neste momento, o desempenho do sistema poderá ser melhorado, quer seja a nível de gestão das comunicações, quer a nível do reconhecimento do padrão da aceleração horizontal, essencial para se detectar os passos. No entanto o sistema é capaz de fornecer a posição através do GPS, quando é possível a sua utilização, e é capaz de fornecer a orientação do movimento.

Hand gesture recognition system based in computer vision and machine learning : applications on human-machine interaction

Sendo uma forma natural de interação homem-máquina, o reconhecimento de gestos implica uma forte componente de investigação em áreas como a visão por computador e a aprendizagem computacional. O reconhecimento gestual é uma área com aplicações muito diversas, fornecendo aos utilizadores uma forma mais natural e mais simples de comunicar com sistemas baseados em computador, sem a necessidade de utilização de dispositivos extras. Assim, o objectivo principal da investigação na área de reconhecimento de gestos aplicada à interacção homemmáquina é o da criação de sistemas, que possam identificar gestos específicos e usálos para transmitir informações ou para controlar dispositivos. Para isso as interfaces baseados em visão para o reconhecimento de gestos, necessitam de detectar a mão de forma rápida e robusta e de serem capazes de efetuar o reconhecimento de gestos em tempo real. Hoje em dia, os sistemas de reconhecimento de gestos baseados em visão são capazes de trabalhar com soluções específicas, construídos para resolver um determinado problema e configurados para trabalhar de uma forma particular. Este projeto de investigação estudou e implementou soluções, suficientemente genéricas, com o recurso a algoritmos de aprendizagem computacional, permitindo a sua aplicação num conjunto alargado de sistemas de interface homem-máquina, para reconhecimento de gestos em tempo real. A solução proposta, Gesture Learning Module Architecture (GeLMA), permite de forma simples definir um conjunto de comandos que pode ser baseado em gestos estáticos e dinâmicos e que pode ser facilmente integrado e configurado para ser utilizado numa série de aplicações. É um sistema de baixo custo e fácil de treinar e usar, e uma vez que é construído unicamente com bibliotecas de código. As experiências realizadas permitiram mostrar que o sistema atingiu uma precisão de 99,2% em termos de reconhecimento de gestos estáticos e uma precisão média de 93,7% em termos de reconhecimento de gestos dinâmicos. Para validar a solução proposta, foram implementados dois sistemas completos. O primeiro é um sistema em tempo real capaz de ajudar um árbitro a arbitrar um jogo de futebol robótico. A solução proposta combina um sistema de reconhecimento de gestos baseada em visão com a definição de uma linguagem formal, o CommLang Referee, à qual demos a designação de Referee Command Language Interface System (ReCLIS). O sistema identifica os comandos baseados num conjunto de gestos estáticos e dinâmicos executados pelo árbitro, sendo este posteriormente enviado para um interface de computador que transmite a respectiva informação para os robôs. O segundo é um sistema em tempo real capaz de interpretar um subconjunto da Linguagem Gestual Portuguesa. As experiências demonstraram que o sistema foi capaz de reconhecer as vogais em tempo real de forma fiável. Embora a solução implementada apenas tenha sido treinada para reconhecer as cinco vogais, o sistema é facilmente extensível para reconhecer o resto do alfabeto. As experiências também permitiram mostrar que a base dos sistemas de interação baseados em visão pode ser a mesma para todas as aplicações e, deste modo facilitar a sua implementação. A solução proposta tem ainda a vantagem de ser suficientemente genérica e uma base sólida para o desenvolvimento de sistemas baseados em reconhecimento gestual que podem ser facilmente integrados com qualquer aplicação de interface homem-máquina. A linguagem formal de definição da interface pode ser redefinida e o sistema pode ser facilmente configurado e treinado com um conjunto de gestos diferentes de forma a serem integrados na solução final.

Hyperspectral imagery framework for unmixing and dimensionality estimation

In hyperspectral imagery a pixel typically consists mixture of spectral signatures of reference substances, also called endmembers. Linear spectral mixture analysis, or linear unmixing, aims at estimating the number of endmembers, their spectral signatures, and their abundance fractions. This paper proposes a framework for hyperpsectral unmixing. A blind method (SISAL) is used for the estimation of the unknown endmember signature and their abundance fractions. This method solve a non-convex problem by a sequence of augmented Lagrangian optimizations, where the positivity constraints, forcing the spectral vectors to belong to the convex hull of the endmember signatures, are replaced by soft constraints. The proposed framework simultaneously estimates the number of endmembers present in the hyperspectral image by an algorithm based on the minimum description length (MDL) principle. Experimental results on both synthetic and real hyperspectral data demonstrate the effectiveness of the proposed algorithm.

Structured Light System Calibration for Perception in Underwater Tanks

The process of visually exploring underwater environments is still a complex problem. Underwater vision systems require complementary means of sensor information to help overcome water disturbances. This work proposes the development of calibration methods for a structured light based system consisting on a camera and a laser with a line beam. Two different calibration procedures that require only two images from different viewpoints were developed and tested in dry and underwater environments. Results obtained show, an accurate calibration for the camera/projector pair with errors close to 1 mm even in the presence of a small stereos baseline.

Time series morphological analysis applied to biomedical signals events detection

Dissertation submitted in the fufillment of the requirements for the Degree of Master in Biomedical Engineering

Human activity recognition with accelerometry: novel time and frequency features

Human Activity Recognition systems require objective and reliable methods that can be used in the daily routine and must offer consistent results according with the performed activities. These systems are under development and offer objective and personalized support for several applications such as the healthcare area. This thesis aims to create a framework for human activities recognition based on accelerometry signals. Some new features and techniques inspired in the audio recognition methodology are introduced in this work, namely Log Scale Power Bandwidth and the Markov Models application. The Forward Feature Selection was adopted as the feature selection algorithm in order to improve the clustering performances and limit the computational demands. This method selects the most suitable set of features for activities recognition in accelerometry from a 423th dimensional feature vector. Several Machine Learning algorithms were applied to the used accelerometry databases – FCHA and PAMAP databases - and these showed promising results in activities recognition. The developed algorithm set constitutes a mighty contribution for the development of reliable evaluation methods of movement disorders for diagnosis and treatment applications.

Hand gesture recognition system based in computer vision and machine learning: Applications on human-machine interaction

Tese de Doutoramento em Engenharia de Eletrónica e de Computadores

MAMOT: hidden Markov modeling tool.

Hidden Markov models (HMMs) are probabilistic models that are well adapted to many tasks in bioinformatics, for example, for predicting the occurrence of specific motifs in biological sequences. MAMOT is a command-line program for Unix-like operating systems, including MacOS X, that we developed to allow scientists to apply HMMs more easily in their research. One can define the architecture and initial parameters of the model in a text file and then use MAMOT for parameter optimization on example data, decoding (like predicting motif occurrence in sequences) and the production of stochastic sequences generated according to the probabilistic model. Two examples for which models are provided are coiled-coil domains in protein sequences and protein binding sites in DNA. A wealth of useful features include the use of pseudocounts, state tying and fixing of selected parameters in learning, and the inclusion of prior probabilities in decoding. AVAILABILITY: MAMOT is implemented in C++, and is distributed under the GNU General Public Licence (GPL). The software, documentation, and example model files can be found at http://bcf.isb-sib.ch/mamot

A new approach to real time traffic sign recognition based on colour information

The automatic interpretation of conventional traffic signs is very complex and time consuming. The paper concerns an automatic warning system for driving assistance. It does not interpret the standard traffic signs on the roadside; the proposal is to incorporate into the existing signs another type of traffic sign whose information will be more easily interpreted by a processor. The type of information to be added is profuse and therefore the most important object is the robustness of the system. The basic proposal of this new philosophy is that the co-pilot system for automatic warning and driving assistance can interpret with greater ease the information contained in the new sign, whilst the human driver only has to interpret the "classic" sign. One of the codings that has been tested with good results and which seems to us easy to implement is that which has a rectangular shape and 4 vertical bars of different colours. The size of these signs is equivalent to the size of the conventional signs (approximately 0.4 m2). The colour information from the sign can be easily interpreted by the proposed processor and the interpretation is much easier and quicker than the information shown by the pictographs of the classic signs

Assessment of Severe Apnoea through Voice Analysis, Automatic Speech, and Speaker Recognition Techniques

This study is part of an ongoing collaborative effort between the medical and the signal processing communities to promote research on applying standard Automatic Speech Recognition (ASR) techniques for the automatic diagnosis of patients with severe obstructive sleep apnoea (OSA). Early detection of severe apnoea cases is important so that patients can receive early treatment. Effective ASR-based detection could dramatically cut medical testing time. Working with a carefully designed speech database of healthy and apnoea subjects, we describe an acoustic search for distinctive apnoea voice characteristics. We also study abnormal nasalization in OSA patients by modelling vowels in nasal and nonnasal phonetic contexts using Gaussian Mixture Model (GMM) pattern recognition on speech spectra. Finally, we present experimental findings regarding the discriminative power of GMMs applied to severe apnoea detection. We have achieved an 81% correct classification rate, which is very promising and underpins the interest in this line of inquiry.

Musical training intensity yields opposite effects on grey matter density in cognitive versus sensorimotor networks.

Using optimized voxel-based morphometry, we performed grey matter density analyses on 59 age-, sex- and intelligence-matched young adults with three distinct, progressive levels of musical training intensity or expertise. Structural brain adaptations in musicians have been repeatedly demonstrated in areas involved in auditory perception and motor skills. However, musical activities are not confined to auditory perception and motor performance, but are entangled with higher-order cognitive processes. In consequence, neuronal systems involved in such higher-order processing may also be shaped by experience-driven plasticity. We modelled expertise as a three-level regressor to study possible linear relationships of expertise with grey matter density. The key finding of this study resides in a functional dissimilarity between areas exhibiting increase versus decrease of grey matter as a function of musical expertise. Grey matter density increased with expertise in areas known for their involvement in higher-order cognitive processing: right fusiform gyrus (visual pattern recognition), right mid orbital gyrus (tonal sensitivity), left inferior frontal gyrus (syntactic processing, executive function, working memory), left intraparietal sulcus (visuo-motor coordination) and bilateral posterior cerebellar Crus II (executive function, working memory) and in auditory processing: left Heschl's gyrus. Conversely, grey matter density decreased with expertise in bilateral perirolandic and striatal areas that are related to sensorimotor function, possibly reflecting high automation of motor skills. Moreover, a multiple regression analysis evidenced that grey matter density in the right mid orbital area and the inferior frontal gyrus predicted accuracy in detecting fine-grained incongruities in tonal music.

Automatic recognition of postural allocations.

A significant part of daily energy expenditure may be attributed to non-exercise activity thermogenesis and exercise activity thermogenesis. Automatic recognition of postural allocations such as standing or sitting can be used in behavioral modification programs aimed at minimizing static postures. In this paper we propose a shoe-based device and related pattern recognition methodology for recognition of postural allocations. Inexpensive technology allows implementation of this methodology as a part of footwear. The experimental results suggest high efficiency and reliability of the proposed approach.

Not one odour but two: A new model for nestmate recognition.

Recognition systems play a key role in a range of biological processes, including mate choice, immune defence and altruistic behaviour. Social insects provide an excellent model for studying recognition systems because workers need to discriminate between nestmates and non-nestmates, enabling them to direct altruistic behaviour towards closer kin and to repel potential invaders. However, the level of aggression directed towards conspecific intruders can vary enormously, even among workers within the same colony. This is usually attributed to differences in the aggression thresholds of individuals or to workers having different roles within the colony. Recent evidence from the weaver ant Oecophylla smaragdina suggests that this does not tell the whole story. Here I propose a new model for nestmate recognition based on a vector template derived from both the individual's innate odour and the shared colony odour. This model accounts for the recent findings concerning weaver ants, and also provides an alternative explanation for why the level of aggression expressed by a colony decreases as the diversity within the colony increases, even when odour is well-mixed. The model makes additional predictions that are easily tested, and represents a significant advance in our conceptualisation of recognition systems.

Dynamics of domain walls in pattern formation under traveling-wave forcing

We study dynamics of domain walls in pattern forming systems that are externally forced by a moving space-periodic modulation close to 2:1 spatial resonance. The motion of the forcing induces nongradient dynamics, while the wave number mismatch breaks explicitly the chiral symmetry of the domain walls. The combination of both effects yields an imperfect nonequilibrium Ising-Bloch bifurcation, where all kinks (including the Ising-like one) drift. Kink velocities and interactions are studied within the generic amplitude equation. For nonzero mismatch, a transition to traveling bound kink-antikink pairs and chaotic wave trains occurs.