A digital signal processing approach for affective sensing of a computer user through pupil diameter monitoring

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.

A multimodal and adaptive real-time human -computer interface for peoples with severe motor disability

This dissertation introduces the design of a multimodal, adaptive real-time assistive system as an alternate human computer interface that can be used by individuals with severe motor disabilities. The proposed design is based on the integration of a remote eye-gaze tracking system, voice recognition software, and a virtual keyboard. The methodology relies on a user profile that customizes eye gaze tracking using neural networks. The user profiling feature facilitates the notion of universal access to computing resources for a wide range of applications such as web browsing, email, word processing and editing. ^ The study is significant in terms of the integration of key algorithms to yield an adaptable and multimodal interface. The contributions of this dissertation stem from the following accomplishments: (a) establishment of the data transport mechanism between the eye-gaze system and the host computer yielding to a significantly low failure rate of 0.9%; (b) accurate translation of eye data into cursor movement through congregate steps which conclude with calibrated cursor coordinates using an improved conversion function; resulting in an average reduction of 70% of the disparity between the point of gaze and the actual position of the mouse cursor, compared with initial findings; (c) use of both a moving average and a trained neural network in order to minimize the jitter of the mouse cursor, which yield an average jittering reduction of 35%; (d) introduction of a new mathematical methodology to measure the degree of jittering of the mouse trajectory; (e) embedding an onscreen keyboard to facilitate text entry, and a graphical interface that is used to generate user profiles for system adaptability. ^ The adaptability nature of the interface is achieved through the establishment of user profiles, which may contain the jittering and voice characteristics of a particular user as well as a customized list of the most commonly used words ordered according to the user's preferences: in alphabetical or statistical order. This allows the system to successfully provide the capability of interacting with a computer. Every time any of the sub-system is retrained, the accuracy of the interface response improves even more. ^

Fast and accurate hand pose detection for human-robot interaction

[EN]Enabling natural human-robot interaction using computer vision based applications requires fast and accurate hand detection. However, previous works in this field assume different constraints, like a limitation in the number of detected gestures, because hands are highly complex objects difficult to locate. This paper presents an approach which integrates temporal coherence cues and hand detection based on wrists using a cascade classifier. With this approach, we introduce three main contributions: (1) a transparent initialization mechanism without user participation for segmenting hands independently of their gesture, (2) a larger number of detected gestures as well as a faster training phase than previous cascade classifier based methods and (3) near real-time performance for hand pose detection in video streams.

Building a Open Source Framework for Virtual Medical Training

This paper presents a framework to build medical training applications by using virtual reality and a tool that helps the class instantiation of this framework. The main purpose is to make easier the building of virtual reality applications in the medical training area, considering systems to simulate biopsy exams and make available deformation, collision detection, and stereoscopy functionalities. The instantiation of the classes allows quick implementation of the tools for such a purpose, thus reducing errors and offering low cost due to the use of open source tools. Using the instantiation tool, the process of building applications is fast and easy. Therefore, computer programmers can obtain an initial application and adapt it to their needs. This tool allows the user to include, delete, and edit parameters in the functionalities chosen as well as storing these parameters for future use. In order to verify the efficiency of the framework, some case studies are presented.

Opportunistic feeding behavior of Diplodus argenteus (Perciformes, Sparidae) : human-fish interaction in two rocky reefs from SE and S Brazil

Comportamento alimentar oportunista do marimbá Diplodus argenteus (Perciformes, Sparidae): interação humana-peixe em dois sistemas de costões rochosos do SE e SU do Brasil. Esse artigo aborda dois tipos de comportamentos alimentares do marimbá Diplodus argenteus em dois sistemas de costões rochosos do Atlântico sul no Brasil. Essa espécie alimentou-se sobre o fundo e exibiu comportamentos ocasionais e oportunísticos.

Is ambient intelligence a truly human-centric paradigm in industry? Current research and application scenario

Based on the paper presented at the International Conference “Autonomous Systems: inter-relations of technical and societal issues”, organized by IET with the support of the Portuguese-German collaboration project on “Technology Assessment of Autonomous Robotics” (DAAD/CRUP) at FCT-UNL, Biblioteca da UNL, Campus de Caparica on 5-6 November 2009.

A Real Time Vision System for Autonomous Systems: Characterization during a Middle Size Match

in RoboCup 2007: Robot Soccer World Cup XI

Simulation Environment for Multi-Robot Cooperative 3D Target Perception

4th International Conference, SIMPAR 2014, Bergamo, Italy, October 20-23, 2014

Uncertainty based Multi-Robot Cooperative Triangulation

The paper presents a multi-robot cooperative framework to estimate the 3D position of dynamic targets, based on bearing-only vision measurements. The uncertainty of the observation provided by each robot equipped with a bearing-only vision system is effectively addressed for cooperative triangulation purposes by weighing the contribution of each monocular bearing ray in a probabilistic manner. The envisioned framework is evaluated in an outdoor scenario with a team of heterogeneous robots composed of an Unmanned Ground and Aerial Vehicle.

Sistema Inteligente de Apoio ao Coach

O coaching é um processo que permite ajudar um ou mais indivíduos a definirem e saberem como concretizar os seus objetivos, sejam eles pessoais ou profissionais. Atualmente, existe um interesse e procura crescente de pessoas com experiência nesta área (designados por coaches) por parte de empresas, equipas desportivas, escolas e outras organizações, com a finalidade de obter um maior rendimento. De forma a ajudar os intervenientes no processo, este documento demonstra a necessidade de existir uma ferramenta de apoio que permite aos coaches gerirem melhor a sua atividade profissional. A pesquisa e estudo efetuados procuram responder a este caso, desenvolvendo um sistema informático inteligente de apoio ao coach dotado de uma interface centrada no utilizador. Antes de iniciar o desenvolvimento de um sistema inteligente é necessário realizar e apresentar um levantamento do estado da arte, mais concretamente sobre a interação homem-computador, modelação do perfil de utilizador e processo de coaching, que apresenta os fundamentos teóricos para a escolha da metodologia de desenvolvimento adequado. São apresentadas posteriormente as fases constituintes do modelo de desenvolvimento de interfaces escolhido, a engenharia de usabilidade, que se inicia com uma análise detalhada, permitindo de seguida uma estruturação dos conhecimentos obtidos e a aplicação de linhas de orientação estipuladas, finalizando com testes de utilização e respetivo feedback dos utilizadores. O protótipo desenvolvido distingue utilizadores com diferentes características, através de uma classificação por níveis e permite gerir todo o processo de coaching efetuado a outras pessoas ou ao próprio utilizador. O facto de existir uma classificação dos utilizadores faz com que a interação entre sistema e utilizadores seja diferente e adaptada às necessidades de cada um. O resultado dos testes de utilização com um caso prático e dos questionários efetuados permite detetar se o modelo foi bem-sucedido e funciona corretamente e o que é necessário alterar no futuro para facilitar a interação e satisfazer as necessidades de cada utilizador.

Visualization and interaction in a simulation system for flood emergencies

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para obtenção do grau de Mestre em Engenharia Informática

Development of advanced human robot interaction systems

O trabalho aqui apresentado é a Dissertação da minha Tese do curso de Mestrado em Engenharia Eletrotécnica e de Computadores do ISEP, realizada em parceria com o INESC TEC. O trabalho consiste no desenvolvimento de um sistema avançado de interação entre homem-robô, usando ferramentas de software livres e de domínio público e hardware pouco dispendioso e facilmente acessível. Pretende-se que o sistema desenvolvido possa ser adotado por pequenas ou micro empresas, daí a restrição monetária. Este tipo de empresas tem, por norma, uma capacidade de investimento pequena, e ficam impossibilitadas de aceder a este tipo de sistemas automatizados se estes forem caros. No entanto, o robô continua a ser um componente fundamental, sendo dispendioso. Os trabalhos realizados pelos sistemas robóticos podem por um lado, ser repetitivos sem necessidade de grandes ajustes; por outro lado, o trabalho a realizar pode ser bastante diverso, sendo necessários bastantes ajustes com (possivelmente) programação do robô. As empresas podem não ter disponível mão-de-obra qualificada para realização da programação do robô. Pretende-se então um sistema de “ensino” que seja simples e rápido. Este trabalho pretende satisfazer as necessidades de um sistema de interação homem-robô intuitivo mesmo para operadores que não estejam familiarizados com a robótica. Para simplificar a transferência de informação da tarefa a desempenhar pelo sistema robótico é usado um sistema de infravermelhos para delinear a operação a desempenhar, neste caso concreto uma operação de soldadura. O operador usa um apontador com marcadores, a posição destes marcadores é detetada usando duas câmaras para permitir o posicionamento tridimensional no espaço. As câmaras possuem filtros infravermelhos para separar o espectro de luz. Para o controlo do sistema e interface com o robô é usado um computador de baixos recursos computacionais e energéticos, e também de baixo custo. O sistema desenvolvido é portanto computacionalmente leve para poder ser executado neste computador.

Robust facial expression analysis for affect-based interaction

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Robots and humans as co-workers? The human-centred perspective of work with autonomous systems

The design of work organisation systems with automated equipment is facing new challenges and the emergence of new concepts. The social aspects that are related with new concepts on the complex work environments (CWE) are becoming more relevant for that design. The work with autonomous systems implies options in the design of workplaces. Especially that happens in such complex environments. The concepts of “agents”, “co-working” or “human-centred technical systems” reveal new dimensions related to human-computer interaction (HCI). With an increase in the number and complexity of those human-technology interfaces, the capacities of human intervention can become limited, originating further problems. The case of robotics is used to exemplify the issues related with automation in working environments and the emergence of new HCI approaches that would include social implications. We conclude that studies on technology assessment of industrial robotics and autonomous agents on manufacturing environment should also focus on the human involvement strategies in organisations. A needed participatory strategy implies a new approach to workplaces design. This means that the research focus must be on the relation between technology and social dimensions not as separate entities, but integrated in the design of an interaction system.

Benefits & drawbacks of different means of interaction for placing objects above a video footage

Public Display Systems (PDS) increasingly have a greater presence in our cities. These systems provide information and advertising specifically tailored to audiences in spaces such as airports, train stations, and shopping centers. A large number of public displays are also being deployed for entertainment reasons. Sometimes designing and prototyping PDS come to be a laborious, complex and a costly task. This dissertation focuses on the design and evaluation of PDS at early development phases with the aim of facilitating low-effort, rapid design and the evaluation of interactive PDS. This study focuses on the IPED Toolkit. This tool proposes the design, prototype, and evaluation of public display systems, replicating real-world scenes in the lab. This research aims at identifying benefits and drawbacks on the use of different means to place overlays/virtual displays above a panoramic video footage, recorded at real-world locations. The means of interaction studied in this work are on the one hand the keyboard and mouse, and on the other hand the tablet with two different techniques of use. To carry out this study, an android application has been developed whose function is to allow users to interact with the IPED Toolkit using the tablet. Additionally, the toolkit has been modified and adapted to tablets by using different web technologies. Finally the users study makes a comparison about the different means of interaction.