A multimodal and adaptive real-time human -computer interface for peoples with severe motor disability

This dissertation introduces the design of a multimodal, adaptive real-time assistive system as an alternate human computer interface that can be used by individuals with severe motor disabilities. The proposed design is based on the integration of a remote eye-gaze tracking system, voice recognition software, and a virtual keyboard. The methodology relies on a user profile that customizes eye gaze tracking using neural networks. The user profiling feature facilitates the notion of universal access to computing resources for a wide range of applications such as web browsing, email, word processing and editing. ^ The study is significant in terms of the integration of key algorithms to yield an adaptable and multimodal interface. The contributions of this dissertation stem from the following accomplishments: (a) establishment of the data transport mechanism between the eye-gaze system and the host computer yielding to a significantly low failure rate of 0.9%; (b) accurate translation of eye data into cursor movement through congregate steps which conclude with calibrated cursor coordinates using an improved conversion function; resulting in an average reduction of 70% of the disparity between the point of gaze and the actual position of the mouse cursor, compared with initial findings; (c) use of both a moving average and a trained neural network in order to minimize the jitter of the mouse cursor, which yield an average jittering reduction of 35%; (d) introduction of a new mathematical methodology to measure the degree of jittering of the mouse trajectory; (e) embedding an onscreen keyboard to facilitate text entry, and a graphical interface that is used to generate user profiles for system adaptability. ^ The adaptability nature of the interface is achieved through the establishment of user profiles, which may contain the jittering and voice characteristics of a particular user as well as a customized list of the most commonly used words ordered according to the user's preferences: in alphabetical or statistical order. This allows the system to successfully provide the capability of interacting with a computer. Every time any of the sub-system is retrained, the accuracy of the interface response improves even more. ^

A Digital Signal Processing Approach for Affective Sensing of a Computer User through Pupil Diameter Monitoring

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.

Face description for perceptual user interfaces

[EN]We investigate mechanisms which can endow the computer with the ability of describing a human face by means of computer vision techniques. This is a necessary requirement in order to develop HCI approaches which make the user feel himself/herself perceived. This paper describes our experiences considering gender, race and the presence of moustache and glasses. This is accomplished comparing, on a set of 6000 facial images, two di erent face representation approaches: Principal Components Analysis (PCA) and Gabor lters. The results achieved using a Support Vector Machine (SVM) based classi er are promising and particularly better for the second representation approach.

User Interface Abstraction for enabling TV set based Inclusive Access to the Information Society

199 p.

Dynamic Image Precompensation for Improving Visual Performance of Computer Users with Ocular Aberrations

With the progress of computer technology, computers are expected to be more intelligent in the interaction with humans, presenting information according to the user's psychological and physiological characteristics. However, computer users with visual problems may encounter difficulties on the perception of icons, menus, and other graphical information displayed on the screen, limiting the efficiency of their interaction with computers. In this dissertation, a personalized and dynamic image precompensation method was developed to improve the visual performance of the computer users with ocular aberrations. The precompensation was applied on the graphical targets before presenting them on the screen, aiming to counteract the visual blurring caused by the ocular aberration of the user's eye. A complete and systematic modeling approach to describe the retinal image formation of the computer user was presented, taking advantage of modeling tools, such as Zernike polynomials, wavefront aberration, Point Spread Function and Modulation Transfer Function. The ocular aberration of the computer user was originally measured by a wavefront aberrometer, as a reference for the precompensation model. The dynamic precompensation was generated based on the resized aberration, with the real-time pupil diameter monitored. The potential visual benefit of the dynamic precompensation method was explored through software simulation, with the aberration data from a real human subject. An "artificial eye'' experiment was conducted by simulating the human eye with a high-definition camera, providing objective evaluation to the image quality after precompensation. In addition, an empirical evaluation with 20 human participants was also designed and implemented, involving image recognition tests performed under a more realistic viewing environment of computer use. The statistical analysis results of the empirical experiment confirmed the effectiveness of the dynamic precompensation method, by showing significant improvement on the recognition accuracy. The merit and necessity of the dynamic precompensation were also substantiated by comparing it with the static precompensation. The visual benefit of the dynamic precompensation was further confirmed by the subjective assessments collected from the evaluation participants.

A user-centered interface for scheduling problem definition

In this paper we present a user-centered interface for a scheduling system. The purpose of this interface is to provide graphical and interactive ways of defining a scheduling problem. To create such user interface an evaluation-centered user interaction development method was adopted: the star life cycle. The created prototype comprises the Task Module and the Scheduling Problem Module. The first one allows users to define a sequence of operations, i.e., a task. The second one enables a scheduling problem definition, which consists in a set of tasks. Both modules are equipped with a set of real time validations to assure the correct definition of the necessary data input for the scheduling module of the system. The usability evaluation allowed us to measure the ease of interaction and observe the different forms of interaction provided by each participant, namely the reactions to the real time validation mechanism.

Opportunistic feeding behavior of Diplodus argenteus (Perciformes, Sparidae) : human-fish interaction in two rocky reefs from SE and S Brazil

Comportamento alimentar oportunista do marimbá Diplodus argenteus (Perciformes, Sparidae): interação humana-peixe em dois sistemas de costões rochosos do SE e SU do Brasil. Esse artigo aborda dois tipos de comportamentos alimentares do marimbá Diplodus argenteus em dois sistemas de costões rochosos do Atlântico sul no Brasil. Essa espécie alimentou-se sobre o fundo e exibiu comportamentos ocasionais e oportunísticos.

Edição gráfica de layout fabril no apoio ao planeamento e controlo da produção

O contributo da área de investigação Interacção Humano-Computador (HCI) está patente não só na qualidade da interacção, mas também na diversificação das formas de interacção. A HCI define-se como sendo uma disciplina que se dedica ao desenho, desenvolvimento e implementação de sistemas de computação interactivos para uso humano e estudo dos fenómenos relevantes que os rodeiam. Pretende-se, no âmbito desta tese de mestrado, o desenvolvimento de um Editor Gráfico de Layout Fabril a integrar num SAD para suporte ao Planeamento e Controlo da Produção. O sistema deve ser capaz de gerar um layout fabril do qual constam, entre outros objectos, as representações gráficas e as respectivas características/atributos do conjunto de recursos (máquinas/processadores) existentes no sistema de produção a modelar. O módulo desenvolvido será integrado no projecto de I&D ADSyS (Adaptative Decision Support System for Interactive Scheduling with MetaCognition and User Modeling Experience), melhorando aspectos de interacção referentes ao sistema AutoDynAgents, um dedicado ao escalonamento, planeamento e controlo de produção. Foi realizada a análise de usabilidade a este módulo com a qual se pretendeu realizar a respectiva avaliação, através da realização de um teste de eficiência e do preenchimento de um inquérito, da qual se identificaram um conjunto de melhorias e sugestões a serem consideradas no refinamento deste módulo.

Learning with tangible interfaces

Trabalho apresentado no âmbito do Mestrado em Engenharia Informática, como requisito parcial para obtenção do grau de Mestre em Engenharia Informática

SketchyDynamics apoio à produção de sistemas baseados em interfaces caligráficas para a simulação da dinâmica de corpos rígidos

Mestrado em Engenharia Informática - Área de Especialização em Sistemas Gráficos e Multimédia

Utilização do Kinect no apoio ao ensino/ aprendizagem dos fundamentos da Matemática

A interação humano-computador passou a desempenhar um papel fundamental no mundo atual. Esta forma de comunicar continua a evoluir, introduzindo novas formas de interação, como por exemplo, a interação natural. Este estilo de interação começou por estar presente na área de jogos. No entanto, atualmente está a ser explorada noutras áreas. Esta dissertação tem como propósito investigar a utilidade das interfaces naturais encontradas em consolas de jogos e conjugar com a área educativa, nomeadamente, o ensino e a aprendizagem dos fundamentos de Matemática. O desenvolvimento deste projeto baseou-se no estudo dos conteúdos programáticos de Matemática referentes ao 1º ciclo do ensino básico, de várias aplicações já existentes que estão relacionadas com o tema abordado e de alguns dispositivos de interação natural. De forma a avaliar a ideia proposta, foi desenvolvido um protótipo, designado Matemática Interativa, no sentido de permitir ao utilizador enriquecer a aprendizagem e também o interesse pela disciplina. São descritas, de uma forma mais aprofundada, as funcionalidades do dispositivo escolhido, o Kinect, de modo a tirar proveito das suas potencialidades e desenvolver um motor de reconhecimento de gestos e respetiva avaliação. Por fim, é feita uma discussão dos resultados de uma avaliação de usabilidade com o objetivo de validar a aplicação Matemática Interativa. Os resultados desta avaliação sugerem que a aplicação foi bem-sucedida e revelam ainda capacidades de melhoria.

Sistema Inteligente de Apoio ao Coach

O coaching é um processo que permite ajudar um ou mais indivíduos a definirem e saberem como concretizar os seus objetivos, sejam eles pessoais ou profissionais. Atualmente, existe um interesse e procura crescente de pessoas com experiência nesta área (designados por coaches) por parte de empresas, equipas desportivas, escolas e outras organizações, com a finalidade de obter um maior rendimento. De forma a ajudar os intervenientes no processo, este documento demonstra a necessidade de existir uma ferramenta de apoio que permite aos coaches gerirem melhor a sua atividade profissional. A pesquisa e estudo efetuados procuram responder a este caso, desenvolvendo um sistema informático inteligente de apoio ao coach dotado de uma interface centrada no utilizador. Antes de iniciar o desenvolvimento de um sistema inteligente é necessário realizar e apresentar um levantamento do estado da arte, mais concretamente sobre a interação homem-computador, modelação do perfil de utilizador e processo de coaching, que apresenta os fundamentos teóricos para a escolha da metodologia de desenvolvimento adequado. São apresentadas posteriormente as fases constituintes do modelo de desenvolvimento de interfaces escolhido, a engenharia de usabilidade, que se inicia com uma análise detalhada, permitindo de seguida uma estruturação dos conhecimentos obtidos e a aplicação de linhas de orientação estipuladas, finalizando com testes de utilização e respetivo feedback dos utilizadores. O protótipo desenvolvido distingue utilizadores com diferentes características, através de uma classificação por níveis e permite gerir todo o processo de coaching efetuado a outras pessoas ou ao próprio utilizador. O facto de existir uma classificação dos utilizadores faz com que a interação entre sistema e utilizadores seja diferente e adaptada às necessidades de cada um. O resultado dos testes de utilização com um caso prático e dos questionários efetuados permite detetar se o modelo foi bem-sucedido e funciona corretamente e o que é necessário alterar no futuro para facilitar a interação e satisfazer as necessidades de cada utilizador.

Visualization and interaction in a simulation system for flood emergencies

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para obtenção do grau de Mestre em Engenharia Informática

Robust facial expression analysis for affect-based interaction

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

A SDK improvement towards gesture support

Human-Computer Interaction have been one of the main focus of the technological community, specially the Natural User Interfaces (NUI) field of research as, since the launch of the Kinect Sensor, the goal to achieve fully natural interfaces just got a lot closer to reality. Taking advantage of this conditions the following research work proposes to compute the hand skeleton in order to recognize Sign Language Shapes. The proposed solution uses the Kinect Sensor to achieve a good segmentation and image analysis algorithms to extend the skeleton from the extraction of high-level features. In order to recognize complex hand shapes the current research work proposes the redefinition of the hand contour making it immutable to translation, rotation and scaling operations, and a set of tools to achieve a good recognition. The validation of the proposed solution extended the Kinects Software Development Kit to allow the developer to access the new set of inferred points and created a template-matching based platform that uses the contour to define the hand shape, this prototype was tested in a set of predefined conditions and showed to have a good success ration and has proven to be eligible for real-time scenarios.