A digital signal processing approach for affective sensing of a computer user through pupil diameter monitoring

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.

A practical system for hands-off computer interaction based on electromyogram (EMG)/eye gaze tracking (EGT)

Effective interaction with personal computers is a basic requirement for many of the functions that are performed in our daily lives. With the rapid emergence of the Internet and the World Wide Web, computers have become one of the premier means of communication in our society. Unfortunately, these advances have not become equally accessible to physically handicapped individuals. In reality, a significant number of individuals with severe motor disabilities, due to a variety of causes such as Spinal Cord Injury (SCI), Amyothrophic Lateral Sclerosis (ALS), etc., may not be able to utilize the computer mouse as a vital input device for computer interaction. The purpose of this research was to further develop and improve an existing alternative input device for computer cursor control to be used by individuals with severe motor disabilities. This thesis describes the development and the underlying principle for a practical hands-off human-computer interface based on Electromyogram (EMG) signals and Eye Gaze Tracking (EGT) technology compatible with the Microsoft Windows operating system (OS). Results of the software developed in this thesis show a significant improvement in the performance and usability of the EMG/EGT cursor control HCI.

A Digital Signal Processing Approach for Affective Sensing of a Computer User through Pupil Diameter Monitoring

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.

Designing for User-facing Uncertainty in Everyday Sensing and Prediction

Thesis (Ph.D.)--University of Washington, 2016-07

Comparative study of human age estimation based on hand-crafted and deep face features

In the past few years, human facial age estimation has drawn a lot of attention in the computer vision and pattern recognition communities because of its important applications in age-based image retrieval, security control and surveillance, biomet- rics, human-computer interaction (HCI) and social robotics. In connection with these investigations, estimating the age of a person from the numerical analysis of his/her face image is a relatively new topic. Also, in problems such as Image Classification the Deep Neural Networks have given the best results in some areas including age estimation. In this work we use three hand-crafted features as well as five deep features that can be obtained from pre-trained deep convolutional neural networks. We do a comparative study of the obtained age estimation results with these features.

A NATURALISTIC COMPUTATIONAL MODEL OF HUMAN BEHAVIOR IN NAVIGATION AND SEARCH TASKS

Planning, navigation, and search are fundamental human cognitive abilities central to spatial problem solving in search and rescue, law enforcement, and military operations. Despite a wealth of literature concerning naturalistic spatial problem solving in animals, literature on naturalistic spatial problem solving in humans is comparatively lacking and generally conducted by separate camps among which there is little crosstalk. Addressing this deficiency will allow us to predict spatial decision making in operational environments, and understand the factors leading to those decisions. The present dissertation is comprised of two related efforts, (1) a set of empirical research studies intended to identify characteristics of planning, execution, and memory in naturalistic spatial problem solving tasks, and (2) a computational modeling effort to develop a model of naturalistic spatial problem solving. The results of the behavioral studies indicate that problem space hierarchical representations are linear in shape, and that human solutions are produced according to multiple optimization criteria. The Mixed Criteria Model presented in this dissertation accounts for global and local human performance in a traditional and naturalistic Traveling Salesman Problem. The results of the empirical and modeling efforts hold implications for basic and applied science in domains such as problem solving, operations research, human-computer interaction, and artificial intelligence.

Dynamic Image Precompensation for Improving Visual Performance of Computer Users with Ocular Aberrations

With the progress of computer technology, computers are expected to be more intelligent in the interaction with humans, presenting information according to the user's psychological and physiological characteristics. However, computer users with visual problems may encounter difficulties on the perception of icons, menus, and other graphical information displayed on the screen, limiting the efficiency of their interaction with computers. In this dissertation, a personalized and dynamic image precompensation method was developed to improve the visual performance of the computer users with ocular aberrations. The precompensation was applied on the graphical targets before presenting them on the screen, aiming to counteract the visual blurring caused by the ocular aberration of the user's eye. A complete and systematic modeling approach to describe the retinal image formation of the computer user was presented, taking advantage of modeling tools, such as Zernike polynomials, wavefront aberration, Point Spread Function and Modulation Transfer Function. The ocular aberration of the computer user was originally measured by a wavefront aberrometer, as a reference for the precompensation model. The dynamic precompensation was generated based on the resized aberration, with the real-time pupil diameter monitored. The potential visual benefit of the dynamic precompensation method was explored through software simulation, with the aberration data from a real human subject. An "artificial eye'' experiment was conducted by simulating the human eye with a high-definition camera, providing objective evaluation to the image quality after precompensation. In addition, an empirical evaluation with 20 human participants was also designed and implemented, involving image recognition tests performed under a more realistic viewing environment of computer use. The statistical analysis results of the empirical experiment confirmed the effectiveness of the dynamic precompensation method, by showing significant improvement on the recognition accuracy. The merit and necessity of the dynamic precompensation were also substantiated by comparing it with the static precompensation. The visual benefit of the dynamic precompensation was further confirmed by the subjective assessments collected from the evaluation participants.

Building a Open Source Framework for Virtual Medical Training

This paper presents a framework to build medical training applications by using virtual reality and a tool that helps the class instantiation of this framework. The main purpose is to make easier the building of virtual reality applications in the medical training area, considering systems to simulate biopsy exams and make available deformation, collision detection, and stereoscopy functionalities. The instantiation of the classes allows quick implementation of the tools for such a purpose, thus reducing errors and offering low cost due to the use of open source tools. Using the instantiation tool, the process of building applications is fast and easy. Therefore, computer programmers can obtain an initial application and adapt it to their needs. This tool allows the user to include, delete, and edit parameters in the functionalities chosen as well as storing these parameters for future use. In order to verify the efficiency of the framework, some case studies are presented.

Acoplamentos: um estudo sobre a intera????o entre humanos e computadores

Este estudo investiga a intera????o dos humanos com os computadores pessoais. A partir dos inc??modos produzidos por esses encontros, na primeira parte estuda-se a produ????o da m??quina ???amig??vel??? ??? que insinua compara????es com a mente humana ??? e seus fortes efeitos nas transforma????es culturais do final do s??culo XX. Na segunda parte do trabalho, a partir de entrevistas de trabalhadores falando sobre sua aprendizagem b??sica do uso de computadores pessoais, faz-se uma an??lise das principais concep????es sobre a rela????o homem-t??cnica que est??o influenciando a intera????o humano-computador. Com isso fica evidenciado o papel do computador na crise do sujeito moderno, e afirma-se a necessidade de se pensar a subjetividade para al??m da cl??ssica divis??o sujeito-objeto, homem m??quina, natureza-artif??cio... o que aponta para a id??ia de acoplamentos flex??veis entre humanos e computadores e pede por uma no????o de aprendizagem que acolha a cria????o de problemas como seu aspecto mais rico.

Estudo preliminar para a avaliação da acessibilidade de um sítio Web

O avanço e difusão da tecnologia, em particular da Internet, obrigaram à disponibilização de informação com qualidade e facilmente acessível. Tais exigências vêm confirmar a relevância do papel da interface como elemento principal na interacção do utilizador com os sistemas de informação. É, por isso, fundamental que a interface seja fácil de usar e que vá ao encontro das expectativas e necessidades de todos os utilizadores. O desenvolvimento de interfaces que satisfaçam utilizadores com necessidades distintas, independentemente das suas capacidades motoras e perceptivas, culturais e sociais não é uma tarefa que possa se considerada simples [1]. De acordo com vários especialistas em Interacção Homem-Computador, [1], [2] e [3] as interfaces devem ser construídas respeitando os princípios de desenho centrado no utilizador, visando um elevado grau de usabilidade e em conformidade com directrizes de acessibilidade básicas. Este trabalho apresenta uma metodologia para avaliação da acessibilidade de um sítio Web, baseado num documento disponibilizado pela Web Accessibility Initiative, WAI, [4], a ser aplicada à secretaria on-line de uma escola do ensino superior.

A user-centered interface for scheduling problem definition

In this paper we present a user-centered interface for a scheduling system. The purpose of this interface is to provide graphical and interactive ways of defining a scheduling problem. To create such user interface an evaluation-centered user interaction development method was adopted: the star life cycle. The created prototype comprises the Task Module and the Scheduling Problem Module. The first one allows users to define a sequence of operations, i.e., a task. The second one enables a scheduling problem definition, which consists in a set of tasks. Both modules are equipped with a set of real time validations to assure the correct definition of the necessary data input for the scheduling module of the system. The usability evaluation allowed us to measure the ease of interaction and observe the different forms of interaction provided by each participant, namely the reactions to the real time validation mechanism.

Safety high accuracy Context-Aware Matrix (CAM) making based on X.509 proxy certificate

Human Computer Interaction (HCl) is to interaction between computers and each person. And context-aware (CA) is very important one of HCI composition. In particular, if there are sequential or continuous tasks between users and devices, among users, and among devices etc, it is important to decide the next action using right CA. And to take perfect decision we have to get together all CA into a structure. We define that structure is Context-Aware Matrix (CAM) in this article. However to make exact decision is too hard for some problems like low accuracy, overhead and bad context by attacker etc. Many researcher has been studying to solve these problems. Moreover, still it has weak point HCI using in safety. In this Article, we propose CAM making include best selecting Server in each area. As a result, moving users could be taken the best way.

Realidade aumentada aplicada ao processo de ensino/aprendizagem. Estudo de caso.

Nas últimas décadas temos assistido a um avanço tecnológico a todos os níveis mas com particular incidência ao nível do hardware e dos dispositivos móveis. Estes tornaram-­‐se cada vez mais leves e mais baratos, e transferiram-­‐se do escritório para o carro, para os equipamentos e para os utensílios. A quantidade de informação (digital) disponível no meio envolvente aumentou de forma exponencial exigindo uma resposta tecnológica com o intuito de melhorar/facilitar o seu acesso e assimilação. É aqui que surge o conceito de Realidade Aumentada a funcionar como uma ponte de ligação entre o real e o digital convidando a novos modelos de interacção com o utilizador. A sua incorporação visa essencialmente tornar os sistemas mais usáveis diminuindo a carga cognitiva inerente à sua utilização. Este trabalho apresenta um estudo de caso, propondo um modelo para a construção de um Objecto de Aprendizagem com recurso a Realidade Aumentada especificamente para a área da saúde. O problema identificado nesta dissertação procura investigar se a integração de técnicas de Realidade Aumentada combinadas com técnicas multimédia e outros materiais convencionais podem contribuir para uma maior motivação e percepção cooperando para a construção de conhecimento.

Edição gráfica de layout fabril no apoio ao planeamento e controlo da produção

O contributo da área de investigação Interacção Humano-Computador (HCI) está patente não só na qualidade da interacção, mas também na diversificação das formas de interacção. A HCI define-se como sendo uma disciplina que se dedica ao desenho, desenvolvimento e implementação de sistemas de computação interactivos para uso humano e estudo dos fenómenos relevantes que os rodeiam. Pretende-se, no âmbito desta tese de mestrado, o desenvolvimento de um Editor Gráfico de Layout Fabril a integrar num SAD para suporte ao Planeamento e Controlo da Produção. O sistema deve ser capaz de gerar um layout fabril do qual constam, entre outros objectos, as representações gráficas e as respectivas características/atributos do conjunto de recursos (máquinas/processadores) existentes no sistema de produção a modelar. O módulo desenvolvido será integrado no projecto de I&D ADSyS (Adaptative Decision Support System for Interactive Scheduling with MetaCognition and User Modeling Experience), melhorando aspectos de interacção referentes ao sistema AutoDynAgents, um dedicado ao escalonamento, planeamento e controlo de produção. Foi realizada a análise de usabilidade a este módulo com a qual se pretendeu realizar a respectiva avaliação, através da realização de um teste de eficiência e do preenchimento de um inquérito, da qual se identificaram um conjunto de melhorias e sugestões a serem consideradas no refinamento deste módulo.

Towards the detection of deception in interactive multimedia environments

A classical application of biosignal analysis has been the psychophysiological detection of deception, also known as the polygraph test, which is currently a part of standard practices of law enforcement agencies and several other institutions worldwide. Although its validity is far from gathering consensus, the underlying psychophysiological principles are still an interesting add-on for more informal applications. In this paper we present an experimental off-the-person hardware setup, propose a set of feature extraction criteria and provide a comparison of two classification approaches, targeting the detection of deception in the context of a role-playing interactive multimedia environment. Our work is primarily targeted at recreational use in the context of a science exhibition, where the main goal is to present basic concepts related with knowledge discovery, biosignal analysis and psychophysiology in an educational way, using techniques that are simple enough to be understood by children of different ages. Nonetheless, this setting will also allow us to build a significant data corpus, annotated with ground-truth information, and collected with non-intrusive sensors, enabling more advanced research on the topic. Experimental results have shown interesting findings and provided useful guidelines for future work. Pattern Recognition