Adaptive Dialogue Strategy Selection through Imprecise Probabilistic Query Answering

In a human-computer dialogue system, the dialogue strategy can range from very restrictive to highly flexible. Each specific dialogue style has its pros and cons and a dialogue system needs to select the most appropriate style for a given user. During the course of interaction, the dialogue style can change based on a user’s response and the system observation of the user. This allows a dialogue system to understand a user better and provide a more suitable way of communication. Since measures of the quality of the user’s interaction with the system can be incomplete and uncertain, frameworks for reasoning with uncertain and incomplete information can help the system make better decisions when it chooses a dialogue strategy. In this paper, we investigate how to select a dialogue strategy based on aggregating the factors detected during the interaction with the user. For this purpose, we use probabilistic logic programming (PLP) to model probabilistic knowledge about how these factors will affect the degree of freedom of a dialogue. When a dialogue system needs to know which strategy is more suitable, an appropriate query can be executed against the PLP and a probabilistic solution with a degree of satisfaction is returned. The degree of satisfaction reveals how much the system can trust the probability attached to the solution.

Adaptive Fusion Of Particle Filtering And Spatio-Temporal Motion Energy For Human Tracking

Object tracking is an active research area nowadays due to its importance in human computer interface, teleconferencing and video surveillance. However, reliable tracking of objects in the presence of occlusions, pose and illumination changes is still a challenging topic. In this paper, we introduce a novel tracking approach that fuses two cues namely colour and spatio-temporal motion energy within a particle filter based framework. We conduct a measure of coherent motion over two image frames, which reveals the spatio-temporal dynamics of the target. At the same time, the importance of both colour and motion energy cues is determined in the stage of reliability evaluation. This determination helps maintain the performance of the tracking system against abrupt appearance changes. Experimental results demonstrate that the proposed method outperforms the other state of the art techniques in the used test datasets.

The Belfast Storytelling Database: A spontaneous social interaction database with laughter focused annotation

To support the endeavor of creating intelligent interfaces between computers and humans the use of training materials based on realistic human-human interactions has been recognized as a crucial task. One of the effects of the creation of these databases is an increased realization of the importance of often overlooked social signals and behaviours in organizing and orchestrating our interactions. Laughter is one of these key social signals; its importance in maintaining the smooth flow of human interaction has only recently become apparent in the embodied conversational agent domain. In turn, these realizations require training data that focus on these key social signals. This paper presents a database that is well annotated and theoretically constructed with respect to understanding laughter as it is used within human social interaction. Its construction, motivation, annotation and availability are presented in detail in this paper.

Artefactos tangíveis e adaptáveis no ambiente doméstico

Recentemente assistimos a uma evolução da relação do Homem com a tecnologia, em larga medida acompanhada por novos modelos de interacção que modificam a forma de conceber os artefactos e constroem novos contextos de uso. Procuramos investigar, na presente tese, uma das abordagens emergentes, o universo dos media tangíveis, articulando a perspectiva do design da tecnologia orientada para a Human-Computer Interation (HCI), com a dimensão social, cultural e estética no uso da tecnologia. Os media tangíveis, ao contrário do que sucede com os conteúdos digitais convencionais, têm espessura e expressão física e, porque são dotados de um corpo que habita o espaço das disposições físicas, estão sujeitos à acção do mundo cultural e das práticas sociais que regem os demais objectos físicos que podemos encontrar no nosso quotidiano. Esta nova relação com a tecnologia digital obrigará as disciplinas que se encontram mais próximas do desenvolvimento tecnológico, tais como o Design de Interacção e a HCI, a abrirem-se aos contributos e abordagens das ciências humanas. Admitindo que a natureza subjacente ao processo da adaptabilidade no ambiente doméstico altera o equilíbrio da relação entre o design e o uso da tecnologia, julgamos ser essencial o desenvolvimento de uma fenomenologia da interação. Por outro lado, a adaptabilidade dos media tangíveis apresenta um conjunto de dificuldades, não apenas de ordem técnica, mas também de natureza conceptual, que têm dificultado o desenvolvimento e a implementação no terreno de tecnologias personalizáveis. Um dos objectivos da presente tese consiste em investigar um quadro conceptual capaz de enquadrar o fenómeno da adaptabilidade dos media tangíveis, e desenvolver uma tecnologia que possa servir de objecto a um estudo empírico com base numa abordagem etnográfica.

Concepção de uma aplicação móvel no âmbito da ecologia e sustentabilidade ambiental

Trabalho de projeto apresentado à Escola Superior de Comunicação Social como parte dos requisitos para obtenção de grau de mestre em Audiovisual e Multimédia.

Modèle novateur de conception d’interface humain-ordinateur centrée sur l’utilisateur : le designer en tant que médiateur

Cette recherche porte sur des questions relatives à la conception des interfaces humain-ordinateur. Elle s’inscrit dans le courant des recherches sur l’utilisabilité et elle s’intéresse particulièrement aux approches centrées sur l’utilisateur. Nous avons été très souvent témoin des difficultés éprouvées par les utilisateurs dans l’usage de certaines interfaces interactives et nous considérons que ces difficultés découlent d’un problème de design. Le design d’interface doit être basé sur les besoins de l’utilisateur dans le cadre de ses activités, dont les caractéristiques devaient être bien comprises et bien prises en considération pour mener à la conception d’interfaces qui respectent les critères d’utilisabilité. De plus, la communauté des chercheurs ainsi que l’industrie admettent maintenant que pour améliorer le design, il est crucial de développer les interfaces humain-ordinateur au sein d’une équipe multidisciplinaire. Malgré les avancées significatives dans le domaine du design centrées sur l’utilisateur, les visées annoncées sont rarement réalisées. La problématique étudiée nous a conduit à poser la question suivante : En tant que designer d’une équipe multidisciplinaire de conception, comment modifier la dynamique de collaboration et créer les conditions d’une conception véritablement centrée sur l’interaction humain-ordinateur ? Notre démarche de recherche a été guidée par l’hypothèse voulant que l’activité de design puisse être le moyen de faciliter la création d’un langage commun, des échanges constructifs entre les disciplines, et une réflexion commune centrée sur l’utilisateur. La formulation de cette hypothèse nous a mené à réfléchir sur le rôle du designer. Pour mener cette recherche, nous avons adopté une méthodologie mixte. Dans un premier temps, nous avons utilisé une approche de recherche par projet (recherche-projet) et notre fonction était celle de designer-chercheur. La recherche-projet est particulièrement appropriée pour les recherches en design. Elle privilégie les méthodes qualitatives et interprétatives ; elle étudie la situation dans sa complexité et de façon engagée. Nous avons effectué trois études de cas successives. L’objectif de la première étude était d’observer notre propre rôle et nos interactions avec les autres membres de l’équipe de projet pendant le processus de design. Dans la seconde étude, notre attention a été portée sur les interactions et la collaboration de l’équipe. Nous avons utilisé le processus de design comme méthode pour la construction d’un langage commun entre les intervenants, pour enrichir les réflexions et pour favoriser leur collaboration menant à redéfinir les objectifs du projet. Les limites de ces deux cas nous ont conduit à une intervention différente que nous avons mise en œuvre dans la troisième étude de cas. Cette intervention est constituée par la mise en place d’un atelier intensif de conception où les intervenants au projet se sont engagés à développer une attitude interdisciplinaire permettant la copratique réflexive pour atteindre les objectifs d’un projet de construction d’un site web complexe centré sur l’utilisateur. L’analyse et l’interprétation des données collectées de ces trois études de cas nous ont conduit à créer un modèle théorique de conception d’interface humain-ordinateur. Ce modèle qui informe et structure le processus de design impliquant une équipe multidisciplinaire a pour objectif d’améliorer l’approche centrée sur l’utilisateur. Dans le cadre de ce modèle, le designer endosse le rôle de médiateur en assurant l’efficacité de la collaboration de l’équipe. Dans un deuxième temps, afin de valider le modèle et éventuellement le perfectionner, nous avons utilisé une approche ethnographique comportant des entrevues avec trois experts dans le domaine. Les données des entrevues confirment la validité du modèle ainsi que son potentiel de transférabilité à d’autres contextes. L’application de ce modèle de conception permet d’obtenir des résultats plus performants, plus durables, et dans un délai plus court.

Human and Raptor Interactions in the Context of a Nomadic Society

This research project focuses on contemporary eagle-taming falconry practice of the Altaic Kazakhs animal herding society in Bayan Ulgii Province in Western Mongolia. It aims to contributing both theoretical and empirical criteria for cultural preservation of Asian falconry. This cultural as well as environmental discourse is illustrated with concentrated field research framed by ecological anthropology and ethno-ornithology from the viewpoint of “Human-Animal Interaction (HAI)” and “Human-Animal Behavior (HAB)”. Part I (Chapter 2 & 3) explores ethno-archaeological and ethno-ornithological dimensions by interpretive research of archaeological artefacts which trace the historical depth of Asian falconry culture. Part II (Chapter 4 & 5) provides an extensive ethnographic narrative of Altaic Kazakh falconry, which is the central part of this research project. The “Traditional Art and Knowledge (TAK)” in human-raptor interactions, comprising the entire cycle of capture, perch, feeding, training, hunting, and release, is presented with specific emphasis on its relation to environmental and societal context. Traditional falconry as integral part of a nomadic lifestyle has to face some critical problems nowadays which necessitate preventing the complete disappearance of this outstanding indigenous cultural heritage. Part III (Chapter 6 & 7) thus focuses on the cultural sustainability of Altaic Kazakh falconry. Changing livelihoods, sedentarisation, and decontextualisation are identified as major threats. The role of Golden Eagle Festivals is critically analysed with regard to positive and negative impact. This part also intends to contribute to the academic definition of eagle falconry as an intangible cultural heritage, and to provide scientific criteria for a preservation master plan, as well as stipulate local resilience by pointing to successive actions needed for conservation. This research project concludes that cultural sustainability of Altaic Kazakh falconry needs to be supported from the angles of three theoretical frameworks; (1) Cultural affairs for protection based on the concept of nature-guardianship in its cultural domain, (2) Sustainable development and improvement of animal herding productivity and herder’s livelihood, (3) Natural resource management, especially supporting the population of Golden Eagles, their potential prey animals, and their nesting environment.

Using Analogy to Acquire Commonsense Knowledge from Human Contributors

The goal of the work reported here is to capture the commonsense knowledge of non-expert human contributors. Achieving this goal will enable more intelligent human-computer interfaces and pave the way for computers to reason about our world. In the domain of natural language processing, it will provide the world knowledge much needed for semantic processing of natural language. To acquire knowledge from contributors not trained in knowledge engineering, I take the following four steps: (i) develop a knowledge representation (KR) model for simple assertions in natural language, (ii) introduce cumulative analogy, a class of nearest-neighbor based analogical reasoning algorithms over this representation, (iii) argue that cumulative analogy is well suited for knowledge acquisition (KA) based on a theoretical analysis of effectiveness of KA with this approach, and (iv) test the KR model and the effectiveness of the cumulative analogy algorithms empirically. To investigate effectiveness of cumulative analogy for KA empirically, Learner, an open source system for KA by cumulative analogy has been implemented, deployed, and evaluated. (The site "1001 Questions," is available at http://teach-computers.org/learner.html). Learner acquires assertion-level knowledge by constructing shallow semantic analogies between a KA topic and its nearest neighbors and posing these analogies as natural language questions to human contributors. Suppose, for example, that based on the knowledge about "newspapers" already present in the knowledge base, Learner judges "newspaper" to be similar to "book" and "magazine." Further suppose that assertions "books contain information" and "magazines contain information" are also already in the knowledge base. Then Learner will use cumulative analogy from the similar topics to ask humans whether "newspapers contain information." Because similarity between topics is computed based on what is already known about them, Learner exhibits bootstrapping behavior --- the quality of its questions improves as it gathers more knowledge. By summing evidence for and against posing any given question, Learner also exhibits noise tolerance, limiting the effect of incorrect similarities. The KA power of shallow semantic analogy from nearest neighbors is one of the main findings of this thesis. I perform an analysis of commonsense knowledge collected by another research effort that did not rely on analogical reasoning and demonstrate that indeed there is sufficient amount of correlation in the knowledge base to motivate using cumulative analogy from nearest neighbors as a KA method. Empirically, evaluating the percentages of questions answered affirmatively, negatively and judged to be nonsensical in the cumulative analogy case compares favorably with the baseline, no-similarity case that relies on random objects rather than nearest neighbors. Of the questions generated by cumulative analogy, contributors answered 45% affirmatively, 28% negatively and marked 13% as nonsensical; in the control, no-similarity case 8% of questions were answered affirmatively, 60% negatively and 26% were marked as nonsensical.

Ergonomía. Fundamentos para el desarrollo de soluciones ergonómicas

En este texto se presentan algunos conceptos y marcos teóricos útiles para el análisis del trabajo en ergonomía. El objetivo es mostrar los conceptos de base para el estudio del trabajo en la tradición de la ergonomía de la actividad, y analizar de manera general algunos de los modelos empleados para el análisis de una actividad de trabajo. Inicialmente se abordan los principios teóricos de la ergonomía y los principios que provienen de la fisiología, la biomecánica, la psicología y la sociología; también se presentan los acercamientos metodológicos empleados en esta misma perspectiva para el análisis de actividades de trabajo. Se parte del principio de que un estudio ergonómico del trabajo se puede llevar a cabo desde una doble perspectiva: la perspectiva analítica y la perspectiva comprensiva.

Generic control interface for networked haptic virtual environments

As Virtual Reality pushes the boundaries of the human computer interface new ways of interaction are emerging. One such technology is the integration of haptic interfaces (force-feedback devices) into virtual environments. This modality offers an improved sense of immersion to that achieved when relying only on audio and visual modalities. The paper introduces some of the technical obstacles such as latency and network traffic that need to be overcome for maintaining a high degree of immersion during haptic tasks. The paper describes the advantages of integrating haptic feedback into systems, and presents some of the technical issues inherent in a networked haptic virtual environment. A generic control interface has been developed to seamlessly mesh with existing networked VR development libraries.

Examining the relationships between Holocene climate change, hydrology, and human society in Ireland

This thesis explores human-environment interactions during the Mid-Late Holocene in raised bogs in central Ireland. The raised bogs of central Ireland are widely-recognised for their considerable palaeoenvironmental and archaeological resources: research over the past few decades has established the potential for such sites to preserve sensitive records of Holocene climatic variability expressed as changes in bog surface wetness (BSW); meanwhile archaeological investigations over the past century have uncovered hundreds of peatland archaeological features dating from the Neolithic through to the Post-Medieval period including wooden trackways, platforms, and deposits of high-status metalwork. Previous studies have attempted to explore the relationship between records of past environmental change and the occurrence of peatland archaeological sites reaching varying conclusions. More recently, environmentally-deterministic models of human-environment interaction in Irish raised bogs at the regional scale have been explicitly tested leading to the conclusion that there is no relationship between BSW and past human activity. These relationships are examined in more detail on a site-by-site basis in this thesis. To that end, testate amoebae-derived BSW records from nine milled former raised bogs in central Ireland were produced from sites with known and dated archaeological records. Relationships between BSW records and environmental conditions within the study area were explored through both the development of a new central Ireland testate amoebae transfer function and through comparisons between recent BSW records and instrumental weather data. Compilation of BSW records from the nine fossil study sites show evidence both for climate forcing, particularly during 3200-2400 cal BP, as well as considerable inter-site variability. Considerable inter-site variability was also evident in the archaeological records of the same sites. Whilst comparisons between BSW and archaeological records do not show a consistent linear relationship, examination of records on a site-by-site basis were shown to reveal interpretatively important contingent relationships. It is concluded therefore, that future research on human-environment interactions should focus on individual sites and should utilise theoretical approaches from the humanities in order to avoid the twin pitfalls of masking important local patterns of change, and of environmental determinism.

Environmental factors and features that influence visual search in a 3D WIMP interface

The challenge of moving past the classic Window Icons Menus Pointer (WIMP) interface, i.e. by turning it ‘3D’, has resulted in much research and development. To evaluate the impact of 3D on the ‘finding a target picture in a folder’ task, we built a 3D WIMP interface that allowed the systematic manipulation of visual depth, visual aides, semantic category distribution of targets versus non-targets; and the detailed measurement of lower-level stimuli features. Across two separate experiments, one large sample web-based experiment, to understand associations, and one controlled lab environment, using eye tracking to understand user focus, we investigated how visual depth, use of visual aides, use of semantic categories, and lower-level stimuli features (i.e. contrast, colour and luminance) impact how successfully participants are able to search for, and detect, the target image. Moreover in the lab-based experiment, we captured pupillometry measurements to allow consideration of the influence of increasing cognitive load as a result of either an increasing number of items on the screen, or due to the inclusion of visual depth. Our findings showed that increasing the visible layers of depth, and inclusion of converging lines, did not impact target detection times, errors, or failure rates. Low-level features, including colour, luminance, and number of edges, did correlate with differences in target detection times, errors, and failure rates. Our results also revealed that semantic sorting algorithms significantly decreased target detection times. Increased semantic contrasts between a target and its neighbours correlated with an increase in detection errors. Finally, pupillometric data did not provide evidence of any correlation between the number of visible layers of depth and pupil size, however, using structural equation modelling, we demonstrated that cognitive load does influence detection failure rates when there is luminance contrasts between the target and its surrounding neighbours. Results suggest that WIMP interaction designers should consider stimulus-driven factors, which were shown to influence the efficiency with which a target icon can be found in a 3D WIMP interface.

Interfaces culturais de Mallarmé à World Wide Web

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Deep learning architecture for gesture recognition

Pós-graduação em Engenharia Mecânica - FEG

Modelling shared attention through relational reinforcement learning

Shared attention is a type of communication very important among human beings. It is sometimes reserved for the more complex form of communication being constituted by a sequence of four steps: mutual gaze, gaze following, imperative pointing and declarative pointing. Some approaches have been proposed in Human-Robot Interaction area to solve part of shared attention process, that is, the most of works proposed try to solve the first two steps. Models based on temporal difference, neural networks, probabilistic and reinforcement learning are methods used in several works. In this article, we are presenting a robotic architecture that provides a robot or agent, the capacity of learning mutual gaze, gaze following and declarative pointing using a robotic head interacting with a caregiver. Three learning methods have been incorporated to this architecture and a comparison of their performance has been done to find the most adequate to be used in real experiment. The learning capabilities of this architecture have been analyzed by observing the robot interacting with the human in a controlled environment. The experimental results show that the robotic head is able to produce appropriate behavior and to learn from sociable interaction.