On-line policy optimisation of Bayesian spoken dialogue systems via human interaction

A partially observable Markov decision process has been proposed as a dialogue model that enables robustness to speech recognition errors and automatic policy optimisation using reinforcement learning (RL). However, conventional RL algorithms require a very large number of dialogues, necessitating a user simulator. Recently, Gaussian processes have been shown to substantially speed up the optimisation, making it possible to learn directly from interaction with human users. However, early studies have been limited to very low dimensional spaces and the learning has exhibited convergence problems. Here we investigate learning from human interaction using the Bayesian Update of Dialogue State system. This dynamic Bayesian network based system has an optimisation space covering more than one hundred features, allowing a wide range of behaviours to be learned. Using an improved policy model and a more robust reward function, we show that stable learning can be achieved that significantly outperforms a simulator trained policy. © 2013 IEEE.

REVERIE: Natural Human Interaction in Virtual Immersive Environments

REVERIE (REal and Virtual Engagement in Realistic Immersive Environments [1]) targets novel research to address the demanding challenges involved with developing state-of-the-art technologies for online human interaction. The REVERIE framework enables users to meet, socialise and share experiences online by integrating cutting-edge technologies for 3D data acquisition and processing, networking, autonomy and real-time rendering. In this paper, we describe the innovative research that is showcased through the REVERIE integrated framework through richly defined use-cases which demonstrate the validity and potential for natural interaction in a virtual immersive and safe environment. Previews of the REVERIE demo and its key research components can be viewed at www.youtube.com/user/REVERIEFP7.

Spiteful Strategies. The ontogeny and the practice of spite in human interaction: an experimental game theory approach and empirical applications

As estratégias de malevolência implicam que um indivíduo pague um custo para infligir um custo superior a um oponente. Como um dos comportamentos fundamentais da sociobiologia, a malevolência tem recebido menos atenção que os seus pares o egoísmo e a cooperação. Contudo, foi estabelecido que a malevolência é uma estratégia viável em populações pequenas quando usada contra indivíduos negativamente geneticamente relacionados pois este comportamento pode i) ser eliminado naturalmente, ou ii) manter-se em equilíbrio com estratégias cooperativas devido à disponibilidade da parte de indivíduos malevolentes de pagar um custo para punir. Esta tese propõe compreender se a propensão para a malevolência nos humanos é inerente ou se esta se desenvolve com a idade. Para esse efeito, considerei duas experiências de teoria de jogos em crianças em ambiente escolar com idades entre os 6 e os 22 anos. A primeira, um jogo 2x2 foi testada com duas variantes: 1) um prémio foi atribuído a ambos os jogadores, proporcionalmente aos pontos acumulados; 2), um prémio foi atribuído ao jogador com mais pontos. O jogo foi desenhado com o intuito de causar o seguinte dilema a cada jogador: i) maximizar o seu ganho e arriscar ter menos pontos que o adversário; ou ii) decidir não maximizar o seu ganho, garantindo que este não era inferior ao do seu adversário. A segunda experiência consistia num jogo do ditador com duas opções: uma escolha egoísta/altruísta (A), onde o ditador recebia mais ganho, mas o seu recipiente recebia mais que ele e uma escolha malevolente (B) que oferecia menos ganhos ao ditador que a A mas mais ganhos que o recipiente. O dilema era que se as crianças se comportassem de maneira egoísta, obtinham mais ganho para si, ao mesmo tempo que aumentavam o ganho do seu colega. Se fossem malevolentes, então prefeririam ter mais ganho que o seu colega ao mesmo tempo que tinham menos para eles próprios. As experiências foram efetuadas em escolas de duas áreas distintas de Portugal (continente e Açores) para perceber se as preferências malevolentes aumentavam ou diminuíam com a idade. Os resultados na primeira experiência sugerem que (1) os alunos compreenderam a primeira variante como um jogo de coordenação e comportaram-se como maximizadores, copiando as jogadas anteriores dos seus adversários; (2) que os alunos repetentes se comportaram preferencialmente como malevolentes, mais frequentemente que como maximizadores, com especial ênfase para os alunos de 14 anos; (3) maioria dos alunos comportou-se reciprocamente desde os 12 até aos 16 anos de idade, após os quais começaram a desenvolver uma maior tolerância às escolhas dos seus parceiros. Os resultados da segunda experiência sugerem que (1) as estratégias egoístas eram prevalentes até aos 6 anos de idade, (2) as tendências altruístas emergiram até aos 8 anos de idade e (3) as estratégias de malevolência começaram a emergir a partir dos 8 anos de idade. Estes resultados complementam a literatura relativamente escassa sobre malevolência e sugerem que este comportamento está intimamente ligado a preferências de consideração sobre os outros, o paroquialismo e os estágios de desenvolvimento das crianças.************************************************************Spite is defined as an act that causes loss of payoff to an opponent at a cost to the actor. As one of the four fundamental behaviours in sociobiology, it has received far less attention than its counterparts selfishness and cooperation. It has however been established as a viable strategy in small populations when used against negatively related individuals. Because of this, spite can either i) disappear or ii) remain at equilibrium with cooperative strategies due to the willingness of spiteful individuals to pay a cost in order to punish. This thesis sets out to understand whether propensity for spiteful behaviour is inherent or if it develops with age. For that effect, two game-theoretical experiments were performed with schoolboys and schoolgirls aged 6 to 22. The first, a 2 x 2 game, was tested in two variants: 1) a prize was awarded to both players, proportional to accumulated points; 2), a prize was given to the player with most points. Each player faced the following dilemma: i) to maximise pay-off risking a lower pay-off than the opponent; or ii) not to maximise pay-off in order to cut down the opponent below their own. The second game was a dictator experiment with two choices, (A) a selfish/altruistic choice affording more payoff to the donor than B, but more to the recipient than to the donor, and (B) a spiteful choice that afforded less payoff to the donor than A, but even lower payoff to the recipient. The dilemma here was that if subjects behaved selfishly, they obtained more payoff for themselves, while at the same time increasing their opponent payoff. If they were spiteful, they would rather have more payoff than their colleague, at the cost of less for themselves. Experiments were run in schools in two different areas in Portugal (mainland and Azores) to understand whether spiteful preferences varied with age. Results in the first experiment suggested that (1) students understood the first variant as a coordination game and engaged in maximising behaviour by copying their opponent’s plays; (2) repeating students preferentially engaged in spiteful behaviour more often than maximising behaviour, with special emphasis on 14 year-olds; (3) most students engaged in reciprocal behaviour from ages 12 to 16, as they began developing higher tolerance for their opponent choices. Results for the second experiment suggested that (1) selfish strategies were prevalent until the age of 6, (2) altruistic tendencies emerged since then, and (3) spiteful strategies began being chosen more often by 8 year-olds. These results add to the relatively scarce body of literature on spite and suggest that this type of behaviour is closely tied with other-regarding preferences, parochialism and the children’s stages of development.

Robot-human interaction - practical experiments with a cyborg

This paper presents results to indicate the potential applications of a direct connection between the human nervous system and a computer network. Actual experimental results obtained from a human subject study are given, with emphasis placed on the direct interaction between the human nervous system and possible extra-sensory input. An brief overview of the general state of neural implants is given, as well as a range of application areas considered. An overall view is also taken as to what may be possible with implant technology as a general purpose human-computer interface for the future.

Fingertips detection for human interaction system

Fingertips of human hand play an important role in hand-based interaction with computers. Identification of fingertips' positions in hand images is vital for developing a human computer interaction system. This paper proposes a novel method for detecting fingertips of a hand image analyzing the concept of the geometrical structural information of fingers. The research is divided into three parts: First, hand image is segmented for detecting hand, Second, invariant features (curvature zero-crossing points) are extracted from the boundary of the hand, Third, fingertips are detected. Experimental results show that the proposed approach is promising.

Recognition: combining human interaction and a digital performing agent

Virtual and augmented environments are often dependent on human intervention for change to occur. However there are times when it would be advantageous for appropriate human-like activity to still occur when there are no humans present. In this paper, we describe the installation art piece Recognition, which uses the movement of human participants to effect change, and the movement of a performing agent when there are no humans present. The agent's Artificial Neural Network has learnt appropriate movements from a dancer and is able to generate suitable movement for the main avatar in the absence of human participants.

Risk assessment for public health from human interaction with ornamental waters

Water is essential to life; nevertheless ingestion of contaminated water could result in death caused by waterborne diseases such as cholera. Pathogens present in the water can cause diseases, other than those resulting from water ingestion, being registered an increase in the number of case reports in recent years. It is not clear if this increase is due either to a better case reporting system or to an increase in microorganism’s virulence. The generalized use of antibiotics in agriculture and animal farming contributed to their dissemination in the environment which promotes microorganism selection and emergence of resistant strains. This phenomenon can be enhanced by the ability of microorganism to persist within complex communities known as biofilms. In the present work we aim to characterize the microbial population present in ornamental waters and perform a risk assessment for public health resulting from human interaction with it.

Influencing human interaction in public schools : studies of IPR effectiveness as an inservice teacher training program /

Research performed under contract with the National Institute of Mental Health.

On-line policy optimisation of spoken dialogue systems via live interaction with human subjects

Statistical dialogue models have required a large number of dialogues to optimise the dialogue policy, relying on the use of a simulated user. This results in a mismatch between training and live conditions, and significant development costs for the simulator thereby mitigating many of the claimed benefits of such models. Recent work on Gaussian process reinforcement learning, has shown that learning can be substantially accelerated. This paper reports on an experiment to learn a policy for a real-world task directly from human interaction using rewards provided by users. It shows that a usable policy can be learnt in just a few hundred dialogues without needing a user simulator and, using a learning strategy that reduces the risk of taking bad actions. The paper also investigates adaptation behaviour when the system continues learning for several thousand dialogues and highlights the need for robustness to noisy rewards. © 2011 IEEE.

Desenvolvimento de uma base de dados para fatores de transcrição de seres humanos e suas redes de interação: Human Transcriptional Regulation Interaction Database (HTRIDB 2.0)

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Designing to maintain human agency in context-aware systems

This paper describes a series of design games, specifically aimed at exploring shifts in human agency, how they are managed, and the impact this will have on the design of future context-aware applications. The games focussed on understanding information handling issues in dental practice with participants from the University of Queensland Dental School playing an active role in the activities. Participatory design activities reveal how technology solution impact on dental practices. By finding methods of representing technological possibilities in ways which can easily be understood we enhance the contribution that dentists can make to the design process.