Emergent behaviour: learning from an artificially intelligent performing software agent

This research investigates the possibility for emergent choreographic behaviour to arise from the interactions between a human dancer and a learning, digital performing agent. The cognitive framework is extended through theories of distributed cognition to take into account the two interacting agents rather than a single agent and its environment. The Artificial Neural Network based performing agent demonstrated emergent dance behaviour when performing live with the human dancer. The agent was able to follow the dancer, create movement phrases based on what the dancer was performing and recognize short movement phrases, as a result of the interaction of the dancer’s motion captured movement data and the agent’s artificial neural network. This emergent behaviour was not explicitly programmed, but emerged as a result of the learning process and the interactions with the human dancer.

Responding to human full-body gestures embedded in motion data streams.

 This research created a neural-network enabled artificially intelligent performing agent that was able to learn to dance and recognise movement through a rehearsal and performance process with a human dancer. The agent exhibited emergent dance behaviour and successfully engaged in a live, semi-improvised dance performance with the human dancer.

Emergence

Emergence, by John McCormick, Steph Hutchison and an emerging performing agent, is a duet performed between a human dancer and an artificially intelligent performing agent. The work premiered on 1 August 2014 at Metanoia Theatre, Brunswick.

Duet: Improvising spatial dialogues with an artificially intelligent agent

This paper presents an experimental framework for a virtual reality artwork, Duet, that employs a combination of live, full body motion capture and Oculus Rift HMD to construct an experience through which a human User can spatially interact with an artificially intelligent Agent. The project explores conceptual notions of embodied knowledge transfer, shared poetics of movement and distortions of the body schema. Within this context, both the User and the Agent become performers, constructing an intimate and spontaneously generated proximal space. The project generates a visualization of the relationship between the User and the Agent without the context of a fixed VR landscape or architecture. The Agent's ability to retain and accumulate movement knowledge in a way that mimics human learning transforms an interactive experience into a collaborative one. The virtual representation of both performers is distorted and amplified in a dynamic manner, enhancing the potential for creative dialogue between the Agent and the User.

Making motes intelligent:an agent-based approach to wireless sensor networks

In-Motes is a mobile agent middleware that generates an intelligent framework for deploying applications in Wireless Sensor Networks (WSNs). In-Motes is based on the injection of mobile agents into the network that can migrate or clone following specific rules and performing application specific tasks. By doing so, each mote is given a certain degree of perception, cognition and control, forming the basis for its intelligence. Our middleware incorporates technologies such as Linda-like tuplespaces and federated system architecture in order to obtain a high degree of collaboration and coordination for the agent society. A set of behavioral rules inspired by a community of bacterial strains is also generated as the means for robustness of the WSN. In this paper, we present In-Motes and provide a detailed evaluation of its implementation for MICA2 motes.

Teaching a digital performing agent: artificial neural network and hidden Markov Model for recognising and performing dance movement

For a Digital Performing Agent to be able to perform live with a human dancer, it would be useful for the agent to be able to contextualize the movement the dancer is performing and to have a suitable movement vocabulary with which to contribute to the performance. In this paper we will discuss our research into the use of Artificial Neural Networks (ANN) as a means of allowing a software agent to learn a shared vocabulary of movement from a dancer. The agent is able to use the learnt movements to form an internal representation of what the dancer is performing, allowing it to follow the dancer, generate movement sequences based on the dancer's current movement and dance independently of the dancer using a shared movement vocabulary. By combining the ANN with a Hidden Markov Model (HMM) the agent is able to recognize short full body movement phrases and respond when the dancer performs these phrases. We consider the relationship between the dancer and agent as a means of supporting the agent's learning and performance, rather than developing the agent's capability in a self-contained fashion.

Recognition: combining human interaction and a digital performing agent

Virtual and augmented environments are often dependent on human intervention for change to occur. However there are times when it would be advantageous for appropriate human-like activity to still occur when there are no humans present. In this paper, we describe the installation art piece Recognition, which uses the movement of human participants to effect change, and the movement of a performing agent when there are no humans present. The agent's Artificial Neural Network has learnt appropriate movements from a dancer and is able to generate suitable movement for the main avatar in the absence of human participants.

Argumentation and artifacts for intelligent multi-agent systems

Reasoning under uncertainty is a human capacity that in software system is necessary and often hidden. Argumentation theory and logic make explicit non-monotonic information in order to enable automatic forms of reasoning under uncertainty. In human organization Distributed Cognition and Activity Theory explain how artifacts are fundamental in all cognitive process. Then, in this thesis we search to understand the use of cognitive artifacts in an new argumentation framework for an agent-based artificial society.

An intelligent broker agent for energy trading:an MDP approach

This paper details the development and evaluation of AstonTAC, an energy broker that successfully participated in the 2012 Power Trading Agent Competition (Power TAC). AstonTAC buys electrical energy from the wholesale market and sells it in the retail market. The main focus of the paper is on the broker’s bidding strategy in the wholesale market. In particular, it employs Markov Decision Processes (MDP) to purchase energy at low prices in a day-ahead power wholesale market, and keeps energy supply and demand balanced. Moreover, we explain how the agent uses Non-Homogeneous Hidden Markov Model (NHHMM) to forecast energy demand and price. An evaluation and analysis of the 2012 Power TAC finals show that AstonTAC is the only agent that can buy energy at low price in the wholesale market and keep energy imbalance low.

Learning Data-Driven Models of Non-Verbal Behaviors for Building Rapport Using an Intelligent Virtual Agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness. Evidence-based patient-centered Brief Motivational Interviewing (BMI) interven- tions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary. Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems. To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].

Learning data-driven models of non-verbal behaviors for building rapport using an intelligent virtual agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness.^ Evidence-based patient-centered Brief Motivational Interviewing (BMI) interventions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary.^ Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems.^ To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].^

Collaborative dance between robot and human

Dance is an inherently embodied activity. The dancer is attuned to the effects of the physical world on her own physicality and the relationship of her presence to other dancers. This research is an investigation into artificially intelligent performing agents and robots and how a human dancer can guide the learning and performance of a robot performer. Using Artificial Neural Networks as the bases for the agent’s computational intelligence, performing agents were created that can perform by collaborating with human dancers through robots.

Building agent-based hybrid intelligent systems : a case study

Many complex problems (e.g., financial investment planning, foreign exchange trading, data mining from large/multiple databases) require hybrid intelligent systems that integrate many intelligent techniques (e.g., fuzzy logic, neural networks, and genetic algorithms). However, hybrid intelligent systems are difficult to develop because they have a large number of parts or components that have many interactions. On the other hand, agents offer a new and often more appropriate route to the development of complex systems, especially in open and dynamic environments. Thus, this paper discusses the development of an agent-based hybrid intelligent system for financial investment planning, in which a great number of heterogeneous computing techniques/packages are easily integrated into a unifying agent framework. This shows that agent technology can indeed facilitate the development of hybrid intelligent systems.

A multi-agent framework for financial investment planning

If a company or person wants to invest a lot of money, where, when, and how should the investment go? A multi-agent based Financial Investment Planner may give some reasonable answers to the above question. Good advice is mainly based on adequate information, rich knowledge, and great
skills to use knowledge and information. To this end, this planner consists of four principal components information gathering agents that are responsible for gathering relevant information on the Internet, data mining agents that are in charge of discovering knowledge from retrieved information as well as other relevant databases, group decision making agents that can effectively use available knowledge and appropriate information to make reasonable decisions (investment advice), and a graphical user interface that interacts with users. This paper is focused on the group decision making part. The design and implementation of an agent-based hybrid intelligent system - agent-based soft computing society are detailed.

Modelling financial investment planning from agent perspectives

Multi-agent Systems (MASs) offer strong models for representing complex and dynamic real-world environments. Taking financial investment planning as an example, this paper describes how to model complex systems from agent perspectives. Different agents and their behaviours are identified for financial investment planning. These agents are put together as an agent-based system. The experimental results show that all agents in the system can work cooperatively to provide reasonable investment advice. The system is very flexible and robust. The success of the system indicates that (MASs) can significantly facilitate the modelling of complex systems.