Building Autonomous Sensitive Artificial Listeners

This paper describes a substantial effort to build a real-time interactive multimodal dialogue system with a focus on emotional and non-verbal interaction capabilities. The work is motivated by the aim to provide technology with competences in perceiving and producing the emotional and non-verbal behaviours required to sustain a conversational dialogue. We present the Sensitive Artificial Listener (SAL) scenario as a setting which seems particularly suited for the study of emotional and non- verbal behaviour, since it requires only very limited verbal understanding on the part of the machine. This scenario allows us to concentrate on non-verbal capabilities without having to address at the same time the challenges of spoken language understanding, task modeling etc. We first report on three prototype versions of the SAL scenario, in which the behaviour of the Sensitive Artificial Listener characters was determined by a human operator. These prototypes served the purpose of verifying the effectiveness of the SAL scenario and allowed us to collect data required for building system components for analysing and synthesising the respective behaviours. We then describe the fully autonomous integrated real-time system we created, which combines incremental analysis of user behaviour, dialogue management, and synthesis of speaker and listener behaviour of a SAL character displayed as a virtual agent. We discuss principles that should underlie the evaluation of SAL-type systems. Since the system is designed for modularity and reuse, and since it is publicly available, the SAL system has potential as a joint research tool in the affective computing research community.

Primitive living chitons: Morphological cladistic analysis as a model for character evaluation

Chitons are often referred to as “living fossils” in part because they are proposed as one of the earliest-diverging groups of living molluscs, but also because the gross morphology of the polyplacophoran shell has been conserved for hundreds of millions of years. As such, the analysis of evolution and radiation within polyplacophorans is of considerable interest not only for resolving the shape of pan-molluscan phylogeny but also as model organisms for the study of character evolution. This study presents a new, rigorous cladistic analysis of the morphological characters used in taxonomic descriptions for chitons in the living suborder Lepidopleurina Thiele, 1910 (the earliest-derived living group of chitons). Shell-based characters alone entirely fail to recover any recognized subdivisions within the group, which may raise serious questions about the application of fossil data (from isolated shell valves). New analysis including characters from girdle armature and gill arrangements recovers some genera within the group but also points to the lack of monophyly within the main genus Leptochiton Gray, 1847. Additional characters from molecular data and soft anatomy, used in combination, are clearly needed to resolve questions of chiton relationships. However, the data sets currently available already provide interesting insights into the analytical power of traditional morphology as well as some knowledge about the early evolution and radiation of this group.

Rotating Brains / Beating Heart:International collaborative performance with Stelarc, the virtual reality ensemble Avatar Orchestra Metaverse, Pauline Oliveros, Martin Parker and saxophonist Franziska Schroeder.

The work ROTATING BRAINS / BEATING HEART was specifically developed for the opening performance of the 2010 DRHA conference. The conference’s theme ‘Sensual Technologies: Collaborative Practices of Interdisciplinarity explored collaborative relationships between the body and sensual/sensing technologies across various disciplines, looking to new approaches offered by various emerging fields and practices that incorporate new and existing technologies. The conference had a specific focus on SecondLife with roundtable events and discussions, led by performance artist Stelarc, as well as international participation via SecondLife.
The collaboration between Stelarc, the Avatar Orchestra Metaverse (AOM) and myself as the DRHA2010 conference program chair was a unique occurrence for this conference.