Building autonomous sensitive artificial listeners (Extended abstract)

This paper describes a substantial effort to build a real-time interactive multimodal dialogue system with a focus on emotional and non-verbal interaction capabilities. The work is motivated by the aim to provide technology with competences in perceiving and producing the emotional and non-verbal behaviours required to sustain a conversational dialogue. We present the Sensitive Artificial Listener (SAL) scenario as a setting which seems particularly suited for the study of emotional and non-verbal behaviour, since it requires only very limited verbal understanding on the part of the machine. This scenario allows us to concentrate on non-verbal capabilities without having to address at the same time the challenges of spoken language understanding, task modeling etc. We first summarise three prototype versions of the SAL scenario, in which the behaviour of the Sensitive Artificial Listener characters was determined by a human operator. These prototypes served the purpose of verifying the effectiveness of the SAL scenario and allowed us to collect data required for building system components for analysing and synthesising the respective behaviours. We then describe the fully autonomous integrated real-time system we created, which combines incremental analysis of user behaviour, dialogue management, and synthesis of speaker and listener behaviour of a SAL character displayed as a virtual agent. We discuss principles that should underlie the evaluation of SAL-type systems. Since the system is designed for modularity and reuse, and since it is publicly available, the SAL system has potential as a joint research tool in the affective computing research community.

Ethical reasoning through simulation: a phenomenological analysis of student experience

Background
Medical students transitioning into professional practice feel underprepared to deal with the emotional complexities of real-life ethical situations. Simulation-based learning (SBL) may provide a safe environment for students to probe the boundaries of ethical encounters. Published studies of ethics simulation have not generated sufficiently deep accounts of student experience to inform pedagogy. The aim of this study was to understand students’ lived experiences as they engaged with the emotional challenges of managing clinical ethical dilemmas within a SBL environment.

Methods
This qualitative study was underpinned by an interpretivist epistemology. Eight senior medical students participated in an interprofessional ward-based SBL activity incorporating a series of ethically challenging encounters. Each student wore digital video glasses to capture point-of-view (PoV) film footage. Students were interviewed immediately after the simulation and the PoV footage played back to them. Interviews were transcribed verbatim. An interpretative phenomenological approach, using an established template analysis approach, was used to iteratively analyse the data.

Results
Four main themes emerged from the analysis: (1) ‘Authentic on all levels?’, (2)‘Letting the emotions flow’, (3) ‘Ethical alarm bells’ and (4) ‘Voices of children and ghosts’. Students recognised many explicit ethical dilemmas during the SBL activity but had difficulty navigating more subtle ethical and professional boundaries. In emotionally complex situations, instances of moral compromise were observed (such as telling an untruth). Some participants felt unable to raise concerns or challenge unethical behaviour within the scenarios due to prior negative undergraduate experiences.

Conclusions
This study provided deep insights into medical students’ immersive and embodied experiences of ethical reasoning during an authentic SBL activity. By layering on the human dimensions of ethical decision-making, students can understand their personal responses to emotion, complexity and interprofessional working. This could assist them in framing and observing appropriate ethical and professional boundaries and help smooth the transition into clinical practice.

Hierarchical Task Network Planning with Common-Sense Reasoning for Multiple-People Behaviour Analysis

Safety on public transport is a major concern for the relevant authorities. We
address this issue by proposing an automated surveillance platform which combines data from video, infrared and pressure sensors. Data homogenisation and integration is achieved by a distributed architecture based on communication middleware that resolves interconnection issues, thereby enabling data modelling. A common-sense knowledge base models and encodes knowledge about public-transport platforms and the actions and activities of passengers. Trajectory data from passengers is modelled as a time-series of human activities. Common-sense knowledge and rules are then applied to detect inconsistencies or errors in the data interpretation. Lastly, the rationality that characterises human behaviour is also captured here through a bottom-up Hierarchical Task Network planner that, along with common-sense, corrects misinterpretations to explain passenger behaviour. The system is validated using a simulated bus saloon scenario as a case-study. Eighteen video sequences were recorded with up to six passengers. Four metrics were used to evaluate performance. The system, with an accuracy greater than 90% for each of the four metrics, was found to outperform a rule-base system and a system containing planning alone.