Robust indoor speaker recognition in a network of audio and video sensors

Situational awareness is achieved naturally by the human senses of sight and hearing in combination. Automatic scene understanding aims at replicating this human ability using microphones and cameras in cooperation. In this paper, audio and video signals are fused and integrated at different levels of semantic abstractions. We detect and track a speaker who is relatively unconstrained, i.e., free to move indoors within an area larger than the comparable reported work, which is usually limited to round table meetings. The system is relatively simple: consisting of just 4 microphone pairs and a single camera. Results show that the overall multimodal tracker is more reliable than single modality systems, tolerating large occlusions and cross-talk. System evaluation is performed on both single and multi-modality tracking. The performance improvement given by the audio–video integration and fusion is quantified in terms of tracking precision and accuracy as well as speaker diarisation error rate and precision–recall (recognition). Improvements vs. the closest works are evaluated: 56% sound source localisation computational cost over an audio only system, 8% speaker diarisation error rate over an audio only speaker recognition unit and 36% on the precision–recall metric over an audio–video dominant speaker recognition method.

Commentary on Publication of Lectures Act 1835

Legislation conferring the exclusive right of printing and publishing certain lectures for the same term of protection provided by the existing copyright legislation (see: Statute of Anne, uk_1710; Copyright Act, uk_1814). This was the first occasion on which the legislature extended copyright protection to works in the oral form. The legislation is of interest in terms of the distinction it draws between lectures delivered within the 'public' and the 'private' spheres (lectures delivered at a University, for example, are not protected), in terms of articulating the nature of the relationship between a speaker and his audience, and in specifically clarifying that newspapers are similarly prohibited from reporting protected lectures. The commentary explores the background to the passing of the Act, and in particular the role which Henry Brougham played in proposing and securing the same.

Building autonomous sensitive artificial listeners (Extended abstract)

This paper describes a substantial effort to build a real-time interactive multimodal dialogue system with a focus on emotional and non-verbal interaction capabilities. The work is motivated by the aim to provide technology with competences in perceiving and producing the emotional and non-verbal behaviours required to sustain a conversational dialogue. We present the Sensitive Artificial Listener (SAL) scenario as a setting which seems particularly suited for the study of emotional and non-verbal behaviour, since it requires only very limited verbal understanding on the part of the machine. This scenario allows us to concentrate on non-verbal capabilities without having to address at the same time the challenges of spoken language understanding, task modeling etc. We first summarise three prototype versions of the SAL scenario, in which the behaviour of the Sensitive Artificial Listener characters was determined by a human operator. These prototypes served the purpose of verifying the effectiveness of the SAL scenario and allowed us to collect data required for building system components for analysing and synthesising the respective behaviours. We then describe the fully autonomous integrated real-time system we created, which combines incremental analysis of user behaviour, dialogue management, and synthesis of speaker and listener behaviour of a SAL character displayed as a virtual agent. We discuss principles that should underlie the evaluation of SAL-type systems. Since the system is designed for modularity and reuse, and since it is publicly available, the SAL system has potential as a joint research tool in the affective computing research community.

Multimodal Education for Very Young Learners

Invited Plenary Speaker

Benthic assemblages associated with native and non-native oysters are similar

Invasive species can impact native species and alter assemblage structure, which affects associated ecosystem functioning. The pervasive Pacific oyster, Crassostrea gigas, has been shown to affect the diversity and composition of many host ecosystems. We tested for effects of the presence of the invasive C. gigas on native assemblages by comparing them directly to assemblages associated with the declining native European oyster, Ostrea edulis. The presence of both oyster species was manipulated in intertidal and subtidal habitats and reefs were constructed at horizontal and vertical orientation to the substratum. After 12 months, species diversity and benthic assemblage structure between assemblages with C. gigas and O. edulis were similar, but differed between habitats and orientation, suggesting that both oyster species were functionally similar in terms of biodiversity facilitation. These findings support evidence, that non-native species could play an important role in maintaining biodiversity in systems with declining populations of native species.