Robust indoor speaker recognition in a network of audio and video sensors

Situational awareness is achieved naturally by the human senses of sight and hearing in combination. Automatic scene understanding aims at replicating this human ability using microphones and cameras in cooperation. In this paper, audio and video signals are fused and integrated at different levels of semantic abstractions. We detect and track a speaker who is relatively unconstrained, i.e., free to move indoors within an area larger than the comparable reported work, which is usually limited to round table meetings. The system is relatively simple: consisting of just 4 microphone pairs and a single camera. Results show that the overall multimodal tracker is more reliable than single modality systems, tolerating large occlusions and cross-talk. System evaluation is performed on both single and multi-modality tracking. The performance improvement given by the audio–video integration and fusion is quantified in terms of tracking precision and accuracy as well as speaker diarisation error rate and precision–recall (recognition). Improvements vs. the closest works are evaluated: 56% sound source localisation computational cost over an audio only system, 8% speaker diarisation error rate over an audio only speaker recognition unit and 36% on the precision–recall metric over an audio–video dominant speaker recognition method.

Reassembled, Slightly Askew:Composition & Sound Design

Reassembled, Slightly Askew is an autobiographical, immersive audio-based artwork based on Shannon Sickels’ experience of falling critically ill with a rare brain infection and her journey of rehabilitation with an acquired brain injury. Audience members experience Reassembled individually, listening to the audio via headphones while lying on a bed. The piece makes use of binaural microphone technology and spatial sound design techniques, causing listeners to feel they are inside Shannon’s head, viscerally experiencing her descent into coma, brain surgeries, early days in the hospital, and re-integration into the world with a hidden disability. It is a new kind of storytelling, never done before about this topic, that places the listener safely in the first-person perspective with the aim of increasing empathy and understanding. Reassembled… was made through a 5-year collaboration with an interdisciplinary team of artists led by Shannon Sickels (writer & performer), Paul Stapleton (composer & sound designer), Anna Newell (director), Hanna Slattne (dramaturgy), Stevie Prickett (choreography), and Shannon’s consultant neurosurgeon and head injury nurse. It’s development and production has been made possible with the support of a Wellcome Trust Arts Award, the Arts Council NI, Sonic Arts Research Centre, Belfast's Metropolitan Arts Centre, and grants from the Arts & Disability Award Ireland scheme. In its 2015 premiere year, Reassembled had 99 shows across Northern Ireland, including at the Cathedral Quarter Arts Festival (the MAC, Belfast) and BOUNCE Arts & Disability Forum Festival (Lyric Theatre, Belfast). It was awarded 5 stars in the Stage, a Hospital Club h100 Theatre & Performance Award, and been shared at medical conferences and trainings across the UK. It continues to be presented in diverse artistic and educational contexts, including as part of A Nation’s Theatre Festival in 2016 at Battersea Arts Centre in London where it was given 4 star reviews in the Guardian, Time Out London and the Evening Standard. "A real-life ordeal, captured by a daring, disorientating artistic collaboration, which works brilliantly on so many levels…It should be available on prescription.” — The Stage ★★★★★ www.reassembled.co.uk Audio clips and documentary footage available here: http://www.paulstapleton.net/portfolio/reassembled-slightly-askew