Time domain wave separation using multiple microphones

Methods of measuring the acoustic behavior of tubular systems can be broadly characterized as steady state measurements, where the measured signals are analyzed in terms of infinite duration sinusoids, and reflectometry measurements which exploit causality to separate the forward and backward going waves in a duct. This paper sets out a multiple microphone reflectometry technique which performs wave separation by using time domain convolution to track the forward and backward going waves in a cylindrical source tube. The current work uses two calibration runs (one for forward going waves and one for backward going waves) to measure the time domain transfer functions for each pair of microphones. These time domain transfer functions encode the time delay, frequency dependent losses and microphone gain ratios for travel between microphones. This approach is applied to the measurement of wave separation, bore profile and input impedance. The work differs from existing frequency domain methods in that it combines the information of multiple microphones within a time domain algorithm, and differs from existing time domain methods in its inclusion of the effect of losses and gain ratios in intermicrophone transfer functions.

Target Detection and Tracking With Heterogeneous Sensors

We present a multimodal detection and tracking algorithm for sensors composed of a camera mounted between two microphones. Target localization is performed on color-based change detection in the video modality and on time difference of arrival (TDOA) estimation between the two microphones in the audio modality. The TDOA is computed by multiband generalized cross correlation (GCC) analysis. The estimated directions of arrival are then postprocessed using a Riccati Kalman filter. The visual and audio estimates are finally integrated, at the likelihood level, into a particle filter (PF) that uses a zero-order motion model, and a weighted probabilistic data association (WPDA) scheme. We demonstrate that the Kalman filtering (KF) improves the accuracy of the audio source localization and that the WPDA helps to enhance the tracking performance of sensor fusion in reverberant scenarios. The combination of multiband GCC, KF, and WPDA within the particle filtering framework improves the performance of the algorithm in noisy scenarios. We also show how the proposed audiovisual tracker summarizes the observed scene by generating metadata that can be transmitted to other network nodes instead of transmitting the raw images and can be used for very low bit rate communication. Moreover, the generated metadata can also be used to detect and monitor events of interest.

Wideband measurement of the acoustic impedance of tubular objects

A method is discussed for measuring the acoustic impedance of tubular objects that gives accurate results for a wide range of frequencies. The apparatus that is employed is similar to that used in many previously developed methods; it consists of a cylindrical measurement duct fitted with several microphones, of which two are active in each measurement session, and a driver at one of its ends. The object under study is fitted at the other end. The impedance of the object is determined from the microphone signals obtained during excitation of the air inside the 1 duct by the driver, and from three coefficients that are pre-determined using four calibration measurements with closed cylindrical tubes. The calibration procedure is based on the simple mathematical relationships between the impedances of the calibration tubes, and does not require knowledge of the propagation constant. Measurements with a cylindrical tube yield an estimate of the attenuation constant for plane waves, which is found to differ from the theoretical prediction by less than 1.4% in the frequency range 1 kHz-20 kHz. Impedance measurements of objects with abrupt changes in diameter are found to be in good agreement with multimodal theory.

A Frequency Domain Algorithm for Wave Separation

We propose a frequency domain adaptive algorithm for
wave separation in wind instruments. Forward and backward travelling waves are obtained from the signals acquired by two microphones placed along the tube, while the
separation ?lter is adapted from the information given by a
third microphone. Working in the frequency domain has a
series of advantages, among which are the ease of design of
the propagation ?lter and its differentiation with respect to
its parameters.
Although the adaptive algorithm was developed as a ?rst
step for the estimation of playing parameters in wind instruments it can also be used, without any modi?cations, for
other applications such as in-air direction of arrival (DOA)
estimation. Preliminary results on these applications will
also be presented.

The SEMAINE Database: Annotated Multimodal Records of Emotionally Colored Conversations between a Person and a Limited Agent

SEMAINE has created a large audiovisual database as a part of an iterative approach to building Sensitive Artificial Listener (SAL) agents that can engage a person in a sustained, emotionally colored conversation. Data used to build the agents came from interactions between users and an operator simulating a SAL agent, in different configurations: Solid SAL (designed so that operators displayed an appropriate nonverbal behavior) and Semi-automatic SAL (designed so that users' experience approximated interacting with a machine). We then recorded user interactions with the developed system, Automatic SAL, comparing the most communicatively competent version to versions with reduced nonverbal skills. High quality recording was provided by five high-resolution, high-framerate cameras, and four microphones, recorded synchronously. Recordings total 150 participants, for a total of 959 conversations with individual SAL characters, lasting approximately 5 minutes each. Solid SAL recordings are transcribed and extensively annotated: 6-8 raters per clip traced five affective dimensions and 27 associated categories. Other scenarios are labeled on the same pattern, but less fully. Additional information includes FACS annotation on selected extracts, identification of laughs, nods, and shakes, and measures of user engagement with the automatic system. The material is available through a web-accessible database. © 2010-2012 IEEE.

Investigation on localisation accuracy for first and higher order ambisonics reproduced sound sources

Ambisonics and higher order ambisonics (HOA) technologies aim at reproducing sound field either synthesised or previously recorded with dedicated microphones. Based on a spherical harmonic decomposition, the sound field is more precisely described when higher-order components are used. The presented study evaluated the perceptual and objective localisation accuracy of the sound field encoded with four microphones of order one to four and decoded over a ring of loudspeakers. A perceptual test showed an improvement of the localisation with higher order ambisonic microphones. Reproduced localisation indices were estimated for the four microphones and the respective synthetic systems of order one to four. The perceptual and objective analysis revealed the same conclusions. The localisation accuracy depends on the ambisonic order as well as the source incidence. Furthermore, impairments linked to the microphones were highlighted.

Adaptive calibration of a three-microphone system for acoustic waveguide characterization under time-varying conditions

The pressure and velocity field in a one-dimensional acoustic waveguide can be sensed in a non-intrusive manner using spatially distributed microphones. Experimental characterization with sensor arrangements of this type has many applications in measurement and control. This paper presents a method for measuring the acoustic variables in a duct under fluctuating propagation conditions with specific focus on in-system calibration and tracking of the system parameters of a three-microphone measurement configuration. The tractability of the non-linear optimization problem that results from taking a parametric approach is investigated alongside the influence of extraneous measurement noise on the parameter estimates. The validity and accuracy of the method are experimentally assessed in terms of the ability of the calibrated system to separate the propagating waves under controlled conditions. The tracking performance is tested through measurements with a time-varying mean flow, including an experiment conducted under propagation conditions similar to those in a wind instrument during playing.

The SEMAINE corpus of emotionally coloured character interactions

We have recorded a new corpus of emotionally coloured conversations. Users were recorded while holding conversations with an operator who adopts in sequence four roles designed to evoke emotional reactions. The operator and the user are seated in separate rooms; they see each other through teleprompter screens, and hear each other through speakers. To allow high quality recording, they are recorded by five high-resolution, high framerate cameras, and by four microphones. All sensor information is recorded synchronously, with an accuracy of 25 μs. In total, we have recorded 20 participants, for a total of 100 character conversational and 50 non-conversational recordings of approximately 5 minutes each. All recorded conversations have been fully transcribed and annotated for five affective dimensions and partially annotated for 27 other dimensions. The corpus has been made available to the scientific community through a web-accessible database.

Petals of Resonance: A performance system for the transformation of small body sounds

A system of software and hardware that combines signal processing and contact microphones using normally inaudible body sounds, including heartbeat/pulse, respiration and internal sounds from the vocal tract that can be heard internally by the performer but not externally by others, to drive resonant filters. Performance at SARC Sonic Lab, Belfast, 19 Feb 2015 in collaboration with Birgit Ulher.

Feedback Networks and Self-Oscillating Objects

A system of self-designed microphones, speakers and transducers creating performable feedback networks and self-oscillating objects. Performance SARC Sonic Lab, Belfast, 18 March 2015

Robust indoor speaker recognition in a network of audio and video sensors

Situational awareness is achieved naturally by the human senses of sight and hearing in combination. Automatic scene understanding aims at replicating this human ability using microphones and cameras in cooperation. In this paper, audio and video signals are fused and integrated at different levels of semantic abstractions. We detect and track a speaker who is relatively unconstrained, i.e., free to move indoors within an area larger than the comparable reported work, which is usually limited to round table meetings. The system is relatively simple: consisting of just 4 microphone pairs and a single camera. Results show that the overall multimodal tracker is more reliable than single modality systems, tolerating large occlusions and cross-talk. System evaluation is performed on both single and multi-modality tracking. The performance improvement given by the audio–video integration and fusion is quantified in terms of tracking precision and accuracy as well as speaker diarisation error rate and precision–recall (recognition). Improvements vs. the closest works are evaluated: 56% sound source localisation computational cost over an audio only system, 8% speaker diarisation error rate over an audio only speaker recognition unit and 36% on the precision–recall metric over an audio–video dominant speaker recognition method.