Robust speech recognition using speech enhancement

Automatic Speech Recognition (ASR) has matured into a technology which is becoming more common in our everyday lives, and is emerging as a necessity to minimise driver distraction when operating in-car systems such as navigation and infotainment. In “noise-free” environments, word recognition performance of these systems has been shown to approach 100%, however this performance degrades rapidly as the level of background noise is increased. Speech enhancement is a popular method for making ASR systems more ro- bust. Single-channel spectral subtraction was originally designed to improve hu- man speech intelligibility and many attempts have been made to optimise this algorithm in terms of signal-based metrics such as maximised Signal-to-Noise Ratio (SNR) or minimised speech distortion. Such metrics are used to assess en- hancement performance for intelligibility not speech recognition, therefore mak- ing them sub-optimal ASR applications. This research investigates two methods for closely coupling subtractive-type enhancement algorithms with ASR: (a) a computationally-efficient Mel-filterbank noise subtraction technique based on likelihood-maximisation (LIMA), and (b) in- troducing phase spectrum information to enable spectral subtraction in the com- plex frequency domain. Likelihood-maximisation uses gradient-descent to optimise parameters of the enhancement algorithm to best fit the acoustic speech model given a word se- quence known a priori. Whilst this technique is shown to improve the ASR word accuracy performance, it is also identified to be particularly sensitive to non-noise mismatches between the training and testing data. Phase information has long been ignored in spectral subtraction as it is deemed to have little effect on human intelligibility. In this work it is shown that phase information is important in obtaining highly accurate estimates of clean speech magnitudes which are typically used in ASR feature extraction. Phase Estimation via Delay Projection is proposed based on the stationarity of sinusoidal signals, and demonstrates the potential to produce improvements in ASR word accuracy in a wide range of SNR. Throughout the dissertation, consideration is given to practical implemen- tation in vehicular environments which resulted in two novel contributions – a LIMA framework which takes advantage of the grounding procedure common to speech dialogue systems, and a resource-saving formulation of frequency-domain spectral subtraction for realisation in field-programmable gate array hardware. The techniques proposed in this dissertation were evaluated using the Aus- tralian English In-Car Speech Corpus which was collected as part of this work. This database is the first of its kind within Australia and captures real in-car speech of 50 native Australian speakers in seven driving conditions common to Australian environments.

A hydrogen tethered inhibitor of matrix metalloproteinases

During wound repair, the balance between matrix metalloproteinases (MMPs) and their natural inhibitors (the TIMPs) is crucial for the normal extra cellular matrix turnover. However, the over expression of several MMPs including MMP-1, 2, 3, 8, 9 and MMP-10, combined with abnormally high levels of activation or low expression of TIMPs, may contribute to excessive degradation of connective tissue and formation of chronic ulcers. There are many groups exploring strategies for promoting wound healing involving delivery of growth factors, cells, ECM components and small molecules. Our approach for improving the balance of MMPs is not to add anything more to the wound, but instead to neutralise the over-expressed MMPs using inhibitors tethered to a bandage-like hydrogel. Our in vitro experiments using designed synthetic pseudo peptide inhibitors have been demonstrated to inhibit MMP activity in standard solutions. These inhibitors have also been tethered to polyethylene glycol hydrogels using a facile reaction between the linker unit on the inhibitor and the gel. After tethering the inhibition of MMPs diminishes to some extent and we postulate that this arises due to poor diffusion of the MMPs into the gels. When the tethered inhibitors were tested against chronic wound fluid obtained against patients we observed over 40% inhibition in proteolytic activity suggesting our approach may prove useful in rebalancing MMPs within chronic wounds.

Analysis and detection of cognitive load and frustration in drivers' speech

Non-driving related cognitive load and variations of emotional state may impact a driver’s capability to control a vehicle and introduces driving errors. Availability of reliable cognitive load and emotion detection in drivers would benefit the design of active safety systems and other intelligent in-vehicle interfaces. In this study, speech produced by 68 subjects while driving in urban areas is analyzed. A particular focus is on speech production differences in two secondary cognitive tasks, interactions with a co-driver and calls to automated spoken dialog systems (SDS), and two emotional states during the SDS interactions - neutral/negative. A number of speech parameters are found to vary across the cognitive/emotion classes. Suitability of selected cepstral- and production-based features for automatic cognitive task/emotion classification is investigated. A fusion of GMM/SVM classifiers yields an accuracy of 94.3% in cognitive task and 81.3% in emotion classification.

A peptidomimetic inhibitor of matrix metalloproteinases containing a tetherable linker group

Successful wound repair and normal turnover of the extracellular matrix relies on a balance between matrix metalloproteinases (MMPs) and their natural inhibitors (the TIMPs). When over-expression of MMPs and abnormally high levels of activation or low expression of TIMPs are encountered, excessive degradation of connective tissue and the formation of chronic ulcers can occur. One strategy to rebalance MMPs and TIMPs is to use inhibitors. We have designed a synthetic pseudopeptide inhibitor with an amine linker group based on a known high-affinity peptidomimetic MMP inhibitor have demonstrated inhibition of MMP-1, -2, -3 and -9 activity in standard solutions. The inhibitor was also tethered to a polyethylene glycol hydrogel using a facile reaction between the linker unit on the inhibitor and the hydrogel precursors. After tethering, we observed inhibition of the MMPs although there was an increase in the IC50s which was attributed to poor diffusion of the MMPs into the hydrogels, reduced activity of the tethered inhibitor or incomplete incorporation of the inhibitor into the hydrogels. When the tethered inhibitors were tested against chronic wound fluid we observed significant inhibition in proteolytic activity suggesting our approach may prove useful in rebalancing MMPs within chronic wounds.

The effect of dialect mismatch on likelihood-maximising speech enhancement for noise-robust speech recognition

Traditional speech enhancement methods optimise signal-level criteria such as signal-to-noise ratio, but these approaches are sub-optimal for noise-robust speech recognition. Likelihood-maximising (LIMA) frameworks are an alternative that optimise parameters of enhancement algorithms based on state sequences generated for utterances with known transcriptions. Previous reports of LIMA frameworks have shown significant promise for improving speech recognition accuracies under additive background noise for a range of speech enhancement techniques. In this paper we discuss the drawbacks of the LIMA approach when multiple layers of acoustic mismatch are present – namely background noise and speaker accent. Experimentation using LIMA-based Mel-filterbank noise subtraction on American and Australian English in-car speech databases supports this discussion, demonstrating that inferior speech recognition performance occurs when a second layer of mismatch is seen during evaluation.

The application of phonetic distribution normalisation to likelihood-maximising speech enhancement for robust ASR

Traditional speech enhancement methods optimise signal-level criteria such as signal-to-noise ratio, but such approaches are sub-optimal for noise-robust speech recognition. Likelihood-maximising (LIMA) frameworks on the other hand, optimise the parameters of speech enhancement algorithms based on state sequences generated by a speech recogniser for utterances of known transcriptions. Previous applications of LIMA frameworks have generated a set of global enhancement parameters for all model states without taking in account the distribution of model occurrence, making optimisation susceptible to favouring frequently occurring models, in particular silence. In this paper, we demonstrate the existence of highly disproportionate phonetic distributions on two corpora with distinct speech tasks, and propose to normalise the influence of each phone based on a priori occurrence probabilities. Likelihood analysis and speech recognition experiments verify this approach for improving ASR performance in noisy environments.

A multi-disciplinary approach for the design and management of airport terminals

Multi-disciplinary approaches to complex problems are becoming more common – they enable criteria manifested in distinct (and potentially conflicting) domains to be jointly balanced and satisfied. In this paper we present airport terminals as a case study which requires multi-disciplinary knowledge in order to balance conflicting security, economic and passenger-driven needs and correspondingly enhance the design, management and operation of airport terminals. The need for a truly multi-disciplinary scientific approach which integrates information, process, people, technology and space domains is highlighted through a brief discussion of two challenges currently faced by airport operators. The paper outlines the approach taken by this project, detailing the aims and objectives of each of seven diverse research programs.

The use of phase in complex spectrum subtraction for robust speech recognition

In this paper we propose a new method for utilising phase information by complementing it with traditional magnitude-only spectral subtraction speech enhancement through Complex Spectrum Subtraction (CSS). The proposed approach has the following advantages over traditional magnitude-only spectral subtraction: (a) it introduces complementary information to the enhancement algorithm; (b) it reduces the total number of algorithmic parameters, and; (c) is designed for improving clean speech magnitude spectra and is therefore suitable for both automatic speech recognition (ASR) and speech perception applications. Oracle-based ASR experiments verify this approach, showing an average of 20% relative word accuracy improvements when accurate estimates of the phase spectrum are available. Based on sinusoidal analysis and assuming stationarity between observations (which is shown to be better approximated as the frame rate is increased), this paper also proposes a novel method for acquiring the phase information called Phase Estimation via Delay Projection (PEDEP). Further oracle ASR experiments validate the potential for the proposed PEDEP technique in ideal conditions. Realistic implementation of CSS with PEDEP shows performance comparable to state of the art spectral subtraction techniques in a range of 15-20 dB signal-to-noise ratio environments. These results clearly demonstrate the potential for using phase spectra in spectral subtractive enhancement applications, and at the same time highlight the need for deriving more accurate phase estimates in a wider range of noise conditions.

Ions in motor vehicle exhaust and their dispersion near busy roads

Measurements in the exhaust plume of a petrol-driven motor car showed that molecular cluster ions of both signs were present in approximately equal amounts. The emission rate increased sharply with engine speed while the charge symmetry remained unchanged. Measurements at the kerbside of nine motorways and five city roads showed that the mean total cluster ion concentration near city roads (603 cm-3) was about one-half of that near motorways (1211 cm-3) and about twice as high as that in the urban background (269 cm-3). Both positive and negative ion concentrations near a motorway showed a significant linear increase with traffic density (R2=0.3 at p<0.05) and correlated well with each other in real time (R2=0.87 at p<0.01). Heavy duty diesel vehicles comprised the main source of ions near busy roads. Measurements were conducted as a function of downwind distance from two motorways carrying around 120-150 vehicles per minute. Total traffic-related cluster ion concentrations decreased rapidly with distance, falling by one-half from the closest approach of 2m to 5m of the kerb. Measured concentrations decreased to background at about 15m from the kerb when the wind speed was 1.3 m s-1, this distance being greater at higher wind speed. The number and net charge concentrations of aerosol particles were also measured. Unlike particles that were carried downwind to distances of a few hundred metres, cluster ions emitted by motor vehicles were not present at more than a few tens of metres from the road.

Autonomous underwater vehicle trajectory design coupled with predictive ocean models : a case study

Data collection using Autonomous Underwater Vehicles (AUVs) is increasing in importance within the oceano- graphic research community. Contrary to traditional moored or static platforms, mobile sensors require intelligent planning strategies to manoeuvre through the ocean. However, the ability to navigate to high-value locations and collect data with specific scientific merit is worth the planning efforts. In this study, we examine the use of ocean model predictions to determine the locations to be visited by an AUV, and aid in planning the trajectory that the vehicle executes during the sampling mission. The objectives are: a) to provide near-real time, in situ measurements to a large-scale ocean model to increase the skill of future predictions, and b) to utilize ocean model predictions as a component in an end-to-end autonomous prediction and tasking system for aquatic, mobile sensor networks. We present an algorithm designed to generate paths for AUVs to track a dynamically evolving ocean feature utilizing ocean model predictions. This builds on previous work in this area by incorporating the predicted current velocities into the path planning to assist in solving the 3-D motion planning problem of steering an AUV between two selected locations. We present simulation results for tracking a fresh water plume by use of our algorithm. Additionally, we present experimental results from field trials that test the skill of the model used as well as the incorporation of the model predictions into an AUV trajectory planner. These results indicate a modest, but measurable, improvement in surfacing error when the model predictions are incorporated into the planner.

Experimental study of the concentration field of discharge from a boat propeller

Two-stroke outboard boat engines using total loss lubrication deposit a significant proportion of their lubricant and fuel directly into the water. The purpose of this work is to document the velocity and concentration field characteristics of a submerged swirling water jet emanating from a propeller in order to provide information on its fundamental characteristics. The properties of the jet were examined far enough downstream to be relevant to the eventual modelling of the mixing problem. Measurements of the velocity and concentration field were performed in a turbulent jet generated by a model boat propeller (0.02 m diameter) operating at 1500 rpm and 3000 rpm in a weak co-flow of 0.04 m/s. The measurements were carried out in the Zone of Established Flow up to 50 propeller diameters downstream of the propeller, which was placed in a glass-walled flume 0.4 m wide with a free surface depth of 0.15 m. The jet and scalar plume development were compared to that of a classical free round jet. Further, results pertaining to radial distribution, self similarity, standard deviation growth, maximum value decay and integral fluxes of velocity and concentration were presented and fitted with empirical correlations. Furthermore, propeller induced mixing and pollutant source concentration from a two-stroke engine were estimated.

Check-in processing : simulation of passengers with advanced traits

In order to tackle the growth of air travelers in airports worldwide, it is important to simulate and understand passenger flows to predict future capacity constraints and levels of service. We discuss the ability of agent-based models to understand complicated pedestrian movement in built environments. In this paper we propose advanced passenger traits to enable more detailed modelling of behaviors in terminal buildings, particularly in the departure hall around the check-in facilities. To demonstrate the concepts, we perform a series of passenger agent simulations in a virtual airport terminal. In doing so, we generate a spatial distribution of passengers within the departure hall to ancillary facilities such as cafes, information kiosks and phone booths as well as common check-in facilities, and observe the effects this has on passenger check-in and departure hall dwell times, and facility utilization.

Including airport duty-free shopping in arriving passenger simulation and the opportunities this presents

Simulating passenger flows within airports is very important as it can provide an indication of queue lengths, bottlenecks, system capacity and overall level of service. To date, visual simulation tools such as agent based models have focused on processing formalities such as check-in, and not incorporate discretionary activities such as duty-free shopping. As airport retail contributes greatly to airport revenue generation, but also has potentially detrimental effects on facilitation efficiency benchmarks, this study developed a simplistic simulation model which captures common duty-free purchasing opportunities, as well as high-level behaviours of passengers. It is argued that such a model enables more realistic simulation of passenger facilitation, and provides a platform for simulating real-time revenue generation as well as more complex passenger behaviours within the airport. Simulations are conducted to verify the suitability of the model for inclusion in the international arrivals process for assessing passenger flow and infrastructure utilization.