Lip detection for audio-visual speech recognition in-car environment

Acoustically, car cabins are extremely noisy and as a consequence audio-only, in-car voice recognition systems perform poorly. As the visual modality is immune to acoustic noise, using the visual lip information from the driver is seen as a viable strategy in circumventing this problem by using audio visual automatic speech recognition (AVASR). However, implementing AVASR requires a system being able to accurately locate and track the drivers face and lip area in real-time. In this paper we present such an approach using the Viola-Jones algorithm. Using the AVICAR [1] in-car database, we show that the Viola- Jones approach is a suitable method of locating and tracking the driver’s lips despite the visual variability of illumination and head pose for audio-visual speech recognition system.

Audio visual automatic speech recognition in vehicles

Acoustically, car cabins are extremely noisy and as a consequence, existing audio-only speech recognition systems, for voice-based control of vehicle functions such as the GPS based navigator, perform poorly. Audio-only speech recognition systems fail to make use of the visual modality of speech (eg: lip movements). As the visual modality is immune to acoustic noise, utilising this visual information in conjunction with an audio only speech recognition system has the potential to improve the accuracy of the system. The field of recognising speech using both auditory and visual inputs is known as Audio Visual Speech Recognition (AVSR). Continuous research in AVASR field has been ongoing for the past twenty-five years with notable progress being made. However, the practical deployment of AVASR systems for use in a variety of real-world applications has not yet emerged. The main reason is due to most research to date neglecting to address variabilities in the visual domain such as illumination and viewpoint in the design of the visual front-end of the AVSR system. In this paper we present an AVASR system in a real-world car environment using the AVICAR database [1], which is publicly available in-car database and we show that the use of visual speech conjunction with the audio modality is a better approach to improve the robustness and effectiveness of voice-only recognition systems in car cabin environments.

Cascading appearance-based features for visual voice activity detection

The detection of voice activity is a challenging problem, especially when the level of acoustic noise is high. Most current approaches only utilise the audio signal, making them susceptible to acoustic noise. An obvious approach to overcome this is to use the visual modality. The current state-of-the-art visual feature extraction technique is one that uses a cascade of visual features (i.e. 2D-DCT, feature mean normalisation, interstep LDA). In this paper, we investigate the effectiveness of this technique for the task of visual voice activity detection (VAD), and analyse each stage of the cascade and quantify the relative improvement in performance gained by each successive stage. The experiments were conducted on the CUAVE database and our results highlight that the dynamics of the visual modality can be used to good effect to improve visual voice activity detection performance.

Recognising audio-visual speech in vehicles using the AVICAR database

Interacting with technology within a vehicle environment using a voice interface can greatly reduce the effects of driver distraction. Most current approaches to this problem only utilise the audio signal, making them susceptible to acoustic noise. An obvious approach to circumvent this is to use the visual modality in addition. However, capturing, storing and distributing audio-visual data in a vehicle environment is very costly and difficult. One current dataset available for such research is the AVICAR [1] database. Unfortunately this database is largely unusable due to timing mismatch between the two streams and in addition, no protocol is available. We have overcome this problem by re-synchronising the streams on the phone-number portion of the dataset and established a protocol for further research. This paper presents the first audio-visual results on this dataset for speaker-independent speech recognition. We hope this will serve as a catalyst for future research in this area.

Speech recognition in adverse environments using lip information

The performance of automatic speech recognition systems deteriorates in the presence of noise. One known solution is to incorporate video information with an existing acoustic speech recognition system. We investigate the performance of the individual acoustic and visual sub-systems and then examine different ways in which the integration of the two systems may be performed. The system is to be implemented in real time on a Texas Instruments' TMS320C80 DSP.

Robust speaker verification via fusion of speech and lip modalities

This paper investigates the use of lip information, in conjunction with speech information, for robust speaker verification in the presence of background noise. It has been previously shown in our own work, and in the work of others, that features extracted from a speaker's moving lips hold speaker dependencies which are complementary with speech features. We demonstrate that the fusion of lip and speech information allows for a highly robust speaker verification system which outperforms the performance of either sub-system. We present a new technique for determining the weighting to be applied to each modality so as to optimize the performance of the fused system. Given a correct weighting, lip information is shown to be highly effective for reducing the false acceptance and false rejection error rates in the presence of background noise

A VARIABLE SWITCHING FREQUENCY PWM TECHNIQUE FOR INVERTER-FED INDUCTION MOTOR TO ACHIEVE SPREAD SPECTRUM

Voltage Source Inverter (VSI) fed induction motors are widely used in variable speed applications. For inverters using fixed switching frequency PWM, the output harmonic spectra are located at a few discrete frequencies. The ac motordrives powered by these inverters cause acoustic noise. This paper proposes a new variable switching frequency pwm technique and compares its performance with constant switching frequency pwm technique. It is shown that the proposed technique leads to spread spectra of voltages and currents. Also this technique ensures that no lower order harmonics are present and the current THD is comparable to that of fixed switching frequency PWM and is even better for higher modulation indices.

Influência do ruído sonoro subaquático na variação dos assobios do boto-cinza, Sotalia guianensis

A poluição sonora é um grave problema nos oceanos devido à eficiência de propagação do som na água e à importância da comunicação acústica para os organismos marinhos. Delfinídeos utilizam o som para comunicação, coordenação de grupo, percepção do hábitat e busca por alimentos, já tendo sido demonstrado que podem alterar suas vocalizações em função do aumento do ruído subaquático. O presente estudo realizou uma comparação dos assobios do boto-cinza Sotalia guianensis em dois ambientes acústicos distintos, um silencioso e um ruidoso, dentro da Baía de Guanabara, Rio de Janeiro, Brasil. Também foram realizadas investigadas as relações dos parâmetros acústicos dos assobios de S. guianensis com os valores de pressão sonora do ruído subaquático. O sistema de gravação foi totalmente calibrado e consistiu de um gravador digital Marantz PMD670 com taxa de amostragem de 96 kHz e um hidrofone HTI-96MIN (5 Hz 30 kHz, sensibilidade média de -170,5 dB re 1 Pa). As gravações realizadas dos assobios e do ruído subaquático ocorreram simultaneamente em duas regiões da baía: a APA de Guapimirim e o Canal central. Durante o período de amostragem os grupos de S. guianensis foram observados em três estados comportamentais: alimentação, deslocamento e socialização; foram anotadas também informações quanto a tamanho e composição de grupo. A análise dos assobios foi realizada no software Raven 1.4 e 10 parâmetros acústicos foram extraídos. Também foi calculada a razão de emissão de assobios. A análise de ruído subaquático foi realizada no software Adobe Audition 1.5, onde foram extraídos valores de pressão sonora do ruído 300ms imediatamente antes de cada assobio analisado, sendo utilizados para análise estatística os maiores valores de pressão sonora dentro de sete intervalos de frequência. Um Teste U de Mann-Whitney foi aplicado para comparar os parâmetros acústicos dos assobios e os valores de pressão sonora das duas regiões amostradas. Esta comparação foi feita para cada estado comportamental observado durante a coleta. Posteriormente foi realizado um teste de correlações de Spearman para investigar a relação entre os parâmetros acústicos e os valores de pressão sonora. Este teste também foi feito separadamente para cada estado comportamental. No comportamento de alimentação foi encontrada diferença na duração, na frequência central e em todos os valores de pressão sonora. Durante o comportamento de socialização foi encontrada diferença na duração e em todos os valores de pressão sonora. Durante o comportamento de alimentação foi encontrada relação entre cinco parâmetros acústicos, a taxa de vocalização e a pressão sonora. Durante o comportamento de socialização foi encontrada relação entre a duração e a pressão sonora. S. guianensis alterou seu comportamento acústico em situações ruidosas, diminuindo a duração e aumentando a taxa de vocalização. Na Baía de Guanabara esta espécie está exposta diariamente a poluição sonora, sendo a APA de Guapimirim o ambiente acústico menos perturbado a que S. guianensis tem acesso.

Discrimination d'événements par analyse des signaux enregistrés par le projet PICASSO

La matière sombre est un mystère dans le domaine de l’astrophysique depuis déjà plusieurs années. De nombreuses observations montrent que jusqu’à 85 % de la masse gravitationnelle totale de l’univers serait composée de cette matière de nature inconnue. Une théorie expliquant cette masse manquante considérerait les WIMPs (Weakly Interacting Massive Particles), particules stables, non chargées, prédites par des extensions du modèle standard, comme candidats. Le projet PICASSO (Projet d’Identification des CAndidats Supersymétriques à la matière Sombre) est une expérience qui tente de détecter directement le WIMP. Le projet utilise des détecteurs à gouttelettes de fréon (C4F10) surchauffées. La collision entre un WIMP et le noyau de fluor crée un recul nucléaire qui cause à son tour une transition de phase de la gouttelette liquide à une bulle gazeuse. Le bruit de ce phénomène est alors capté par des senseurs piézoélectriques montés sur les parois des détecteurs. Le WIMP n’est cependant pas la seule particule pouvant causer une telle transition de phase. D’autres particules environnantes peuvent former des bulles, telles les particules alpha où même des rayons gamma . Le système d’acquisition de données (DAQ) est aussi en proie à du bruit électronique qui peut être enregistré, ainsi que sensible à du bruit acoustique extérieur au détecteur. Finalement, des fractures dans le polymère qui tient les gouttelettes en place peut également causer des transitions de phase spontanées. Il faut donc minimiser l’impact de tous ces différents bruit de fond. La pureté du matériel utilisé dans la fabrication des détecteurs devient alors très importante. On fait aussi appel à des méthodes qui impliquent l’utilisation de variables de discrimination développées dans le but d’améliorer les limites d’exclusion de détection du WIMP.

Nouvelles limites sur la détection directe de la matière sombre avec l’expérience PICASSO

Les observations astronomiques et cosmologiques suggèrent fortement la présence d’une matière exotique, non-relativiste et non-baryonique qui représenterait 26% du contenu de masse-énergie de l’Univers actuel. Cette matière dite sombre et froide serait compo- sée de particules neutres, massives et interagissant faiblement avec la matière ordinaire (WIMP : Weakly Interactive Massive Particles). Le projet PICASSO (Projet d’Identification des CAndidats Supersymétriques de la matière SOmbre) est une des expériences installées dans le site souterrain de SNOLAB à Sudbury en Ontario, qui tente de détecter directement un des candidats de la matière sombre, proposé dans le cadre des extensions supersymétriques du modèle standard : le neutralino. Pour cela, PICASSO utilise des détecteurs à gouttelettes surchauffées de C4F10, basés sur le principe de la chambre à bulles. Les transitions de phase dans les liquides surchauffés peuvent être déclenchées par le recul du 19 F, causé par une collision élastique avec les neutralinos. La nucléation de la gouttelette génère une onde sonore enregistrée par des senseurs piézo-électriques. Cette thèse présentera les récents progrès de l’expérience PICASSO qui ont conduit à une augmentation substantielle de sa sensibilité dans la recherche du neutralino. En effet, de nouvelles procédures de fabrication et de purification ont permis de réduire à un facteur de 10, la contamination majeure des détecteurs, causée par les émetteurs alpha. L’étude de cette contamination dans les détecteurs a permis de localiser la source de ces émetteurs. Les efforts effectués dans le cadre de l’analyse des données, ont permis d’améliorer l’effet de discrimination entre des évènements engendrés par les particules alpha et par les reculs nucléaires. De nouveaux outils d’analyse ont également été implémentés dans le but de discriminer les évènements générés par des particules de ceux générés par des bruits de fond électroniques ou acoustiques. De plus, un mécanisme important de suppression de bruit de fond indésirable à haute température, a permis à l’expérience PICASSO d’être maintenant sensible aux WIMPs de faibles masses.

The sight of silence : the conceptualisation of silence in the visual cultures of sign language communities

Deaf people are perceived by hearing people as living in a silent world. Yet, silence cannot exist without sound, so if sound is not heard, can there be silence? From a linguistic point of view silence is the absence of, or intermission in, communication. Silence can be communicative or noncommunicative. Thus, silence must exist in sign languages as well. Sign languages are based on visual perception and production through movement and sight. Silence must, therefore, be visually perceptible; and, if there is such a thing as visual silence, how does it look? The paper will analyse the topic of silence from a Deaf perspective. The main aspects to be explored are the perception and evaluation of acoustic noise and silence by Deaf people; the conceptualisation of silence in visual languages, such as sign languages; the qualities of visual silence; the meaning of silence as absence of communication (particularly between hearing and Deaf people); social rules for silence; and silencing strategies.

Energy method to calculate the density of liquids using ultrasonic reflection techniques

In this work it is introduced a new approach to calculate the density of liquids in terms of the energies of the acoustic signals. This method is compared to other methods in the time domain (peak-to-peak amplitudes) and frequency domain magnitudes at a single frequency. It is used a measurement cell based on a multiple reflection technique, and it is developed an acoustic model for the cell. Simulations and experiments using several liquids are presented, showing that the energy method a less sensitive to noise than the other techniques. The relative errors in the density are smaller than 0.2% when compared to the values measured with a pycnometer.

A sound education: Becoming aware of the sound around us

In this article it is intended to discuss the issue of noise pollution from an unusual point of view: noise pollution is not only the result of sound increase worldwide, but, particularly, the poor quality of our listening habits in modern life as well. In contemporary society we are subject to a considerable amount of stimulus to all our senses: vision, scent, taste and hearing which are becoming more and more insensible due to over exposure in our environment. These increased stimuli make us look for alternatives to reduce our ability to perceive them and be protected from injuries. However, our sensitivity will also decrease. In the specific case of environment noise, over exposure has made us forget the enchantment of certain sounds that used to give us pleasure or evoke good feelings by many ways, making us recall certain good things, bringing particular moments of our lives to our memory or even filling us with strong emotion. The Canadian composer and music educator, R. Murray Schafer, believes that noise pollution is the result of a society who became deaf. Closing our ears to noise protect us from noise pollution but also prevent us from grasping subtleties of listening. Contemporary world does not help us to be aware of sound in the space around us; acquiring this hearing ability is a matter of focus, interest and practice. Sound education exercises are aimed at children, teenagers and adults who want to improve their listening ability to environmental sounds, perceive its proprieties and learn how sound affects us and touches our feelings. The results are easy to accomplish and contribute to our awareness of the sound environment around us and to the conception of the environmental sound as a composition made by everybody and everything through positive actions, strong will and high sensitivity. Copyright © (2011) by the International Institute of Acoustics & Vibration.

Estudio de la propagación del ruido de tráfico en diversas calles de la ciudad de Madrid

El ruido de tráfico generado en la ciudad es uno de los principales problemas ambientales que afecta notablemente a la calidad de vida de los ciudadanos. Actualmente, la manera de abordar el problema de la contaminación acústica se basa principalmente en medidas correctoras que se aplican a posteriori; cuando el problema ya existe. El problema del ruido debería abordarse además, con medidas preventivas aplicables en la fase de diseño de la ciudad. Sin embargo existen pocos estudios acústicos que puedan aportar conclusiones concretas sobre cómo afectan acústicamente las decisiones tomadas en el planeamiento urbano, ni sobre cómo podrían optimizarse. El trabajo realizado consiste en el estudio de la propagación de ruido en diversas calles representativas de la ciudad de Madrid que pertenecen a diversas tipologías urbanas. De él se concluye que existe una relación directa entre las características tipológicas urbanas y la propagación del ruido. Este estudio representa la base para la investigación acústica sobre múltiples aspectos urbanos y se encuadra en esta nueva área de investigación dentro de la acústica, que podría estar al servicio del planeamiento urbanístico, aportándole las herramientas que precisa para optimizar el diseño de las ciudades teniendo en consideración la problemática del ruido. ABSTRACT. Traffic noise generated in the city has become one of the main environmental problems that significantly affects the quality of life of its citizens. Currently, the approach to the problem of acoustic noise pollution is mainly based on corrective methods that are applied retrospectively; when the problem already exists. The problem of noise pollution in the city should also be dealt with preventive methods, developed in the design phase of the city. However there are few studies that can provide concrete conclusions on how urban planning decisions can affect acoustically the noise problem, or how to optimize it. This work consists in studying noise propagation in several representative streets in the city of Madrid. These streets are a selection belonging to different urban typologies. This study reveals that a direct relation exists between the urban typological characteristics and the noise propagation. This conclusion represents the base for acoustic research on multiple urban aspects. The work fits into this new area of research in acoustics, which could be at the service of the urban planning, giving it the tools it needs to improve urban designing taking into account the problem of noise.

Trade-off aproaches for the vibroacoustic analysis of trains

Passengers’ comfort in terms of acoustic noise levels is a key design driver for train design. The problem is especially relevant for high speed trains, where the aerodynamic induced noise is dominant, but it is also important for medium speed trains where the mechanical sources of noise may have more influence. The numerical interior noise prediction inside the train is a very comp lex problem, involving many different parameters: complex geometries and materials, different noise sources, com- plex interactions among those sources, broad range of frequencies where the phenomenon is important, etc. In this paper, the main findings of this work developed at IDR/UPM (Instituto de Microgravedad “Ignacio Da Riva”, Universidad Politécnica de Madrid) are presented, concentrat ing on the different modelling methodologies used for the different frequency ranges of interest, from FEM-BEM models, hybrid FEM-SEA to pure SEA models. The advantages and disadvantages of the different approaches are summarized. Different modelling techniques have also been evaluated and compared, taking into account the various and specific geometrical configurations typical in this type of structures, and the material properties used in the models. The critical configuration of the train inside a tunnel is studied in order to evaluate the external loads due to noise sources of the train. In this work, a SEA-model composed by periodic characteristic sections of a high spee d train is analysed inside a tunnel.