Lip detection for audio-visual speech recognition in-car environment

Acoustically, car cabins are extremely noisy and as a consequence audio-only, in-car voice recognition systems perform poorly. As the visual modality is immune to acoustic noise, using the visual lip information from the driver is seen as a viable strategy in circumventing this problem by using audio visual automatic speech recognition (AVASR). However, implementing AVASR requires a system being able to accurately locate and track the drivers face and lip area in real-time. In this paper we present such an approach using the Viola-Jones algorithm. Using the AVICAR [1] in-car database, we show that the Viola- Jones approach is a suitable method of locating and tracking the driver’s lips despite the visual variability of illumination and head pose for audio-visual speech recognition system.

Speech recognition in adverse environments using lip information

The performance of automatic speech recognition systems deteriorates in the presence of noise. One known solution is to incorporate video information with an existing acoustic speech recognition system. We investigate the performance of the individual acoustic and visual sub-systems and then examine different ways in which the integration of the two systems may be performed. The system is to be implemented in real time on a Texas Instruments' TMS320C80 DSP.

An approach to statistical lip modelling for speaker identification via chromatic feature extraction

This paper presents a novel technique for the tracking of moving lips for the purpose of speaker identification. In our system, a model of the lip contour is formed directly from chromatic information in the lip region. Iterative refinement of contour point estimates is not required. Colour features are extracted from the lips via concatenated profiles taken around the lip contour. Reduction of order in lip features is obtained via principal component analysis (PCA) followed by linear discriminant analysis (LDA). Statistical speaker models are built from the lip features based on the Gaussian mixture model (GMM). Identification experiments performed on the M2VTS¹ database, show encouraging results

Robust speaker verification via fusion of speech and lip modalities

This paper investigates the use of lip information, in conjunction with speech information, for robust speaker verification in the presence of background noise. It has been previously shown in our own work, and in the work of others, that features extracted from a speaker's moving lips hold speaker dependencies which are complementary with speech features. We demonstrate that the fusion of lip and speech information allows for a highly robust speaker verification system which outperforms the performance of either sub-system. We present a new technique for determining the weighting to be applied to each modality so as to optimize the performance of the fused system. Given a correct weighting, lip information is shown to be highly effective for reducing the false acceptance and false rejection error rates in the presence of background noise

Learning object dynamics for smooth tracking of moving lip contours

A new technique is proposed for learning the dynamic characteristics of a deformable object, applied in particular to the problem of lip-tracking. Experimental results are given which demonstrate that the use of dynamic models allows the system to track more robustly under adverse conditions and to correct spurious, poorly tracked frames

The use of temporal speech and lip information for multi-modal speaker identification via multi-stream HMMs

Investigates the use of temporal lip information, in conjunction with speech information, for robust, text-dependent speaker identification. We propose that significant speaker-dependent information can be obtained from moving lips, enabling speaker recognition systems to be highly robust in the presence of noise. The fusion structure for the audio and visual information is based around the use of multi-stream hidden Markov models (MSHMM), with audio and visual features forming two independent data streams. Recent work with multi-modal MSHMMs has been performed successfully for the task of speech recognition. The use of temporal lip information for speaker identification has been performed previously (T.J. Wark et al., 1998), however this has been restricted to output fusion via single-stream HMMs. We present an extension to this previous work, and show that a MSHMM is a valid structure for multi-modal speaker identification

The use of speech and lip modalities for robust speaker verification under adverse conditions

Investigates the use of lip information, in conjunction with speech information, for robust speaker verification in the presence of background noise. We have previously shown (Int. Conf. on Acoustics, Speech and Signal Proc., vol. 6, pp. 3693-3696, May 1998) that features extracted from a speaker's moving lips hold speaker dependencies which are complementary with speech features. We demonstrate that the fusion of lip and speech information allows for a highly robust speaker verification system which outperforms either subsystem individually. We present a new technique for determining the weighting to be applied to each modality so as to optimize the performance of the fused system. Given a correct weighting, lip information is shown to be highly effective for reducing the false acceptance and false rejection error rates in the presence of background noise

Early-mid Cretaceous tectonic evolution of eastern Gondwana : from silicic LIP magmatism to continental rupture

The Early–mid Cretaceous marks the confluence of three major continental-scale events in eastern Gondwana: (1) the emplacement of a Silicic Large Igneous Province (LIP) near the continental margin; (2) the volcaniclastic fill, transgression and regression of a major epicontinental seaway developed over at least a quarter of the Australian continent; and (3) epeirogenic uplift, exhumation and continental rupturing culminating in the opening of the Tasman Basin c. 84 Ma. The Whitsunday Silicic LIP event had widespread impact, producing both substantial extrusive volumes of dominantly silicic pyroclastic material and coeval first-cycle volcanogenic sediment that accumulated within many eastern Australian sedimentary basins, and principally in the Great Australian Basin system (>2 Mkm3 combined volume). The final pulse of volcanism and volcanogenic sedimentation at c. 105–95 Ma coincided with epicontinental seaway regression, which shows a lack of correspondence with the global sea-level curve, and alternatively records a wider, continental-scale effect of volcanism and rift tectonism. Widespread igneous underplating related to this LIP event is evident from high paleogeothermal gradients and regional hydrothermal fluid flow detectable in the shallow crust and over a broad region. Enhanced CO2 fluxing through sedimentary basins also records indirectly, large-scale, LIP-related mafic underplating. A discrete episode of rapid crustal cooling and exhumation began c. 100–90 Ma along the length of the eastern Australian margin, related to an enhanced phase of continental rifting that was largely amagmatic, and probably a switch from wide–more narrow rift modes. Along-margin variations in detachment fault architecture produced narrow (SE Australia) and wide continental margins with marginal, submerged continental plateaux (NE Australia). Long-lived NE-trending cross-orogen lineaments controlled the switch from narrow to wide continental margin geometries.

How do supercontinents break up? — assessing the continental large igneous province (LIP) record of the break-up of Pangea

LIP emplacement is linked to the timing and evolution of supercontinental break-up. LIP-related break-up produces volcanic rifted margins, new and large (up to 108 km2) ocean basins, and new, smaller continents that undergo dispersal and potentially reassembly (e.g., India). However, not all continental LIPs lead to continental rupture. We analysed the <330 Ma continental LIP record(following final assembly of Pangea) to find relationships between LIP event attributes (e.g., igneous volume, extent, distance from pre-existing continental margin) and ocean basin attributes (e.g., length of new ocean basin/rifted margin) and how these varied during the progressive break up of Pangea. No correlation exists between LIP magnitude and size of the subsequent ocean basin or rifted margin. Our review suggests a three-phased break-up history of Pangea: 1) “Preconditioning” phase (∼330–200 Ma): LIP events (n=7) occurred largely around the supercontinental margin clustering today in Asia, with a low (<20%) rifting success rate. The Panjal Traps at ∼280 Ma may represent the first continental rupturing event of Pangea, resulting in continental ribboning along the Tethyan margin; 2) “Main Break-up” phase (∼200–100 Ma): numerous large LIP events(n=10) in the supercontinent interior, resulting in highly successful fragmentation (90%) and large, new ocean basins(e.g., Central/South Atlantic, Indian, >3000 km long); 3) “Waning” phase (∼100–0 Ma): Declining LIP magnitudes (n=6), greater proximity to continental margins (e.g., Madagascar, North Atlantic, Afro-Arabia, Sierra Madre) producing smaller ocean basins (<2600 km long). How Pangea broke up may thus have implications for earlier supercontinent reconstructions and LIP record.

Self-Concept and School Achievement of Pupils with Cleft Lip, Cleft Palate or Both : A longitudinal study

The goal of this research was to survey the self-concept and school achievement of pupils with cleft lip, cleft palate or both from juvenile age to adolescence. Longitudinal researches of self-concept and school achievement among pupils with cleft lip, cleft palate or both are uncommon. This research was the first longitudinal research ever conducted in Finland among this population. This research can be considered to be a special educational study because of the target group involved. Self-concept consists of the person s entire personality. Personality is biological and deterministic. Self-concept includes concepts, attitudes and feelings that the person has about him or her qualities, abilities and relations to the environment. The individual associates experiences to this personality with earlier observations through the social interaction. The individual will have the consciousness of the person s existence and action. The target group in this study consisted of Finnish children with clefts, who were comprised of four different age groups. The questionnaire was sent to all subjects (N1 = 419) both times. A total of 74 % of children returned the questionnaire in 1988 (N2=305). 48 % of children returned the questionnaire in 1993 (N3=203). 42% of children returned the questionnaire both times (N4=175) . These 175 children formed the research subjects. The survey was conducted in 1988, and again in 1993. In 1988, the pupils surveyed were 9 to 12 years of age, while in 1993 they were between 14 and 17 years old. The data was collected through the use of a questionnaire, which consisted of common questions and a personality inventory test that was developed for Finnish students by professor Maija-Liisa Rauste-von Wright. Quantitative analysis methods were used to examine the structure of self-concept and school achievement. Structures found in this research were observed in relation to disorder, gender and maturation. According to these results, structures of self-concepts and school achievement are in fact stable. Basic self-concept elements are seen to be formed at an early age. The developmental aspects of self-concept following puberty are observed as the stability of self-concept and as the forming of a general self. The level of school achievement is stable, but the structure of school achievement changes. From these results, it is possible to state that the gender of the child has a statistical significance regarding self-concept and school achievement. However, the experienced disorder does not have statistical significance as regards to self-concept and school achievement. Results of self-concept support the research of self-concept conducted earlier in Finland.

Wear lip formation during dry sliding

A pin-on-disc test configuration has been used to examine the formation of the strain-hardened projection, or wear lips, especially at the trailing edge of the pin during dry sliding of aluminium alloys against steel discs. The mechanism of formation of such wear lips is studied with the aid of optical and electron microscopes. The plastic deformation of the pin, growth and eventual removal of the wear lip as wear debris are elucidated. The size and shape of the wear lips in pins of different shapes, i.e. square, rectangular, triangular and circular cross-sections, are described.

Singularity characteristics for a lip-shaped crack subjected to remote biaxial loading

In this paper, the conformal mapping method was adopted to solve the problem of an infinite plate containing a central lip-shaped crack subjected to remote biaxial loading. A kind of leaf-shaped configuration was also constructed in order to solve the problem. The analytical result showed that the singularity order of the stress field at the tip of a lip-shaped crack remains -1/2, despite the difference in notch-crack width.