Robust speaker verification via fusion of speech and lip modalities

This paper investigates the use of lip information, in conjunction with speech information, for robust speaker verification in the presence of background noise. It has been previously shown in our own work, and in the work of others, that features extracted from a speaker's moving lips hold speaker dependencies which are complementary with speech features. We demonstrate that the fusion of lip and speech information allows for a highly robust speaker verification system which outperforms the performance of either sub-system. We present a new technique for determining the weighting to be applied to each modality so as to optimize the performance of the fused system. Given a correct weighting, lip information is shown to be highly effective for reducing the false acceptance and false rejection error rates in the presence of background noise

The use of speech and lip modalities for robust speaker verification under adverse conditions

Investigates the use of lip information, in conjunction with speech information, for robust speaker verification in the presence of background noise. We have previously shown (Int. Conf. on Acoustics, Speech and Signal Proc., vol. 6, pp. 3693-3696, May 1998) that features extracted from a speaker's moving lips hold speaker dependencies which are complementary with speech features. We demonstrate that the fusion of lip and speech information allows for a highly robust speaker verification system which outperforms either subsystem individually. We present a new technique for determining the weighting to be applied to each modality so as to optimize the performance of the fused system. Given a correct weighting, lip information is shown to be highly effective for reducing the false acceptance and false rejection error rates in the presence of background noise

Object tracking using multiple motion modalities

This paper presents an object tracking system that utilises a hybrid multi-layer motion segmentation and optical flow algorithm. While many tracking systems seek to combine multiple modalities such as motion and depth or multiple inputs within a fusion system to improve tracking robustness, current systems have avoided the combination of motion and optical flow. This combination allows the use of multiple modes within the object detection stage. Consequently, different categories of objects, within motion or stationary, can be effectively detected utilising either optical flow, static foreground or active foreground information. The proposed system is evaluated using the ETISEO database and evaluation metrics and compared to a baseline system utilising a single mode foreground segmentation technique. Results demonstrate a significant improvement in tracking results can be made through the incorporation of the additional motion information.

Lip detection for audio-visual speech recognition in-car environment

Acoustically, car cabins are extremely noisy and as a consequence audio-only, in-car voice recognition systems perform poorly. As the visual modality is immune to acoustic noise, using the visual lip information from the driver is seen as a viable strategy in circumventing this problem by using audio visual automatic speech recognition (AVASR). However, implementing AVASR requires a system being able to accurately locate and track the drivers face and lip area in real-time. In this paper we present such an approach using the Viola-Jones algorithm. Using the AVICAR [1] in-car database, we show that the Viola- Jones approach is a suitable method of locating and tracking the driver’s lips despite the visual variability of illumination and head pose for audio-visual speech recognition system.

Speech recognition in adverse environments using lip information

The performance of automatic speech recognition systems deteriorates in the presence of noise. One known solution is to incorporate video information with an existing acoustic speech recognition system. We investigate the performance of the individual acoustic and visual sub-systems and then examine different ways in which the integration of the two systems may be performed. The system is to be implemented in real time on a Texas Instruments' TMS320C80 DSP.

An approach to statistical lip modelling for speaker identification via chromatic feature extraction

This paper presents a novel technique for the tracking of moving lips for the purpose of speaker identification. In our system, a model of the lip contour is formed directly from chromatic information in the lip region. Iterative refinement of contour point estimates is not required. Colour features are extracted from the lips via concatenated profiles taken around the lip contour. Reduction of order in lip features is obtained via principal component analysis (PCA) followed by linear discriminant analysis (LDA). Statistical speaker models are built from the lip features based on the Gaussian mixture model (GMM). Identification experiments performed on the M2VTS¹ database, show encouraging results

Learning object dynamics for smooth tracking of moving lip contours

A new technique is proposed for learning the dynamic characteristics of a deformable object, applied in particular to the problem of lip-tracking. Experimental results are given which demonstrate that the use of dynamic models allows the system to track more robustly under adverse conditions and to correct spurious, poorly tracked frames

The use of temporal speech and lip information for multi-modal speaker identification via multi-stream HMMs

Investigates the use of temporal lip information, in conjunction with speech information, for robust, text-dependent speaker identification. We propose that significant speaker-dependent information can be obtained from moving lips, enabling speaker recognition systems to be highly robust in the presence of noise. The fusion structure for the audio and visual information is based around the use of multi-stream hidden Markov models (MSHMM), with audio and visual features forming two independent data streams. Recent work with multi-modal MSHMMs has been performed successfully for the task of speech recognition. The use of temporal lip information for speaker identification has been performed previously (T.J. Wark et al., 1998), however this has been restricted to output fusion via single-stream HMMs. We present an extension to this previous work, and show that a MSHMM is a valid structure for multi-modal speaker identification

Early-mid Cretaceous tectonic evolution of eastern Gondwana : from silicic LIP magmatism to continental rupture

The Early–mid Cretaceous marks the confluence of three major continental-scale events in eastern Gondwana: (1) the emplacement of a Silicic Large Igneous Province (LIP) near the continental margin; (2) the volcaniclastic fill, transgression and regression of a major epicontinental seaway developed over at least a quarter of the Australian continent; and (3) epeirogenic uplift, exhumation and continental rupturing culminating in the opening of the Tasman Basin c. 84 Ma. The Whitsunday Silicic LIP event had widespread impact, producing both substantial extrusive volumes of dominantly silicic pyroclastic material and coeval first-cycle volcanogenic sediment that accumulated within many eastern Australian sedimentary basins, and principally in the Great Australian Basin system (>2 Mkm3 combined volume). The final pulse of volcanism and volcanogenic sedimentation at c. 105–95 Ma coincided with epicontinental seaway regression, which shows a lack of correspondence with the global sea-level curve, and alternatively records a wider, continental-scale effect of volcanism and rift tectonism. Widespread igneous underplating related to this LIP event is evident from high paleogeothermal gradients and regional hydrothermal fluid flow detectable in the shallow crust and over a broad region. Enhanced CO2 fluxing through sedimentary basins also records indirectly, large-scale, LIP-related mafic underplating. A discrete episode of rapid crustal cooling and exhumation began c. 100–90 Ma along the length of the eastern Australian margin, related to an enhanced phase of continental rifting that was largely amagmatic, and probably a switch from wide–more narrow rift modes. Along-margin variations in detachment fault architecture produced narrow (SE Australia) and wide continental margins with marginal, submerged continental plateaux (NE Australia). Long-lived NE-trending cross-orogen lineaments controlled the switch from narrow to wide continental margin geometries.

How do supercontinents break up? — assessing the continental large igneous province (LIP) record of the break-up of Pangea

LIP emplacement is linked to the timing and evolution of supercontinental break-up. LIP-related break-up produces volcanic rifted margins, new and large (up to 108 km2) ocean basins, and new, smaller continents that undergo dispersal and potentially reassembly (e.g., India). However, not all continental LIPs lead to continental rupture. We analysed the <330 Ma continental LIP record(following final assembly of Pangea) to find relationships between LIP event attributes (e.g., igneous volume, extent, distance from pre-existing continental margin) and ocean basin attributes (e.g., length of new ocean basin/rifted margin) and how these varied during the progressive break up of Pangea. No correlation exists between LIP magnitude and size of the subsequent ocean basin or rifted margin. Our review suggests a three-phased break-up history of Pangea: 1) “Preconditioning” phase (∼330–200 Ma): LIP events (n=7) occurred largely around the supercontinental margin clustering today in Asia, with a low (<20%) rifting success rate. The Panjal Traps at ∼280 Ma may represent the first continental rupturing event of Pangea, resulting in continental ribboning along the Tethyan margin; 2) “Main Break-up” phase (∼200–100 Ma): numerous large LIP events(n=10) in the supercontinent interior, resulting in highly successful fragmentation (90%) and large, new ocean basins(e.g., Central/South Atlantic, Indian, >3000 km long); 3) “Waning” phase (∼100–0 Ma): Declining LIP magnitudes (n=6), greater proximity to continental margins (e.g., Madagascar, North Atlantic, Afro-Arabia, Sierra Madre) producing smaller ocean basins (<2600 km long). How Pangea broke up may thus have implications for earlier supercontinent reconstructions and LIP record.

Combining multiple sensor modalities for a localisation robust to smoke

This paper proposes an approach to obtain a localisation that is robust to smoke by exploiting multiple sensing modalities: visual and infrared (IR) cameras. This localisation is based on a state-of-the-art visual SLAM algorithm. First, we show that a reasonably accurate localisation can be obtained in the presence of smoke by using only an IR camera, a sensor that is hardly affected by smoke, contrary to a visual camera (operating in the visible spectrum). Second, we demonstrate that improved results can be obtained by combining the information from the two sensor modalities (visual and IR cameras). Third, we show that by detecting the impact of smoke on the visual images using a data quality metric, we can anticipate and mitigate the degradation in performance of the localisation by discarding the most affected data. The experimental validation presents multiple trajectories estimated by the various methods considered, all thoroughly compared to an accurate dGPS/INS reference.

Invasive and non-invasive modalities of imaging carotid stenosis

Despite recent therapeutic advances, acute ischemic complications of atherosclerosis remain the primary cause of morbidity and mortality in Western countries, with carotid atherosclerotic disease one of the major preventable causes of stroke. As the impact of this disease challenges our healthcare systems, we are becoming aware that factors influencing this disease are more complex than previously realized. In current clinical practice, risk stratification relies primarily on evaluation of the degree of luminal stenosis and patient symptomatology. Adequate investigation and optimal imaging are important factors that affect the quality of a carotid endarterectomy (CEA) service and are fundamental to patient selection. Digital subtraction angiography is still perceived as the most accurate imaging modality for carotid stenosis and historically has been the cornerstone of most of the major CEA trials but concerns regarding potential neurological complications have generated substantial interest in non-invasive modalities, such as contrast-enhanced magnetic resonance angiography. The purpose of this review is to give an overview to the vascular specialist of the current imaging modalities in clinical practice to identify patients with carotid stenosis. Advantages and disadvantages of each technique are outlined. Finally, limitations of assessing luminal stenosis in general are discussed. This article will not cover imaging of carotid atheroma morphology, function and other emerging imaging modalities of assessing plaque risk, which look beyond simple luminal measurements.

Self-Concept and School Achievement of Pupils with Cleft Lip, Cleft Palate or Both : A longitudinal study

The goal of this research was to survey the self-concept and school achievement of pupils with cleft lip, cleft palate or both from juvenile age to adolescence. Longitudinal researches of self-concept and school achievement among pupils with cleft lip, cleft palate or both are uncommon. This research was the first longitudinal research ever conducted in Finland among this population. This research can be considered to be a special educational study because of the target group involved. Self-concept consists of the person s entire personality. Personality is biological and deterministic. Self-concept includes concepts, attitudes and feelings that the person has about him or her qualities, abilities and relations to the environment. The individual associates experiences to this personality with earlier observations through the social interaction. The individual will have the consciousness of the person s existence and action. The target group in this study consisted of Finnish children with clefts, who were comprised of four different age groups. The questionnaire was sent to all subjects (N1 = 419) both times. A total of 74 % of children returned the questionnaire in 1988 (N2=305). 48 % of children returned the questionnaire in 1993 (N3=203). 42% of children returned the questionnaire both times (N4=175) . These 175 children formed the research subjects. The survey was conducted in 1988, and again in 1993. In 1988, the pupils surveyed were 9 to 12 years of age, while in 1993 they were between 14 and 17 years old. The data was collected through the use of a questionnaire, which consisted of common questions and a personality inventory test that was developed for Finnish students by professor Maija-Liisa Rauste-von Wright. Quantitative analysis methods were used to examine the structure of self-concept and school achievement. Structures found in this research were observed in relation to disorder, gender and maturation. According to these results, structures of self-concepts and school achievement are in fact stable. Basic self-concept elements are seen to be formed at an early age. The developmental aspects of self-concept following puberty are observed as the stability of self-concept and as the forming of a general self. The level of school achievement is stable, but the structure of school achievement changes. From these results, it is possible to state that the gender of the child has a statistical significance regarding self-concept and school achievement. However, the experienced disorder does not have statistical significance as regards to self-concept and school achievement. Results of self-concept support the research of self-concept conducted earlier in Finland.