Lip detection for audio-visual speech recognition in-car environment

Acoustically, car cabins are extremely noisy and as a consequence audio-only, in-car voice recognition systems perform poorly. As the visual modality is immune to acoustic noise, using the visual lip information from the driver is seen as a viable strategy in circumventing this problem by using audio visual automatic speech recognition (AVASR). However, implementing AVASR requires a system being able to accurately locate and track the drivers face and lip area in real-time. In this paper we present such an approach using the Viola-Jones algorithm. Using the AVICAR [1] in-car database, we show that the Viola- Jones approach is a suitable method of locating and tracking the driver’s lips despite the visual variability of illumination and head pose for audio-visual speech recognition system.

Person-independent facial expression detection using constrained local models

In automatic facial expression detection, very accurate registration is desired which can be achieved via a deformable model approach where a dense mesh of 60-70 points on the face is used, such as an active appearance model (AAM). However, for applications where manually labeling frames is prohibitive, AAMs do not work well as they do not generalize well to unseen subjects. As such, a more coarse approach is taken for person-independent facial expression detection, where just a couple of key features (such as face and eyes) are tracked using a Viola-Jones type approach. The tracked image is normally post-processed to encode for shift and illumination invariance using a linear bank of filters. Recently, it was shown that this preprocessing step is of no benefit when close to ideal registration has been obtained. In this paper, we present a system based on the Constrained Local Model (CLM) which is a generic or person-independent face alignment algorithm which gains high accuracy. We show these results against the LBP feature extraction on the CK+ and GEMEP datasets.

Visual voice activity detection using frontal versus profile views

Visual activity detection of lip movements can be used to overcome the poor performance of voice activity detection based solely in the audio domain, particularly in noisy acoustic conditions. However, most of the research conducted in visual voice activity detection (VVAD) has neglected addressing variabilities in the visual domain such as viewpoint variation. In this paper we investigate the effectiveness of the visual information from the speaker’s frontal and profile views (i.e left and right side views) for the task of VVAD. As far as we are aware, our work constitutes the first real attempt to study this problem. We describe our visual front end approach and the Gaussian mixture model (GMM) based VVAD framework, and report the experimental results using the freely available CUAVE database. The experimental results show that VVAD is indeed possible from profile views and we give a quantitative comparison of VVAD based on frontal and profile views The results presented are useful in the development of multi-modal Human Machine Interaction (HMI) using a single camera, where the speaker’s face may not always be frontal.

Towards pose-robust face recognition on video

This thesis investigates face recognition in video under the presence of large pose variations. It proposes a solution that performs simultaneous detection of facial landmarks and head poses across large pose variations, employs discriminative modelling of feature distributions of faces with varying poses, and applies fusion of multiple classifiers to pose-mismatch recognition. Experiments on several benchmark datasets have demonstrated that improved performance is achieved using the proposed solution.

Vibration characteristics and damage detection in a suspension bridge

Suspension bridges are flexible and vibration sensitive structures that exhibit complex and multi-modal vibration. Due to this, the usual vibration based methods could face a challenge when used for damage detection in these structures. This paper develops and applies a mode shape component specific damage index (DI) to detect and locate damage in a suspension bridge with pre-tensioned cables. This is important as suspension bridges are large structures and damage in them during their long service lives could easily go un-noticed. The capability of the proposed vibration based DI is demonstrated through its application to detect and locate single and multiple damages with varied locations and severity in the cables of the suspension bridge. The outcome of this research will enhance the safety and performance of these bridges which play an important role in the transport network.

Robust Observer Design with Application to Fault Detection

A new scheme for robust estimation of the partial state of linear time-invariant multivariable systems is presented, and it is shown how this may be used for the detection of sensor faults in such systems. We consider an observer to be robust if it generates a faithful estimate of the plant state in the face of modelling uncertainty or plant perturbations. Using the Stable Factorization approach we formulate the problem of optimal robust observer design by minimizing an appropriate norm on the estimation error. A logical candidate is the 2-norm, corresponding to an H�¿ optimization problem, for which solutions are readily available. In the special case of a stable plant, the optimal fault diagnosis scheme reduces to an internal model control architecture.

Ultrasonic Guided Wave Characterization and Damage Detection in Foam-core Sandwich Panel using PWAS and LDV

Lamb wave type guided wave propagation in foam core sandwich structures and detectability of damages using spectral analysis method are reported in this paper. An experimental study supported by theoretical evaluation of the guided wave characteristics is presented here that shows the applicability of Lamb wave type guided ultrasonic wave for detection of damage in foam core sandwich structures. Sandwich beam specimens were fabricated with 10 mm thick foam core and 0.3 mm thick aluminum face sheets. Thin piezoelectric patch actuators and sensors are used to excite and sense guided wave. Group velocity dispersion curves and frequency response of sensed signal are obtained experimentally. The nature of damping present in the sandwich panel is monitored by measuring the sensor signal amplitude at various different distances measured from the center of the linear phased array. Delaminations of increasing width are created and detected experimentally by pitch-catch interrogation with guided waves and wavelet transform of the sensed signal. Signal amplitudes are analyzed for various different sizes of damages to differentiate the damage size/severity. A sandwich panel is also fabricated with a planer dimension of 600 mm x 400 mm. Release film delamination is introduced during fabrication. Non-contact Laser Doppler Vibrometer (LDV) is used to scan the panel while exciting with a surface bonded piezoelectric actuator. Presence of damage is confirmed by the reflected wave fringe pattern obtained from the LDV scan. With this approach it is possible to locate and monitor the damages by tracking the wave packets scattered from the damages.

Guided Wave based Damage Detection in a Composite T-joint using 3D Scanning Laser Doppler Vibrometer

Composite T-joints are commonly used in modern composite airframe, pressure vessels and piping structures, mainly to increase the bending strength of the joint and prevents buckling of plates and shells, and in multi-cell thin-walled structures. Here we report a detailed study on the propagation of guided ultrasonic wave modes in a composite T-joint and their interactions with delamination in the co-cured co-bonded flange. A well designed guiding path is employed wherein the waves undergo a two step mode conversion process, one is due to the web and joint filler on the back face of the flange and the other is due to the delamination edges close to underneath the accessible surface of the flange. A 3D Laser Doppler Vibrometer is used to obtain the three components of surface displacements/velocities of the accessible face of the flange of the T-joint. The waves are launched by a piezo ceramic wafer bonded on to the back surface of the flange. What is novel in the proposed method is that the location of any change in material/geometric properties can be traced by computing a frequency domain power flow along a scan line. The scan line can be chosen over a grid either during scan or during post-processing of the scan data off-line. The proposed technique eliminates the necessity of baseline data and disassembly of structure for structural interrogation.

Guided wave based detection of damage in honeycomb core sandwich structures

We report on the Lamb wave type guided wave propagation in honeycomb core sandwich structures. An experimental study supported by theoretical evaluation of the guided wave characteristics is presented that proves the potential of Lamb wave type guided wave for detection of damage in sandwich structures. A sandwich panel is fabricated with planar dimension of 600 mm x 600 mm, having a core thickness of 7 mm, cell size of 5 mm and 0.1 mm thick aluminum face sheets. Thin piezoelectric patch actuators and sensors are used to excite and sense a frequency band limited guided wave with a central frequency. A linear phased array of piezoelectric patch actuators is used to achieve higher signal strength and directivity. Group velocity dispersion curves and corresponding frequency response of sensed signal are obtained experimentally. Linearity between the excitation signal amplitude and the corresponding sensed signal amplitude is found for certain range of parameters. The nature of damping present in the sandwich panel is monitored by measuring the sensor signal amplitude at various different distances measured from the center of the linear phased array. Indentation and low velocity impact induced damages of increasing diameter covering several honeycomb cells are created. Crushing of honeycomb core with rupture of face sheet is observed while introducing the damage. The damages are then detected experimentally by pitch-catch interrogation with guided waves and wavelet transform of the sensed signal. Signal amplitudes are analyzed for various different sizes of damages to differentiate the damage size/severity. Monotonic changes in the sensor signal amplitude due to increase in the damage size has been established successfully. With this approach it is possible to locate and monitor the damages with the help of phased array and by tracking the wave packets scattered from the damages. (C) 2012 Elsevier Ltd. All rights reserved.

Green synthesis of biopolymer-silver nanoparticle nanocomposite: An optical sensor for ammonia detection

Biopolymer used for the production of nanoparticles (NPs) has attracted increasing attention. In the presence article we use aqueous solution of polysaccharide Cyamopsis tetragonaloba commonly known as guar gum (GG), from plants. GG acts as reductive preparation of silver nanoparticles which are found to be <10. nm in size. The uniformity of the NPs size was measured by the SEM and TEM, while a face centered cubic structure of crystalline silver nanoparticles was characterized using powder X-ray diffraction technique. Aqueous ammonia sensing study of polymer/silver nanoparticles nanocomposite (GG/AgNPs NC) was performed by optical method based on surface plasmon resonance (SPR). The performances of optical sensor were investigated which provide the excellent result. The response time of 2-3. s and the detection limit of ammonia solution, 1. ppm were found at room temperature. Thus, in future this room temperature optical ammonia sensor can be used for clinical and medical diagnosis for detecting low ammonia level in biological fluids, such as plasma, sweat, saliva, cerebrospinal liquid or biological samples in general for various biomedical applications in human. © 2012 Elsevier B.V.

Guided-wave-based damage detection in a composite T-joint using 3D scanning laser Doppler vibrometer

Composite T-joints are commonly used in modern composite airframe, pressure vessels and piping structures, mainly to increase the bending strength of the joint and prevents buckling of plates and shells, and in multi-cell thin-walled structures. Here we report a detailed study on the propagation of guided ultrasonic wave modes in a composite T-joint and their interactions with delamination in the co-cured co-bonded flange. A well designed guiding path is employed wherein the waves undergo a two step mode conversion process, one is due to the web and joint filler on the back face of the flange and the other is due to the delamination edges close to underneath the accessible surface of the flange. A 3D Laser Doppler Vibrometer is used to obtain the three components of surface displacements/velocities of the accessible face of the flange of the T-joint. The waves are launched by a piezo ceramic wafer bonded on to the back surface of the flange. What is novel in the proposed method is that the location of any change in material/geometric properties can be traced by computing a frequency domain power flow along a scan line. The scan line can be chosen over a grid either during scan or during post-processing of the scan data off-line. The proposed technique eliminates the necessity of baseline data and disassembly of structure for structural interrogation.

Visual-Context Boosting for Eye Detection

Eye detection plays an important role in many practical applications. This paper presents a novel two-step scheme for eye detection. The first step models an eye by a newly defined visual-context pattern (VCP), and the second step applies semisupervised boosting for precise detection. VCP describes both the space and appearance relations between an eye region (region of eye) and a reference region (region of reference). The context feature of a VCP is extracted by using the integral image. Aiming to reduce the human labeling efforts, we apply semisupervised boosting, which integrates the context feature and the Haar-like features for precise eye detection. Experimental results on several standard face data sets demonstrate that the proposed approach is effective, robust, and efficient. We finally show that this approach is ready for practical applications.

Vectorizing Face Images by Interpreting Shape and Texture Computations

The correspondence problem in computer vision is basically a matching task between two or more sets of features. In this paper, we introduce a vectorized image representation, which is a feature-based representation where correspondence has been established with respect to a reference image. This representation has two components: (1) shape, or (x, y) feature locations, and (2) texture, defined as the image grey levels mapped onto the standard reference image. This paper explores an automatic technique for "vectorizing" face images. Our face vectorizer alternates back and forth between computation steps for shape and texture, and a key idea is to structure the two computations so that each one uses the output of the other. A hierarchical coarse-to-fine implementation is discussed, and applications are presented to the problems of facial feature detection and registration of two arbitrary faces.