Comparison of Image Transform-Based Features for Visual Speech Recognition in Clean and Corrupted Videos

We present results of a study into the performance of a variety of different image transform-based feature types for speaker-independent visual speech recognition of isolated digits. This includes the first reported use of features extracted using a discrete curvelet transform. The study will show a comparison of some methods for selecting features of each feature type and show the relative benefits of both static and dynamic visual features. The performance of the features will be tested on both clean video data and also video data corrupted in a variety of ways to assess each feature type's robustness to potential real-world conditions. One of the test conditions involves a novel form of video corruption we call jitter which simulates camera and/or head movement during recording.

The human prion protein residue 129 polymorphism lies within a cluster of epitopes for T cell recognition.

T cell immune responses to central nervous system-derived and other self-antigens are commonly described in both healthy and autoimmune individuals. However, in the case of the human prion protein (PrP), it has been argued that immunologic tolerance is uncommonly robust. Although development of an effective vaccine for prion disease requires breaking of tolerance to PrP, the extent of immune tolerance to PrP and the identity of immunodominant regions of the protein have not previously been determined in humans. We analyzed PrP T cell epitopes both by using a predictive algorithm and by measuring functional immune responses from healthy donors. Interestingly, clusters of epitopes were focused around the area of the polymorphic residue 129, previously identified as an indicator of susceptibility to prion disease, and in the C-terminal region. Moreover, responses were seen to PrP peptide 121-134 containing methionine at position 129, whereas PrP 121-134 [129V] was not immunogenic. The residue 129 polymorphism was also associated with distinct patterns of cytokine response: PrP 128-141 [129M] inducing IL-4 and IL-6 production, which was not seen in response to PrP 128-141 [129V]. Our data suggest that the immunogenic regions of human PrP lie between residue 107 and the C-terminus and that, like with many other central nervous system antigens, healthy individuals carry responses to PrP within the T cell repertoire and yet do not experience deleterious autoimmune reactions.

Robust face recognition using posterior union model based neural networks

Face recognition with unknown, partial distortion and occlusion is a practical problem, and has a wide range of applications, including security and multimedia information retrieval. The authors present a new approach to face recognition subject to unknown, partial distortion and occlusion. The new approach is based on a probabilistic decision-based neural network, enhanced by a statistical method called the posterior union model (PUM). PUM is an approach for ignoring severely mismatched local features and focusing the recognition mainly on the reliable local features. It thereby improves the robustness while assuming no prior information about the corruption. We call the new approach the posterior union decision-based neural network (PUDBNN). The new PUDBNN model has been evaluated on three face image databases (XM2VTS, AT&T and AR) using testing images subjected to various types of simulated and realistic partial distortion and occlusion. The new system has been compared to other approaches and has demonstrated improved performance.

Hidden Conditional Random Fields for Visual Speech Recognition

In this paper we present the application of Hidden Conditional Random Fields (HCRFs) to modelling speech for visual speech recognition. HCRFs may be easily adapted to model long range dependencies across an observation sequence. As a result visual word recognition performance can be improved as the model is able to take more of a contextual approach to generating state sequences. Results are presented from a speaker-dependent, isolated digit, visual speech recognition task using comparisons with a baseline HMM system. We firstly illustrate that word recognition rates on clean video using HCRFs can be improved by increasing the number of past and future observations being taken into account by each state. Secondly we compare model performances using various levels of video compression on the test set. As far as we are aware this is the first attempted use of HCRFs for visual speech recognition.