Robust face recognition with partial occlusion, illumination variation and limited training data by optimal feature selection

This study investigates face recognition with partial occlusion, illumination variation and their combination, assuming no prior information about the mismatch, and limited training data for each person. The authors extend their previous posterior union model (PUM) to give a new method capable of dealing with all these problems. PUM is an approach for selecting the optimal local image features for recognition to improve robustness to partial occlusion. The extension is in two stages. First, authors extend PUM from a probability-based formulation to a similarity-based formulation, so that it operates with as little as one single training sample to offer robustness to partial occlusion. Second, they extend this new formulation to make it robust to illumination variation, and to combined illumination variation and partial occlusion, by a novel combination of multicondition relighting and optimal feature selection. To evaluate the new methods, a number of databases with various simulated and realistic occlusion/illumination mismatches have been used. The results have demonstrated the improved robustness of the new methods.

Toward Unconstrained Ear Recognition From Two-Dimensional Images

Ear recognition, as a biometric, has several advantages. In particular, ears can be measured remotely and are also relatively static in size and structure for each individual. Unfortunately, at present, good recognition rates require controlled conditions. For commercial use, these systems need to be much more robust. In particular, ears have to be recognized from different angles ( poses), under different lighting conditions, and with different cameras. It must also be possible to distinguish ears from background clutter and identify them when partly occluded by hair, hats, or other objects. The purpose of this paper is to suggest how progress toward such robustness might be achieved through a technique that improves ear registration. The approach focuses on 2-D images, treating the ear as a planar surface that is registered to a gallery using a homography transform calculated from scale-invariant feature-transform feature matches. The feature matches reduce the gallery size and enable a precise ranking using a simple 2-D distance algorithm. Analysis on a range of data sets demonstrates the technique to be robust to background clutter, viewing angles up to +/- 13 degrees, and up to 18% occlusion. In addition, recognition remains accurate with masked ear images as small as 20 x 35 pixels.

Other-race faces : limitations of expert face recognition /

Adults' expert face recognition is limited to the kinds of faces they encounter on a daily basis (typically upright human faces of the same race). Adults process own-race faces holistically (Le., as a gestalt) and are exquisitely sensitive to small differences among faces in the spacing of features, the shape of individual features and the outline or contour of the face (Maurer, Le Grand, & Mondloch, 2002), however this expertise does not seem to extend to faces from other races. The goal of the current study was to investigate the extent to which the mechanisms that underlie expert face processing of own-race faces extend to other-race faces. Participants from rural Pennsylvania that had minimal exposure to other-race faces were tested on a battery of tasks. They were tested on a memory task, two measures of holistic processing (the composite task and the part/whole task), two measures of spatial and featural processing (the JanelLing task and the scrambledlblurred faces task) and a test of contour processing (JanelLing task) for both own-and other-race faces. No study to date has tested the same participants on all of these tasks. Participants had minimal experience with other-race faces; they had no Chinese family members, friends or had ever traveled to an Asian country. Results from the memory task did not reveal an other-race effect. In the present study, participants also demonstrated holistic processing of both own- and other-race faces on both the composite task and the part/whole task. These findings contradict previous findings that Caucasian adults process own-race faces more holistically than other-race faces. However participants did demonstrate an own-race advantage for processing the spacing among features, consistent with two recent studies that used different manipulations of spacing cues (Hayward et al. 2007; Rhodes et al. 2006). They also demonstrated an other-race effect for the processing of individual features for the Jane/Ling task (a direct measure of featural processing) consistent with previous findings (Rhodes, Hayward, & Winkler, 2006), but not for the scrambled faces task (an indirect measure offeatural processing). There was no own-race advantage for contour processing. Thus, these results lead to the conclusion that individuals may show less sensitivity to the appearance of individual features and the spacing among them in other-race faces, despite processing other-race faces holistically.

Individual discriminative subspace for face recognition under uncontrolled conditions

Most face recognition (FR) algorithms require the face images to satisfy certain restrictions in various aspects like view angle, illumination, occlusion, etc. But what is needed in general is the techniques that can recognize any face images recognizable by human beings. This paper provides one potential solution to this problem. A method named Individual Discriminative Subspace (IDS) is proposed for robust face recognition under uncontrolled conditions. IDS is the subspace where only the images from one particular person converge around the origin while those from others scatter. Each IDS can be used to distinguish one individual from others. There is no restriction on the face images fed into the algorithm, which makes it practical for real-life applications. In the experiments, IDS is tested on two large face databases with extensive variations and performs significantly better than 12 existing FR techniques.

Qualitative estimation of camera motion parameters from video sequences

We propose a simple technique for extracting camera motion parameters from a sequence of images. The method can estimate qualitatively camera pan, tilt, zoom, roll, and horizontal and vertical tracking. Unlike most other comparable techniques, the present method can distinguish pan from horizontal tracking, and tilt from vertical tracking. The technique can be applied to the automated indexing of video and film sequences.

Face recognition using kernel ridge regression

In this paper, we present novel ridge regression (RR) and kernel ridge regression (KRR) techniques for multivariate labels and apply the methods to the problem of face recognition. Motivated by the fact that the regular simplex vertices are separate points with highest degree of symmetry, we choose such vertices as the targets for the distinct individuals in recognition and apply RR or KRR to map the training face images into a face subspace where the training images from each individual will locate near their individual targets. We identify the new face image by mapping it into this face subspace and comparing its distance to all individual targets. An efficient cross-validation algorithm is also provided for selecting the regularization and kernel parameters. Experiments were conducted on two face databases and the results demonstrate that the proposed algorithm significantly outperforms the three popular linear face recognition techniques (Eigenfaces, Fisherfaces and Laplacianfaces) and also performs comparably with the recently developed Orthogonal Laplacianfaces with the advantage of computational speed. Experimental results also demonstrate that KRR outperforms RR as expected since KRR can utilize the nonlinear structure of the face images. Although we concentrate on face recognition in this paper, the proposed method is general and may be applied for general multi-category classification problems.

Face recognition via the overlapping energy histogram

In this paper we investigate the face recognition problem via the overlapping energy histogram of the DCT coefficients. Particularly, we investigate some important issues relating to the recognition performance, such as the issue of selecting threshold and the number of bins. These selection methods utilise information obtained from the training dataset. Experimentation is conducted on the Yale face database and results indicate that the proposed parameter selection methods perform well in selecting the threshold and number of bins. Furthermore, we show that the proposed overlapping energy histogram approach outperforms the Eigenfaces, 2DPCA and energy histogram significantly.

Learning Boltzmann distance metric for face recognition

We introduce a new method for face recognition using a versatile probabilistic model known as Restricted Boltzmann Machine (RBM). In particular, we propose to regularise the standard data likelihood learning with an information-theoretic distance metric defined on intra-personal images. This results in an effective face representation which captures the regularities in the face space and minimises the intra-personal variations. In addition, our method allows easy incorporation of multiple feature sets with controllable level of sparsity. Our experiments on a high variation dataset show that the proposed method is competitive against other metric learning rivals. We also investigated the RBM method under a variety of settings, including fusing facial parts and utilising localised feature detectors under varying resolutions. In particular, the accuracy is boosted from 71.8% with the standard whole-face pixels to 99.2% with combination of facial parts, localised feature extractors and appropriate resolutions.

Making the most of the self-quotient image in face recognition

The self-quotient image is a biologically inspired representation which has been proposed as an illumination invariant feature for automatic face recognition. Owing to the lack of strong domain specific assumptions underlying this representation, it can be readily extracted from raw images irrespective of the persons's pose, facial expression etc. What makes the self-quotient image additionally attractive is that it can be computed quickly and in a closed form using simple low-level image operations. However, it is generally accepted that the self-quotient is insufficiently robust to large illumination changes which is why it is mainly used in applications in which low precision is an acceptable compromise for high recall (e.g. retrieval systems). Yet, in this paper we demonstrate that the performance of this representation in challenging illuminations has been greatly underestimated. We show that its error rate can be reduced by over an order of magnitude, without any changes to the representation itself. Rather, we focus on the manner in which the dissimilarity between two self-quotient images is computed. By modelling the dominant sources of noise affecting the representation, we propose and evaluate a series of different dissimilarity measures, the best of which reduces the initial error rate of 63.0% down to only 5.7% on the notoriously challenging YaleB data set.

Infrared face recognition: a literature review

Automatic face recognition (AFR) is an area with immense practical potential which includes a wide range of commercial and law enforcement applications, and it continues to be one of the most active research areas of computer vision. Even after over three decades of intense research, the state-of-the-art in AFR continues to improve, benefiting from advances in a range of different fields including image processing, pattern recognition, computer graphics and physiology. However, systems based on visible spectrum images continue to face challenges in the presence of illumination, pose and expression changes, as well as facial disguises, all of which can significantly decrease their accuracy. Amongst various approaches which have been proposed in an attempt to overcome these limitations, the use of infrared (IR) imaging has emerged as a particularly promising research direction. This paper presents a comprehensive and timely review of the literature on this subject.

Vesselness features and the inverse compositional AAM for robust face recognition using thermal IR

Over the course of the last decade, infrared (IR) and particularly thermal IR imaging based face recognition has emerged as a promising complement to conventional, visible spectrum based approaches which continue to struggle when applied in the real world. While inherently insensitive to visible spectrum illumination changes, IR images introduce specific challenges of their own, most notably sensitivity to factors which affect facial heat emission patterns, e.g. emotional state, ambient temperature, and alcohol intake. In addition, facial expression and pose changes are more difficult to correct in IR images because they are less rich in high frequency detail which is an important cue for fitting any deformable model. In this paper we describe a novel method which addresses these major challenges. Specifically, to normalize for pose and facial expression changes we generate a synthetic frontal image of a face in a canonical, neutral facial expression from an image of the face in an arbitrary pose and facial expression. This is achieved by piecewise affine warping which follows active appearance model (AAM) fitting. This is the first publication which explores the use of an AAM on thermal IR images; we propose a pre-processing step which enhances detail in thermal images, making AAM convergence faster and more accurate. To overcome the problem of thermal IR image sensitivity to the exact pattern of facial temperature emissions we describe a representation based on reliable anatomical features. In contrast to previous approaches, our representation is not binary; rather, our method accounts for the reliability of the extracted features. This makes the proposed representation much more robust both to pose and scale changes. The effectiveness of the proposed approach is demonstrated on the largest public database of thermal IR images of faces on which it achieved 100% identification rate, significantly outperforming previously described methods

Recognition from appearance subspaces across image sets of variable scale

Linear subspace representations of appearance variation are pervasive in computer vision. In this paper we address the problem of robustly matching them (computing the similarity between them) when they correspond to sets of images of different (possibly greatly so) scales. We show that the naïve solution of projecting the low-scale subspace into the high-scale image space is inadequate, especially at large scale discrepancies. A successful approach is proposed instead. It consists of (i) an interpolated projection of the low-scale subspace into the high-scale space, which is followed by (ii) a rotation of this initial estimate within the bounds of the imposed “downsampling constraint”. The optimal rotation is found in the closed-form which best aligns the high-scale reconstruction of the low-scale subspace with the reference it is compared to. The proposed method is evaluated on the problem of matching sets of face appearances under varying illumination. In comparison to the naïve matching, our algorithm is shown to greatly increase the separation of between-class and within-class similarities, as well as produce far more meaningful modes of common appearance on which the match score is based.

Unfolding a face : from singular to manifold

Face recognition from a single image remains an important task in many practical applications and a significant research challenge. Some of the challenges are inherent to the problem, for example due to changing lighting conditions. Others, no less significant, are of a practical nature – face recognition algorithms cannot be assumed to operate on perfect data, but rather often on data that has already been subject to pre-processing errors (e.g. localization and registration errors). This paper introduces a novel method for face recognition that is both trained and queried using only a single image per subject. The key concept, motivated by abundant prior work on face appearance manifolds, is that of face part manifolds – it is shown that the appearance seen through a sliding window overlaid over an image of a face, traces a trajectory over a 2D manifold embedded in the image space. We present a theoretical argument for the use of this representation and demonstrate how it can be effectively exploited in the single image based recognition. It is shown that while inheriting the advantages of local feature methods, it also implicitly captures the geometric relationship between discriminative facial features and is naturally robust to face localization errors. Our theoretical arguments are verified in an experimental evaluation on the Yale Face Database.

Face set classification using maximally probable mutual modes

In this paper we consider face recognition from sets of face images and, in particular, recognition invariance to illumination. The main contribution is an algorithm based on the novel concept of maximally probable mutual modes (MMPM). Specifically: (i) we discuss and derive a local manifold illumination invariant and (ii) show how the invariant naturally leads to a formulation of "common modes" of two face appearance distributions. Recognition is then performed by finding the most probable mode, which is shown to be an eigenvalue problem. The effectiveness of the proposed method is demonstrated empirically on a challenging database containing the total of 700 video sequences of 100 individuals

A new look at filtering techniques for illumination invariance in automatic face recognition

Illumination invariance remains the most researched, yet the most challenging aspect of automatic face recognition. In this paper we propose a novel, general recognition framework for efficient matching of individual face images, sets or sequences. The framework is based on simple image processing filters that compete with unprocessed greyscale input to yield a single matching score between individuals. It is shown how the discrepancy between illumination conditions between novel input and the training data set can be estimated and used to weigh the contribution of two competing representations. We describe an extensive empirical evaluation of the proposed method on 171 individuals and over 1300 video sequences with extreme illumination, pose and head motion variation. On this challenging data set our algorithm consistently demonstrated a dramatic performance improvement over traditional filtering approaches. We demonstrate a reduction of 50-75% in recognition error rates, the best performing method-filter combination correctly recognizing 96% of the individuals.