Super-realistic rendering using real-time tweening

The realism of contemporary computer graphics (and especially Virtual Reality {VR}) is limited by the great computational cost of rendering objects of appropriate complexity with convincing lighting and surface effects. We introduce a framework that allows rendering of objects in true photographic quality using tweening. The simple but effective design of our system allows us not only to perform the necessary operations in real-time on standard hardware, but also achieve other effects like morphing. Furthermore, it is shown how our system can be gainfully employed in non-VR contexts like extreme low-bandwidth video-conferencing and others.

Robust learning of discriminative projection for multicategory classification on the Stiefel manifold

Learning a robust projection with a small number of training samples is still a challenging problem in face recognition, especially when the unseen faces have extreme variation in pose, illumination, and facial expression. To address this problem, we propose a framework formulated under statistical learning theory that facilitates robust learning of a discriminative projection. Dimensionality reduction using the projection matrix is combined with a linear classifier in the regularized framework of lasso regression. The projection matrix in conjunction with the classifier parameters are then found by solving an optimization problem over the Stiefel manifold. The experimental results on standard face databases suggest that the proposed method outperforms some recent regularized techniques when the number of training samples is small.

Content validity of visual analog scales to assess symptom severity of acute angioedema attacks in adults with hereditary angioedema : an interview study

Background: Hereditary angioedema (HAE) is a rare, debilitating, potentially life-threatening condition characterized by recurrent acute attacks of edema of the skin, face/upper airway, and gastrointestinal and urogenital tracts. During a laryngeal attack, people with HAE may be at risk of suffocation, while other attacks are often associated with intense pain, disfigurement, disability, and/or vomiting. The intensity of some symptoms is known only to the person experiencing them. Thus, interview studies are needed to explore such experience and patient-reported outcome measures (PROMs) are required for systematic assessment of symptoms in the clinical setting and in clinical trials of treatments for acute HAE attacks.

Objective: The aim of this interview study was to assess the content validity and suitability of four visual analog scale (VAS) instruments for use in clinical studies. The VAS instruments were designed to assess symptoms at abdominal, oro-facial-pharyngeal-laryngeal, peripheral, and urogenital attack locations. This is the first known study to report qualitative data about the patient's experience of the rare disorder, HAE.

Methods: Semi-structured exploratory and cognitive debriefing interviews were conducted with 27 adults with a confirmed clinical/laboratory diagnosis of HAE (baseline plasma level of functional plasma protein C1 esterase inhibitor [C1INH] <50% of normal without evidence for acquired angioedema). There were 17 participants from the US and 10 from Italy, with mean age 42.5 (SD 14.5) years, range 18–72 years, mean HAE duration 21.3 (SD 14.1) years, range 1–45 years, 67% female, and 44% VAS-naïve. Experience of acute angioedema attacks was first explored, noting spontaneous mentions by participants of HAE symptomatology. Cognitive debriefing of the VAS instruments was undertaken to assess the suitability, comprehensibility, and relevance of the VAS items. Asymptomatic participants completed the VAS instruments relevant to their angioedema experience, reporting as if they were experiencing an acute angioedema attack at the time. Interviews were conducted in the clinic setting in the US and Italy over an 8-month period.

Results: Participants mentioned spontaneously almost all aspects of acute angioedema attacks covered by the four VAS instruments, thus providing strong support for inclusion of nearly all VAS items, with no important symptoms missing. Predominant symptoms found to be associated with acute angioedema attacks were edema and pain, and there was evidence of varying degrees of disruption to everyday activities supporting the inclusion of an overall severity item reflecting the disabling effects of HAE symptoms. VAS item wording was understood by participants.

Conclusion: This interview study explored and reported the patient experience of HAE attacks. It demonstrated the content validity of the four anatomical location HAE VAS instruments and their suitability for use in clinical trials of recombinant human C1INH (rhC1INH) treatment for ascertaining trial participants' assessments of the severity of acute angioedema symptoms.

Utility of short-term evaluation of presbyopic contact lens performance

Objectives: To establish if evaluations of multifocal contact lens performance conducted at dispensing are representative of behavior after a moderate adaptation period.

Methods: Eighty-eight presbyopic subjects, across four clinical sites, wore each of four multifocal soft contact lenses (ACUVUE BIFOCAL, Focus Progressives, Proclear Multifocal, and SofLens Multifocal) for 4 days of daily wear. Comprehensive performance assessments were conducted at dispensing and after 4 days wear and included the following objective metrics: LogMAR acuity (contrast, 90% and 10%; illumination, 250 and 10 cd/m2; distance, 6 m, 100 cm, and 40 cm), stereopsis (RANDOT), reading critical print size and maximum speed and range of clear vision at near. Subjective assessments were made, with 100-point numerical rating scales, of comfort, ghosting (distance, near), visual quality (distance, intermediate, and near), and the appearance of haloes. At two sites, subjects (n = 39) also rated visual fluctuation (distance, intermediate, and near), facial recognition, and overall satisfaction.

Results: Among the objective variables, significant differences (paired t test, P<0.05) between dispensing and 4 days were found only for range of clear vision at near (2.9 ± 2.0 cm; mean difference ± standard deviation) and high contrast near acuity in low illumination (-0.013 ± 0.011 LogMAR). With the exception of insertion comfort, all subjective variables showed significant decrements over the same period. Overall satisfaction declined by an average of 10.9 ± 5.1 points.

Conclusions: Early assessment is relatively unrepresentative of performance later on during multifocal contact lens wear. Acuity based measures of vision remain substantially unchanged over the medium term, apparently because these metrics are insensitive indicators of performance compared with subjective alternatives.

Learning Boltzmann distance metric for face recognition

We introduce a new method for face recognition using a versatile probabilistic model known as Restricted Boltzmann Machine (RBM). In particular, we propose to regularise the standard data likelihood learning with an information-theoretic distance metric defined on intra-personal images. This results in an effective face representation which captures the regularities in the face space and minimises the intra-personal variations. In addition, our method allows easy incorporation of multiple feature sets with controllable level of sparsity. Our experiments on a high variation dataset show that the proposed method is competitive against other metric learning rivals. We also investigated the RBM method under a variety of settings, including fusing facial parts and utilising localised feature detectors under varying resolutions. In particular, the accuracy is boosted from 71.8% with the standard whole-face pixels to 99.2% with combination of facial parts, localised feature extractors and appropriate resolutions.

Making the most of the self-quotient image in face recognition

The self-quotient image is a biologically inspired representation which has been proposed as an illumination invariant feature for automatic face recognition. Owing to the lack of strong domain specific assumptions underlying this representation, it can be readily extracted from raw images irrespective of the persons's pose, facial expression etc. What makes the self-quotient image additionally attractive is that it can be computed quickly and in a closed form using simple low-level image operations. However, it is generally accepted that the self-quotient is insufficiently robust to large illumination changes which is why it is mainly used in applications in which low precision is an acceptable compromise for high recall (e.g. retrieval systems). Yet, in this paper we demonstrate that the performance of this representation in challenging illuminations has been greatly underestimated. We show that its error rate can be reduced by over an order of magnitude, without any changes to the representation itself. Rather, we focus on the manner in which the dissimilarity between two self-quotient images is computed. By modelling the dominant sources of noise affecting the representation, we propose and evaluate a series of different dissimilarity measures, the best of which reduces the initial error rate of 63.0% down to only 5.7% on the notoriously challenging YaleB data set.

Illumination-invariant face recognition from a single image across extreme pose using a dual dimension AAM ensemble in the thermal infrared spectrum

Over the course of the last decade, infrared (IR) and particularly thermal IR imaging based face recognition has emerged as a promising complement to conventional, visible spectrum based approaches which continue to struggle when applied in practice. While inherently insensitive to visible spectrum illumination changes, IR data introduces specific challenges of its own, most notably sensitivity to factors which affect facial heat emission patterns, e.g. emotional state, ambient temperature, and alcohol intake. In addition, facial expression and pose changes are more difficult to correct in IR images because they are less rich in high frequency detail which is an important cue for fitting any deformable model. In this paper we describe a novel method which addresses these major challenges. Specifically, when comparing two thermal IR images of faces, we mutually normalize their poses and facial expressions by using an active appearance model (AAM) to generate synthetic images of the two faces with a neutral facial expression and in the same view (the average of the two input views). This is achieved by piecewise affine warping which follows AAM fitting. A major contribution of our work is the use of an AAM ensemble in which each AAM is specialized to a particular range of poses and a particular region of the thermal IR face space. Combined with the contributions from our previous work which addressed the problem of reliable AAM fitting in the thermal IR spectrum, and the development of a person-specific representation robust to transient changes in the pattern of facial temperature emissions, the proposed ensemble framework accurately matches faces across the full range of yaw from frontal to profile, even in the presence of scale variation (e.g. due to the varying distance of a subject from the camera). The effectiveness of the proposed approach is demonstrated on the largest public database of thermal IR images of faces and a newly acquired data set of thermal IR motion videos. Our approach achieved perfect recognition performance on both data sets, significantly outperforming the current state of the art methods even when they are trained with multiple images spanning a range of head views.

Infrared face recognition: a literature review

Automatic face recognition (AFR) is an area with immense practical potential which includes a wide range of commercial and law enforcement applications, and it continues to be one of the most active research areas of computer vision. Even after over three decades of intense research, the state-of-the-art in AFR continues to improve, benefiting from advances in a range of different fields including image processing, pattern recognition, computer graphics and physiology. However, systems based on visible spectrum images continue to face challenges in the presence of illumination, pose and expression changes, as well as facial disguises, all of which can significantly decrease their accuracy. Amongst various approaches which have been proposed in an attempt to overcome these limitations, the use of infrared (IR) imaging has emerged as a particularly promising research direction. This paper presents a comprehensive and timely review of the literature on this subject.

Vesselness features and the inverse compositional AAM for robust face recognition using thermal IR

Over the course of the last decade, infrared (IR) and particularly thermal IR imaging based face recognition has emerged as a promising complement to conventional, visible spectrum based approaches which continue to struggle when applied in the real world. While inherently insensitive to visible spectrum illumination changes, IR images introduce specific challenges of their own, most notably sensitivity to factors which affect facial heat emission patterns, e.g. emotional state, ambient temperature, and alcohol intake. In addition, facial expression and pose changes are more difficult to correct in IR images because they are less rich in high frequency detail which is an important cue for fitting any deformable model. In this paper we describe a novel method which addresses these major challenges. Specifically, to normalize for pose and facial expression changes we generate a synthetic frontal image of a face in a canonical, neutral facial expression from an image of the face in an arbitrary pose and facial expression. This is achieved by piecewise affine warping which follows active appearance model (AAM) fitting. This is the first publication which explores the use of an AAM on thermal IR images; we propose a pre-processing step which enhances detail in thermal images, making AAM convergence faster and more accurate. To overcome the problem of thermal IR image sensitivity to the exact pattern of facial temperature emissions we describe a representation based on reliable anatomical features. In contrast to previous approaches, our representation is not binary; rather, our method accounts for the reliability of the extracted features. This makes the proposed representation much more robust both to pose and scale changes. The effectiveness of the proposed approach is demonstrated on the largest public database of thermal IR images of faces on which it achieved 100% identification rate, significantly outperforming previously described methods

Towards person authentication by fusing visual and thermal face biometrics

In this chapter we focus on face appearance-based biometrics. The cheap and readily available hardware used to acquire data, their non-invasiveness and the ease of employing them from a distance and without the awareness of the user, are just some of the reasons why these continue to be of great practical interest. However, a number of research challenges remain. Specifically, face biometrics have traditionally focused on images acquired in the visible light spectrum and these are greatly affected by such extrinsic factors such as the illumination, camera angle (or, equivalently, head pose) and occlusion. In practice, the effects of changing pose are usually least problematic and can oftentimes be overcome by acquiring data over a time period, e.g., by tracking a face in a surveillance video. Consequently, image sequence or image set matching has recently gained a lot of attention in the literature [137–139] and is the paradigm adopted in this chapter as well. In other words, we assume that the training image set for each individual contains some variability in pose, but is not obtained in scripted conditions or in controlled illumination. In contrast, illumination is much more difficult to deal with: the illumination setup is in most cases not practical to control and its physics is difficult to accurately model. Thermal spectrum imagery is useful in this regard as it is virtually insensitive to illumination changes, as illustrated in Fig. 6.1. On the other hand, it lacks much of the individual, discriminating facial detail contained in visual images. In this sense, the two modalities can be seen as complementing each other. The key idea behind the system presented in this chapter is that robustness to extreme illumination changes can be achieved by fusing the two. This paradigm will further prove useful when we consider the difficulty of recognition in the presence of occlusion caused by prescription glasses.

Unfolding a face : from singular to manifold

Face recognition from a single image remains an important task in many practical applications and a significant research challenge. Some of the challenges are inherent to the problem, for example due to changing lighting conditions. Others, no less significant, are of a practical nature – face recognition algorithms cannot be assumed to operate on perfect data, but rather often on data that has already been subject to pre-processing errors (e.g. localization and registration errors). This paper introduces a novel method for face recognition that is both trained and queried using only a single image per subject. The key concept, motivated by abundant prior work on face appearance manifolds, is that of face part manifolds – it is shown that the appearance seen through a sliding window overlaid over an image of a face, traces a trajectory over a 2D manifold embedded in the image space. We present a theoretical argument for the use of this representation and demonstrate how it can be effectively exploited in the single image based recognition. It is shown that while inheriting the advantages of local feature methods, it also implicitly captures the geometric relationship between discriminative facial features and is naturally robust to face localization errors. Our theoretical arguments are verified in an experimental evaluation on the Yale Face Database.

Gradient edge map features for frontal face recognition under extreme illumination changes

Our aim in this paper is to robustly match frontal faces in the presence of extreme illumination changes, using only a single training image per person and a single probe image. In the illumination conditions we consider, which include those with the dominant light source placed behind and to the side of the user, directly above and pointing downwards or indeed below and pointing upwards, this is a most challenging problem. The presence of sharp cast shadows, large poorly illuminated regions of the face, quantum and quantization noise and other nuisance effects, makes it difficult to extract a sufficiently discriminative yet robust representation. We introduce a representation which is based on image gradient directions near robust edges which correspond to characteristic facial features. Robust edges are extracted using a cascade of processing steps, each of which seeks to harness further discriminative information or normalize for a particular source of extra-personal appearance variability. The proposed representation was evaluated on the extremely difficult YaleB data set. Unlike most of the previous work we include all available illuminations, perform training using a single image per person and match these also to a single probe image. In this challenging evaluation setup, the proposed gradient edge map achieved 0.8% error rate, demonstrating a nearly perfect receiver-operator characteristic curve behaviour. This is by far the best performance achieved in this setup reported in the literature, the best performing methods previously proposed attaining error rates of approximately 6–7%.

Multi-sensory face biometric fusion (for personal identification)

The objective of this work is to recognize faces using sets of images in visual and thermal spectra. This is challenging because the former is greatly affected by illumination changes, while the latter frequently contains occlusions due to eye-wear and is inherently less discriminative. Our method is based on a fusion of the two modalities. Specifically: we examine (i) the effects of preprocessing of data in each domain, (ii) the fusion of holistic and local facial appearance, and (iii) propose an algorithm for combining the similarity scores in visual and thermal spectra in the presence of prescription glasses and significant pose variations, using a small number of training images (5-7). Our system achieved a high correct identification rate of 97% on a freely available test set of 29 individuals and extreme illumination changes.