Toward a more robust facial expression recognition in occluded images using randomly sampled Gabor based templates

Occlusion is a big challenge for facial expression recognition (FER) in real-world situations. Previous FER efforts to address occlusion suffer from loss of appearance features and are largely limited to a few occlusion types and single testing strategy. This paper presents a robust approach for FER in occluded images and addresses these issues. A set of Gabor based templates is extracted from images in the gallery using a Monte Carlo algorithm. These templates are converted into distance features using template matching. The resulting feature vectors are robust to occlusion. Occluded eyes and mouth regions and randomly places occlusion patches are used for testing. Two testing strategies analyze the effects of these occlusions on the overall recognition performance as well as each facial expression. Experimental results on the Cohn-Kanade database confirm the high robustness of our approach and provide useful insights about the effects of occlusion on FER. Performance is also compared with previous approaches.

Random Gabor based templates for facial expression recognition in images with facial occlusion

Robust facial expression recognition (FER) under occluded face conditions is challenging. It requires robust algorithms of feature extraction and investigations into the effects of different types of occlusion on the recognition performance to gain insight. Previous FER studies in this area have been limited. They have spanned recovery strategies for loss of local texture information and testing limited to only a few types of occlusion and predominantly a matched train-test strategy. This paper proposes a robust approach that employs a Monte Carlo algorithm to extract a set of Gabor based part-face templates from gallery images and converts these templates into template match distance features. The resulting feature vectors are robust to occlusion because occluded parts are covered by some but not all of the random templates. The method is evaluated using facial images with occluded regions around the eyes and the mouth, randomly placed occlusion patches of different sizes, and near-realistic occlusion of eyes with clear and solid glasses. Both matched and mis-matched train and test strategies are adopted to analyze the effects of such occlusion. Overall recognition performance and the performance for each facial expression are investigated. Experimental results on the Cohn-Kanade and JAFFE databases demonstrate the high robustness and fast processing speed of our approach, and provide useful insight into the effects of occlusion on FER. The results on the parameter sensitivity demonstrate a certain level of robustness of the approach to changes in the orientation and scale of Gabor filters, the size of templates, and occlusions ratios. Performance comparisons with previous approaches show that the proposed method is more robust to occlusion with lower reductions in accuracy from occlusion of eyes or mouth.

Feature-domain super-resolution framework for gabor-based face and iris recognition

The low resolution of images has been one of the major limitations in recognising humans from a distance using their biometric traits, such as face and iris. Superresolution has been employed to improve the resolution and the recognition performance simultaneously, however the majority of techniques employed operate in the pixel domain, such that the biometric feature vectors are extracted from a super-resolved input image. Feature-domain superresolution has been proposed for face and iris, and is shown to further improve recognition performance by capitalising on direct super-resolving the features which are used for recognition. However, current feature-domain superresolution approaches are limited to simple linear features such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), which are not the most discriminant features for biometrics. Gabor-based features have been shown to be one of the most discriminant features for biometrics including face and iris. This paper proposes a framework to conduct super-resolution in the non-linear Gabor feature domain to further improve the recognition performance of biometric systems. Experiments have confirmed the validity of the proposed approach, demonstrating superior performance to existing linear approaches for both face and iris biometrics.

Gabor-based region covariance matrices for face recognition

This paper presents a new method for human face recognition by utilizing Gabor-based region covariance matrices as face descriptors. Both pixel locations and Gabor coefficients are employed to form the covariance matrices. Experimental results demonstrate the advantages of this proposed method.

Towards spontaneous facial expression recognition in real-world video

Facial expression is an important channel of human social communication. Facial expression recognition (FER) aims to perceive and understand emotional states of humans based on information in the face. Building robust and high performance FER systems that can work in real-world video is still a challenging task, due to the various unpredictable facial variations and complicated exterior environmental conditions, as well as the difficulty of choosing a suitable type of feature descriptor for extracting discriminative facial information. Facial variations caused by factors such as pose, age, gender, race and occlusion, can exert profound influence on the robustness, while a suitable feature descriptor largely determines the performance. Most present attention on FER has been paid to addressing variations in pose and illumination. No approach has been reported on handling face localization errors and relatively few on overcoming facial occlusions, although the significant impact of these two variations on the performance has been proved and highlighted in many previous studies. Many texture and geometric features have been previously proposed for FER. However, few comparison studies have been conducted to explore the performance differences between different features and examine the performance improvement arisen from fusion of texture and geometry, especially on data with spontaneous emotions. The majority of existing approaches are evaluated on databases with posed or induced facial expressions collected in laboratory environments, whereas little attention has been paid on recognizing naturalistic facial expressions on real-world data. This thesis investigates techniques for building robust and high performance FER systems based on a number of established feature sets. It comprises of contributions towards three main objectives: (1) Robustness to face localization errors and facial occlusions. An approach is proposed to handle face localization errors and facial occlusions using Gabor based templates. Template extraction algorithms are designed to collect a pool of local template features and template matching is then performed to covert these templates into distances, which are robust to localization errors and occlusions. (2) Improvement of performance through feature comparison, selection and fusion. A comparative framework is presented to compare the performance between different features and different feature selection algorithms, and examine the performance improvement arising from fusion of texture and geometry. The framework is evaluated for both discrete and dimensional expression recognition on spontaneous data. (3) Evaluation of performance in the context of real-world applications. A system is selected and applied into discriminating posed versus spontaneous expressions and recognizing naturalistic facial expressions. A database is collected from real-world recordings and is used to explore feature differences between standard database images and real-world images, as well as between real-world images and real-world video frames. The performance evaluations are based on the JAFFE, CK, Feedtum, NVIE, Semaine and self-collected QUT databases. The results demonstrate high robustness of the proposed approach to the simulated localization errors and occlusions. Texture and geometry have different contributions to the performance of discrete and dimensional expression recognition, as well as posed versus spontaneous emotion discrimination. These investigations provide useful insights into enhancing robustness and achieving high performance of FER systems, and putting them into real-world applications.

ART: An ontology based tool for the translation of papers into Semantic Web format

To be presented at SIG/ISMB07 ontology workshop: http://bio-ontologies.org.uk/index.php To be published in BMC Bioinformatics. Sponsorship: JISC

Site Selection for Restoration Planning: A Protocol With Landscape and Legislation Based Alternatives

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Accurate urban road model reconstruction from high resolution remotely sensed imagery based on Support Vector Machine and Gabor filters

In this paper, we presented an automatic system for precise urban road model reconstruction based on aerial images with high spatial resolution. The proposed approach consists of two steps: i) road surface detection and ii) road pavement marking extraction. In the first step, support vector machine (SVM) was utilized to classify the images into two categories: road and non-road. In the second step, road lane markings are further extracted on the generated road surface based on 2D Gabor filters. The experiments using several pan-sharpened aerial images of Brisbane, Queensland have validated the proposed method.

Phosphorus-based functional groups as hydrogen bonding templates for rotaxane formation

We report on the use of the hydrogen bond acceptor properties of some phosphorus-containing functional groups for the assembly of a series of [2]rotaxanes. Phosphinamides, and the homologous thio– and selenophosphinamides, act as hydrogen bond acceptors that, in conjunction with an appropriately positioned amide group on the thread, direct the assembly of amide-based macrocycles around the axle to form rotaxanes in up to 60% yields. Employing solely phosphorus-based functional groups as the hydrogen bond accepting groups on the thread, a bis(phosphinamide) template and a phosphine oxide-phosphinamide template afforded the corresponding rotaxanes in 18 and 15 % yields, respectively. X-Ray crystallography of the rotaxanes shows the presence of up to four intercomponent hydrogen bonds between the amide groups of the macrocycle and various hydrogen bond accepting groups on the thread, including rare examples of amide-to-phosphonamide, -thiophosphinamide and -selenophosphinamide groups. With a phosphine oxide-phosphinamide thread, the solid state structure of the rotaxane is remarkable, featuring no direct intercomponent hydrogen bonds but rather a hydrogen bond network involving water molecules that bridge the H-bonding groups of the macrocycle and thread through bifurcated hydrogen bonds. The incorporation of phosphorus-based functional groups into rotaxanes may prove useful for the development of molecular shuttles in which the macrocycle can be used to hinder or expose binding ligating sites for metal-based catalysts.

Design of mechanism-based inhibitors of transthyretin amyloidosis:Studies with biphenyl ethers and new structural templates

Transthyretin (TTR), a tetrameric thyroxine (T4) carrier protein, is associated with a variety of amyloid diseases. In this study, we explore the potential of biphenyl ethers (BPE), which are shown to interact with a high affinity to its T4 binding site thereby preventing its aggregation and fibrillogenesis. They prevent fibrillogenesis by stabilizing the tetrameric ground state of transthyretin. Additionally, we identify two new structural templates (2-(5-mercapto-[1,3,4]oxadiazol-2-yl)-phenol and 2,3,6-trichloro-N-(4H-[1,2,4]triazol-3-yl) represented as compounds 11 and 12, respectively, throughout the manuscript) exhibiting the ability to arrest TTR amyloidosis. The dissociation constants for the binding of BPEs and compound 11 and 12 to TTR correlate with their efficacies of inhibiting amyloidosis. They also have the ability to inhibit the elongation of intermediate fibrils as well as show nearly complete (> 90%) disruption of the preformed fibrils. The present study thus establishes biphenyl ethers and compounds 11 and 12 as very potent inhibitors of TTR fibrillization and inducible cytotoxicity.

Design of mechanism-based inhibitors of transthyretin amyloidosis: Studies with biphenyl ethers and new structural templates

Transthyretin (TTR), a tetrameric thyroxine (T4) carrier protein, is associated with a variety of amyloid diseases. In this study, we explore the potential of biphenyl ethers (BPE), which are shown to interact with a high affinity to its T4 binding site thereby preventing its aggregation and fibrillogenesis. They prevent fibrillogenesis by stabilizing the tetrameric ground state of transthyretin. Additionally, we identify two new structural templates (2-(5-mercapto-[1,3,4]oxadiazol-2-yl)-phenol and 2,3,6-trichloro-N-(4H-[1,2,4]triazol-3-yl) represented as compounds 11 and 12, respectively, throughout the manuscript) exhibiting the ability to arrest TTR amyloidosis. The dissociation constants for the binding of BPEs and compound 11 and 12 to TTR correlate with their efficacies of inhibiting amyloidosis. They also have the ability to inhibit the elongation of intermediate fibrils as well as show nearly complete (> 90%) disruption of the preformed fibrils. The present study thus establishes biphenyl ethers and compounds 11 and 12 as very potent inhibitors of TTR fibrillization and inducible cytotoxicity.