Exploring visual features through Gabor representations for facial expression detection

Gabor representations have been widely used in facial analysis (face recognition, face detection and facial expression detection) due to their biological relevance and computational properties. Two popular Gabor representations used in literature are: 1) Log-Gabor and 2) Gabor energy filters. Even though these representations are somewhat similar, they also have distinct differences as the Log-Gabor filters mimic the simple cells in the visual cortex while the Gabor energy filters emulate the complex cells, which causes subtle differences in the responses. In this paper, we analyze the difference between these two Gabor representations and quantify these differences on the task of facial action unit (AU) detection. In our experiments conducted on the Cohn-Kanade dataset, we report an average area underneath the ROC curve (A`) of 92.60% across 17 AUs for the Gabor energy filters, while the Log-Gabor representation achieved an average A` of 96.11%. This result suggests that small spatial differences that the Log-Gabor filters pick up on are more useful for AU detection than the differences in contours and edges that the Gabor energy filters extract.

Temporal-based support vector machines for facial expression recognition

When classifying a signal, ideally we want our classifier to trigger a large response when it encounters a positive example and have little to no response for all other examples. Unfortunately in practice this does not occur with responses fluctuating, often causing false alarms. There exists a myriad of reasons why this is the case, most notably not incorporating the dynamics of the signal into the classification. In facial expression recognition, this has been highlighted as one major research question. In this paper we present a novel technique which incorporates the dynamics of the signal which can produce a strong response when the peak expression is found and essentially suppresses all other responses as much as possible. We conducted preliminary experiments on the extended Cohn-Kanade (CK+) database which shows its benefits. The ability to automatically and accurately recognize facial expressions of drivers is highly relevant to the automobile. For example, the early recognition of “surprise” could indicate that an accident is about to occur; and various safeguards could immediately be deployed to avoid or minimize injury and damage. In this paper, we conducted initial experiments on the extended Cohn-Kanade (CK+) database which shows its benefits.

Effect of poly(acrylic acid) molecular mass and end-group functionality on calcium oxalate crystal morphology and growth

A number of series of poly(acrylic acids) (PAA) of differing end-groups and molecular mass were used to study the inhibition of calcium oxalate crystallization. The effects of the end-group on crystal speciation and morphology were significant and dramatic, with hexyl-isobutyrate end groups giving preferential formation of calcium oxalate dihydrate (COD) rather than the more stable calcium oxalate monohydrate (COM), while both more hydrophobic end-groups and less-hydrophobic end groups led predominantly to formation of the least thermodynamically stable form of calcium oxalate, calcium oxalate trihydrate. Conversely, molecular mass had little impact on calcium oxalate speciation or crystal morphology. It is probable that the observed effects are related to the rate of desorption of the PAA moiety from the crystal (lite) surfaces and that the results point to a major role for end-group as well as molecular mass in controlling desorption rate.

Person-independent facial expression detection using constrained local models

In automatic facial expression detection, very accurate registration is desired which can be achieved via a deformable model approach where a dense mesh of 60-70 points on the face is used, such as an active appearance model (AAM). However, for applications where manually labeling frames is prohibitive, AAMs do not work well as they do not generalize well to unseen subjects. As such, a more coarse approach is taken for person-independent facial expression detection, where just a couple of key features (such as face and eyes) are tracked using a Viola-Jones type approach. The tracked image is normally post-processed to encode for shift and illumination invariance using a linear bank of filters. Recently, it was shown that this preprocessing step is of no benefit when close to ideal registration has been obtained. In this paper, we present a system based on the Constrained Local Model (CLM) which is a generic or person-independent face alignment algorithm which gains high accuracy. We show these results against the LBP feature extraction on the CK+ and GEMEP datasets.

Toward a more robust facial expression recognition in occluded images using randomly sampled Gabor based templates

Occlusion is a big challenge for facial expression recognition (FER) in real-world situations. Previous FER efforts to address occlusion suffer from loss of appearance features and are largely limited to a few occlusion types and single testing strategy. This paper presents a robust approach for FER in occluded images and addresses these issues. A set of Gabor based templates is extracted from images in the gallery using a Monte Carlo algorithm. These templates are converted into distance features using template matching. The resulting feature vectors are robust to occlusion. Occluded eyes and mouth regions and randomly places occlusion patches are used for testing. Two testing strategies analyze the effects of these occlusions on the overall recognition performance as well as each facial expression. Experimental results on the Cohn-Kanade database confirm the high robustness of our approach and provide useful insights about the effects of occlusion on FER. Performance is also compared with previous approaches.

Automatically detecting pain in video through facial action units

In a clinical setting, pain is reported either through patient self-report or via an observer. Such measures are problematic as they are: 1) subjective, and 2) give no specific timing information. Coding pain as a series of facial action units (AUs) can avoid these issues as it can be used to gain an objective measure of pain on a frame-by-frame basis. Using video data from patients with shoulder injuries, in this paper, we describe an active appearance model (AAM)-based system that can automatically detect the frames in video in which a patient is in pain. This pain data set highlights the many challenges associated with spontaneous emotion detection, particularly that of expression and head movement due to the patient's reaction to pain. In this paper, we show that the AAM can deal with these movements and can achieve significant improvements in both the AU and pain detection performance compared to the current-state-of-the-art approaches which utilize similarity-normalized appearance features only.

Frequency decomposition techniques for increased discriminative 3D facial information capture

Eigen-based techniques and other monolithic approaches to face recognition have long been a cornerstone in the face recognition community due to the high dimensionality of face images. Eigen-face techniques provide minimal reconstruction error and limit high-frequency content while linear discriminant-based techniques (fisher-faces) allow the construction of subspaces which preserve discriminatory information. This paper presents a frequency decomposition approach for improved face recognition performance utilising three well-known techniques: Wavelets; Gabor / Log-Gabor; and the Discrete Cosine Transform. Experimentation illustrates that frequency domain partitioning prior to dimensionality reduction increases the information available for classification and greatly increases face recognition performance for both eigen-face and fisher-face approaches.

The patella morphology in trochlear dysplasia : a comparative MRI study

BACKGROUND: Trochlear dysplasia is suspected to have a genetic basis and causes recurrent patellar instability due to insufficient anatomical geometry. Numerous studies about trochlear morphology and the optimal surgical treatment have been carried out, but no attention has been paid to the corresponding patellar morphology.----- ----- PURPOSE: The aim of this study was the evaluation of the patellar morphology in normal and trochlear dysplastic knees. ----- ----- STUDY DESIGN: Biometric analysis. ----- ----- METHODS: Twenty two patellae with underlying trochlear dysplasia (study group--SG) were compared with 22 matched knees with normal trochlear shape (control group--CG) on transverse and sagittal MRI slices. We compared transverse diameter, cartilaginous thickness, Wiberg-index and -angle, length and radius of lateral and medial facet, patellar shape and angle, retropatellar length, and type of trochlear dysplasia. For statistical analysis we used the Wilcoxon signed ranks test. ----- ----- RESULTS: The transverse and sagittal diameter, mean length of medial patellar facet, and mean cartilaginous and subchondral Wiberg-index showed statistical differences between the two groups. ----- ----- CONCLUSIONS: Although the insufficient trochlear depth and decreased lateral trochlear slope are responsible for patellofemoral instability, the patella shows morphological changes in trochlear dysplastic knees. Its overall size and the medial facet are smaller. Although the femoral sulcus angle is larger, the Wiberg-angle and -index are equal to the control group. This may indicate that the patellar morphology may not be a result of missing medial patellofemoral pressure in trochlear dysplastic knees, but a decreased medial patellofemoral traction. This seems to be caused by hypotrophic medial patellofemoral restraints in combination with an increased lateral patellar tilt, both resulting in a decreased tension onto the medial patella facet. Whether there is a genetic component to the patellar morphology remains open.

Plantar fasciitis : are pain and fascial thickness associated with arch shape and loading?

Background and Purpose Although plantar fascial thickening is a sonographic criterion for the diagnosis of plantar fasciitis, the effect of local loading and structural factors on fascial morphology are unknown. The purposes of this study were to compare sonographic measures of fascial thickness and radiographic measures of arch shape and regional loading of the foot during gait in individuals with and without unilateral plantar fasciitis and to investigate potential relationships between these loading and structural factors and the morphology of the plantar fascia in individuals with and without heel pain. Subjects The participants were 10 subjects with unilateral plantar fasciitis and 10 matched asymptomatic controls. Methods Heel pain on weight bearing was measured by a visual analog scale. Fascial thickness and static arch angle were determined from bilateral sagittal sonograms and weight-bearing lateral foot roentgenograms. Regional plantar loading was estimated from a pressure plate. Results On average, the plantar fascia of the symptomatic limb was thicker than the plantar fascia of the asymptomatic limb (6.1±1.4 mm versus 4.2±0.5 mm), which, in turn, was thicker than the fascia of the matched control limbs (3.4±0.5 mm and 3.5±0.6 mm). Pain was correlated with fascial thickness, arch angle, and midfoot loading in the symptomatic foot. Fascial thickness, in turn, was positively correlated with arch angle in symptomatic and asymptomatic feet and with peak regional loading of the midfoot in the symptomatic limb. Discussion and Conclusion The findings indicate that fascial thickness and pain in plantar fasciitis are associated with the regional loading and static shape of the arch.

Facial expression recognition using facial movement features

Facial expression is an important channel for human communication and can be applied in many real applications. One critical step for facial expression recognition (FER) is to accurately extract emotional features. Current approaches on FER in static images have not fully considered and utilized the features of facial element and muscle movements, which represent static and dynamic, as well as geometric and appearance characteristics of facial expressions. This paper proposes an approach to solve this limitation using ‘salient’ distance features, which are obtained by extracting patch-based 3D Gabor features, selecting the ‘salient’ patches, and performing patch matching operations. The experimental results demonstrate high correct recognition rate (CRR), significant performance improvements due to the consideration of facial element and muscle movements, promising results under face registration errors, and fast processing time. The comparison with the state-of-the-art performance confirms that the proposed approach achieves the highest CRR on the JAFFE database and is among the top performers on the Cohn-Kanade (CK) database.

Improving the performance of facial expression recognition using dynamic, subtle and regional features

Human facial expression is a complex process characterized of dynamic, subtle and regional emotional features. State-of-the-art approaches on facial expression recognition (FER) have not fully utilized this kind of features to improve the recognition performance. This paper proposes an approach to overcome this limitation using patch-based ‘salient’ Gabor features. A set of 3D patches are extracted to represent the subtle and regional features, and then inputted into patch matching operations for capturing the dynamic features. Experimental results show a significant performance improvement of the proposed approach due to the use of the dynamic features. Performance comparison with pervious work also confirms that the proposed approach achieves the highest CRR reported to date on the JAFFE database and a top-level performance on the Cohn-Kanade (CK) database.

Urban maze : A typological investigation in porous morphology and dynamic pedestrian networks

Mixed use typologies and pedestrian networks are two strategies commonly applied in design of the contemporary city. These approaches, aimed towards the creation of a more sustainalble urban environment, have their roots in the traditional, pre-industrial towns; they characterize urban form, articulating the tension between privaate and public realms through a series of typological variations as well as stimulating commercial activity in the city centre. Arcades, loggias and verandas are just some of the elements which can mediate this tension. Historically they have defined physical and social spaces with particular character; in the contemporary city these features are applied to deform the urban form and create a porous, dynamic morphology. This paper, comparing case studies from Italy, Japan and Australia, investigates how the design of the transition zone can define hybrid pedestrian networks, where a clear distinction between the public and private realms is no longer applicable. Pedestrians use the city in a dynamic way, combining trajectories on the public street with ones on the fringe or inside of the private built environment. In some cases, cities offer different pedestrian network possibilities at different times, as the commercial precints are subject to variations in accessibility across various timeframes. These walkable systems have an impact on the urban form and identity of places, redefining typologies and requiring an in depth analysis through plan, section and elevation diagrams.

Evaluation of texture and geometry for dimensional facial expression recognition

Facial expression recognition (FER) algorithms mainly focus on classification into a small discrete set of emotions or representation of emotions using facial action units (AUs). Dimensional representation of emotions as continuous values in an arousal-valence space is relatively less investigated. It is not fully known whether fusion of geometric and texture features will result in better dimensional representation of spontaneous emotions. Moreover, the performance of many previously proposed approaches to dimensional representation has not been evaluated thoroughly on publicly available databases. To address these limitations, this paper presents an evaluation framework for dimensional representation of spontaneous facial expressions using texture and geometric features. SIFT, Gabor and LBP features are extracted around facial fiducial points and fused with FAP distance features. The CFS algorithm is adopted for discriminative texture feature selection. Experimental results evaluated on the publicly accessible NVIE database demonstrate that fusion of texture and geometry does not lead to a much better performance than using texture alone, but does result in a significant performance improvement over geometry alone. LBP features perform the best when fused with geometric features. Distributions of arousal and valence for different emotions obtained via the feature extraction process are compared with those obtained from subjective ground truth values assigned by viewers. Predicted valence is found to have a more similar distribution to ground truth than arousal in terms of covariance or Bhattacharya distance, but it shows a greater distance between the means.