Automatically detecting pain in video through facial action units

In a clinical setting, pain is reported either through patient self-report or via an observer. Such measures are problematic as they are: 1) subjective, and 2) give no specific timing information. Coding pain as a series of facial action units (AUs) can avoid these issues as it can be used to gain an objective measure of pain on a frame-by-frame basis. Using video data from patients with shoulder injuries, in this paper, we describe an active appearance model (AAM)-based system that can automatically detect the frames in video in which a patient is in pain. This pain data set highlights the many challenges associated with spontaneous emotion detection, particularly that of expression and head movement due to the patient's reaction to pain. In this paper, we show that the AAM can deal with these movements and can achieve significant improvements in both the AU and pain detection performance compared to the current-state-of-the-art approaches which utilize similarity-normalized appearance features only.

Frequency decomposition techniques for increased discriminative 3D facial information capture

Eigen-based techniques and other monolithic approaches to face recognition have long been a cornerstone in the face recognition community due to the high dimensionality of face images. Eigen-face techniques provide minimal reconstruction error and limit high-frequency content while linear discriminant-based techniques (fisher-faces) allow the construction of subspaces which preserve discriminatory information. This paper presents a frequency decomposition approach for improved face recognition performance utilising three well-known techniques: Wavelets; Gabor / Log-Gabor; and the Discrete Cosine Transform. Experimentation illustrates that frequency domain partitioning prior to dimensionality reduction increases the information available for classification and greatly increases face recognition performance for both eigen-face and fisher-face approaches.

Improving the performance of facial expression recognition using dynamic, subtle and regional features

Human facial expression is a complex process characterized of dynamic, subtle and regional emotional features. State-of-the-art approaches on facial expression recognition (FER) have not fully utilized this kind of features to improve the recognition performance. This paper proposes an approach to overcome this limitation using patch-based ‘salient’ Gabor features. A set of 3D patches are extracted to represent the subtle and regional features, and then inputted into patch matching operations for capturing the dynamic features. Experimental results show a significant performance improvement of the proposed approach due to the use of the dynamic features. Performance comparison with pervious work also confirms that the proposed approach achieves the highest CRR reported to date on the JAFFE database and a top-level performance on the Cohn-Kanade (CK) database.

Influence of internal fixator stiffness on murine fracture healing : two types of fracture healing lead to two distinct cellular events and FGF-2 expressions

This study aimed to clarify the relationship between the mechanical environment at the fracture site and endogenous fibroblast growth factor-2 (FGF-2). We compared two types of fracture healing with different callus formations and cellular events using MouseFix(TM) plate fixation systems for murine fracture models. Left femoral fractures were induced in 72 ten-week-old mice and then fixed with a flexible (Group F) or rigid (Group R) Mouse Fix(TM) plate. Mice were sacrificed on days 3, 5, 7, 10, 14, and 21. The callus volumes were measured by 3D micro-CT and tissues were histologically stained with hematoxylin & eosin or safranin-O. Sections from days 3, 5, and 7 were immunostained for FGF-2 and Proliferating Cell Nuclear Antigen (PCNA). The callus in Group F was significantly larger than that in Group R. The rigid plate allowed bone union without a marked external callus or chondrogenesis. The flexible plate formed a large external callus as a result of endochondral ossification. Fibroblastic cells in the granulation tissue on days 5 and 7 in Group F showed marked FGF-2 expression compared with Group R. Fibroblastic cells showed ongoing proliferation in granulation tissue in group F, as indicated by PCNA expression, which explained the relative granulation tissue increase in group F. There were major differences in early phase endogenous FGF-2 expression between these two fracture healing processes, due to different mechanical environments.

Data flow analysis of embedded program expressions

Data flow analysis techniques can be used to help assess threats to data confidentiality and integrity in security critical program code. However, a fundamental weakness of static analysis techniques is that they overestimate the ways in which data may propagate at run time. Discounting large numbers of these false-positive data flow paths wastes an information security evaluator's time and effort. Here we show how to automatically eliminate some false-positive data flow paths by precisely modelling how classified data is blocked by certain expressions in embedded C code. We present a library of detailed data flow models of individual expression elements and an algorithm for introducing these components into conventional data flow graphs. The resulting models can be used to accurately trace byte-level or even bit-level data flow through expressions that are normally treated as atomic. This allows us to identify expressions that safely downgrade their classified inputs and thereby eliminate false-positive data flow paths from the security evaluation process. To validate the approach we have implemented and tested it in an existing data flow analysis toolkit.

Discovering the best feature extraction and selection algorithms for spontaneous facial expression recognition

Feature extraction and selection are critical processes in developing facial expression recognition (FER) systems. While many algorithms have been proposed for these processes, direct comparison between texture, geometry and their fusion, as well as between multiple selection algorithms has not been found for spontaneous FER. This paper addresses this issue by proposing a unified framework for a comparative study on the widely used texture (LBP, Gabor and SIFT) and geometric (FAP) features, using Adaboost, mRMR and SVM feature selection algorithms. Our experiments on the Feedtum and NVIE databases demonstrate the benefits of fusing geometric and texture features, where SIFT+FAP shows the best performance, while mRMR outperforms Adaboost and SVM. In terms of computational time, LBP and Gabor perform better than SIFT. The optimal combination of SIFT+FAP+mRMR also exhibits a state-of-the-art performance.

On the application of the probabilistic linear discriminant analysis to face recognition across expression

Facial expression is one of the main issues of face recognition in uncontrolled environments. In this paper, we apply the probabilistic linear discriminant analysis (PLDA) method to recognize faces across expressions. Several PLDA approaches are tested and cross-evaluated on the Cohn-Kanade and JAFFE databases. With less samples per gallery subject, high recognition rates comparable to previous works have been achieved indicating the robustness of the approaches. Among the approaches, the mixture of PLDAs has demonstrated better performances. The experimental results also indicate that facial regions around the cheeks, eyes, and eyebrows are more discriminative than regions around the mouth, jaw, chin, and nose.

Improved facial expression recognition via uni-hyperplane classification

Large margin learning approaches, such as support vector machines (SVM), have been successfully applied to numerous classification tasks, especially for automatic facial expression recognition. The risk of such approaches however, is their sensitivity to large margin losses due to the influence from noisy training examples and outliers which is a common problem in the area of affective computing (i.e., manual coding at the frame level is tedious so coarse labels are normally assigned). In this paper, we leverage the relaxation of the parallel-hyperplanes constraint and propose the use of modified correlation filters (MCF). The MCF is similar in spirit to SVMs and correlation filters, but with the key difference of optimizing only a single hyperplane. We demonstrate the superiority of MCF over current techniques on a battery of experiments.

Robust facial feature extraction and matching

Automated feature extraction and correspondence determination is an extremely important problem in the face recognition community as it often forms the foundation of the normalisation and database construction phases of many recognition and verification systems. This paper presents a completely automatic feature extraction system based upon a modified volume descriptor. These features form a stable descriptor for faces and are utilised in a reversible jump Markov chain Monte Carlo correspondence algorithm to automatically determine correspondences which exist between faces. The developed system is invariant to changes in pose and occlusion and results indicate that it is also robust to minor face deformations which may be present with variations in expression.

Gradual training of cascaded shape regression for facial landmark localization and pose estimation

Facial landmarks play an important role in face recognition. They serve different steps of the recognition such as pose estimation, face alignment, and local feature extraction. Recently, cascaded shape regression has been proposed to accurately locate facial landmarks. A large number of weak regressors are cascaded in a sequence to fit face shapes to the correct landmark locations. In this paper, we propose to improve the method by applying gradual training. With this training, the regressors are not directly aimed to the true locations. The sequence instead is divided into successive parts each of which is aimed to intermediate targets between the initial and the true locations. We also investigate the incorporation of pose information in the cascaded model. The aim is to find out whether the model can be directly used to estimate head pose. Experiments on the Annotated Facial Landmarks in the Wild database have shown that the proposed method is able to improve the localization and give accurate estimates of pose.

Random Gabor based templates for facial expression recognition in images with facial occlusion

Robust facial expression recognition (FER) under occluded face conditions is challenging. It requires robust algorithms of feature extraction and investigations into the effects of different types of occlusion on the recognition performance to gain insight. Previous FER studies in this area have been limited. They have spanned recovery strategies for loss of local texture information and testing limited to only a few types of occlusion and predominantly a matched train-test strategy. This paper proposes a robust approach that employs a Monte Carlo algorithm to extract a set of Gabor based part-face templates from gallery images and converts these templates into template match distance features. The resulting feature vectors are robust to occlusion because occluded parts are covered by some but not all of the random templates. The method is evaluated using facial images with occluded regions around the eyes and the mouth, randomly placed occlusion patches of different sizes, and near-realistic occlusion of eyes with clear and solid glasses. Both matched and mis-matched train and test strategies are adopted to analyze the effects of such occlusion. Overall recognition performance and the performance for each facial expression are investigated. Experimental results on the Cohn-Kanade and JAFFE databases demonstrate the high robustness and fast processing speed of our approach, and provide useful insight into the effects of occlusion on FER. The results on the parameter sensitivity demonstrate a certain level of robustness of the approach to changes in the orientation and scale of Gabor filters, the size of templates, and occlusions ratios. Performance comparisons with previous approaches show that the proposed method is more robust to occlusion with lower reductions in accuracy from occlusion of eyes or mouth.

Automatic inference of driving task demand from visual cues of emotion and attention

Sensing the mental, physical and emotional demand of a driving task is of primary importance in road safety research and for effectively designing in-vehicle information systems (IVIS). Particularly, the need of cars capable of sensing and reacting to the emotional state of the driver has been repeatedly advocated in the literature. Algorithms and sensors to identify patterns of human behavior, such as gestures, speech, eye gaze and facial expression, are becoming available by using low cost hardware: This paper presents a new system which uses surrogate measures such as facial expression (emotion) and head pose and movements (intention) to infer task difficulty in a driving situation. 11 drivers were recruited and observed in a simulated driving task that involved several pre-programmed events aimed at eliciting emotive reactions, such as being stuck behind slower vehicles, intersections and roundabouts, and potentially dangerous situations. The resulting system, combining face expressions and head pose classification, is capable of recognizing dangerous events (such as crashes and near misses) and stressful situations (e.g. intersections and way giving) that occur during the simulated drive.

The roles of JK-1 (FAM134B) expressions in colorectal cancer

The aims of the present study are to investigate the clinicopathological correlations of JK-1(FAM134B) expression and its relationship to carcinogenesis in a colorectal adenoma-adenocarcinoma model. JK-1(FAM134B) protein expression was studied in a colon cancer cell line by Western blot and immunocytochemistry. JK-1(FAM134B) expression profiles at mRNA and protein levels were investigated in cancer tissues from 236 patients with colorectal adenocarcinoma and 32 patients with colorectal adenoma using real-time polymerase chain reaction and immunohistochemistry. The findings were then correlated with the clinicopathological features of these tumours. JK-1(FAM134B) protein was demonstrated in the colon cancer cells by Western blot. The protein was located in the nuclei of the tumour cells at both cellular and tissue levels. In colorectal adenocarcinomas, lower levels of JK-1(FAM134B) protein expression were associated with younger age (p=0.032), larger tumour size (p=0.004), advanced cancer stages (p=0.016) and higher rates of cancer recurrence (p=0.04). Also, lower levels of JK-1(FAM134B) mRNA expression were associated with advanced cancer stages (p=0.02) and presence of lymphovascular invasion (p=0.014). Higher JK-1(FAM134B) mRNA and protein expression levels were identified in adenomas and non-neoplastic mucosae, compared to carcinomas (p=0.005). To conclude, JK-1(FAM134B) mRNA expression and JK1 (FAM134B) protein levels varied with the different stages of progression of colorectal tumours. The expression levels of the gene were associated with clinicopathological features in patients with colorectal adenocarcinoma suggesting that JK-1(FAM134B) gene has roles in controlling some steps in the development of the invasive phenotypes from colorectal adenoma to early staged as well as advanced staged colorectal adenocarcinomas.

Predicting movie ratings from audience behaviors

We propose a method of representing audience behavior through facial and body motions from a single video stream, and use these features to predict the rating for feature-length movies. This is a very challenging problem as: i) the movie viewing environment is dark and contains views of people at different scales and viewpoints; ii) the duration of feature-length movies is long (80-120 mins) so tracking people uninterrupted for this length of time is still an unsolved problem, and; iii) expressions and motions of audience members are subtle, short and sparse making labeling of activities unreliable. To circumvent these issues, we use an infrared illuminated test-bed to obtain a visually uniform input. We then utilize motion-history features which capture the subtle movements of a person within a pre-defined volume, and then form a group representation of the audience by a histogram of pair-wise correlations over a small-window of time. Using this group representation, we learn our movie rating classifier from crowd-sourced ratings collected by rottentomatoes.com and show our prediction capability on audiences from 30 movies across 250 subjects (> 50 hrs).