Toward a more robust facial expression recognition in occluded images using randomly sampled Gabor based templates

Occlusion is a big challenge for facial expression recognition (FER) in real-world situations. Previous FER efforts to address occlusion suffer from loss of appearance features and are largely limited to a few occlusion types and single testing strategy. This paper presents a robust approach for FER in occluded images and addresses these issues. A set of Gabor based templates is extracted from images in the gallery using a Monte Carlo algorithm. These templates are converted into distance features using template matching. The resulting feature vectors are robust to occlusion. Occluded eyes and mouth regions and randomly places occlusion patches are used for testing. Two testing strategies analyze the effects of these occlusions on the overall recognition performance as well as each facial expression. Experimental results on the Cohn-Kanade database confirm the high robustness of our approach and provide useful insights about the effects of occlusion on FER. Performance is also compared with previous approaches.

Automatically detecting pain in video through facial action units

In a clinical setting, pain is reported either through patient self-report or via an observer. Such measures are problematic as they are: 1) subjective, and 2) give no specific timing information. Coding pain as a series of facial action units (AUs) can avoid these issues as it can be used to gain an objective measure of pain on a frame-by-frame basis. Using video data from patients with shoulder injuries, in this paper, we describe an active appearance model (AAM)-based system that can automatically detect the frames in video in which a patient is in pain. This pain data set highlights the many challenges associated with spontaneous emotion detection, particularly that of expression and head movement due to the patient's reaction to pain. In this paper, we show that the AAM can deal with these movements and can achieve significant improvements in both the AU and pain detection performance compared to the current-state-of-the-art approaches which utilize similarity-normalized appearance features only.

Facial expression recognition using facial movement features

Facial expression is an important channel for human communication and can be applied in many real applications. One critical step for facial expression recognition (FER) is to accurately extract emotional features. Current approaches on FER in static images have not fully considered and utilized the features of facial element and muscle movements, which represent static and dynamic, as well as geometric and appearance characteristics of facial expressions. This paper proposes an approach to solve this limitation using ‘salient’ distance features, which are obtained by extracting patch-based 3D Gabor features, selecting the ‘salient’ patches, and performing patch matching operations. The experimental results demonstrate high correct recognition rate (CRR), significant performance improvements due to the consideration of facial element and muscle movements, promising results under face registration errors, and fast processing time. The comparison with the state-of-the-art performance confirms that the proposed approach achieves the highest CRR on the JAFFE database and is among the top performers on the Cohn-Kanade (CK) database.

Improving the performance of facial expression recognition using dynamic, subtle and regional features

Human facial expression is a complex process characterized of dynamic, subtle and regional emotional features. State-of-the-art approaches on facial expression recognition (FER) have not fully utilized this kind of features to improve the recognition performance. This paper proposes an approach to overcome this limitation using patch-based ‘salient’ Gabor features. A set of 3D patches are extracted to represent the subtle and regional features, and then inputted into patch matching operations for capturing the dynamic features. Experimental results show a significant performance improvement of the proposed approach due to the use of the dynamic features. Performance comparison with pervious work also confirms that the proposed approach achieves the highest CRR reported to date on the JAFFE database and a top-level performance on the Cohn-Kanade (CK) database.

Least-squares congealing for large numbers of images

In this paper we pursue the task of aligning an ensemble of images in an unsupervised manner. This task has been commonly referred to as “congealing” in literature. A form of congealing, using a least-squares criteria, has been recently demonstrated to have desirable properties over conventional congealing. Least-squares congealing can be viewed as an extension of the Lucas & Kanade (LK)image alignment algorithm. It is well understood that the alignment performance for the LK algorithm, when aligning a single image with another, is theoretically and empirically equivalent for additive and compositional warps. In this paper we: (i) demonstrate that this equivalence does not hold for the extended case of congealing, (ii) characterize the inherent drawbacks associated with least-squares congealing when dealing with large numbers of images, and (iii) propose a novel method for circumventing these limitations through the application of an inverse-compositional strategy that maintains the attractive properties of the original method while being able to handle very large numbers of images.

On the application of the probabilistic linear discriminant analysis to face recognition across expression

Facial expression is one of the main issues of face recognition in uncontrolled environments. In this paper, we apply the probabilistic linear discriminant analysis (PLDA) method to recognize faces across expressions. Several PLDA approaches are tested and cross-evaluated on the Cohn-Kanade and JAFFE databases. With less samples per gallery subject, high recognition rates comparable to previous works have been achieved indicating the robustness of the approaches. Among the approaches, the mixture of PLDAs has demonstrated better performances. The experimental results also indicate that facial regions around the cheeks, eyes, and eyebrows are more discriminative than regions around the mouth, jaw, chin, and nose.

In the pursuit of effective affective computing : the relationship between features and registration

For facial expression recognition systems to be applicable in the real world, they need to be able to detect and track a previously unseen person's face and its facial movements accurately in realistic environments. A highly plausible solution involves performing a "dense" form of alignment, where 60-70 fiducial facial points are tracked with high accuracy. The problem is that, in practice, this type of dense alignment had so far been impossible to achieve in a generic sense, mainly due to poor reliability and robustness. Instead, many expression detection methods have opted for a "coarse" form of face alignment, followed by an application of a biologically inspired appearance descriptor such as the histogram of oriented gradients or Gabor magnitudes. Encouragingly, recent advances to a number of dense alignment algorithms have demonstrated both high reliability and accuracy for unseen subjects [e.g., constrained local models (CLMs)]. This begs the question: Aside from countering against illumination variation, what do these appearance descriptors do that standard pixel representations do not? In this paper, we show that, when close to perfect alignment is obtained, there is no real benefit in employing these different appearance-based representations (under consistent illumination conditions). In fact, when misalignment does occur, we show that these appearance descriptors do work well by encoding robustness to alignment error. For this work, we compared two popular methods for dense alignment-subject-dependent active appearance models versus subject-independent CLMs-on the task of action-unit detection. These comparisons were conducted through a battery of experiments across various publicly available data sets (i.e., CK+, Pain, M3, and GEMEP-FERA). We also report our performance in the recent 2011 Facial Expression Recognition and Analysis Challenge for the subject-independent task.

The artful life story : the oral history interview as fiction

This practice-led PhD project consists of two parts. The first is an exegesis documenting how a fiction writer can enter a dialogue with the oral history project in Australia. I identify two philosophical mandates of the oral history project in Australia that have shaped my creative practice: an emphasis on the analysis of the interviewee’s subjective experience as a means of understanding the past, and the desire to engage a wide audience in order to promote empathy towards the subject. The discussion around fiction in the oral history project is in its infancy. In order to deepen the debate, I draw on the more mature discussion in ethnographic fiction. I rely on literary theorists Steven Greenblatt, Dorrit Cohn and Gerard Genette to develop a clear understanding of the distinct narrative qualities of fiction, in order to explore how fiction can re-present and explore an interviewee’s subjective experience, and engage a wide readership. I document my own methodology for producing a work of fiction that is enriched by oral history methodology and theory, and responds to the mandates of the project. I demonstrate the means by which fiction and the oral history project can enter a dialogue in the truest sense of the word: a two-way conversation that enriches and augments practice in both fields. The second part of the PhD is a novel, set in Brisbane and based on oral history interviews and archival material I gathered over the course of the project. The novel centres on Brisbane artist Evelyn, who has been given an impossible task: a derelict old house is about to be demolished, and she must capture its history in a sculpture that will be built on the site. Evelyn struggles to come up with ideas and create the sculpture, realising that she has no way to discover who inhabited the house. What follows is a series of stories, each set in a different era in Brisbane’s history, which take the reader backwards through the house’s history. Hidden Objects is a novel about the impossibility of grasping the past and the powerful pull of storytelling. The novel is an experiment in a hybrid form and is accompanied by an appendix that identifies the historically accurate sources informing the fiction. The decisions about the aesthetics of the novel were a direct result of my engagement with the mandates of the oral history project in Australia. The novel was shortlisted in the 2012 Queensland Literary Awards, unpublished manuscript category.

Anatomic fitting of total artificial hearts for in vivo evaluation

Successful anatomic fitting of a total artificial heart (TAH) is vital to achieve optimal pump hemodynamics after device implantation. Although many anatomic fitting studies have been completed in humans prior to clinical trials, few reports exist that detail the experience in animals for in vivo device evaluation. Optimal hemodynamics are crucial throughout the in vivo phase to direct design iterations and ultimately validate device performance prior to pivotal human trials. In vivo evaluation in a sheep model allows a realistically sized representation of a smaller patient, for which smaller third-generation TAHs have the potential to treat. Our study aimed to assess the anatomic fit of a single device rotary TAH in sheep prior to animal trials and to use the data to develop a threedimensional, computer-aided design (CAD)-operated anatomic fitting tool for future TAH development. Following excision of the native ventricles above the atrio-ventricular groove, a prototype TAH was inserted within the chest cavity of six sheep (28–40 kg).Adjustable rods representing inlet and outlet conduits were oriented toward the center of each atrial chamber and the great vessels, with conduit lengths and angles recorded for future analysis. A threedimensional, CAD-operated anatomic fitting tool was then developed, based on the results of this study, and used to determine the inflow and outflow conduit orientation of the TAH. The mean diameters of the sheep left atrium, right atrium, aorta, and pulmonary artery were 39, 33, 12, and 11 mm, respectively. The center-to-center distance and outer-edge-to-outer-edge distance between the atria, found to be 39 ± 9 mm and 72 ± 17 mm in this study, were identified as the most critical geometries for successful TAH connection. This geometric constraint restricts the maximum separation allowable between left and right inlet ports of a TAH to ensure successful alignment within the available atrial circumference.

Unsupervised temporal ensemble alignment for rapid annotation

This paper presents a novel framework for the unsupervised alignment of an ensemble of temporal sequences. This approach draws inspiration from the axiom that an ensemble of temporal signals stemming from the same source/class should have lower rank when "aligned" rather than "misaligned". Our approach shares similarities with recent state of the art methods for unsupervised images ensemble alignment (e.g. RASL) which breaks the problem into a set of image alignment problems (which have well known solutions i.e. the Lucas-Kanade algorithm). Similarly, we propose a strategy for decomposing the problem of temporal ensemble alignment into a similar set of independent sequence problems which we claim can be solved reliably through Dynamic Time Warping (DTW). We demonstrate the utility of our method using the Cohn-Kanade+ dataset, to align expression onset across multiple sequences, which allows us to automate the rapid discovery of event annotations.

Random Gabor based templates for facial expression recognition in images with facial occlusion

Robust facial expression recognition (FER) under occluded face conditions is challenging. It requires robust algorithms of feature extraction and investigations into the effects of different types of occlusion on the recognition performance to gain insight. Previous FER studies in this area have been limited. They have spanned recovery strategies for loss of local texture information and testing limited to only a few types of occlusion and predominantly a matched train-test strategy. This paper proposes a robust approach that employs a Monte Carlo algorithm to extract a set of Gabor based part-face templates from gallery images and converts these templates into template match distance features. The resulting feature vectors are robust to occlusion because occluded parts are covered by some but not all of the random templates. The method is evaluated using facial images with occluded regions around the eyes and the mouth, randomly placed occlusion patches of different sizes, and near-realistic occlusion of eyes with clear and solid glasses. Both matched and mis-matched train and test strategies are adopted to analyze the effects of such occlusion. Overall recognition performance and the performance for each facial expression are investigated. Experimental results on the Cohn-Kanade and JAFFE databases demonstrate the high robustness and fast processing speed of our approach, and provide useful insight into the effects of occlusion on FER. The results on the parameter sensitivity demonstrate a certain level of robustness of the approach to changes in the orientation and scale of Gabor filters, the size of templates, and occlusions ratios. Performance comparisons with previous approaches show that the proposed method is more robust to occlusion with lower reductions in accuracy from occlusion of eyes or mouth.

International hermelin brain tumor symposium on matricellular proteins in normal and cancer cell-matrix interactions

The second of the Hermelin Brain Tumor Center Symposia was held once again at Henry Ford Hospital in Detroit, Michigan on October 24th and 25th, 2003. A public conference was held on the 24th while a closed-door session took place on the 25th. The purpose of these symposia is to bring together experts in a particular field of study with the aim to share information with each other and the public, but then to meet privately to present novel data, hold discussions, and share concepts. While the interaction is intended to benefit all involved, the incentive is the expectation that the shared information will aid researchers at the Hermelin Brain Tumor Center in their quest to identify potential therapeutic targets and explore translational therapeutic strategies for the treatment of patients suffering nervous system tumors...

Parent feeding interactions and practices during childhood cancer treatment: A qualitative investigation

In the general population it is evident that parent feeding practices can directly shape a child’s life long dietary intake. Young children undergoing childhood cancer treatment may experience feeding difficulties and limited food intake, due to the inherent side effects of their anti-cancer treatment. What is not clear is how these treatment side effects are influencing the parent–child feeding relationship during anti-cancer treatment. This retrospective qualitative study collected telephone based interview data from 38 parents of childhood cancer patients who had recently completed cancer treatment (child’s mean age: 6.98 years). Parents described a range of treatment side effects that impacted on their child’s ability to eat, often resulting in weight loss. Sixty-one percent of parents (n = 23) reported high levels of stress in regard to their child’s eating and weight loss during treatment. Parents reported stress, feelings of helplessness, and conflict and/or tension between parent and the child during feeding/eating interactions. Parents described using both positive and negative feeding practices, such as: pressuring their child to eat, threatening the insertion of a nasogastric feeding tube, encouraging the child to eat and providing home cooked meals in hospital. Results indicated that parent stress may lead to the use of coping strategies such as positive or negative feeding practices to entice their child to eat during cancer treatment. Future research is recommended to determine the implication of parent feeding practice on the long term diet quality and food preferences of childhood cancer survivors.