Towards robust automatic affective classification of images using facial expressions for practical applications

Affect is an important feature of multimedia content and conveys valuable information for multimedia indexing and retrieval. Most existing studies for affective content analysis are limited to low-level features or mid-level representations, and are generally criticized for their incapacity to address the gap between low-level features and high-level human affective perception. The facial expressions of subjects in images carry important semantic information that can substantially influence human affective perception, but have been seldom investigated for affective classification of facial images towards practical applications. This paper presents an automatic image emotion detector (IED) for affective classification of practical (or non-laboratory) data using facial expressions, where a lot of “real-world” challenges are present, including pose, illumination, and size variations etc. The proposed method is novel, with its framework designed specifically to overcome these challenges using multi-view versions of face and fiducial point detectors, and a combination of point-based texture and geometry. Performance comparisons of several key parameters of relevant algorithms are conducted to explore the optimum parameters for high accuracy and fast computation speed. A comprehensive set of experiments with existing and new datasets, shows that the method is effective despite pose variations, fast, and appropriate for large-scale data, and as accurate as the method with state-of-the-art performance on laboratory-based data. The proposed method was also applied to affective classification of images from the British Broadcast Corporation (BBC) in a task typical for a practical application providing some valuable insights.

Evolution of self-reporting methods for identifying discrete emotions in science classrooms

Emotion researchers have grappled with challenging methodological issues in capturing emotions of participants in naturalistic settings such as school or university classrooms. Self-reporting methods have been used frequently, yet these methods are inadequate when used alone. We argue that the self-reporting methods of emotion diaries and cogenerative dialogues can be helpful in identifying in-the-moment emotions when used in conjunction with the microanalysis of video recordings of classroom events. We trace the evolution of our use of innovative self-reporting methods through three cases from our research projects, and propose new directions for our ongoing development and application of these methods in both school and university classrooms.

A genome-wide association study identifies five loci influencing facial morphology in Europeans

Inter-individual variation in facial shape is one of the most noticeable phenotypes in humans, and it is clearly under genetic regulation; however, almost nothing is known about the genetic basis of normal human facial morphology. We therefore conducted a genome-wide association study for facial shape phenotypes in multiple discovery and replication cohorts, considering almost ten thousand individuals of European descent from several countries. Phenotyping of facial shape features was based on landmark data obtained from three-dimensional head magnetic resonance images (MRIs) and two-dimensional portrait images. We identified five independent genetic loci associated with different facial phenotypes, suggesting the involvement of five candidate genes-PRDM16, PAX3, TP63, C5orf50, and COL17A1-in the determination of the human face. Three of them have been implicated previously in vertebrate craniofacial development and disease, and the remaining two genes potentially represent novel players in the molecular networks governing facial development. Our finding at PAX3 influencing the position of the nasion replicates a recent GWAS of facial features. In addition to the reported GWA findings, we established links between common DNA variants previously associated with NSCL/P at 2p21, 8q24, 13q31, and 17q22 and normal facial-shape variations based on a candidate gene approach. Overall our study implies that DNA variants in genes essential for craniofacial development contribute with relatively small effect size to the spectrum of normal variation in human facial morphology. This observation has important consequences for future studies aiming to identify more genes involved in the human facial morphology, as well as for potential applications of DNA prediction of facial shape such as in future forensic applications.

The role of emotions in service customer decision-making

This chapter explains how customers make purchase decisions and how these decisions are influenced not only by the service marketer but also by the customers own emotions. While decision-making is described from the perspective of purchasing services rather than purchasing goods, we challenge the traditional notion that customers make informed, rational and well-thought-out decisions. Rather customers are often driven by subjective feelings such as emotions. We present evidence of how these emotions influence the behavior of customers, their attitudes and evaluation of the service, as well as final decision-making processes.

Electronic Control of Facial Selection in Additions to Sterically Unbiased Ketones and Olefins

By definition, the two faces of a pi bond are equivalent.1 However, they are rendered nonequivalent in most molecules because of the absence of a plane of symmetry encompassing the double bond and the adjacent substituents. As a result, additions to trigonal centers from the two faces need not be equally facile. Exploiting this stereodifferentiation in a controlled manner represents one of the core problems in organic synthesis. Evidently, the factors which determine such diastereoselection need to be delineated in as much detail as possible.

A flexible hierarchical approach for facial age estimation based on multiple features

Age estimation from facial images is increasingly receiving attention to solve age-based access control, age-adaptive targeted marketing, amongst other applications. Since even humans can be induced in error due to the complex biological processes involved, finding a robust method remains a research challenge today. In this paper, we propose a new framework for the integration of Active Appearance Models (AAM), Local Binary Patterns (LBP), Gabor wavelets (GW) and Local Phase Quantization (LPQ) in order to obtain a highly discriminative feature representation which is able to model shape, appearance, wrinkles and skin spots. In addition, this paper proposes a novel flexible hierarchical age estimation approach consisting of a multi-class Support Vector Machine (SVM) to classify a subject into an age group followed by a Support Vector Regression (SVR) to estimate a specific age. The errors that may happen in the classification step, caused by the hard boundaries between age classes, are compensated in the specific age estimation by a flexible overlapping of the age ranges. The performance of the proposed approach was evaluated on FG-NET Aging and MORPH Album 2 datasets and a mean absolute error (MAE) of 4.50 and 5.86 years was achieved respectively. The robustness of the proposed approach was also evaluated on a merge of both datasets and a MAE of 5.20 years was achieved. Furthermore, we have also compared the age estimation made by humans with the proposed approach and it has shown that the machine outperforms humans. The proposed approach is competitive with current state-of-the-art and it provides an additional robustness to blur, lighting and expression variance brought about by the local phase features.

Microneurovascular free muscle transfer with cross-over nerve grafts in facial reanimation

Microneurovascular free muscle transfer with cross-over nerve grafts in facial reanimation Loss of facial symmetry and mimetic function as seen in facial paralysis has an enormous impact on the psychosocial conditions of the patients. Patients with severe long-term facial paralysis are often reanimated with a two-stage procedure combining cross-facial nerve grafting, and 6 to 8 months later with microneurovascular (MNV) muscle transfer. In this thesis, we recorded the long-term results of MNV surgery in facial paralysis and observed the possible contributing factors to final functional and aesthetic outcome after this procedure. Twenty-seven out of forty patients operated on were interviewed, and the functional outcome was graded. Magnetic resonance imaging (MRI) of MNV muscle flaps was done, and nerve graft samples (n=37) were obtained in second stage of the operation and muscle biopsies (n=18) were taken during secondary operations.. The structure of MNV muscles and nerve grafts was evaluated using histological and immunohistochemical methods ( Ki-67, anti-myosin fast, S-100, NF-200, CD-31, p75NGFR, VEGF, Flt-1, Flk-1). Statistical analysis was performed. In our studies, we found that almost two-thirds of the patients achieved good result in facial reanimation. The longer the follow-up time after muscle transfer the weaker was the muscle function. A majority of the patients (78%) defined their quality of life improved after surgery. In MRI study, the free MNV flaps were significantly smaller than originally. A correlation was found between good functional outcome and normal muscle structure in MRI. In muscle biopsies, the mean muscle fiber diameter was diminished to 40% compared to control values. Proliferative activity of satellite cells was seen in 60% of the samples and it tended to decline with an increase of follow-up time. All samples showed intramuscular innervation. Severe muscle atrophy correlated with prolonged intraoperative ischaemia. The good long-term functional outcome correlated with dominance of fast fibers in muscle grafts. In nerve grafts, the mean number of viable axons amounted to 38% of that in control samples. The grafted nerves characterized by fibrosis and regenerated axons were thinner than in control samples although they were well vascularized. A longer time between cross facial nerve grafting and biopsy sampling correlated with a higher number of viable axons. P75Nerve Growth Factor Receptor (p75NGFR) was expressed in every nerve graft sample. The expression of p75NGFR was lower in older than in younger patients. A high expression of p75NGFR was often seen with better function of the transplanted muscle. In grafted nerve Vascular Endothelial Growth Factor (VEGF) and its receptors were expressed in nervous tissue. In conclusion, most of the patients achieved good result in facial reanimation and were satisfied with the functional outcome. The mimic function was poorer in patients with longer follow-up time. MRI can be used to evaluate the structure of the microneurovascular muscle flaps. Regeneration of the muscle flaps was still going on many years after the transplantation and reinnervation was seen in all muscle samples. Grafted nerves were characterized by fibrosis and fewer, thinner axons compared to control nerves although they were well vascularized. P75NGFR and VEGF were expressed in human nerve grafts with higher intensity than in control nerves which is described for the first time.

Decision-making in a death investigation: Emotions, families and the coroner

The role of the coroner in common law countries such as Australia, England, Canada and New Zealand is to preside over death investigations where there is uncertainty as to the manner of death, a need to identify the deceased, a death of unknown cause, or a violent or unnatural death. The vast majority of these deaths are not suspicious and thus require coroners to engage with grieving families who have been thrust into a legal process through the misfortune of a loved one's sudden or unexpected death. In this research, 10 experienced coroners discussed how they negotiated the grief and trauma evident in a death investigation. In doing so, they articulated two distinct ways in which legal officers engaged with emotions, which are also evident in the literature. The first engages the script of judicial dispassion, articulating a hierarchical relationship between reason and emotion, while the second introduces an ethic of care via the principles of therapeutic jurisprudence, and thus offers a challenge to the role of emotion in the personae of the professional judicial officer. By using Hochschild's work on the sociology of emotions, this article discusses the various ways in which coroners manage the emotion of a death investigation through emotion work. While emotional distance may be an understandable response by coroners to the grief and trauma experienced by families and directed at cleaner coronial decision-making, the article concludes that coroners may be better served by offering emotions such as sympathy, consideration and compassion directly to the family in those situations where families are struggling to accept, or are resistant to, coroners' decisions.

Peripheral facial palsy : Grading, Etiology, and Melkersson-Rosenthal Syndrome

The objective of this study was to assess the utility of two subjective facial grading systems, to evaluate the etiologic role of human herpesviruses in peripheral facial palsy (FP), and to explore characteristics of Melkersson-Rosenthal syndrome (MRS). Intrarater repeatability and interrater agreement were assessed for Sunnybrook (SFGS) and House-Brackmann facial grading systems (H-B FGS). Eight video-recorded FP patients were graded in two sittings by 26 doctors. Repeatability for SFGS was from good to excellent and agreement between doctors from moderate to excellent by intraclass correlation coefficient and coefficient of repeatability. For H-B FGS, repeatability was from fair to good and agreement from poor to fair by agreement percentage and kappa coefficients. Because SFGS was at least as good in repeatability as H-B FGS and showed more reliable results in agreement between doctors, we encourage the use of SFGS over H-B FGS. Etiologic role of human herpesviruses in peripheral FP was studied by searching DNA of herpes simplex virus (HSV) -1 and -2, varicella-zoster virus (VZV), human herpesvirus (HHV) -6A, -6B, and -7, Epstein-Barr virus (EBV), and cytomegalovirus (CMV) by PCR/microarray methods in cerebrospinal fluid (CSF) of 33 peripheral FP patients and 36 controls. Three patients and five controls had HHV-6 or -7 DNA in CSF. No DNA of HSV-1 or -2, VZV, EBV, or CMV was found. Detecting HHV-7 and dual HHV-6A and -6B DNA in CSF of FP patients is intriguing, but does not allow etiologic conclusions as such. These DNA findings in association with FP and the other diseases that they accompanied require further exploration. MRS is classically defined as a triad of recurrent labial or oro-facial edema, recurrent peripheral FP, and plicated tongue. All three signs are present in the minority of patients. Edema-dominated forms are more common in the literature, while MRS with FP has received little attention. The etiology and true incidence of MRS are unknown. Characteristics of MRS were evaluated at the Departments of Otorhinolaryngology and Dermatology focusing on patients with FP. There were 35 MRS patients, 20 with FP and they were mailed a questionnaire (17 answered) and were clinically examined (14 patients). At the Department of Otorhinolaryngology, every MRS patient had FP and half had the triad form of MRS. Two patients, whose tissue biopsies were taken during an acute edema episode, revealed nonnecrotizing granulomatous findings typical for MRS, the other without persisting edema and with symptoms for less than a year. A peripheral blood DNA was searched for gene mutations leading to UNC-93B protein deficiency predisposing to HSV-1 infections; no gene mutations were found. Edema in most MRS FP patients did not dominate the clinical picture, and no progression of the disease was observed, contrary to existing knowledge. At the Department of Dermatology, two patients had triad MRS and 15 had monosymptomatic granulomatous cheilitis with frequent or persistent edema and typical MRS tissue histology. The clinical picture of MRS varied according to the department where the patient was treated. More studies from otorhinolaryngology departments and on patients with FP would clarify the actual incidence and clinical picture of the syndrome.

Mesh Processing for Computerized Facial Anthropometry

Understanding of the shape and size of different features of the human body from scanned data is necessary for automated design and evaluation of product ergonomics. In this paper, a computational framework is presented for automatic detection and recognition of important facial feature regions, from scanned head and shoulder polyhedral models. A noise tolerant methodology is proposed using discrete curvature computations, band-pass filtering, and morphological operations for isolation of the primary feature regions of the face, namely, the eyes, nose, and mouth. Spatial disposition of the critical points of these isolated feature regions is analyzed for the recognition of these critical points as the standard landmarks associated with the primary facial features. A number of clinically identified landmarks lie on the facial midline. An efficient algorithm for detection and processing of the midline, using a point sampling technique, is also presented. The results obtained using data of more than 20 subjects are verified through visualization and physical measurements. A color based and triangle skewness based schemes for isolation of geometrically nonprominent features and ear region are also presented. [DOI: 10.1115/1.3330420]

The role of polar and facial amphipathic character in determining lipopolysaccharide-binding properties in synthetic cationic peptides

Two series of peptides, designated K and NK were synthesized and tested for lipid A binding and neutralizing properties. K-2, which has an 11-residue amphiphilic core, and a branched N-terminus bearing two branched lysinyl residues does not bind lipid A, while NK2, also with an 11-residue amphiphilic core comprised entirely of non-ionizable residues, and a similarly branched, cationic N-terminus, binds lipid A very weakly. Both peptides do not inhibit lipopolysaccharide (LPS) activity in the Limulus assay, nor do they inhibit LPS-induced TNF-alpha and NO production in 5774 cells. These results are entirely unlike a homologous peptide with an exclusively hydrophobic core whose LPS-binding and neutralizing properties are very similar to that of polymyxin B [David SA, Awasthi SK, Wiese A et al. Characterization of the interactions of a polycationic, amphiphilic, terminally branched oligopeptide with lipid A and lipopolysaccharide from the deep rough mutant of Salmonella minnesota. J Endotoxin Res 1996; 3: 369-379]. These data suggest that a clear segregation of charged and apolar domains is crucial in molecules designed for purposes of LPS sequestration and that head-tail (polar) orientation of the cationic/hydrophobic regions is preferable to molecules with mixed or facial cationic/amphipathic character.

Written Cancer Narratives : An Ethnomedical Study of Cancer Patients' Thoughts, Emotions and Experiences

Since the second half of the 20th century, cancer has become a dominant disease in Western countries, endangering people regardless of age, gender, race or social status. Every year almost eight million people die of cancer worldwide. In Finland every fourth person is expected to fall ill with cancer at some stage of his or her life. During the 20th century, along with rapid changes in the medical system, people s awareness of cancer has increased a great deal. This has also influenced the image of cancer in popular discourse over the past decades. However, from the scientific point of view there is still much that is unclear about the disease. This thesis shows that this is a big problem for ordinary people, as, according to culture-bound illness ideology, people need an explanation about the origin of their illness in order to help them cope. The main aim of this thesis is to examine the process of being ill with cancer from the patient s point of view, in order to analyse attitudes and behaviour towards cancer and its significance and culture-bound images. This narrative-based study concentrates on patients voicings , which are important in understanding the cancer experience and when attempting to make it more open within current cultural and societal settings. The Kun sairastuin syöpään ( when I fell ill with cancer ) writing competition organised by Suomen Syöpäpotilaat ry (the Finnish Cancer Patients Association), Suomen Syöpäyhdistys ry (the Finnish Cancer Union), and Suomalaisen Kirjallisuuden Seuran kansanrunousarkisto (the Finnish Literary Society Folklore Archive) was announced on the 1st of May 1994 and lasted until the 30th of September 1994. As a result, a total of 672 cancer narratives, totalling 6384 pages, were received, filled with experiences relating to cancer. Written cancer narratives form a body of empirical data that is suitable for content or textual analysis. In this thesis, content analysis is adopted in order to become familiar with the texts and to preselect the themes and analytical units for further examination. I use multiple perspectives in order to interpret cancer patients ideas and reasoning. The ethnomedical approach unites popular health beliefs that originated in Finnish folk medicine, as well as connecting alternative medicine, which patients make use of, with biomedicine, the dominant form of medicine today. In addition to this, patients narratives, which are composed of various structural segments, are approached from the folklorist s perspective. In this way they can be seen as short pathographies, reconstructions of self-negotiation and individual decision making during the illness process. Above all, cancer patients writing describe their feelings, thoughts and experiences. Factors that appear insignificant to modern medicine, overwhelmed as it is by medical technologies that concentrate on dysfunctional tissue within diseased bodies. Ethnomedical study of cancer patients writings gives access to the human side of cancer discourse, and combines both medical, and popular, knowledge of cancer. In my view, the natural world and glimpses of tradition are bound together with one general aim within cancer narratives: to tackle the illness and mediate its meanings. Furthermore, the narrative approach reveals that participants write with the hope of offering a different interpretation of the cancer experience, and thus of confronting culturally pre-defined images and ideologies.

3D shape extraction of human face in presence of facial hair: A profilometric approach

The increasing use of 3D modeling of Human Face in Face Recognition systems, User Interfaces, Graphics, Gaming and the like has made it an area of active study. Majority of the 3D sensors rely on color coded light projection for 3D estimation. Such systems fail to generate any response in regions covered by Facial Hair (like beard, mustache), and hence generate holes in the model which have to be filled manually later on. We propose the use of wavelet transform based analysis to extract the 3D model of Human Faces from a sinusoidal white light fringe projected image. Our method requires only a single image as input. The method is robust to texture variations on the face due to space-frequency localization property of the wavelet transform. It can generate models to pixel level refinement as the phase is estimated for each pixel by a continuous wavelet transform. In cases of sparse Facial Hair, the shape distortions due to hairs can be filtered out, yielding an estimate for the underlying face. We use a low-pass filtering approach to estimate the face texture from the same image. We demonstrate the method on several Human Faces both with and without Facial Hairs. Unseen views of the face are generated by texture mapping on different rotations of the obtained 3D structure. To the best of our knowledge, this is the first attempt to estimate 3D for Human Faces in presence of Facial hair structures like beard and mustache without generating holes in those areas.