Automatically detecting pain in video through facial action units

In a clinical setting, pain is reported either through patient self-report or via an observer. Such measures are problematic as they are: 1) subjective, and 2) give no specific timing information. Coding pain as a series of facial action units (AUs) can avoid these issues as it can be used to gain an objective measure of pain on a frame-by-frame basis. Using video data from patients with shoulder injuries, in this paper, we describe an active appearance model (AAM)-based system that can automatically detect the frames in video in which a patient is in pain. This pain data set highlights the many challenges associated with spontaneous emotion detection, particularly that of expression and head movement due to the patient's reaction to pain. In this paper, we show that the AAM can deal with these movements and can achieve significant improvements in both the AU and pain detection performance compared to the current-state-of-the-art approaches which utilize similarity-normalized appearance features only.

FACSGen: a tool to synthesize emotional facial expressions through systematic manipulation of facial action units

To investigate the perception of emotional facial expressions, researchers rely on shared sets of photos or videos, most often generated by actor portrayals. The drawback of such standardized material is a lack of flexibility and controllability, as it does not allow the systematic parametric manipulation of specific features of facial expressions on the one hand, and of more general properties of the facial identity (age, ethnicity, gender) on the other. To remedy this problem, we developed FACSGen: a novel tool that allows the creation of realistic synthetic 3D facial stimuli, both static and dynamic, based on the Facial Action Coding System. FACSGen provides researchers with total control over facial action units, and corresponding informational cues in 3D synthetic faces. We present four studies validating both the software and the general methodology of systematically generating controlled facial expression patterns for stimulus presentation.

Evaluation of texture and geometry for dimensional facial expression recognition

Facial expression recognition (FER) algorithms mainly focus on classification into a small discrete set of emotions or representation of emotions using facial action units (AUs). Dimensional representation of emotions as continuous values in an arousal-valence space is relatively less investigated. It is not fully known whether fusion of geometric and texture features will result in better dimensional representation of spontaneous emotions. Moreover, the performance of many previously proposed approaches to dimensional representation has not been evaluated thoroughly on publicly available databases. To address these limitations, this paper presents an evaluation framework for dimensional representation of spontaneous facial expressions using texture and geometric features. SIFT, Gabor and LBP features are extracted around facial fiducial points and fused with FAP distance features. The CFS algorithm is adopted for discriminative texture feature selection. Experimental results evaluated on the publicly accessible NVIE database demonstrate that fusion of texture and geometry does not lead to a much better performance than using texture alone, but does result in a significant performance improvement over geometry alone. LBP features perform the best when fused with geometric features. Distributions of arousal and valence for different emotions obtained via the feature extraction process are compared with those obtained from subjective ground truth values assigned by viewers. Predicted valence is found to have a more similar distribution to ground truth than arousal in terms of covariance or Bhattacharya distance, but it shows a greater distance between the means.

Social signal processing for pain monitoring using a hidden conditional random field

utomatic pain monitoring has the potential to greatly improve patient diagnosis and outcomes by providing a continuous objective measure. One of the most promising methods is to do this via automatically detecting facial expressions. However, current approaches have failed due to their inability to: 1) integrate the rigid and non-rigid head motion into a single feature representation, and 2) incorporate the salient temporal patterns into the classification stage. In this paper, we tackle the first problem by developing a “histogram of facial action units” representation using Active Appearance Model (AAM) face features, and then utilize a Hidden Conditional Random Field (HCRF) to overcome the second issue. We show that both of these methods improve the performance on the task of pain detection in sequence level compared to current state-of-the-art-methods on the UNBC-McMaster Shoulder Pain Archive.

Global Sensitivity Analysis for MAP Inference in Graphical Models

We study the sensitivity of a MAP configuration of a discrete probabilistic graphical model with respect to perturbations of its parameters. These perturbations are global, in the sense that simultaneous perturbations of all the parameters (or any chosen subset of them) are allowed. Our main contribution is an exact algorithm that can check whether the MAP configuration is robust with respect to given perturbations. Its complexity is essentially the same as that of obtaining the MAP configuration itself, so it can be promptly used with minimal effort. We use our algorithm to identify the largest global perturbation that does not induce a change in the MAP configuration, and we successfully apply this robustness measure in two practical scenarios: the prediction of facial action units with posed images and the classification of multiple real public data sets. A strong correlation between the proposed robustness measure and accuracy is verified in both scenarios.

FACSGen 2.0 animation software: generating three-dimensional FACS-valid facial expressions for emotion research

In this article, we present FACSGen 2.0, new animation software for creating static and dynamic threedimensional facial expressions on the basis of the Facial Action Coding System (FACS). FACSGen permits total control over the action units (AUs), which can be animated at all levels of intensity and applied alone or in combination to an infinite number of faces. In two studies, we tested the validity of the software for the AU appearance defined in the FACS manual and the conveyed emotionality of FACSGen expressions. In Experiment 1, four FACS-certified coders evaluated the complete set of 35 single AUs and 54 AU combinations for AU presence or absence, appearance quality, intensity, and asymmetry. In Experiment 2, lay participants performed a recognition task on emotional expressions created with FACSGen software and rated the similarity of expressions displayed by human and FACSGen faces. Results showed good to excellent classification levels for all AUs by the four FACS coders, suggesting that the AUs are valid exemplars of FACS specifications. Lay participants’ recognition rates for nine emotions were high, and comparisons of human and FACSGen expressions were very similar. The findings demonstrate the effectiveness of the software in producing reliable and emotionally valid expressions, and suggest its application in numerous scientific areas, including perception, emotion, and clinical and euroscience research.

Individual group efficacy beliefs and performance motivation: A mediational approach

Introduction: According to the theoretical model of Cranach, Ochsenbein, and Valach (1986) understanding group actions needs consideration of aspects at both the group level and the level of individual members. For example individual action units constituting group actions are motivated at the individual level while potentially being affected by characteristics of the group. Theoretically, group efficacy beliefs could be a part of this motivational process as they are an individual’s cognitive contents about group-level abilities to perform well in a specific task. Positive relations between group level efficacy-beliefs and group performance have been reported and Bandura and Locke (2003) argue that this relationship is being mediated by motivational processes and goal setting. The aims of this study were a) to examine the effects of group characteristics on individual performance motivation and b) to test if those are mediated by individual group efficacy beliefs. Methods: Forty-seven students (M=22.83 years, SD=2.83, 34% women) of the university of Berne participated in this scenario based experiment. Data were collected on two collection points. Subjects were provided information about fictive team members with whom they had to perform a group triathlon. Three values (low, medium, high) of the other team members’ abilities to perform in their parts of the triathlon (swimming and biking respectively) were combined in a 3x3 full factorial design (Anderson, 1982) yielding nine groups. Subjects were asked how confident they were that the teams would perform well in the task (individual group efficacy beliefs), and to provide information about their motivation to perform at their best in the respective group contexts (performance motivation). Multilevel modeling (Mplus) was used to estimate the effects of the factors swim and bike, and the context-varying covariate individual group efficacy beliefs on performance motivation. Further analyses were undertaken to test if the effects of group contexts on performance motivation are mediated by individual group efficacy beliefs. Results: Significant effects were reported for both the group characteristics (βswim = 7.86; βbike = 8.57; both p < .001) and the individual group efficacy beliefs (βigeb; .40, p < .001) on performance motivation. The subsequent mediation model indicated that the effects of group characteristics on performance motivation were partly mediated by the individual group efficacy beliefs of the subjects with significant mediation effects for both factors swim and bike. Discussion/Conclusion: The results of the study provide further support for the motivational character of efficacy beliefs and point out a mechanism by which team characteristics influence performance relevant factors at the level of individual team members. The study indicates that high team abilities lead to augmented performance motivation, adding a psychological advantage to teams already high on task relevant abilities. Future investigations will be aiming at possibilities to keep individual performance motivation high in groups with low task relevant abilities. One possibility could be the formulation of individual task goals. References: Anderson, N. H. (1982). Methods of information integration theory. New York: Academic Press. Bandura, A. & Locke, E. A. (2003). Negative self-efficacy and goal effects revisited. Journal of Applied Psychology, 88, 87-99. Cranach, M. von, Ochsenbein, G. & Valach, L. (1986). The group as a self-active system: Outline of a theory of group action. European Journal of Social Psychology, 16, 193-229.

Desarrollo de una herramienta para analizar métodos de reconocimiento de emociones

Desde hace más de 20 años, muchos grupos de investigación trabajan en el estudio de técnicas de reconocimiento automático de expresiones faciales. En los últimos años, gracias al avance de las metodologías, ha habido numerosos avances que hacen posible una rápida detección de las caras presentes en una imagen y proporcionan algoritmos de clasificación de expresiones. En este proyecto se realiza un estudio sobre el estado del arte en reconocimiento automático de emociones, para conocer los diversos métodos que existen en el análisis facial y en el reconocimiento de la emoción. Con el fin de poder comparar estos métodos y otros futuros, se implementa una herramienta modular y ampliable y que además integra un método de extracción de características que consiste en la obtención de puntos de interés en la cara y dos métodos para clasificar la expresión, uno mediante comparación de desplazamientos de los puntos faciales, y otro mediante detección de movimientos específicos llamados unidades de acción. Para el entrenamiento del sistema y la posterior evaluación del mismo, se emplean las bases de datos Cohn-Kanade+ y JAFFE, de libre acceso a la comunidad científica. Después, una evaluación de estos métodos es llevada a cabo usando diferentes parámetros, bases de datos y variando el número de emociones. Finalmente, se extraen conclusiones del trabajo y su evaluación, proponiendo las mejoras necesarias e investigación futura. ABSTRACT. Currently, many research teams focus on the study of techniques for automatic facial expression recognition. Due to the appearance of digital image processing, in recent years there have been many advances in the field of face detection, feature extraction and expression classification. In this project, a study of the state of the art on automatic emotion recognition is performed to know the different methods existing in facial feature extraction and emotion recognition. To compare these methods, a user friendly tool is implemented. Besides, a feature extraction method is developed which consists in obtaining 19 facial feature points. Those are passed to two expression classifier methods, one based on point displacements, and one based on the recognition of facial Action Units. Cohn-Kanade+ and JAFFE databases, both freely available to the scientific community, are used for system training and evaluation. Then, an evaluation of the methods is performed with different parameters, databases and varying the number of emotions. Finally, conclusions of the work and its evaluation are extracted, proposing some necessary improvements and future research.

A framework for investigating the use of face features to identify spontaneous emotions

Emotion-based analysis has raised a lot of interest, particularly in areas such as forensics, medicine, music, psychology, and human-machine interface. Following this trend, the use of facial analysis (either automatic or human-based) is the most common subject to be investigated once this type of data can easily be collected and is well accepted in the literature as a metric for inference of emotional states. Despite this popularity, due to several constraints found in real world scenarios (e.g. lightning, complex backgrounds, facial hair and so on), automatically obtaining affective information from face accurately is a very challenging accomplishment. This work presents a framework which aims to analyse emotional experiences through naturally generated facial expressions. Our main contribution is a new 4-dimensional model to describe emotional experiences in terms of appraisal, facial expressions, mood, and subjective experiences. In addition, we present an experiment using a new protocol proposed to obtain spontaneous emotional reactions. The results have suggested that the initial emotional state described by the participants of the experiment was different from that described after the exposure to the eliciting stimulus, thus showing that the used stimuli were capable of inducing the expected emotional states in most individuals. Moreover, our results pointed out that spontaneous facial reactions to emotions are very different from those in prototypic expressions due to the lack of expressiveness in the latter.

How do the learning outcomes of students compare in units with different levels of practical work? A comparative study about the relation between Problem-­Based Learning and assessment approaches

The literature supporting the notion that active, student-centered learning is superior to passive, teacher-centered instruction is encyclopedic (Bonwell & Eison, 1991; Bruning, Schraw, & Ronning, 1999; Haile, 1997a, 1997b, 1998; Johnson, Johnson, & Smith, 1999). Previous action research demonstrated that introducing a learning activity in class improved the learning outcomes of students (Mejias, 2010). People acquire knowledge and skills through practice and reflection, not by watching and listening to others telling them how to do something. In this context, this project aims to find more insights about the level of interactivity in the curriculum a class should have and its alignment with assessment so the intended learning outcomes (ILOs) are achieved. In this project, interactivity is implemented in the form of problem- based learning (PBL). I present the argument that a more continuous formative feedback when implemented with the correct amount of PBL stimulates student engagement bringing enormous benefits to student learning. Different levels of practical work (PBL) were implemented together with two different assessment approaches in two subjects. The outcomes were measured using qualitative and quantitative data to evaluate the levels of student engagement and satisfaction in the terms of ILOs.

Action research and unit improvement in higher education : a case study approach

Student satisfaction data has been collected on a national basis in Australia since 1972. In recent years this data has been used by federal government agencies to allocate funding, and by students in selecting their universities of choice. The purpose of this paper is to present the findings of an action research project designed to identify and implement unit improvement initiatives over a three year period for an underperforming unit. This research utilises student survey data and teacher reflections to identify areas of unit improvement, with a view to aligning learning experiences, teaching and assessment to learning outcomes and improved student satisfaction. This research concludes that whilst a voluntary student survey system may be imperfect, it nevertheless provides important data that can be utilised to the benefit of the unit, learning outcomes and student satisfaction ratings, as well as wider course related outcomes. Extrapolation of these findings is recommended to other underperforming units.

Determination of management units for grey mackerel fisheries in northern Australia.

The requirement for Queensland, Northern Territory and Western Australian jurisdictions to ensure sustainable harvest of fish resources and their optimal use relies on robust information on the resource status. For grey mackerel (Scomberomorus semifasciatus) fisheries, each of these jurisdictions has their own management regime in their corresponding waters. The lack of information on stock structure of grey mackerel, however, means that the appropriate spatial scale of management is not known. As well, fishers require assurance of future sustainability to encourage investment and long-term involvement in a fishery that supplies lucrative overseas markets. These management and fisher-unfriendly circumstances must be viewed in the context of recent 3-fold increases in catches of grey mackerel along the Queensland east coast, combined with significant and increasing catches in other parts of the species' northern Australian range. Establishing the stock structure of grey mackerel would also immensely improve the relevance of resource assessments for fishery management of grey mackerel across northern Australia. This highlighted the urgent need for stock structure information for this species. The impetus for this project came from the strategic recommendations of the FRDC review by Ward and Rogers (2003), "Northern mackerel (Scombridae: Scomberomorus): current and future research needs" (Project No. 2002/096), which promoted the urgency for information on the stock structure of grey mackerel. In following these recommendations this project adopted a multi-technique and phased sampling approach as carried out by Buckworth et al (2007), who examined the stock structure of Spanish mackerel, Scomberomorus commerson, across northern Australia. The project objectives were to determine the stock structure of grey mackerel across their northern Australian range, and use this information to define management units and their appropriate spatial scales. We used multiple techniques concurrently to determine the stock structure of grey mackerel. These techniques were: genetic analyses (mitochondrial DNA and microsatellite DNA), otolith (ear bones) isotope ratios, parasite abundances, and growth parameters. The advantage of using this type of multi-technique approach was that each of the different methods is informative about the fish’s life history at different spatial and temporal scales. Genetics can inform about the evolutionary patterns as well as rates of mixing of fish from adjacent areas, while parasites and otolith microchemistry are directly influenced by the environment and so will inform about the patterns of movement during the fishes lifetime. Growth patterns are influenced by both genetic and environmental factors. Due to these differences the use of these techniques concurrently increases the likelihood of detecting different stocks where they exist. We adopted a phased sampling approach whereby sampling was carried out at broad spatial scales in the first year: east coast, eastern Gulf of Carpentaria (GoC), western GoC, and the NW Northern Territory (NW NT). By comparing the fish samples from each of these locations, and using each of the techniques, we tested the null hypothesis that grey mackerel were comprised of a single homogeneous population across northern Australia. Having rejected the null hypothesis we re-sampled the 1st year locations to test for temporal stability in stock structure, and to assess stock structure at finer spatial scales. This included increased spatial coverage on the east coast, the GoC, and WA. From genetic approaches we determined that there at least four genetic stocks of grey mackerel across northern Australia: WA, NW NT (Timor/Arafura), the GoC and the east Grey mackerel management units in northern Australia ix coast. All markers revealed concordant patterns showing WA and NW NT to be clearly divergent stocks. The mtDNA D-loop fragment appeared to have more power to resolve stock boundaries because it was able to show that the GoC and east coast QLD stocks were genetically differentiated. Patterns of stock structure on a finer scale, or where stock boundaries are located, were less clear. From otolith stable isotope analyses four major groups of S. semifasciatus were identified: WA, NT/GoC, northern east coast and central east coast. Differences in the isotopic composition of whole otoliths indicate that these groups must have spent their life history in different locations. The magnitude of the difference between the groups suggests a prolonged separation period at least equal to the fish’s life span. The parasite abundance analyses, although did not include samples from WA, suggest the existence of at least four stocks of grey mackerel in northern Australia: NW NT, the GoC, northern east coast and central east coast. Grey mackerel parasite fauna on the east coast suggests a separation somewhere between Townsville and Mackay. The NW NT region also appears to comprise a separate stock while within the GoC there exists a high degree of variability in parasite faunas among the regions sampled. This may be due to 1. natural variation within the GoC and there is one grey mackerel stock, or 2. the existence of multiple localised adult sub-stocks (metapopulations) within the GoC. Growth parameter comparisons were only possible from four major locations and identified the NW NT, the GoC, and the east coast as having different population growth characteristics. Through the use of multiple techniques, and by integrating the results from each, we were able to determine that there exist at least five stocks of grey mackerel across northern Australia, with some likelihood of additional stock structuring within the GoC. The major management units determined from this study therefore were Western Australia, NW Northern Territory (Timor/Arafura), the Gulf of Carpentaria, northern east Queensland coast and central east Queensland coast. The management implications of these results indicate the possible need for management of grey mackerel fisheries in Australia to be carried out on regional scales finer than are currently in place. In some regions the spatial scales of management might continue as is currently (e.g. WA), while in other regions, such as the GoC and the east coast, managers should at least monitor fisheries on a more local scale dictated by fishing effort and assess accordingly. Stock assessments should also consider the stock divisions identified, particularly on the east coast and for the GoC, and use life history parameters particular to each stock. We also emphasise that where we have not identified different stocks does not preclude the possibility of the occurrence of further stock division. Further, this study did not, nor did it set out to, assess the status of each of the stocks identified. This we identify as a high priority action for research and development of grey mackerel fisheries, as well as a management strategy evaluation that incorporates the conclusions of this work. Until such time that these priorities are addressed, management of grey mackerel fisheries should be cognisant of these uncertainties, particularly for the GoC and the Queensland east coast.

Action Spectra of P25 TiO2 and a visible light absorbing, carbon-modified titania in the photocatalytic degradation of stearic acid

The photonic efficiencies of films of Evonik (formerly Degussa) P25 TiO2 and carbon-modified TiO2 Kronos VLP 7000 samples are reported as a function of excitation wavelength (300–430 nm; FWHM ∼ 7.5 nm), i.e. the action spectra, for the degradation of stearic acid, a model organic for the photocatalytic destruction of solid surface organic pollutants. For each of these semiconductor photocatalysts, at 365 nm (FWHM = 18 nm), the dependence of the rate of degradation of stearic acid, upon the irradiance, I, is determined and the rate is found to be proportional to I0.65 and I0.82 for P25 and Kronos titania, respectively. Assuming this relationship holds at all wavelengths, the action spectra for two different semiconductor photocatalysts is modified by plotting, (RSA (rate of stearic acid destruction, units: molecules cm−2 s−1)/Iθ) vs. wavelength of excitation (λexcit), and both differ noticeably from those of the original (unmodified) action spectra, which are plots of (RSA/I = photonic efficiency, ξ) vs. λexcit. The shape of the modified action spectrum for P25 TiO2 is consistent with that reported by others for other organic mineralisation reactions and correlates well with diffuse reflectance data for P25 TiO2 (Kubelka–Munk plot), although there is some evidence that the active phase, in the photodegradation of stearic acid, is the anatase form present in P25. The unmodified and modified action spectra of the beige Kronos VLP 7000 TiO2 compound exhibits little or no activity in the visible i.e. (λexcit > 400 nm) and a peak at 350 nm. The Kronos powder contains a yellow/brown conjugated, extractable, organic sensitiser which has been identified by others as the species responsible for its reported photocatalytic visible light activity. But, irradiation of the Kronos powder film, with and without a stearic acid coating, in air, using UVA or visible light, bleaches rapidly (<60 min) most, if not all, of the little colour exhibited by the original Kronos powder. The photobleached form of the Kronos has a similar action spectrum to that of the unbleached form, which, in turn, appears very similar to that of P25 titania, at wavelengths >350 nm. It is proposed that the difference between the Kronos and P25 powder films at wavelengths <350 nm is due to a photodegradation-resistant, previously unidentified (but extractable using MeCN) UV-absorbing organic species in the former which screens the titania particles at these lower wavelengths. The implications of these observations are discussed briefly.

Architecture for real-time coordination of multiple autonomous mobile units

Interest on using teams of mobile robots has been growing, due to their potential to cooperate for diverse purposes, such as rescue, de-mining, surveillance or even games such as robotic soccer. These applications require a real-time middleware and wireless communication protocol that can support an efficient and timely fusion of the perception data from different robots as well as the development of coordinated behaviours. Coordinating several autonomous robots towards achieving a common goal is currently a topic of high interest, which can be found in many application domains. Despite these different application domains, the technical problem of building an infrastructure to support the integration of the distributed perception and subsequent coordinated action is similar. This problem becomes tougher with stronger system dynamics, e.g., when the robots move faster or interact with fast objects, leading to tighter real-time constraints. This thesis work addressed computing architectures and wireless communication protocols to support efficient information sharing and coordination strategies taking into account the real-time nature of robot activities. The thesis makes two main claims. Firstly, we claim that despite the use of a wireless communication protocol that includes arbitration mechanisms, the self-organization of the team communications in a dynamic round that also accounts for variable team membership, effectively reduces collisions within the team, independently of its current composition, significantly improving the quality of the communications. We will validate this claim in terms of packet losses and communication latency. We show how such self-organization of the communications can be achieved in an efficient way with the Reconfigurable and Adaptive TDMA protocol. Secondly, we claim that the development of distributed perception, cooperation and coordinated action for teams of mobile robots can be simplified by using a shared memory middleware that replicates in each cooperating robot all necessary remote data, the Real-Time Database (RTDB) middleware. These remote data copies, which are updated in the background by the selforganizing communications protocol, are extended with age information automatically computed by the middleware and are locally accessible through fast primitives. We validate our claim showing a parsimonious use of the communication medium, improved timing information with respect to the shared data and the simplicity of use and effectiveness of the proposed middleware shown in several use cases, reinforced with a reasonable impact in the Middle Size League of RoboCup.