Dynamic facial expressions evoke distinct activation in the face perception network:a connectivity analysis study

Very little is known about the neural structures involved in the perception of realistic dynamic facial expressions. In the present study, a unique set of naturalistic dynamic facial emotional expressions was created. Through fMRI and connectivity analysis, a dynamic face perception network was identified, which is demonstrated to extend Haxby et al.'s [Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. The distributed human neural system for face perception. Trends in Cognitive Science, 4, 223–233, 2000] distributed neural system for face perception. This network includes early visual regions, such as the inferior occipital gyrus, which is identified as insensitive to motion or affect but sensitive to the visual stimulus, the STS, identified as specifically sensitive to motion, and the amygdala, recruited to process affect. Measures of effective connectivity between these regions revealed that dynamic facial stimuli were associated with specific increases in connectivity between early visual regions, such as the inferior occipital gyrus and the STS, along with coupling between the STS and the amygdala, as well as the inferior frontal gyrus. These findings support the presence of a distributed network of cortical regions that mediate the perception of different dynamic facial expressions.

Recognition of Facial Expressions with Autoencoders and Convolutional-Nets

Les humains communiquent via différents types de canaux: les mots, la voix, les gestes du corps, des émotions, etc. Pour cette raison, un ordinateur doit percevoir ces divers canaux de communication pour pouvoir interagir intelligemment avec les humains, par exemple en faisant usage de microphones et de webcams. Dans cette thèse, nous nous intéressons à déterminer les émotions humaines à partir d’images ou de vidéo de visages afin d’ensuite utiliser ces informations dans différents domaines d’applications. Ce mémoire débute par une brève introduction à l'apprentissage machine en s’attardant aux modèles et algorithmes que nous avons utilisés tels que les perceptrons multicouches, réseaux de neurones à convolution et autoencodeurs. Elle présente ensuite les résultats de l'application de ces modèles sur plusieurs ensembles de données d'expressions et émotions faciales. Nous nous concentrons sur l'étude des différents types d’autoencodeurs (autoencodeur débruitant, autoencodeur contractant, etc) afin de révéler certaines de leurs limitations, comme la possibilité d'obtenir de la coadaptation entre les filtres ou encore d’obtenir une courbe spectrale trop lisse, et étudions de nouvelles idées pour répondre à ces problèmes. Nous proposons également une nouvelle approche pour surmonter une limite des autoencodeurs traditionnellement entrainés de façon purement non-supervisée, c'est-à-dire sans utiliser aucune connaissance de la tâche que nous voulons finalement résoudre (comme la prévision des étiquettes de classe) en développant un nouveau critère d'apprentissage semi-supervisé qui exploite un faible nombre de données étiquetées en combinaison avec une grande quantité de données non-étiquetées afin d'apprendre une représentation adaptée à la tâche de classification, et d'obtenir une meilleure performance de classification. Finalement, nous décrivons le fonctionnement général de notre système de détection d'émotions et proposons de nouvelles idées pouvant mener à de futurs travaux.

Intact imitation of emotional facial actions in autism spectrum conditions.

It has been proposed that there is a core impairment in autism spectrum conditions (ASC) to the mirror neuron system (MNS): If observed actions cannot be mapped onto the motor commands required for performance, higher order sociocognitive functions that involve understanding another person's perspective, such as theory of mind, may be impaired. However, evidence of MNS impairment in ASC is mixed. The present study used an 'automatic imitation' paradigm to assess MNS functioning in adults with ASC and matched controls, when observing emotional facial actions. Participants performed a pre-specified angry or surprised facial action in response to observed angry or surprised facial actions, and the speed of their action was measured with motion tracking equipment. Both the ASC and control groups demonstrated automatic imitation of the facial actions, such that responding was faster when they acted with the same emotional expression that they had observed. There was no difference between the two groups in the magnitude of the effect. These findings suggest that previous apparent demonstrations of impairments to the MNS in ASC may be driven by a lack of visual attention to the stimuli or motor sequencing impairments, and therefore that there is, in fact, no MNS impairment in ASC. We discuss these findings with reference to the literature on MNS functioning and imitation in ASC, as well as theories of the role of the MNS in sociocognitive functioning in typical development.

Beta event-related desynchronization as an index of individual differences in processing human facial expression: further investigations of autistic traits in typically developing adults

The human mirror neuron system (hMNS) has been associated with various forms of social cognition and affective processing including vicarious experience. It has also been proposed that a faulty hMNS may underlie some of the deficits seen in the autism spectrum disorders (ASDs). In the present study we set out to investigate whether emotional facial expressions could modulate a putative EEG index of hMNS activation (mu suppression) and if so, would this differ according to the individual level of autistic traits [high versus low Autism Spectrum Quotient (AQ) score]. Participants were presented with 3 s films of actors opening and closing their hands (classic hMNS mu-suppression protocol) while simultaneously wearing happy, angry, or neutral expressions. Mu-suppression was measured in the alpha and low beta bands. The low AQ group displayed greater low beta event-related desynchronization (ERD) to both angry and neutral expressions. The high AQ group displayed greater low beta ERD to angry than to happy expressions. There was also significantly more low beta ERD to happy faces for the low than for the high AQ group. In conclusion, an interesting interaction between AQ group and emotional expression revealed that hMNS activation can be modulated by emotional facial expressions and that this is differentiated according to individual differences in the level of autistic traits. The EEG index of hMNS activation (mu suppression) seems to be a sensitive measure of the variability in facial processing in typically developing individuals with high and low self-reported traits of autism.

Recognition accuracy and response bias to happy and sad facial expressions in patients with major depression

Impaired facial expression recognition has been associated with features of major depression, which could underlie some of the difficulties in social interactions in these patients. Patients with major depressive disorder and age- and gender-matched healthy volunteers judged the emotion of 100 facial stimuli displaying different intensities of sadness and happiness and neutral expressions presented for short (100 ms) and long (2,000 ms) durations. Compared with healthy volunteers, depressed patients demonstrated subtle impairments in discrimination accuracy and a predominant bias away from the identification as happy of mildly happy expressions. The authors suggest that, in depressed patients, the inability to accurately identify subtle changes in facial expression displayed by others in social situations may underlie the impaired interpersonal functioning.

Social anxiety disorder women easily recognize fearfull, sad and happy faces: The influence of gender

Background: It has been suggested that individuals with social anxiety disorder (SAD) are exaggeratedly concerned about approval and disapproval by others. Therefore, we assessed the recognition of facial expressions by individuals with SAD, in an attempt to overcome the limitations of previous studies. Methods: The sample was formed by 231 individuals (78 SAD patients and 153 healthy controls). All individuals were treatment naive, aged 18-30 years and with similar socioeconomic level. Participants judged which emotion (happiness, sadness, disgust, anger, fear, and surprise) was presented in the facial expression of stimuli displayed on a computer screen. The stimuli were manipulated in order to depict different emotional intensities, with the initial image being a neutral face (0%) and, as the individual moved on across images, the expressions increased their emotional intensity until reaching the total emotion (100%). The time, accuracy, and intensity necessary to perform judgments were evaluated. Results: The groups did not show statistically significant differences in respect to the number of correct judgments or to the time necessary to respond. However, women with SAD required less emotional intensity to recognize faces displaying fear (p = 0.002), sadness (p = 0.033) and happiness (p = 0.002), with no significant differences for the other emotions or men with SAD. Conclusions: The findings suggest that women with SAD are hypersensitive to threat-related and approval-related social cues. Future studies investigating the neural basis of the impaired processing of facial emotion in SAD using functional neuroimaging would be desirable and opportune. (C) 2009 Elsevier Ltd. All rights reserved.

Responsividade sensível da cuidadora e processamento de expressões faciais emocionais em crianças institucionalizadas: um estudo com potenciais evocados

Dissertação de mestrado em Psicologia Aplicada

Behavioural and neural correlates of emotion recognition as a function of psychopathic personality traits

Psychopathy is associated with well-known characteristics such as a lack of empathy and impulsive behaviour, but it has also been associated with impaired recognition of emotional facial expressions. The use of event-related potentials (ERPs) to examine this phenomenon could shed light on the specific time course and neural activation associated with emotion recognition processes as they relate to psychopathic traits. In the current study we examined the PI , N170, and vertex positive potential (VPP) ERP components and behavioural performance with respect to scores on the Self-Report Psychopathy (SRP-III) questionnaire. Thirty undergraduates completed two tasks, the first of which required the recognition and categorization of affective face stimuli under varying presentation conditions. Happy, angry or fearful faces were presented under with attention directed to the mouth, nose or eye region and varied stimulus exposure duration (30, 75, or 150 ms). We found that behavioural performance to be unrelated to psychopathic personality traits in all conditions, but there was a trend for the Nl70 to peak later in response to fearful and happy facial expressions for individuals high in psychopathic traits. However, the amplitude of the VPP was significantly negatively associated with psychopathic traits, but only in response to stimuli presented under a nose-level fixation. Finally, psychopathic traits were found to be associated with longer N170 latencies in response to stimuli presented under the 30 ms exposure duration. In the second task, participants were required to inhibit processing of irrelevant affective and scrambled face distractors while categorizing unrelated word stimuli as living or nonliving. Psychopathic traits were hypothesized to be positively associated with behavioural performance, as it was proposed that individuals high in psychopathic traits would be less likely to automatically attend to task-irrelevant affective distractors, facilitating word categorization. Thus, decreased interference would be reflected in smaller N170 components, indicating less neural activity associated with processing of distractor faces. We found that overall performance decreased in the presence of angry and fearful distractor faces as psychopathic traits increased. In addition, the amplitude of the N170 decreased and the latency increased in response to affective distractor faces for individuals with higher levels of psychopathic traits. Although we failed to find the predicted behavioural deficit in emotion recognition in Task 1 and facilitation effect in Task 2, the findings of increased N170 and VPP latencies in response to emotional faces are consistent wi th the proposition that abnormal emotion recognition processes may in fact be inherent to psychopathy as a continuous personality trait.

Les sens des cris de personnes âgées vivant avec une démence en centre d’hébergement et de soins de longue durée

Dans les centres d’hébergement et de soins de longue durée (CHSLD), jusqu’à 80 % des personnes admises vivent avec une démence (Conseil des aînés, 2007). Parmi ces personnes âgées, plusieurs crient. Ce comportement a fait l’objet de quelques études, mais ces études ne fournissent pas une compréhension des sens de ces cris qui pourrait orienter le choix d’interventions face à ces personnes et leur entourage. À l’aide de la perspective infirmière de Leininger (2001) et de la théorie de la communication et des interactions à l’intérieur de triades d’Adams et Gardiner (2005), le but de cette étude était de décrire les sens des cris de personnes âgées vivant avec une démence et les facteurs les influençant en tenant compte de la perspective des personnes âgées, de leurs aidants familiaux et de leurs soignants. Pour atteindre ce but, l’ethnographie critique a été choisie comme méthode. L’étude s’est découlée dans un CHSLD au sein duquel sept triades, composées de personnes âgées vivant avec une démence qui crient, d’aidants familiaux principaux et de soignants, ont été recrutées selon un échantillonnage théorique. Diverses méthodes de collecte des données ont été utilisées dont l’observation des personnes âgées et du milieu et des entrevues semi-dirigées auprès des aidants et soignants. L’analyse des données a été effectuée à l’aide des quatre approches proposées par Spradley (1979) soit l’analyse : des domaines, taxonomique, componentielle et thématique. Les résultats ont permis d’identifier des thèmes sur les sens des cris et sur les facteurs influençant ceux-ci. Les cris peuvent avoir pour sens la vulnérabilité, la souffrance et la perte de sens vécues par la personne âgée. Ce comportement peut aussi indiquer diverses finalités, par exemple l’expression d’insatisfactions ou d’émotions. Ces finalités peuvent être distinguées à partir de critères de modulation tels que la prévisibilité des cris ou un faciès émotif. Par ailleurs, divers aspects rendent les sens des cris singuliers. La stabilité et la flexibilité dans la façon d’organiser les soins ainsi que l’effet réciproque entre les personnes âgées qui crient et les autres personnes dans le CHSLD se sont dégagées comme étant des facteurs qui influencent les sens des cris. Il s’est aussi révélé que les cris de chaque personne âgée peuvent être considérés comme un langage unique que les aidants et les soignants sont en mesure d’apprendre et qui influence l’interprétation des sens des cris. L’accompagnement de la personne âgée, en tenant compte de ses volontés, ses besoins et de sa personnalité, les fluctuations dans les relations de pouvoir au sein des triades personnes âgées-aidants-soignants et les sentiments d’impuissance et de culpabilité des aidants et des soignants sont d’autres facteurs qui influencent les cris. Les connaissances découlant de cette étude augmentent la compréhension sur les sens des cris des personnes âgées vivant avec une démence. Elles ont des implications pour les divers champs d’activités des infirmières et qui pourront contribuer à offrir des soins culturellement cohérents et caring pour les personnes âgées vivant avec une démence et leur entourage.

Visual information processing during conscious and non-conscious face perception

Les stimuli naturels projetés sur nos rétines nous fournissent de l’information visuelle riche. Cette information varie le long de propriétés de « bas niveau » telles que la luminance, le contraste, et les fréquences spatiales. Alors qu’une partie de cette information atteint notre conscience, une autre partie est traitée dans le cerveau sans que nous en soyons conscients. Les propriétés de l’information influençant l’activité cérébrale et le comportement de manière consciente versus non-consciente demeurent toutefois peu connues. Cette question a été examinée dans les deux derniers articles de la présente thèse, en exploitant les techniques psychophysiques développées dans les deux premiers articles. Le premier article présente la boîte à outils SHINE (spectrum, histogram, and intensity normalization and equalization), développée afin de permettre le contrôle des propriétés de bas niveau de l'image dans MATLAB. Le deuxième article décrit et valide la technique dite des bulles fréquentielles, qui a été utilisée tout au long des études de cette thèse pour révéler les fréquences spatiales utilisées dans diverses tâches de perception des visages. Cette technique offre les avantages d’une haute résolution au niveau des fréquences spatiales ainsi que d’un faible biais expérimental. Le troisième et le quatrième article portent sur le traitement des fréquences spatiales en fonction de la conscience. Dans le premier cas, la méthode des bulles fréquentielles a été utilisée avec l'amorçage par répétition masquée dans le but d’identifier les fréquences spatiales corrélées avec les réponses comportementales des observateurs lors de la perception du genre de visages présentés de façon consciente versus non-consciente. Les résultats montrent que les mêmes fréquences spatiales influencent de façon significative les temps de réponse dans les deux conditions de conscience, mais dans des sens opposés. Dans le dernier article, la méthode des bulles fréquentielles a été combinée à des enregistrements intracrâniens et au Continuous Flash Suppression (Tsuchiya & Koch, 2005), dans le but de cartographier les fréquences spatiales qui modulent l'activation de structures spécifiques du cerveau (l'insula et l'amygdale) lors de la perception consciente versus non-consciente des expressions faciales émotionnelles. Dans les deux régions, les résultats montrent que la perception non-consciente s'effectue plus rapidement et s’appuie davantage sur les basses fréquences spatiales que la perception consciente. La contribution de cette thèse est donc double. D’une part, des contributions méthodologiques à la recherche en perception visuelle sont apportées par l'introduction de la boîte à outils SHINE ainsi que de la technique des bulles fréquentielles. D’autre part, des indications sur les « corrélats de la conscience » sont fournies à l’aide de deux approches différentes.

Evaluating a realistic agent in an advice-giving task

The aim of this study was to empirically evaluate an embodied conversational agent called GRETA in an effort to answer two main questions: (1) What are the benefits (and costs) of presenting information via an animated agent, with certain characteristics, in a 'persuasion' task, compared to other forms of display? (2) How important is it that emotional expressions are added in a way that is consistent with the content of the message, in animated agents? To address these questions, a positively framed healthy eating message was created which was variously presented via GRETA, a matched human actor, GRETA's voice only (no face) or as text only. Furthermore, versions of GRETA were created which displayed additional emotional facial expressions in a way that was either consistent or inconsistent with the content of the message. Overall, it was found that although GRETA received significantly higher ratings for helpfulness and likability, presenting the message via GRETA led to the poorest memory performance among users. Importantly, however, when GRETA's additional emotional expressions were consistent with the content of the verbal message, the negative effect on memory performance disappeared. Overall, the findings point to the importance of achieving consistency in animated agents. (c) 2005 Elsevier Ltd. All rights reserved.

A alexitimia na categorização de emoções em expressões faciais dinâmicas

A alexitimia é o termo usado para caracterizar as pessoas com défices no processamento, na compreensão e na descrição de emoções. Definida também como um traço de personalidade multifatorial, a alexitimia tem uma prevalência de 10% na população em geral e está associada a uma menor qualidade de vida. Apesar de este estudo incidir-se numa população saudável, a alexitimia está frequentemente associada a patologias físicas e perturbações mentais. O objetivo deste estudo consistiu em verificar a influência da alexitimia na categorização das expressões faciais emocionais, numa tarefa de apresentação de faces dinâmicas. 87 participantes com diferentes índices de alexitimia completaram uma tarefa que consistia na apresentação de faces dinâmicas cujo o intuito era categorizar as expressões faciais de raiva, nojo e alegria. A precisão das respostas e os níveis de intensidade emocional foram manipulados. Os resultados indicaram que os indivíduos, no geral, identificaram mais precisamente e com menores níveis de intensidade emocional a expressão de alegria comparativamente com as expressões negativas (raiva e nojo). Verificou-se também diferenças significativas entre a alexitimia e as expressões faciais emocionais relativamente à precisão na categorização. Deste modo, foi possível verificar neste estudo, que indivíduos com altos níveis de alexitimia foram significativamente menos precisos na identificação da expressão facial de raiva comparativamente com indivíduos com baixos níveis de alexitimia.

Testosterone and attention deficits as possible mechanisms underlying impaired emotion recognition in intimate partner violence perpetrators

Several studies have reported impairments in decoding emotional facial expressions in intimate partner violence (IPV) perpetrators. However, the mechanisms that underlie these impaired skills are not well known. Given this gap in the literature, we aimed to establish whether IPV perpetrators (n = 18) differ in their emotion decoding process, attentional skills, and testosterone (T), cortisol (C) levels and T/C ratio in comparison with controls (n = 20), and also to examine the moderating role of the group and hormonal parameters in the relationship between attention skills and the emotion decoding process. Our results demonstrated that IPV perpetrators showed poorer emotion recognition and higher attention switching costs than controls. Nonetheless, they did not differ in attention to detail and hormonal parameters. Finally, the slope predicting emotion recognition from deficits in attention switching became steeper as T levels increased, especially in IPV perpetrators, although the basal C and T/C ratios were unrelated to emotion recognition and attention deficits for both groups. These findings contribute to a better understanding of the mechanisms underlying emotion recognition deficits. These factors therefore constitute the target for future interventions.