Facial identity and facial expression matching in 5 - 12-year-old children and adults

Facial identity and facial expression matching tasks were completed by 5–12-year-old children and adults using stimuli extracted from the same set of normalized faces. Configural and feature processing were examined using speed and accuracy of responding and facial feature selection, respectively. Facial identity matching was slower than face expression matching for all age groups. Large age effects were found on both speed and accuracy of responding and feature use in both identity and expression matching tasks. Eye region preference was found on the facial identity task and mouth region preference on the facial expression task. Use of mouth region information for facial expression matching increased with age, whereas use of eye region information for facial identity matching peaked early. The feature use information suggests that the specific use of primary facial features to arrive at identity and emotion matching judgments matures across middle childhood.

Electrophysiological and behavioural correlates of facial identity and expression perception deficits following a closed head injury /

A large variety of social signals, such as facial expression and body language, are conveyed in everyday interactions and an accurate perception and interpretation of these social cues is necessary in order for reciprocal social interactions to take place successfully and efficiently. The present study was conducted to determine whether impairments in social functioning that are commonly observed following a closed head injury, could at least be partially attributable to disruption in the ability to appreciate social cues. More specifically, an attempt was made to determine whether face processing deficits following a closed head injury (CHI) coincide with changes in electrophysiological responsivity to the presentation of facial stimuli. A number of event-related potentials (ERPs) that have been linked specifically to various aspects of visual processing were examined. These included the N170, an index of structural encoding ability, the N400, an index of the ability to detect differences in serially presented stimuli, and the Late Positivity (LP), an index of the sensitivity to affective content in visually-presented stimuli. Electrophysiological responses were recorded while participants with and without a closed head injury were presented with pairs of faces delivered in a rapid sequence and asked to compare them on the basis of whether they matched with respect to identity or emotion. Other behavioural measures of identity and emotion recognition were also employed, along with a small battery of standard neuropsychological tests used to determine general levels of cognitive impairment. Participants in the CHI group were impaired in a number of cognitive domains that are commonly affected following a brain injury. These impairments included reduced efficiency in various aspects of encoding verbal information into memory, general slower rate of information processing, decreased sensitivity to smell, and greater difficulty in the regulation of emotion and a limited awareness of this impairment. Impairments in face and emotion processing were clearly evident in the CHI group. However, despite these impairments in face processing, there were no significant differences between groups in the electrophysiological components examined. The only exception was a trend indicating delayed N170 peak latencies in the CHI group (p = .09), which may reflect inefficient structural encoding processes. In addition, group differences were noted in the region of the N100, thought to reflect very early selective attention. It is possible, then, that facial expression and identity processing deficits following CHI are secondary to (or exacerbated by) an underlying disruption of very early attentional processes. Alternately the difficulty may arise in the later cognitive stages involved in the interpretation of the relevant visual information. However, the present data do not allow these alternatives to be distinguished. Nonetheless, it was clearly evident that individuals with CHI are more likely than controls to make face processing errors, particularly for the more difficult to discriminate negative emotions. Those working with individuals who have sustained a head injury should be alerted to this potential source of social monitoring difficulties which is often observed as part of the sequelae following a CHI.

Aplicação de sistemas imunológicos artificiais para biometria facial: Reconhecimento de identidade baseado nas características de padrões binários

Pós-graduação em Engenharia Elétrica - FEIS

La plateforme Bubbles : un outil d'investigation des différences individuelles de stratégies de reconnaissance de l'identité des visages.

L’objectif de cette recherche est la création d’une plateforme en ligne qui permettrait d’examiner les différences individuelles de stratégies de traitement de l’information visuelle dans différentes tâches de catégorisation des visages. Le but d’une telle plateforme est de récolter des données de participants géographiquement dispersés et dont les habiletés en reconnaissance des visages sont variables. En effet, de nombreuses études ont montré qu’il existe de grande variabilité dans le spectre des habiletés à reconnaître les visages, allant de la prosopagnosie développementale (Susilo & Duchaine, 2013), un trouble de reconnaissance des visages en l’absence de lésion cérébrale, aux super-recognizers, des individus dont les habiletés en reconnaissance des visages sont au-dessus de la moyenne (Russell, Duchaine & Nakayama, 2009). Entre ces deux extrêmes, les habiletés en reconnaissance des visages dans la population normale varient. Afin de démontrer la faisabilité de la création d’une telle plateforme pour des individus d’habiletés très variables, nous avons adapté une tâche de reconnaissance de l’identité des visages de célébrités utilisant la méthode Bubbles (Gosselin & Schyns, 2001) et avons recruté 14 sujets contrôles et un sujet présentant une prosopagnosie développementale. Nous avons pu mettre en évidence l’importance des yeux et de la bouche dans l’identification des visages chez les sujets « normaux ». Les meilleurs participants semblent, au contraire, utiliser majoritairement le côté gauche du visage (l’œil gauche et le côté gauche de la bouche).

Facial emotion and identity processing development in 5- to 15-year-old children

Most developmental studies of emotional face processing to date have focused on infants and very young children. Additionally, studies that examine emotional face processing in older children do not distinguish development in emotion and identity face processing from more generic age-related cognitive improvement. In this study, we developed a paradigm that measures processing of facial expression in comparison to facial identity and complex visual stimuli. The three matching tasks were developed (i.e., facial emotion matching, facial identity matching, and butterfly wing matching) to include stimuli of similar level of discriminability and to be equated for task difficulty in earlier samples of young adults. Ninety-two children aged 5–15 years and a new group of 24 young adults completed these three matching tasks. Young children were highly adept at the butterfly wing task relative to their performance on both face-related tasks. More importantly, in older children, development of facial emotion discrimination ability lagged behind that of facial identity discrimination.

Symptom correlates of static and dynamic facial affect processing in schizophrenia : evidence of a double dissociation?

Schizophrenia patients have been shown to be compromised in their ability to recognize facial emotion. This deficit has been shown to be related to negative symptoms severity. However, to date, most studies have used static rather than dynamic depictions of faces. Nineteen patients with schizophrenia were compared with seventeen controls on 2 tasks; the first involving the discrimination of facial identity, emotion, and butterfly wings; the second testing emotion recognition using both static and dynamic stimuli. In the first task, the patients performed more poorly than controls for emotion discrimination only, confirming a specific deficit in facial emotion recognition. In the second task, patients performed more poorly in both static and dynamic facial emotion processing. An interesting pattern of associations suggestive of a possible double dissociation emerged in relation to correlations with symptom ratings: high negative symptom ratings were associated with poorer recognition of static displays of emotion, whereas high positive symptom ratings were associated with poorer recognition of dynamic displays of emotion. However, while the strength of associations between negative symptom ratings and accuracy during static and dynamic facial emotion processing was significantly different, those between positive symptom ratings and task performance were not. The results confirm a facial emotion-processing deficit in schizophrenia using more ecologically valid dynamic expressions of emotion. The pattern of findings may reflect differential patterns of cortical dysfunction associated with negative and positive symptoms of schizophrenia in the context of differential neural mechanisms for the processing of static and dynamic displays of facial emotion.

Multiview gait biometrics for human identity recognition

We propose a novel multiview fusion scheme for recognizing human identity based on gait biometric data. The gait biometric data is acquired from video surveillance datasets from multiple cameras. Experiments on publicly available CASIA dataset show the potential of proposed scheme based on fusion towards development and implementation of automatic identity recognition systems.

FACSGen: a tool to synthesize emotional facial expressions through systematic manipulation of facial action units

To investigate the perception of emotional facial expressions, researchers rely on shared sets of photos or videos, most often generated by actor portrayals. The drawback of such standardized material is a lack of flexibility and controllability, as it does not allow the systematic parametric manipulation of specific features of facial expressions on the one hand, and of more general properties of the facial identity (age, ethnicity, gender) on the other. To remedy this problem, we developed FACSGen: a novel tool that allows the creation of realistic synthetic 3D facial stimuli, both static and dynamic, based on the Facial Action Coding System. FACSGen provides researchers with total control over facial action units, and corresponding informational cues in 3D synthetic faces. We present four studies validating both the software and the general methodology of systematically generating controlled facial expression patterns for stimulus presentation.

The development and evaluation of a computer-based program to test and to teach the recognition of facial affect

Autism is a chronic pervasive neurodevelopmental disorder characterized by the early onset of social and communicative impairments as well as restricted, ritualized, stereotypic behavior. The endophenotype of autism includes neuropsychological deficits, for instance a lack of "Theory of Mind" and problems recognizing facial affect. In this study, we report the development and evaluation of a computer-based program to teach and test the ability to identify basic facially expressed emotions. 10 adolescent or adult subjects with high-functioning autism or Asperger-syndrome were included in the investigation. A priori the facial affect recognition test had shown good psychometric properties in a normative sample (internal consistency: rtt=.91-.95; retest reliability: rtt=.89-.92). In a prepost design, one half of the sample was randomly assigned to receive computer treatment while the other half of the sample served as control group. The training was conducted for five weeks, consisting of two hours training a week. The trained individuals improved significantly on the affect recognition task, but not on any other measure. Results support the usefulness of the program to teach the detection of facial affect. However, the improvement found is limited to a circumscribed area of social-communicative function and generalization is not ensured.

Steady state visually evoked potential correlates of static and dynamic emotional face processing

While the neural regions associated with facial identity recognition are considered to be well defined, the neural correlates of non-moving and moving images of facial emotion processing are less clear. This study examined the brain electrical activity changes in 26 participants (14 males M = 21.64, SD = 3.99; 12 females M = 24.42, SD = 4.36), during a passive face viewing task, a scrambled face task and separate emotion and gender face discrimination tasks. The steady state visual evoked potential (SSVEP) was recorded from 64-electrode sites. Consistent with previous research, face related activity was evidenced at scalp regions over the parieto-temporal region approximately 170 ms after stimulus presentation. Results also identified different SSVEP spatio-temporal changes associated with the processing of static and dynamic facial emotions with respect to gender, with static stimuli predominately associated with an increase in inhibitory processing within the frontal region. Dynamic facial emotions were associated with changes in SSVEP response within the temporal region, which are proposed to index inhibitory processing. It is suggested that static images represent non-canonical stimuli which are processed via different mechanisms to their more ecologically valid dynamic counterparts.

Social saliency : visual psychophysics and single-neuron recordings in humans

My thesis studies how people pay attention to other people and the environment. How does the brain figure out what is important and what are the neural mechanisms underlying attention? What is special about salient social cues compared to salient non-social cues? In Chapter I, I review social cues that attract attention, with an emphasis on the neurobiology of these social cues. I also review neurological and psychiatric links: the relationship between saliency, the amygdala and autism. The first empirical chapter then begins by noting that people constantly move in the environment. In Chapter II, I study the spatial cues that attract attention during locomotion using a cued speeded discrimination task. I found that when the motion was expansive, attention was attracted towards the singular point of the optic flow (the focus of expansion, FOE) in a sustained fashion. The more ecologically valid the motion features became (e.g., temporal expansion of each object, spatial depth structure implied by distribution of the size of the objects), the stronger the attentional effects. However, compared to inanimate objects and cues, people preferentially attend to animals and faces, a process in which the amygdala is thought to play an important role. To directly compare social cues and non-social cues in the same experiment and investigate the neural structures processing social cues, in Chapter III, I employ a change detection task and test four rare patients with bilateral amygdala lesions. All four amygdala patients showed a normal pattern of reliably faster and more accurate detection of animate stimuli, suggesting that advantageous processing of social cues can be preserved even without the amygdala, a key structure of the “social brain”. People not only attend to faces, but also pay attention to others’ facial emotions and analyze faces in great detail. Humans have a dedicated system for processing faces and the amygdala has long been associated with a key role in recognizing facial emotions. In Chapter IV, I study the neural mechanisms of emotion perception and find that single neurons in the human amygdala are selective for subjective judgment of others’ emotions. Lastly, people typically pay special attention to faces and people, but people with autism spectrum disorders (ASD) might not. To further study social attention and explore possible deficits of social attention in autism, in Chapter V, I employ a visual search task and show that people with ASD have reduced attention, especially social attention, to target-congruent objects in the search array. This deficit cannot be explained by low-level visual properties of the stimuli and is independent of the amygdala, but it is dependent on task demands. Overall, through visual psychophysics with concurrent eye-tracking, my thesis found and analyzed socially salient cues and compared social vs. non-social cues and healthy vs. clinical populations. Neural mechanisms underlying social saliency were elucidated through electrophysiology and lesion studies. I finally propose further research questions based on the findings in my thesis and introduce my follow-up studies and preliminary results beyond the scope of this thesis in the very last section, Future Directions.

面孔身份与面部表情识别中的区分度效应

Whether facial identity and facial expression was processed independently has long been a controversy. Studies at levels of experimental, neuropsychological, functional imaging and cell-recording all failed to consistently support either independent or interdependent processing. Present study proposed that familiarity and discriminability of facial identity and expression was important variable in mediating the relation between facial identity and facial expression recognition. Effects of familiarity on recognition of facial identity and expression had been examined (e.g. Ganel & Goshen-Gottstein, 2004) but the role of the discriminability in recognition of facial identity and expression has not yet been carefully examined. To examine the role of discriminability of facial identity and expression, 8 experiments were conducted with Garner’s speeded classification task in recognition of identity and expression of unfamiliar faces. The discriminability of facial identity and expression was manipulated, and the measurements of Garner interference and facilitation indicated that: 1. The discriminability of facial identity and expression mediate the relation between facial identity and expression recognition. Four possible discriminability combinations between identity and expression predicted 4 interference patterns between them. Low discriminability accounted for the interference either in facial identity judgment or in facial expression judgment task. 2. The measurements of eye movements indicated that either in facial identity or in facial expression recognition low discriminability led to a narrowly-distributed eye fixation pattern while high discriminability led to a widely-distributed eye fixation pattern. 3. By combining the morphing technique with the Garner paradigm, study 2 successfully demonstrated the linar relation between discriminability and Garner facilitation effects, confirmed the discriminability effects in the measurements of Garner facilitation effects.. 4. By providing the varying information of facial expression, study 2 revealed that varying information improved the discriminability of facial expression, and then enhanced the recognition of facial expression. All the results indicated that the discriminability of facial identity and expression could mediate the independent or interdependent processing between them, and the discriminability effects on recognition of identity and expression of unfamiliar faces was identified. The results from interference effects and facilitation effects both indicated that the dimensional relation between facial identity and expression was separable but not asymmetric claimed by previous studies（Schweinberger et al, 1998, 1999）. Absolutedly independent or interdependent processing between facial identity and expression recognition were both impossible, discriminability of identity and expression mediated the relation between them. The discriminability effects revealed in present study could explain the conflicts between existing findings well.

L’utilisation de l’information visuelle en reconnaissance d’expressions faciales d’émotion

L’aptitude à reconnaitre les expressions faciales des autres est cruciale au succès des interactions sociales. L’information visuelle nécessaire à la catégorisation des expressions faciales d’émotions de base présentées de manière statique est relativement bien connue. Toutefois, l’information utilisée pour discriminer toutes les expressions faciales de base entre elle demeure encore peu connue, et ce autant pour les expressions statiques que dynamiques. Plusieurs chercheurs assument que la région des yeux est particulièrement importante pour arriver à « lire » les émotions des autres. Le premier article de cette thèse vise à caractériser l’information utilisée par le système visuel pour discriminer toutes les expressions faciales de base entre elles, et à vérifier l’hypothèse selon laquelle la région des yeux est cruciale pour cette tâche. La méthode des Bulles (Gosselin & Schyns, 2001) est utilisée avec des expressions faciales statiques (Exp. 1) et dynamiques (Exp. 2) afin de trouver quelles régions faciales sont utilisées (Exps. 1 et 2), ainsi que l’ordre temporel dans lequel elles sont utilisées (Exp. 2). Les résultats indiquent que, contrairement à la croyance susmentionnée, la région de la bouche est significativement plus utile que la région des yeux pour discriminer les expressions faciales de base. Malgré ce rôle prépondérant de la bouche, c’est toute de même la région des yeux qui est sous-utilisée chez plusieurs populations cliniques souffrant de difficultés à reconnaitre les expressions faciales. Cette observation pourrait suggérer que l’utilisation de la région des yeux varie en fonction de l’habileté pour cette tâche. Le deuxième article de cette thèse vise donc à vérifier comment les différences individuelles en reconnaissance d’expressions faciales sont reliées aux stratégies d’extraction de l’information visuelle pour cette tâche. Les résultats révèlent une corrélation positive entre l’utilisation de la région de la bouche et l’habileté, suggérant la présence de différences qualitatives entre la stratégie des patients et celle des normaux. De plus, une corrélation positive est retrouvée entre l’utilisation de l’œil gauche et l’habileté des participants, mais aucune corrélation n’est retrouvée entre l’utilisation de l’œil droit et l’habileté. Ces résultats indiquent que la stratégie des meilleurs participants ne se distingue pas de celle des moins bons participants simplement par une meilleure utilisation de l’information disponible dans le stimulus : des différences qualitatives semblent exister même au sein des stratégies des participants normaux.

Relative Contributions of Internal and External Features to Face Recognition

The central challenge in face recognition lies in understanding the role different facial features play in our judgments of identity. Notable in this regard are the relative contributions of the internal (eyes, nose and mouth) and external (hair and jaw-line) features. Past studies that have investigated this issue have typically used high-resolution images or good-quality line drawings as facial stimuli. The results obtained are therefore most relevant for understanding the identification of faces at close range. However, given that real-world viewing conditions are rarely optimal, it is also important to know how image degradations, such as loss of resolution caused by large viewing distances, influence our ability to use internal and external features. Here, we report experiments designed to address this issue. Our data characterize how the relative contributions of internal and external features change as a function of image resolution. While we replicated results of previous studies that have shown internal features of familiar faces to be more useful for recognition than external features at high resolution, we found that the two feature sets reverse in importance as resolution decreases. These results suggest that the visual system uses a highly non-linear cue-fusion strategy in combining internal and external features along the dimension of image resolution and that the configural cues that relate the two feature sets play an important role in judgments of facial identity.