The Belfast Induced Natural Emotion Database

For many years psychological research on facial expression of emotion has relied heavily on a recognition paradigm based on posed static photographs. There is growing evidence that there may be fundamental differences between the expressions depicted in such stimuli and the emotional expressions present in everyday life. Affective computing, with its pragmatic emphasis on realism, needs examples of natural emotion. This paper describes a unique database containing recordings of mild to moderate emotionally coloured responses to a series of laboratory based emotion induction tasks. The recordings are accompanied by information on self-report of emotion and intensity, continuous trace-style ratings of valence and intensity, the sex of the participant, the sex of the experimenter, the active or passive nature of the induction task and it gives researchers the opportunity to compare expressions from people from more than one culture.

Emotional intelligence, personality and decoding non-verbal expressions of emotion

Previous research has highlighted theoretical and empirical links between measures of both personality and trait emotional intelligence (EI), and the ability to decode facial expressions of emotion. Research has also found that the posed, static characteristics of the photographic stimuli used to explore these links affects the decoding process and differentiates them from the natural expressions they represent. This undermines the ecological validity of established trait-emotion decoding relationships. This study addresses these methodological shortcomings by testing relationships between the reliability of participant ratings of dynamic, spontaneously elicited expressions of emotion with personality and trait EI. Fifty participants completed personality and self-report EI questionnaires, and used a computer-logging program to continuously rate change in emotional intensity expressed in video clips. Each clip was rated twice to obtain an intra-rater reliability score. The results provide limited support for links between both trait EI and personality variables and how reliably we decode natural expressions of emotion. Limitations and future directions are discussed.

AdaBoost Multiple Feature Selection and Combination for Face Recognition

Gabor features have been recognized as one of the most successful face representations. Encouraged by the results given by this approach, other kind of facial representations based on Steerable Gaussian first order kernels and Harris corner detector are proposed in this paper. In order to reduce the high dimensional feature space, PCA and LDA techniques are employed. Once the features have been extracted, AdaBoost learning algorithm is used to select and combine the most representative features. The experimental results on XM2VTS database show an encouraging recognition rate, showing an important improvement with respect to face descriptors only based on Gabor filters.

Real-life Social Attention in Children with Autism Spectrum Disorders

Autism is a neuro-developmental disorder defined by atypical social behaviour, of which atypical social attention behaviours are among the earliest clinical markers (Volkmar et al., 1997). Eye tracking studies using still images and movie clips have provided a method for the precise quantification of atypical social attention in ASD. This is generally characterised by diminished viewing of the most socially pertinent regions (eyes), and increased viewing of less socially informative regions (body, background, objects) (Klin et al., 2002; Riby & Hancock, 2008, 2009). Ecological validity within eye tracking studies has become an increasingly important issue. As of yet, however, little is known about the precise nature of the atypicalities of social attention in ASD in real-life. Objectives: To capture and quantify gaze patterns for children with an ASD within a real life setting, compared to two Typically Developing (TD) comparison groups. Methods: Nine children with an ASD were compared to two age matched TD groups – a verbal (N=9) and a non-verbal (N=9) comparison group. A real-life scenario was created involving an experimenter posing as a magician, and consisted of 3 segments: a conversation segment; a magic trick segment; and a puppet segment. The first segment explored children’s attentional preferences during a real-life conversation; the magic trick segment explored children’s use of the eyes as a communicative cue, and the puppet segment explored attention capture. Finally, part of the puppet section explored children’s use of facial information in response to an unexpected event. Results: The most striking difference between the groups was the diminished viewing of the eyes by the ASD group in comparison to both control groups. This was found particularly during the conversation segment, but also during the magic trick segment, and during the puppet segment. When in conversation, participants with ASD were found to spend a greater proportion time looking off-screen, in comparison to TD participants. There was also a tendency for the ASD group to spend a greater proportion of time looking to the mouth of the experimenter. During the magic trick segment, despite the fact that the eyes were not predictive of a correct location, both TD comparison groups continued to use the eyes as a communicative cue, whereas the ASD group did not. In the puppet segment, all three groups spent a similar amount of time looking between the puppet and regions of the experimenter’s face. However, in response to an unexpected event, the ASD group were significantly slower to fixate back on the experimenter’s face. Conclusions: The results demonstrate the reduced salience of socially pertinent information for children with ASD in real life, and they provide support for the findings from previous eye tracking studies involving scene viewing. However, the results also highlight a pattern looking off-screen for both the TD and ASD groups. This eye movement behaviour is likely to be associated specifically with real-life interaction, as it has functional relevance (Doherty-Sneddon et al., 2002). However, the fact that it is significantly increased in the ASD group has implications for their understanding of real life social interactions.

Tracking the Links to Social Functioning: Asperger Syndrome and Gaze Patterns

The application of Eye Tracking (ET) to the study of social functioning in Asperger Syndrome (AS) provides a unique perspective into social attention and cognition in this atypical neurodevelopmental group. Research in this area has shown how ET can capture social attention atypicalities within this group, such as diminished fixations to the eye region when viewing still images and movie clips; increased fixation to the mouth region; reduced face gaze. Issues exist, however, within the literature, where the type (static/dynamic) and the content (ecological validity) of stimuli used appear to affect the nature of the gaze patterns reported. Objectives: Our research aims were: using the same group of adolescents with AS, to compare their viewing patterns to age and IQ matched typically developing (TD) adolescents using stimuli considered to represent a hierarchy of ecological validity, building from static facial images; through a non-verbal movie clip; through verbal footage from real-life conversation; to eye tracking during real-life conversation. Methods: Eleven participants with AS were compared to 11 TD adolescents, matched for age and IQ. In Study 1, participants were shown 2 sets of static facial images (emotion faces, still images taken from the dynamic clips). In Study 2, three dynamic clips were presented (1 non-verbal movie clip, 2 verbal footage from real-life conversation). Study 3 was an exploratory study of eye tracking during a real-life conversation. Eye movements were recorded via a HiSpeeed (240Hz) SMI eye tracker fitted with chin and forehead rests. Various methods of analysis were used, including a paradigm for temporal analysis of the eye movement data. Results: Results from these studies confirmed that the atypical nature of social attention in AS was successfully captured by this paradigm. While results differed across stimulus sets,
collectively they demonstrated how individuals with AS failed to focus on the most socially relevant aspects of the various stimuli presented. There was also evidence that the eye movements of the AS group were atypically affected by the presence of motion and verbal information. Discriminant Function Analysis demonstrated that the ecological validity of stimuli was an important factor in identifying atypicalities associated with AS, with more accurate classifications of AS and TD groups occurring for more naturalistic stimuli (dynamic rather than static). Graphical analysis of temporal sequences of eye movements revealed the atypical manner in which AS participants followed interactions within the dynamic stimuli. Taken together with data on the order of gaze patterns, more subtle atypicalities were detected in the gaze behaviour of AS individuals towards more socially pertinent regions of the dynamic stimuli. Conclusions: These results have potentially important implications for our understanding of deficits in Asperger Syndrome, as they show that, with more naturalistic stimuli, subtle differences in social attention can be detected that

Initial validation of the Behavioral Indicators of Infant Pain (BIIP)

Accurate pain assessment in preterm infants in the neonatal intensive care unit (NICU) is complex. Infants who are born at early gestational ages (GA), and who have had greater early pain exposure, have dampened facial responses which may lead to under-treatment. Since behavioral and physiological responses to pain in infants are often dissociated, using multidimensional scales which combine these indicators into a single score may limit our ability to determine the effects of interventions on each system. Our aim was to design a unidimensional scale which would combine the relatively most specific, individual, behavioral indicators for assessing acute pain in this population. The Behavioral Indicators of Infant Pain (BIIP) combines sleep/wake states, 5 facial actions and 2 hand actions. Ninety-two infants born between 23 and 32 weeks GA were assessed during 3, 1 min Phases of blood collection. Outcome measures included changes in BIIP and in Neonatal Infant Pain Scale (NIPS) scores coded in real time from continuous bedside video recordings; changes in heart rate (HR) were obtained using custom physiological processing software. Scores on the BIIP changed significantly across Phases of blood collection (p

Neonatal procedural pain exposure predicts lower cortisol and behavioral reactivity in preterm infants in the NICU

Data from animal models indicate that neonatal stress or pain can permanently alter subsequent behavioral and/or physiological reactivity to stressors. However, cumulative effects of pain related to acute procedures in the neonatal intensive care unit (NICU) on later stress and/or pain reactivity has received limited attention. The objective of this study is to examine relationships between prior neonatal pain exposure (number of skin breaking procedures), and subsequent stress and pain reactivity in preterm infants in the NICU. Eighty-seven preterm infants were studied at 32 (+/-1 week) postconceptional age (PCA). Infants who received analgesia or sedation in the 72 h prior to each study, or any postnatal dexamethasone, were excluded. Outcomes were infant responses to two different stressors studied on separate days in a repeated measures randomized crossover design: (1) plasma cortisol to stress of a fixed series of nursing procedures; (2) behavioral (Neonatal Facial Coding System; NFCS) and cardiac reactivity to pain of blood collection. Among infants born

Are there developmentally distinct motor indicators of pain in preterm infants?

The aims of this study were to examine preterm infant reactions to pain in detail over prolonged time periods using multiple measures, and to assess the value of including specific body movements of the Neonatal Individualized Developmental Care and Assessment Program (NIDCAP) system to evaluate pain. Ten preterm infants born at 31 weeks mean gestational age (GA) and mean birth weight 1676 g were studied during a routine blood collection in a Level III neonatal intensive care unit (NICU). At 32-week post-conceptional age, computerized physiologic and video recordings were obtained continuously for 60 min (prior to, during and after lance). Motor and facial behaviors were coded independently, using the NIDCAP and the NFCS (Neonatal Facial Coding System), respectively, and compared with heart rate (HR) and oxygen saturation responses. Of the movements hypothesized to be stress cues in the NIDCAP model, extension of arms and legs (80%) and finger splay (70%) were the most common following lance. Contrary to the model, most infants (70%) had lower incidence of twitches and startles post-lance compared to baseline. Whereas all infants showed some NFCS response to lance, for three infants, the magnitude was low. HR increased and oxygen saturation decreased post-lance. Infants with more prior pain exposure, lower Apgar, and lower GA at birth, displayed more motor stress cues but less facial activity post-lance. Extension of extremities and finger splay, but not twitches and startles, from the NIDCAP, appear to be stress cues and show promise as clinical pain indicators to supplement facial and physiological pain measures in preterm infants.

Children's judgements about pain at age 8-10 years:do extremely low birthweight (<or = 1000 g) children differ from full birthweight peers?

Children's judgements about pain at age 8-10 years were examined comparing two groups of children who had experienced different exposure to nociceptive procedures in the neonatal period: extremely low birthweight (ELBW) <or = 1000 g (N = 47) and full birthweight (FBW) > or = 2500 g (N = 37). The 24 pictures that comprise the Pediatric Pain Inventory, depicting events in four settings: medical, recreational, daily living, and psychosocial, were used as the pain stimuli. The subjects rated pain intensity using the Color Analog Scale and pain affect using the Facial Affective Scale. Child IQ and maternal education were statistically adjusted in group comparisons. Pain intensity and pain affect related to activities of daily living and recreation were significantly higher than psychosocial and medically related pain on both scales in both groups of children. Although the two groups of children did not differ overall in their perceptions of pain intensity or affect, the ELBW children rated medical pain intensity significantly higher than psychosocial pain, unlike the FBW group. Also, duration of neonatal intensive care unit stay for the ELBW children was related to increased pain affect ratings in recreational and daily living settings. Despite altered response to pain in the early years reported by parents, on the whole at 8-10 years of age ELBW children judged pain in pictures similarly to their term peers. However, differences were evident, which suggests that studies are needed of biobehavioural reactivity to pain beyond infancy, as well as research into beliefs, attitudes, and perceptions about pain during the course of childhood in formerly ELBW children.

Judging pain in infants:behavioural, contextual, and developmental determinants

Caretakers intuitively use various sources of evidence when judging infant pain, but the relative importance of salient cues has received little attention. This investigation examined the predictive significance for judgements of painful discomfort in preterm and full-term neonates of behavioural (facial activity and body movement), contextual (invasiveness of the procedure), and developmental (gestational age) information. Judges viewed videotapes showing infants varying in the foregoing characteristics undergoing heel incisions for routine blood sampling purposes. Findings indicated all but the contextual information contributed uniquely to judgements of pain, with facial activity accounting for the most unique variance (35%), followed by bodily activity and gestational age, each accounting for 3% and 1% of the judgmental variance, respectively. Generally, 71% of the variance in ratings of pain could be predicted using facial activity alone, compared to 30% of the variance using bodily activity alone, 19% by relying on context alone, and 8% by referring to gestational age alone. Noteworthy was the tendency to judge early preterm infants to be experiencing less pain even though they were subjected to the same invasive procedure as the older infants. This finding also runs counter to evidence from developmental neurobiology which indicates that preterm newborns may be hypersensitive to invasive procedures.

Pain in the preterm neonate:behavioural and physiological indices

The impact of invasive procedures on preterm neonates has received little systematic attention. We examined facial activity, body movements, and physiological measures in 56 preterm and full-term newborns in response to heel lancing, along with comparison preparatory and recovery intervals. The measures were recorded in special care and full-term nurseries during routine blood sampling. Data analyses indicated that in all measurement categories reactions of greatest magnitude were to the lancing procedure. Neonates with gestational ages as short as 25-27 weeks displayed physiological responsivity to the heel lance, but only in the heart rate measure did this vary with gestational age. Bodily activity was diminished in preterm neonates in general, relative to full-term newborns. Facial activity increased with the gestational age of the infant. Specificity of the response to the heel lance was greatest on the facial activity measure. Identification of pain requires attention to gestational age in the preterm neonate.

Developmental changes in pain expression in premature, full-term, two- and four-month-old infants

The purpose of this study was to examine the behavioural responses of infants to pain stimuli across different developmental ages. Eighty infants were included in this cross-sectional design. Four subsamples of 20 infants each included: (1) premature infants between 32 and 34 weeks gestational age undergoing heel-stick procedure; (2) full-term infants receiving intramuscular vitamin K injection; (3) 2-month-old infants receiving subcutaneous injection for immunisation against DPT; and (4) 4-month-old infants receiving subcutaneous injection for immunisation against DPT. Audio and video recordings were made for 15 sec from stimulus. Cry analysis was conducted on the first full expiratory cry by FFT with time and frequency measures. Facial action was coded using the Neonatal Facial Action Coding System (NFCS). Results from multivariate analysis showed that premature infants were different from older infants, that full-term newborns were different from others, but that 2- and 4-month-olds were similar. The specific variables contributing to the significance were higher pitched cries and more horizontal mouth stretch in the premature group and more taut tongue in the full-term newborns. The results imply that the premature infant has the basis for communicating pain via facial actions but that these are not well developed. The full-term newborn is better equipped to interact with his caretakers and express his distress through specific facial actions. The cries of the premature infant, however, have more of the characteristics that are arousing to the listener which serve to alert the caregiver of the state of distress from pain.

Robust Multimodal Person Identification With Limited Training Data

This paper presents a novel method of audio-visual feature-level fusion for person identification where both the speech and facial modalities may be corrupted, and there is a lack of prior knowledge about the corruption. Furthermore, we assume there are limited amount of training data for each modality (e.g., a short training speech segment and a single training facial image for each person). A new multimodal feature representation and a modified cosine similarity are introduced to combine and compare bimodal features with limited training data, as well as vastly differing data rates and feature sizes. Optimal feature selection and multicondition training are used to reduce the mismatch between training and testing, thereby making the system robust to unknown bimodal corruption. Experiments have been carried out on a bimodal dataset created from the SPIDRE speaker recognition database and AR face recognition database with variable noise corruption of speech and occlusion in the face images. The system's speaker identification performance on the SPIDRE database, and facial identification performance on the AR database, is comparable with the literature. Combining both modalities using the new method of multimodal fusion leads to significantly improved accuracy over the unimodal systems, even when both modalities have been corrupted. The new method also shows improved identification accuracy compared with the bimodal systems based on multicondition model training or missing-feature decoding alone.

Modeling continuous self-report measures of perceived emotion using generalized additive mixed models

Emotion research has long been dominated by the “standard method” of displaying posed or acted static images of facial expressions of emotion. While this method has been useful it is unable to investigate the dynamic nature of emotion expression. Although continuous self-report traces have enabled the measurement of dynamic expressions of emotion, a consensus has not been reached on the correct statistical techniques that permit inferences to be made with such measures. We propose Generalized Additive Models and Generalized Additive Mixed Models as techniques that can account for the dynamic nature of such continuous measures. These models allow us to hold constant shared components of responses that are due to perceived emotion across time, while enabling inference concerning linear differences between groups. The mixed model GAMM approach is preferred as it can account for autocorrelation in time series data and allows emotion decoding participants to be modelled as random effects. To increase confidence in linear differences we assess the methods that address interactions between categorical variables and dynamic changes over time. In addition we provide comments on the use of Generalized Additive Models to assess the effect size of shared perceived emotion and discuss sample sizes. Finally we address additional uses, the inference of feature detection, continuous variable interactions, and measurement of ambiguity.

“Shall we playe the good girles”: Playing Girls, Performing Girlhood on Early Modern Stages

This essay investigates the extent to which girlhood functions as a queer category in two theatrical representations of schoolgirls in early seventeenth-century England. It focuses on the depictions of schoolgirls in the anonymous The Wit of a Woman (1604), written for the all-male stage of the professional theatre, and in Robert White’s masque, Cupid’s Banishment (1617), performed by the young Ladies of Deptford Hall before Queen Anna of Denmark, to examine the intersections of age, gender, sexuality and education in early modern concepts of girlhood. Situating these plays within wider debates about female education and the history of the contested role of performance in the schooling of early modern girls, it argues that they deploy the category of girlhood to demonstrate the subversive potential of educating girls. Yet, this essay proposes, these plays simultaneously reveal the potential agency of young women who manipulate girlhood to claim their distinct sexual, aged and gendered states as girls. It argues that early modern girlhood is a state that might be performed by young women to disrupt normative expectations of feminine behaviour and desire. Placing dramatic representations of schoolgirls and the experiences of schoolgirls on the early modern stage side by side, this essay demonstrates that the schoolroom and performance are sites in which this transgressive potential is realised.