A Digital Signal Processing Approach for Affective Sensing of a Computer User through Pupil Diameter Monitoring

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.

A framework for investigating the use of face features to identify spontaneous emotions

Emotion-based analysis has raised a lot of interest, particularly in areas such as forensics, medicine, music, psychology, and human-machine interface. Following this trend, the use of facial analysis (either automatic or human-based) is the most common subject to be investigated once this type of data can easily be collected and is well accepted in the literature as a metric for inference of emotional states. Despite this popularity, due to several constraints found in real world scenarios (e.g. lightning, complex backgrounds, facial hair and so on), automatically obtaining affective information from face accurately is a very challenging accomplishment. This work presents a framework which aims to analyse emotional experiences through naturally generated facial expressions. Our main contribution is a new 4-dimensional model to describe emotional experiences in terms of appraisal, facial expressions, mood, and subjective experiences. In addition, we present an experiment using a new protocol proposed to obtain spontaneous emotional reactions. The results have suggested that the initial emotional state described by the participants of the experiment was different from that described after the exposure to the eliciting stimulus, thus showing that the used stimuli were capable of inducing the expected emotional states in most individuals. Moreover, our results pointed out that spontaneous facial reactions to emotions are very different from those in prototypic expressions due to the lack of expressiveness in the latter.

The Consequences of Fiscal Stimulus on Public Debt : A Historical Perspective

Peer reviewed

The Consequences of Fiscal Stimulus on Public Debt : A Historical Perspective

Peer reviewed

A Study of the Behavioural and Affective Profile of Inpatient Adolescent Girls Presenting a Restrictive Anorexia Nervosa

Essai doctoral d'intégration présenté à la Faculté des Études Supérieures et Postdoctorales en vue de l'obtention du grade de Docteur en psychologie (D.Psy.), en psychologie clinique

The Anxious Heart of Injustice: Negative Affective Responses to Disabilities

Essai / Essay

Stimulus Integration and Parsing in the Primate Auditory Midbrain

Integrating information from multiple sources is a crucial function of the brain. Examples of such integration include multiple stimuli of different modalties, such as visual and auditory, multiple stimuli of the same modality, such as auditory and auditory, and integrating stimuli from the sensory organs (i.e. ears) with stimuli delivered from brain-machine interfaces.

The overall aim of this body of work is to empirically examine stimulus integration in these three domains to inform our broader understanding of how and when the brain combines information from multiple sources.

First, I examine visually-guided auditory, a problem with implications for the general problem in learning of how the brain determines what lesson to learn (and what lessons not to learn). For example, sound localization is a behavior that is partially learned with the aid of vision. This process requires correctly matching a visual location to that of a sound. This is an intrinsically circular problem when sound location is itself uncertain and the visual scene is rife with possible visual matches. Here, we develop a simple paradigm using visual guidance of sound localization to gain insight into how the brain confronts this type of circularity. We tested two competing hypotheses. 1: The brain guides sound location learning based on the synchrony or simultaneity of auditory-visual stimuli, potentially involving a Hebbian associative mechanism. 2: The brain uses a ‘guess and check’ heuristic in which visual feedback that is obtained after an eye movement to a sound alters future performance, perhaps by recruiting the brain’s reward-related circuitry. We assessed the effects of exposure to visual stimuli spatially mismatched from sounds on performance of an interleaved auditory-only saccade task. We found that when humans and monkeys were provided the visual stimulus asynchronously with the sound but as feedback to an auditory-guided saccade, they shifted their subsequent auditory-only performance toward the direction of the visual cue by 1.3-1.7 degrees, or 22-28% of the original 6 degree visual-auditory mismatch. In contrast when the visual stimulus was presented synchronously with the sound but extinguished too quickly to provide this feedback, there was little change in subsequent auditory-only performance. Our results suggest that the outcome of our own actions is vital to localizing sounds correctly. Contrary to previous expectations, visual calibration of auditory space does not appear to require visual-auditory associations based on synchrony/simultaneity.

My next line of research examines how electrical stimulation of the inferior colliculus influences perception of sounds in a nonhuman primate. The central nucleus of the inferior colliculus is the major ascending relay of auditory information before it reaches the forebrain, and thus an ideal target for understanding low-level information processing prior to the forebrain, as almost all auditory signals pass through the central nucleus of the inferior colliculus before reaching the forebrain. Thus, the inferior colliculus is the ideal structure to examine to understand the format of the inputs into the forebrain and, by extension, the processing of auditory scenes that occurs in the brainstem. Therefore, the inferior colliculus was an attractive target for understanding stimulus integration in the ascending auditory pathway.

Moreover, understanding the relationship between the auditory selectivity of neurons and their contribution to perception is critical to the design of effective auditory brain prosthetics. These prosthetics seek to mimic natural activity patterns to achieve desired perceptual outcomes. We measured the contribution of inferior colliculus (IC) sites to perception using combined recording and electrical stimulation. Monkeys performed a frequency-based discrimination task, reporting whether a probe sound was higher or lower in frequency than a reference sound. Stimulation pulses were paired with the probe sound on 50% of trials (0.5-80 µA, 100-300 Hz, n=172 IC locations in 3 rhesus monkeys). Electrical stimulation tended to bias the animals’ judgments in a fashion that was coarsely but significantly correlated with the best frequency of the stimulation site in comparison to the reference frequency employed in the task. Although there was considerable variability in the effects of stimulation (including impairments in performance and shifts in performance away from the direction predicted based on the site’s response properties), the results indicate that stimulation of the IC can evoke percepts correlated with the frequency tuning properties of the IC. Consistent with the implications of recent human studies, the main avenue for improvement for the auditory midbrain implant suggested by our findings is to increase the number and spatial extent of electrodes, to increase the size of the region that can be electrically activated and provide a greater range of evoked percepts.

My next line of research employs a frequency-tagging approach to examine the extent to which multiple sound sources are combined (or segregated) in the nonhuman primate inferior colliculus. In the single-sound case, most inferior colliculus neurons respond and entrain to sounds in a very broad region of space, and many are entirely spatially insensitive, so it is unknown how the neurons will respond to a situation with more than one sound. I use multiple AM stimuli of different frequencies, which the inferior colliculus represents using a spike timing code. This allows me to measure spike timing in the inferior colliculus to determine which sound source is responsible for neural activity in an auditory scene containing multiple sounds. Using this approach, I find that the same neurons that are tuned to broad regions of space in the single sound condition become dramatically more selective in the dual sound condition, preferentially entraining spikes to stimuli from a smaller region of space. I will examine the possibility that there may be a conceptual linkage between this finding and the finding of receptive field shifts in the visual system.

In chapter 5, I will comment on these findings more generally, compare them to existing theoretical models, and discuss what these results tell us about processing in the central nervous system in a multi-stimulus situation. My results suggest that the brain is flexible in its processing and can adapt its integration schema to fit the available cues and the demands of the task.

A Study of the Behavioural and Affective Profile of Inpatient Adolescent Girls Presenting a Restrictive Anorexia Nervosa

Essai doctoral d'intégration présenté à la Faculté des Études Supérieures et Postdoctorales en vue de l'obtention du grade de Docteur en psychologie (D.Psy.), en psychologie clinique

The Anxious Heart of Injustice: Negative Affective Responses to Disabilities

Essai / Essay

EXHAUSTING AFFECTS: THE NEGOTIATION OF AFFECTIVE LABOUR, NEOLIBERAL SUBJECTIVITY, AND FEMINIST POLITICS AMONG ANTI-VIOLENCE WORKERS

This thesis explores the affective and political life of anti-violence labour, with particular attention to the ways that neoliberalism comes to bear on subjectivity, embodiment, and relationality among women responding to violence. In fall of 2015, I conducted qualitative interviews with six women engaged in the work of responding to violence. The participants in this project articulated rich descriptions of the affective life of neoliberalism and the demands of neoliberal subjectivity, drawing particular attention to the affective labour involved in navigating the political complexities of anti-violence organizations, negotiating burnout, and affectively self-managing in order to meet norms of professionalism. Bringing participant narratives into conversations with feminist theories of affect, I argue for an account affective labour that centers the specific, materially embodied experiences of that work and for an account of neoliberalism as a system of embodied and affective pressures.

Moments of Collusion? Close readings of affective, hidden moments within feminist research

This paper addresses affective ‘moments of collusion’ present in feminist research relationships, and contextualises these seemingly personal encounters within a wider systematic framework of the early career researcher and the increasingly neoliberal climate of academia. Focusing on the temporal transition from doctoral research to postdoctorate research positions immediately post-PhD, this paper questions the concept of collusion within (immersive) fieldwork, and examines the delicate and complex question of who is colluding with whom, and for what purpose at different times within the early career academic journey. Specifically, this paper focuses on how the increasing pressures of the neoliberal university play out on our emotions and bodies during fieldwork, an area which still requires attention within the growing critiques of the affects of neo-liberalism in Higher Education. Using personal case studies as springboard for a far wider and important discussion, this paper situates such methodological dilemmas within a broader temporal framework of the increasingly precarious nature of early career academics, where ‘moments of collusion’ may be the only way to keep your head above water.

'Dancing the Downward Slide: Spaces of Affective Adjacency in Rian and Bird with Boy'

This chapter examines how the choreography of affect in two dance theatre works creates a space of affective adjacency—a space in which the building of an alternative structure of feeling and an alternative economy of the body can be experienced. Focusing on the choreographic use of repetition in Junk Ensemble’s Bird With Boy (2011) and Fabulous Beast Dance Theatre’s Rian (2011), it shows how the work required to build an alternative affective space can become visible. Although affect is most often viewed as a preconscious, ephemeral phenomenon (a passage of intensities), that can have little or no lasting impact on socio-political action, theorists such as Megan Watkins have argued for a consideration of the ‘cumulative aspects of affect’. Highlighting Spinoza’s distinction between affectus (the capacity for a body to affect and be affected), and affectio (the impact the affecting body leaves on the affected), Watkins points out that affectio can ‘leave a residue’ allowing for the ‘capacity of affect to be retained, to accumulate, to form dispositions and thus shape subjectivities’. The choreography of repetition in Bird With Boy and Rian presents sites for an examination of this accumulation of affect and its capacity not only to form and shape dispositions, but also, as Lauren Berlant suggests, ‘to move along and make worlds, situations, and environments’.

Confluent Learning: Using a Design Approach to Develop Cognitive Abilities and Enhance Affective Capacities Through Change Management Curriculum

Purpose The aim of the study is to explore the role of confluent learning in supporting the development of change management knowledge, skills and attitudes and to inform the creation of a conceptual model based upon a priori and a posteriori knowledge gained from literature and the research. Design/methodology/approach The research adopts qualitative approach based on reflective inquiry methodology. There are two primary data sources, interviews with learners and the researchers’ reflective journals on learners’ opinions. Findings The confluent learning approach helped to stimulate affective states (e.g. interest and appreciation) to further reinforce cognitive gains (e.g. retention of knowledge) as a number of higher order thinking skills were further developed. The instructional design premised upon confluent learning enabled learners to further appreciate the complexities of change management. Research implications/ limitations The confluent learning approach offers another explanation to how learning takes place, contingent upon the use of a problem solving framework, instructional design and active learning in developing inter- and trans-disciplinary competencies. Practical implications This study not only explains how effective learning takes place but is also instructive to learning and teaching, and human resource development (HRD) professionals in curriculum design and the potential benefits of confluent learning. Social implications The adoption of a confluent learning approach helps to re-naturalise learning that appeals to learners affect. Originality/value This research is one of the few studies that provide an in-depth exploration of the use of confluent learning and how this approach co-develops cognitive abilities and affective capacity in the creation of a conceptual model.