Social signal processing for pain monitoring using a hidden conditional random field

utomatic pain monitoring has the potential to greatly improve patient diagnosis and outcomes by providing a continuous objective measure. One of the most promising methods is to do this via automatically detecting facial expressions. However, current approaches have failed due to their inability to: 1) integrate the rigid and non-rigid head motion into a single feature representation, and 2) incorporate the salient temporal patterns into the classification stage. In this paper, we tackle the first problem by developing a “histogram of facial action units” representation using Active Appearance Model (AAM) face features, and then utilize a Hidden Conditional Random Field (HCRF) to overcome the second issue. We show that both of these methods improve the performance on the task of pain detection in sequence level compared to current state-of-the-art-methods on the UNBC-McMaster Shoulder Pain Archive.

Digital social signal processing - theorectical underpinning and research agenda

Organizations make increasingly use of social media in order to compete for customer awareness and improve the quality of their goods and services. Multiple techniques of social media analysis are already in use. Nevertheless, theoretical underpinnings and a sound research agenda are still unavailable in this field at the present time. In order to contribute to setting up such an agenda, we introduce digital social signal processing (DSSP) as a new research stream in IS that requires multi-facetted investigations. Our DSSP concept is founded upon a set of four sequential activities: sensing digital social signals that are emitted by individuals on social media; decoding online data of social media in order to reconstruct digital social signals; matching the signals with consumers’ life events; and configuring individualized goods and service offerings tailored to the individual needs of customers. We further contribute to tying loose ends of different research areas together, in order to frame DSSP as a field for further investigation. We conclude with developing a research agenda.

Politeness and social signals

In the literature, politeness has been researched within many disciplines. Although Brown and Levinson’s theory of politeness (1978, 1987) is often cited, it is primarily a linguistic theory and has been criticized for its lack of generalizability to all cultures. Consequently, there is a need for a more comprehensive approach to understand and explain politeness. We suggest applying a social signal framework that considers politeness as a communicative state. By doing so, we aim to unify and explain politeness and its corresponding research and identify further research needed in this area.

The use of eye-tracking to explore social difficulties in cognitively able students with autism spectrum disorder: A pilot study

Individuals with autism spectrum disorder do not just 'grow out of' their early difficulties in understanding the social world. Even for those who are cognitively able, autism-related difficulties continue into adulthood. Atypicalities attending to and interpreting communicative signals from others can provide barriers to success in education, employment and relationships. In the current study, we use eye-tracking during real social interaction to explore attention to social cues (e.g. face, eyes, mouth) and links to social awareness in a group of cognitively able University students with autism spectrum disorder and typically developing students from the same University. During the interaction, students with autism spectrum disorder showed less eye fixation and more mouth fixation than typically developing students. Importantly, while 63% of typically developing participants reported thinking they were deceived about the true nature of the interaction, only 9% of autism spectrum disorder participants picked up this subtle social signal. We argue that understanding how these social attentional and social awareness difficulties manifest during adulthood is important given the growing number of adults with autism spectrum disorder who are attending higher level education. These adults may be particularly susceptible to drop-out due to demands of coping in situations where social awareness is so important.

Laughter Induction Techniques Suitable for Generating Motion Capture Data of Laughter Associated Body Movements

Laughter is a frequently occurring social signal and an important part of human non-verbal communication. However it is often overlooked as a serious topic of scientific study. While the lack of research in this area is mostly due to laughter’s non-serious nature, it is also a particularly difficult social signal to produce on demand in a convincing manner; thus making it a difficult topic for study in laboratory settings. In this paper we provide some techniques and guidance for inducing both hilarious laughter and conversational laughter. These techniques were devised with the goal of capturing mo- tion information related to laughter while the person laughing was either standing or seated. Comments on the value of each of the techniques and general guidance as to the importance of atmosphere, environment and social setting are provided.

An Event Driven Fusion Approach for Enjoyment Recognition in Real-time

Social signals and interpretation of carried information is of high importance in Human Computer Interaction. Often used for affect recognition, the cues within these signals are displayed in various modalities. Fusion of multi-modal signals is a natural and interesting way to improve automatic classification of emotions transported in social signals. Throughout most present studies, uni-modal affect recognition as well as multi-modal fusion, decisions are forced for fixed annotation segments across all modalities. In this paper, we investigate the less prevalent approach of event driven fusion, which indirectly accumulates asynchronous events in all modalities for final predictions. We present a fusion approach, handling short-timed events in a vector space, which is of special interest for real-time applications. We compare results of segmentation based uni-modal classification and fusion schemes to the event driven fusion approach. The evaluation is carried out via detection of enjoyment-episodes within the audiovisual Belfast Story-Telling Corpus.

Human Perception of Laughter from Context-free Whole Body Motion Dynamic Stimuli

Laughter is a ubiquitous social signal in human interactions yet it remains understudied from a scientific point of view. The need to understand laughter and its role in human interactions has become more pressing as the ability to create conversational agents capable of interacting with humans has come closer to a reality. This paper reports on three aspects of the human perception of laughter when context has been removed and only the body information from the laughter episode remains. We report on ability to categorise the laugh type and the sex of the laugher; the relationship between personality factors with laughter categorisation and perception; and finally the importance of intensity in the perception and categorisation of laughter.

Food intake, growth and social interactions of signal crayfish, Pacifastacus leniusculus (Dana)

The knowledge about the optimal rearing conditions, such as water temperature and quality, photoperiod and density, with the understanding of animal nutritional requirements forms the basis of economically stable aquaculture for freshwater crayfish. However, the shift from a natural environment to effective culture conditions induces several changes, not only at the population level, but also at the individual level. The social contacts between conspecifics increase with increasing animal density. The competition for limited resources (e.g. food, shelter, mates) is more severe with the presence of agonistic behaviour and may lead to unequal distribution of these. The objectives of this study were to: 1) study the distribution of a common food resource between communally reared signal crayfish (Pacifastacus leniusculus) and to assign potential feeding hierarchy on the basis of individual food intake measurements, 2) explore the possibilities of size distribution manipulations to affect population dynamics and food intake to improve growth and survival in culture and 3) study the effect of food ration and spatial distribution on food intake and to explore the effect of temperature and food ration on growth and body composition of freshwater crayfish. The feeding ranks between animals were assigned with a new method for individual food intake measurement of communally reared crayfish. This technique has a high feasibility and a great potential to be applied in crayfish aquaculture studies. In this study, signal crayfish showed high size-related variability in food consumption both among individuals within a group (inter-individual) and within individual day-to-day variation (intra-individual). Increased competition for food led to an unequal distribution of this resource and this may be a reason for large growth differences between animals. The consumption was significantly higher when reared individually in comparison with communal housing. These results suggest that communally housed crayfish form a feeding hierarchy and that the animal size is the major factor controlling the position in this hierarchy. The optimisation of the social environment ( social conditions ) was evaluated in this study as a new approach to crayfish aquaculture. The results showed that the absence of conspecifics (individual rearing vs. communal housing) affects growth rate, food intake and the proportion of injured animals, whereas size variation between animals influences the number and duration of agonistic encounters. In addition, animal size had a strong influence on the fighting success of signal crayfish reared in a social milieu with a wide size variation of conspecifics. Larger individuals initiated and won most of the competitions, which suggests size-based social hierarchy of P. leniusculus. This is further supported by the fact that the length and weight gain of smaller animals increased after size grading, maybe because of a better access to the food resource due to diminished social pressure. However, the high dominance index was not based on size under conditions of limited size variation, e.g. those characteristic of restocked natural populations and aquaculture, indicating the important role of behaviour on social hierarchy.

“Signal, image and video processing” special issue: Semantic representations for social behavior analysis in video surveillance systems

No abstract available

Social and environmental regulation of signal plasticity and signal reliability in the electric fish Brachyhypopomus gauderio

The balance between the costs and benefits of conspicuous signals ensures that the expression of those signals is related to the quality of the bearer. Plastic signals could enable males to maximize conspicuous traits to impress mates and competitors, but reduce the expression of those traits to minimize signaling costs, potentially compromising the information conveyed by the signals. ^ I investigated the effect of signal enhancement on the information coded by the biphasic electric signal pulse of the gymnotiform fish Brachyhypopomus gauderio. Increases in population density drive males to enhance the amplitude of their signals. I found that signal amplitude enhancement improves the information about the signaler's size. Furthermore, I found that the elongation of the signal's second phase conveys information about androgen levels in both sexes, gonad size in males and estrogen levels in females. Androgens link the duration of the signal's second phase to other androgen-mediated traits making the signal an honest indicator of reproductive state and aggressive motivation. ^ Signal amplitude enhancement facilitates the assessment of the signaler's resource holding potential, important for male-male interactions, while signal duration provides information about aggressive motivation to same-sex competitors and reproductive state to the opposite sex. Moreover, I found that female signals also change in accordance to the social environment. Females also increase the amplitude of their signal when population density increases and elongate the duration of their signal's second phase when the sex ratio becomes female-biased. Indicating that some degree of sexual selection operates in females. ^ I studied whether male B. gauderio use signal plasticity to reduce the cost of reproductive signaling when energy is limited. Surprisingly, I found that food limitation promotes the investment in reproduction manifested as signal enhancement and elevated androgen levels. The short lifespan and single breeding season of B. gauderio diminishes the advantage of energy savings and gives priority to sustaining reproduction. I conclude that the electric signal of B. gauderio provides reliable information about the signaler, the quality of this information is reinforced rather than degraded with signal enhancement.^

Of dogs, fish, birds and turds : the social construction of sewage pollution by recreational boaters

Animals are often used as ‘evidence’ of marine pollution. Take for instance the ubiquitous images of miserable oil-soaked marine birds following high profile oil spills such as the Exxon Valdez, Pacific Adventurer and Deepwater Horizon incidents or the images of bloated floating fish carcasses which are used to signal the presence of toxic pollutants. In recent years waste discharges from vessels have come under increased public and regulatory scrutiny both in Australia and around the world. International, regional, national and local restrictions are becoming more stringent for high profile marine pollutants such as oil as well as previously overlooked vessel-sourced pollutants such as sewage. Drawing upon media reports and recreational boater responses to government attempts to regulate the discharge of sewage from recreational vessels, this paper considers the important role played by animals in constructions of marine pollution by sewage and attributions of blame for this pollution. Specifically, this study found that recreational boat owners disputed claims their sewage management practices posed an environmental threat arguing that the sewage discharged was readily and eagerly consumed by fish in the receiving environment. Boat owners also argued that increased levels of bacteria which indicate the presence of faeces within the marine environment could be directly attributed to the excrement of marine mammals and birds or were the result of dog faeces being washed through municipal storm water systems rather than the result of vessel discharges. By contrast the contamination of oysters was provided as evidence of sewage pollution by other stakeholders.

Characterising an ECG signal using statistical modelling : a feasibility study

For clinical use, in electrocardiogram (ECG) signal analysis it is important to detect not only the centre of the P wave, the QRS complex and the T wave, but also the time intervals, such as the ST segment. Much research focused entirely on qrs complex detection, via methods such as wavelet transforms, spline fitting and neural networks. However, drawbacks include the false classification of a severe noise spike as a QRS complex, possibly requiring manual editing, or the omission of information contained in other regions of the ECG signal. While some attempts were made to develop algorithms to detect additional signal characteristics, such as P and T waves, the reported success rates are subject to change from person-to-person and beat-to-beat. To address this variability we propose the use of Markov-chain Monte Carlo statistical modelling to extract the key features of an ECG signal and we report on a feasibility study to investigate the utility of the approach. The modelling approach is examined with reference to a realistic computer generated ECG signal, where details such as wave morphology and noise levels are variable.