"What" versus "how" in nonvisual whole-body movement

Dissociable processes for conscious perception (“what” processing) and guidance of action (“how” processing) have been identified in visual, auditory, and somatosensory systems. The present study was designed to find similar dissociation within whole-body movements in which the presence of vestibular information creates a unique perceptual condition. In two experiments, blindfolded participants walked along a linear path and specified the walked distance by verbally estimating it (“what” measure) and by pulling a length of tape that matched the walked distance (“how” measure). Although these two measures yielded largely comparable responses under a normal walking condition, variability in verbal estimates showed a qualitatively different pattern from that in tape-pulling when sensory input into walking was altered by having participants wear a heavy backpack. This suggests that the “what” versus “how” dissociation exists in whole-body movements as well, supporting a claim that it is a general principle with which perceptual systems are organized.

Expertise is perceived from both sound and body movement in musical performance

Music is a rich form of nonverbal communication, in which the movements that expert musicians make during performance can influence the perception of expressive and structural features of the music. Whether the actual skill of a musician is perceivable from vision of movement was examined. In Experiment 1, musicians and non-musicians rated performances by novice, intermediate and expert clarinettists from point-light animations of their movements, sound recordings, or both. Performances by clarinettists of more advanced skill level were rated significantly higher from vision of movements, although this effect was stronger when sound was also presented. In Experiment 2, movements and sound from the novice and expert clarinettists' performances were switched for half the presentations, and were matched for the rest. Ratings of novice music were significantly higher when presented with expert movements, although the opposite was not found for expert sound presented with novice movements. No perceptual effect of raters' own level of musicianship was found in either experiment. These results suggest that expertise is perceivable from vision of musicians' body movements, although perception of skill from sound is dominant. The results from Experiment 2 further indicate a cross-modal effect of vision and audition on the perception of musical expertise. © 2012 Elsevier B.V.

The perception of emotion from body movement in point-light displays of interpersonal dialogue

We examined whether it is possible to identify the emotional content of behaviour from point-light displays where pairs of actors are engaged in interpersonal communication. These actors displayed a series of emotions, which included sadness, anger, joy, disgust, fear, and romantic love. In experiment 1, subjects viewed brief clips of these point-light displays presented the right way up and upside down. In experiment 2, the importance of the interaction between the two figures in the recognition of emotion was examined. Subjects were shown upright versions of (i) the original pairs (dyads), (ii) a single actor (monad), and (iii) a dyad comprising a single actor and his/her mirror image (reflected dyad). In each experiment, the subjects rated the emotional content of the displays by moving a slider along a horizontal scale. All of the emotions received a rating for every clip. In experiment 1, when the displays were upright, the correct emotions were identified in each case except disgust; but, when the displays were inverted, performance was significantly diminished for some ernotions. In experiment 2, the recognition of love and joy was impaired by the absence of the acting partner, and the recognition of sadness, joy, and fear was impaired in the non-veridical (mirror image) displays. These findings both support and extend previous research by showing that biological motion is sufficient for the perception of emotion, although inversion affects performance. Moreover, emotion perception from biological motion can be affected by the veridical or non-veridical social context within the displays.

The contribution of upper limb and total body movement to adolescents’ energy expenditure whilst playing Nintendo Wii

Little research documents the contribution of upper limb and total body movement to energy expenditure (EE) during active video gaming. To address this, EE, heart rate (HR), and, upper limb and total body movement were assessed in 11- to 17-year-old adolescents whilst playing three active (Nintendo Wii) and one sedentary (XBOX 360) video games. Non-dominant upper limb activity, EE and HR were significantly greater during Wii Sports boxing [mean 267.2 (SD 115.8) J kg−1 min−1; 136.7 (24.5) beats min−1] than tennis or bowling (P ≤ 0.044). For all active games hip activity best predicted EE (R 2 ≥ 0.53), with two-measure models of HR and single-site activity data, and multi-site activity data, similarly explaining the variance in EE (R 2 ≥ 0.64). The physiological cost of upper-body orientated active video games increased when movement of both upper limbs was encouraged. Improvements in EE explanatory power provide support for multi-site activity monitoring during unique, non-ambulatory activities.

Video-based quantification of body movement during social interaction indicates the severity of negative symptoms in patients with schizophrenia

In schizophrenia, nonverbal behavior, including body movement, is of theoretical and clinical importance. Although reduced nonverbal expressiveness is a major component of the negative symptoms encountered in schizophrenia, few studies have objectively assessed body movement during social interaction. In the present study, 378 brief, videotaped role-play scenes involving 27 stabilized outpatients diagnosed with paranoid-type schizophrenia were analyzed using Motion Energy Analysis (MEA). This method enables the objective measuring of body movement in conjunction with ordinary video recordings. Correlations between movement parameters (percentage of time in movement, movement speed) and symptom ratings from independent PANSS interviews were calculated. Movement parameters proved to be highly reliable. In keeping with predictions, reduced movement and movement speed correlated with negative symptoms. Accordingly, in patients who exhibited noticeable movement for less than 20% of the observation time, prominent negative symptoms were highly probable. As a control measure, the percentage of movement exhibited by the patients during role-play scenes was compared to that of their normal interactants. Patients with negative symptoms differed from normal interactants by showing significantly reduced head and body movement. Two specific positive symptoms were possibly related to movement parameters: suspiciousness tended to correlate with reduced head movement, and the expression of unusual thought content tended to relate to increased movement. Overall, a close and theoretically meaningful association between the objective movement parameters and the symptom profiles was found. MEA appears to be an objective, reliable and valid method for quantifying nonverbal behavior, an aspect which may furnish new insights into the processes related to reduced expressiveness in schizophrenia.

EPA-1380 – Coordinated body-movement reflects psychotherapy outcome

Social interaction is a core aspect of human life that affects individuals’ physical and mental health. Social interaction usually leads to mutual engagement in diverse areas of mental, emotional, physiological and physical activity involving both interacting persons and subsequently impacting the outcome of interactions. A common approach to the analysis of social interaction is the study of the verbal content transmitted between sender and receiver. However, additional important processes and dynamics are occurring in other domains too, for example in the area of nonverbal behaviour: In a series of studies, we have looked at nonverbal synchrony – the coordination of two persons’ movement patterns – and it‘s association with relationship quality and with the outcome of interactions. Using a computer-based algorithm (Motion Energy Analysis, MEA: Ramseyer & Tschacher, 2011), which automatically quantifies a person‘s body-movement, we were able to objectively calculate nonverbal synchrony in a large number of dyads interacting in various settings. In a first step, we showed that the phenomenon of nonverbal synchrony exists at a level that is significantly higher than expected by chance. In a second step, we ascertained that across different settings – including patient-therapist dyads and healthy dyads – more synchronized movement was associated with better relationship quality and better interactional outcomes. The quality of a relationship is thus embodied by the synchronized movement patterns emerging between partners. Our studies suggest that embodied cognition is a valuable approach to research in social interaction, providing important clues for an improved understanding of interaction dynamics.

Video-based quantification of body movement indicates negative symptoms: a replication

Objective: In schizophrenia, abnormalities in nonverbal behaviors have always been considered as highly relevant. However, due to methodological limitations, nonverbal behavior was rarely quantified objectively. Recent methodological advances now allow a quantification of body movement from ordinary video recordings. We showed that patients’ objectively measured amount of movement in social role-play interactions was closely associated with their symptom profiles (Kupper, Ramseyer, Hoffmann, & Tschacher, Schizophrenia Research 2010). In the present study, a replication of these results in the context of semi-standardized PANSS (Positive and Negative Syndrome Scale) interviews was intended. Methods: 17 patients with schizophrenia were analyzed during the initial 15-min sequence of a videotaped PANSS interview using Motion Energy Analysis (MEA). The amount of patients’ movement was then correlated with their PANSS symptom scores. Results: Sizeable and significant correlations between negative symptoms and reduced movements (r = -.68, p<0.01) and reduced movement speed (r = -.80, p<0.001) were found. Moreover, cognitive symptoms were related to reduced movement speed (r = -.70, p<.01). Conclusion: Negative symptoms were reliably indicated by patients’ nonverbal behavior in psychopathology interviews. Hence, the main result of our earlier study, examining patients’ nonverbal behavior in role play tests, was replicated for the less structured interactions in psychopathological interviews. Results could encourage the use of MEA in a wide range of videotaped social interactions of patients with schizophrenia and other psychiatric disorders.

Force sensing for measuring human body movement

The research developed in this thesis explores the sensing and inference of human movement in a dynamic way, as opposed to conventional measurement systems, that are only concerned with discrete evaluations of stimuli in sequential time. Typically, conventional approaches are used to infer the dynamic movement of the body; such as vision and motion tracking devices, with either a human diagnosis or complex image processing algorithm to classify the movement. This research is therefore the first of its kind to attempt and provide a movement classifying algorithm through the use of minimal sensing points, with the application for this novel system, to classify human movement during a golf swing. There are two main categories of force sensing. Firstly, array-type systems consisting of many sensing elements, and are the most commonly researched and commercially available. Secondly, reduced force sensing element systems (RFSES) also known as distributive systems have only been recently exploited in the academic world. The fundamental difference between these systems is that array systems handle the data captured from each sensor as unique outputs and suffer the effects of resolution. The effect of resolution, is the error in the load position measurement between sensing elements, as the output is quantized in terms of position. This can be compared to a reduced sensor element system that maximises that data received through the coupling of data from a distribution of sensing points to describe the output in discrete time. Also this can be extended to a coupling of transients in the time domain to describe an activity or dynamic movement. It is the RFSES that is to be examined and exploited in the commercial sector due to its advantages over array-based approaches such as reduced design, computational complexity and cost.

Towards an Ecological Pedagogy: Body Movement for Ecological Consciousness

This paper examines the moving body as a vehicle for raising ecological consciousness. Due to the modern over-preoccupation with the pursuit of rational aims, human interactions with the surrounding environment increasingly lack conscious awareness. Consequently, in the modern world people tend to lack an ecological consciousness. Nevertheless, the human body is a rich reservoir of ecological significance. From birth, humans are woven into tremendous interconnection with the world. However, humans thrive when their sensitivity to the physical world exists in harmony with their ability to pursue their rational aims. It is the combination of these characteristics that enables humans to survive in capricious surroundings and prosper in a wide array of contexts. Today, the human species faces an unprecedented crisis that threatens to collapse the reciprocality of the ecological bonds bolstering the prosperity of all worldly beings. This paper proposes that it is no longer a rational strategy for people to remain inattentive to their embodied ecological resonance, and that the moving body is an adequate pedagogical site for raising ecological consciousness. Ritualized body movements derived from Chinese traditional cultivation systems such as Taijiquan could orient practitioners to reestablish a perceptual intimacy with the larger cosmic world, thereby raising their ecological consciousness.

Determination of a Total Body Model of Efficiency Applied to a Rowing Movement in Humans

Efficiency represents the ratio of work done to energy expended. In human movement, it is desirable to maximise the work done or minimise the energy expenditure. Whilst research has examined the efficiency of human movement for the lower and upper body, there is a paucity of research which considers the efficiency of a total body movement. Rowing is a movement which encompasses all parts of the body to generate locomotion and is a useful modality to measure total body efficiency. It was the aim of this research to develop a total body model of efficiency and explore how skill level of participants and assumptions of the modelling process affected the efficiency estimates Three studies were used to develop and evaluate the efficiency model. Firstly, the efficiency of ten healthy males was established using rowing, cycling and arm cranking. The model included internal work from motion capture and efficiency estimates were comparable to published literature, indicating the suitability of the model to estimate efficiency. Secondly, the model was developed to include a multi-segmented trunk and twelve novice and twelve skilled participants were assessed for efficiency. Whilst the efficiency estimates were similar to published results, novice participants were assessed as more efficient. Issues such as the unique physiology of trained rowers and a lack of energy transfers in the model were considered contributing factors. Finally the model was redeveloped to account for energy transfers, where skilled participants had higher efficiency at large workloads. This work presents a novel model for estimating efficiency during a rowing motion. The specific inclusion of energy transfers expands previous knowledge of internal work and efficiency, demonstrating a need to include energy transfers in the assessment of efficiency of a total body action.

Sensory and intrinsic coordination of movement

A recently generalized theory of perceptual guidance (general tau theory) was used to analyse coordination in skilled movement. The theory posits that (i) guiding movement entails controlling closure of spatial and/or force gaps between effecters and goals, by sensing and regulating the tau s of the gaps (the time-to-closure at current closure rate), (ii) a principal way of coordinating movements is keeping the rs of different gaps in constant ratio (known as tau-coupling), and (iii) intrinsically paced movements are guided and coordinated by tau-coupling onto a tau-guide, tau(g), generated in the nervous system and described by the equation tau(g) = 0.5(t-T-2/t) where T is the duration of the body movement and t is the time from the start of the movement. Kinematic analysis of hand to mouth movements by human adults, with eyes open or closed, indicated that hand guidance was achieved by maintaining, during 80-85% of the movement, the tau-couplings tau(alpha)-tau(t) and tau(t)-tau(g), where tau(t) is tau of the hand-mouth gap, tau(alpha) is tau of the angular gap to be closed by steering the hand and tau(g) is an intrinsic tau-guide.

Detecting Deception in Movement: The Case of the Side-Step in Rugby

Although coordinated patterns of body movement can be used to communicate action intention, they can also be used to deceive. Often known as deceptive movements, these unpredictable patterns of body movement can give a competitive advantage to an attacker when trying to outwit a defender. In this particular study, we immersed novice and expert rugby players in an interactive virtual rugby environment to understand how the dynamics of deceptive body movement influence a defending player’s decisions about how and when to act. When asked to judge final running direction, expert players who were found to tune into prospective tau-based information specified in the dynamics of ‘honest’ movement signals (Centre of Mass), performed significantly better than novices who tuned into the dynamics of ‘deceptive’ movement signals (upper trunk yaw and out-foot placement) (p<.001). These findings were further corroborated in a second experiment where players were able to move as if to intercept or ‘tackle’ the virtual attacker. An analysis of action responses showed that experts waited significantly longer before initiating movement (p<.001). By waiting longer and picking up more information that would inform about future running direction these experts made significantly fewer errors (p<.05). In this paper we not only present a mathematical model that describes how deception in body-based movement is detected, but we also show how perceptual expertise is manifested in action expertise. We conclude that being able to tune into the ‘honest’ information specifying true running action intention gives a strong competitive advantage.

Laughter Type Recognition from Whole Body Motion

Despite the importance of laughter in social interactions it remains little studied in affective computing. Respiratory, auditory, and facial laughter signals have been investigated but laughter-related body movements have received almost no attention. The aim of this study is twofold: first an investigation into observers' perception of laughter states (hilarious, social, awkward, fake, and non-laughter) based on body movements alone, through their categorization of avatars animated with natural and acted motion capture data. Significant differences in torso and limb movements were found between animations perceived as containing laughter and those perceived as nonlaughter. Hilarious laughter also differed from social laughter in the amount of bending of the spine, the amount of shoulder rotation and the amount of hand movement. The body movement features indicative of laughter differed between sitting and standing avatar postures. Based on the positive findings in this perceptual study, the second aim is to investigate the possibility of automatically predicting the distributions of observer's ratings for the laughter states. The findings show that the automated laughter recognition rates approach human rating levels, with the Random Forest method yielding the best performance.