Inertial motion capture and live performance (with a focus on dance)

3D Motion capture is a fast evolving field and recent inertial technology may expand the artistic possibilities for its use in live performance. Inertial motion capture has three attributes that make it suitable for use with live performance; it is portable, easy to use and can operate in real-time. Using four projects, this paper discusses the suitability of inertial motion capture to live performance with a particular emphasis on dance. Dance is an artistic application of human movement and motion capture is the means to record human movement as digital data. As such, dance is clearly a field in which the use of real-time motion capture is likely to become more common, particularly as projected visual effects including real-time video are already often used in dance performances. Understandably, animation generated in real-time using motion capture is not as extensive or as clean as the highly mediated animation used in movies and games, but the quality is still impressive and the ‘liveness’ of the animation has compensating features that offer new ways of communicating with an audience.

A world elsewhere : virtual reality, motion capture and early modern theatre

This paper investigates virtual reality representations of the 1599 Boar’s Head Theatre and the Rose Theatre, two renaissance places and spaces. These models become a “world elsewhere” in that they represent virtual recreations of these venues in as much detail as possible. The models are based on accurate archeological and theatre historical records and are easy to navigate particularly for current use. This paper demonstrates the ways in which these models can be instructive for reading theatre today. More importantly we introduce human figures onto the stage via motion capture which allows us to explore the potential between space, actor and environment. This facilitates a new way of thinking about early modern playwrights’ “attitudes to locality and localities large and small”. These venues are thus activated to intersect productively with early modern studies so that the paper can test the historical and contemporary limits of such research.

Reproduction, mediation, and experience : virtual reality, motion capture and early modern theatre

This article investigates virtual reality representations of performance in London’s late sixteenth-century Rose Theatre, a venue that, by means of current technology, can once again challenge perceptions of space, performance, and memory. The VR model of The Rose represents a virtual recreation of this venue in as much detail as possible and attempts to recover graphic demonstrations of the trace memories of the performance modes of the day. The VR model is based on accurate archeological and theatre historical records and is easy to navigate. The introduction of human figures onto The Rose’s stage via motion capture allows us to explore the relationships between space, actor and environment. The combination of venue and actors facilitates a new way of thinking about how the work of early modern playwrights can be stored and recalled. This virtual theatre is thus activated to intersect productively with contemporary studies in performance; as such, our paper provides a perspective on and embodiment of the relation between technology, memory and experience. It is, at its simplest, a useful archiving project for theatrical history, but it is directly relevant to contemporary performance practice as well. Further, it reflects upon how technology and ‘re-enactments’ of sorts mediate the way in which knowledge and experience are transferred, and even what may be considered ‘knowledge.’ Our work provides opportunities to begin addressing what such intermedial confrontations might produce for ‘remembering, experiencing, thinking and imagining.’ We contend that these confrontations will enhance live theatre performance rather than impeding or disrupting contemporary performance practice. Our ‘paper’ is in the form of a video which covers the intellectual contribution while also permitting a demonstration of the interventions we are discussing.

Technology, memory and experience : virtual reality, motion capture and early modern theatre

This paper investigates virtual reality representations of performance in London’s late sixteenth-century Rose Theatre, a venue that, by means of current technology, can once again challenge perceptions of space, performance, and memory. The VR model of The Rose becomes a Camillo device in that it represents a virtual recreation of this venue in as much detail as possible and attempts to recover graphic demonstrations of the trace memories of the performance modes of the day. The VR model is based on accurate archeological and theatre historical records and is easy to navigate. The introduction of human figures onto The Rose’s stage via motion capture allows us to explore the relationships between space, actor and environment. The combination of venue and actors facilitates a new way of thinking about how the work of early modern playwrights can be stored and recalled. This virtual theatre is thus activated to intersect productively with contemporary studies in performance; as such, our paper provides a perspective on and embodiment of the relation between technology, memory and experience. It is, at its simplest, a useful archiving project for theatrical history, but it is directly relevant to contemporary performance practice as well. Further, it reflects upon how technology and ‘re-enactments’ of sorts mediate the way in which knowledge and experience are transferred, and even what may be considered ‘knowledge.’ Our work provides opportunities to begin addressing what such intermedial confrontations might produce for ‘remembering, experiencing, thinking and imagining.’ We contend that these confrontations will enhance live theatre performance rather than impeding or disrupting contemporary performance practice. This paper intersects with the CFP’s ‘Performing Memory’ and ‘Memory Lab’ themes. Our presentation (which includes a demonstration of the VR model and the motion capture it requires) takes the form of two closely linked papers that share a single abstract. The two papers will be given by two people, one of whom will be physically present in Utrecht, the other participating via Skype.

Preparing the actor for performance capture

Notions of capture and manipulation infer the existence of an interface that combines performer with system, an interface that separates (or intervenes in) the space between performance, intention and history. It is precisely the effect and provided opportunity of the intermediary device on the practice and craft of the actor, that is particularly examined in this work. Defining the scope of current practice for the contemporary actor is a key construct of this challenge with the most appropriate definition revolving around the pursuit of providing a required mixture of performance and content for live, mediated, framed and variously captured formats that exist in the present day. One of these particular formats is Performance Capture which this paper interrogate in more detail.

Improved Subject Identification in Surveillance Video using Super Resolution

The time consuming and labour intensive task of identifying individuals in surveillance video is often challenged by poor resolution and the sheer volume of stored video. Faces or identifying marks such as tattoos are often too coarse for direct matching by machine or human vision. Object tracking and super-resolution can then be combined to facilitate the automated detection and enhancement of areas of interest. The object tracking process enables the automatic detection of people of interest, greatly reducing the amount of data for super-resolution. Smaller regions such as faces can also be tracked. A number of instances of such regions can then be utilized to obtain a super-resolved version for matching. Performance improvement from super-resolution is demonstrated using a face verification task. It is shown that there is a consistent improvement of approximately 7% in verification accuracy, using both Eigenface and Elastic Bunch Graph Matching approaches for automatic face verification, starting from faces with an eye to eye distance of 14 pixels. Visual improvement in image fidelity from super-resolved images over low-resolution and interpolated images is demonstrated on a small database. Current research and future directions in this area are also summarized.

Measuring reef complexity and rugosity from monocular video bathymetric reconstruction

Topographic structural complexity of a reef is highly correlated to coral growth rates, coral cover and overall levels of biodiversity, and is therefore integral in determining ecological processes. Modeling these processes commonly includes measures of rugosity obtained from a wide range of different survey techniques that often fail to capture rugosity at different spatial scales. Here we show that accurate estimates of rugosity can be obtained from video footage captured using underwater video cameras (i.e., monocular video). To demonstrate the accuracy of our method, we compared the results to in situ measurements of a 2m x 20m area of forereef from Glovers Reef atoll in Belize. Sequential pairs of images were used to compute fine scale bathymetric reconstructions of the reef substrate from which precise measurements of rugosity and reef topographic structural complexity can be derived across multiple spatial scales. To achieve accurate bathymetric reconstructions from uncalibrated monocular video, the position of the camera for each image in the video sequence and the intrinsic parameters (e.g., focal length) must be computed simultaneously. We show that these parameters can be often determined when the data exhibits parallax-type motion, and that rugosity and reef complexity can be accurately computed from existing video sequences taken from any type of underwater camera from any reef habitat or location. This technique provides an infinite array of possibilities for future coral reef research by providing a cost-effective and automated method of determining structural complexity and rugosity in both new and historical video surveys of coral reefs.

Re-targeting choreographic enquiry through motion capture

This creative work is the outcome of preliminary experiments through practice aiming to explore the collaboration of a Dancer/choreographer with an Animator, along with enquiry into the intergeneration of motion capture technologies within the work-flow. The animated visuals derived from the motion capture data is not aimed at just re-targeting of movement from one source to another but looks at describing the thought and emotions of the choreographed dance through visual aesthetics.

Preparing for performance capture #1

This video article articulates two exercises that have been developed to respond to the need for preparedness in the growing field of Performance Capture. The first is called Walking Through (Delbridge 2013), where the actor navigates a series of objects that exist in screen space through a developed sense of the existing physical particularities of the studio and an interaction with a screen (or feedback loop). The second exercise is called The Donut (Delbridge 2013), where the performer continues to navigate screen space but this time does so through the literal stepping through of a Torus in the virtual – again with nothing but the studio infrastructure and the screen as a guide. Notions of Motion Captured performance infer the existence of an interface that combines performer with system, separating (or intervening in) the space between performance and the screen. It is precisely the effect and provided opportunity of the intermediary device on the practice, craft and preparedness of the actor as they navigate the connection between 3D screen space and the physical properties of the studio that is examined here. Defining the scope of current practice for the contemporary actor is a key construct of this challenge, with the most appropriate definition revolving around the provision of a required mixture of performance and content for live, mediated, framed and variously captured formats. One of these particular formats is Performance Capture. The exercises presented here are two from a series, developed over a three year study that contribute to our understanding of the potential for a training regimen to be developed for the rigors of Performance Capture.

Using a video camera to measure the radius of the Earth

A simple but accurate method for measuring the Earth’s radius using a video camera is described. A video camera was used to capture a shadow rising up the wall of a tall building at sunset. A free program called ImageJ was used to measure the time it took the shadow to rise a known distance up the building. The time, distance and length of the sidereal day were used to calculate the radius of the Earth. The radius was measured as 6394.3 +/- 118 km, which is within 1.8% of the accepted average value of 6371 km and well within the experimental error. The experiment is suitable as a high school or university project and should produce a value for Earth’s radius within a few per cent at latitudes towards the equator, where at some times of the year the ecliptic is approximately normal to the horizon.

Effects of a pediatric weight management program with and without active video games

Importance Active video games may offer an effective strategy to increase physical activity in overweight and obese children. However, the specific effects of active gaming when delivered within the context of a pediatric weight management program are unknown. Objective To evaluate the effects of active video gaming on physical activity and weight loss in children participating in an evidence-based weight management program delivered in the community. Design, Setting, and Participants Group-randomized clinical trial conducted during a 16-week period in YMCAs and schools located in Massachusetts, Rhode Island, and Texas. Seventy-five overweight or obese children (41 girls [55%], 34 whites [45%], 20 Hispanics [27%], and 17 blacks [23%]) enrolled in a community-based pediatric weight management program. Mean (SD) age of the participants was 10.0 (1.7) years; body mass index (BMI) z score, 2.15 (0.40); and percentage overweight from the median BMI for age and sex, 64.3% (19.9%). Interventions All participants received a comprehensive family-based pediatric weight management program (JOIN for ME). Participants in the program and active gaming group received hardware consisting of a game console and motion capture device and 1 active game at their second treatment session and a second game in week 9 of the program. Participants in the program-only group were given the hardware and 2 games at the completion of the 16-week program. Main Outcomes and Measures Objectively measured daily moderate-to-vigorous and vigorous physical activity, percentage overweight, and BMI z score. Results Participants in the program and active gaming group exhibited significant increases in moderate-to-vigorous (mean [SD], 7.4 [2.7] min/d) and vigorous (2.8 [0.9] min/d) physical activity at week 16 (P < .05). In the program-only group, a decline or no change was observed in the moderate-to-vigorous (mean [SD] net difference, 8.0 [3.8] min/d; P = .04) and vigorous (3.1 [1.3] min/d; P = .02) physical activity. Participants in both groups exhibited significant reductions in percentage overweight and BMI z scores at week 16. However, the program and active gaming group exhibited significantly greater reductions in percentage overweight (mean [SD], −10.9% [1.6%] vs −5.5% [1.5%]; P = .02) and BMI z score (−0.25 [0.03] vs −0.11 [0.03]; P < .001). Conclusions and Relevance Incorporating active video gaming into an evidence-based pediatric weight management program has positive effects on physical activity and relative weight.

Using viewer’s facial expression and heart rate for sports video highlights detection

Viewer interests, evoked by video content, can potentially identify the highlights of the video. This paper explores the use of facial expressions (FE) and heart rate (HR) of viewers captured using camera and non-strapped sensor for identifying interesting video segments. The data from ten subjects with three videos showed that these signals are viewer dependent and not synchronized with the video contents. To address this issue, new algorithms are proposed to effectively combine FE and HR signals for identifying the time when viewer interest is potentially high. The results show that, compared with subjective annotation and match report highlights, ‘non-neutral’ FE and ‘relatively higher and faster’ HR is able to capture 60%-80% of goal, foul, and shot-on-goal soccer video events. FE is found to be more indicative than HR of viewer’s interests, but the fusion of these two modalities outperforms each of them.

Effect of position on precision within the recording field of a markerless motion capture system

Introduction Markerless motion capture systems are relatively new devices that can significantly speed up capturing full body motion. A precision of the assessment of the finger’s position with this type of equipment was evaluated at 17.30 ± 9.56 mm when compare to an active marker system [1]. The Microsoft Kinect was proposed to standardized and enhanced clinical evaluation of patients with hemiplegic cerebral palsy [2]. Markerless motion capture systems have the potential to be used in a clinical setting for movement analysis, as well as for large cohort research. However, the precision of such system needs to be characterized. Global objectives • To assess the precision within the recording field of the markerless motion capture system Openstage 2 (Organic Motion, NY). • To compare the markerless motion capture system with an optoelectric motion capture system with active markers. Specific objectives • To assess the noise of a static body at 13 different location within the recording field of the markerless motion capture system. • To assess the smallest oscillation detected by the markerless motion capture system. • To assess the difference between both systems regarding the body joint angle measurement. Methods Equipment • OpenStage® 2 (Organic Motion, NY) o Markerless motion capture system o 16 video cameras (acquisition rate : 60Hz) o Recording zone : 4m * 5m * 2.4m (depth * width * height) o Provide position and angle of 23 different body segments • VisualeyezTM VZ4000 (PhoeniX Technologies Incorporated, BC) o Optoelectric motion capture system with active markers o 4 trackers system (total of 12 cameras) o Accuracy : 0.5~0.7mm Protocol & Analysis • Static noise: o Motion recording of an humanoid mannequin was done in 13 different locations o RMSE was calculated for each segment in each location • Smallest oscillation detected: o Small oscillations were induced to the humanoid mannequin and motion was recorded until it stopped. o Correlation between the displacement of the head recorded by both systems was measured. A corresponding magnitude was also measured. • Body joints angle: o Body motion was recorded simultaneously with both systems (left side only). o 6 participants (3 females; 32.7 ± 9.4 years old) • Tasks: Walk, Squat, Shoulder flexion & abduction, Elbow flexion, Wrist extension, Pronation / supination (not in results), Head flexion & rotation (not in results), Leg rotation (not in results), Trunk rotation (not in results) o Several body joint angles were measured with both systems. o RMSE was calculated between signals of both systems. Results Conclusion Results show that the Organic Motion markerless system has the potential to be used for assessment of clinical motor symptoms or motor performances However, the following points should be considered: • Precision of the Openstage system varied within the recording field. • Precision is not constant between limb segments. • The error seems to be higher close to the range of motion extremities.