Experimental arts practice incorporating motion capture

My practice-led research explores and maps workflows for generating experimental creative work involving inertia based motion capture technology. Motion capture has often been used as a way to bridge animation and dance resulting in abstracted visuals outcomes. In early works this process was largely done by rotoscoping, reference footage and mechanical forms of motion capture. With the evolution of technology, optical and inertial forms of motion capture are now more accessible and able to accurately capture a larger range of complex movements. Made by Motion is a collaboration between digital artist Paul Van Opdenbosch and performer and choreographer Elise May; a series of studies on captured motion data used to generate experimental visual forms that reverberate in space and time. The project investigates the invisible forces generated by and influencing the movement of a dancer. Along with how the forces can be captured and applied to generating visual outcomes that surpass simple data visualisation, projecting the intent of the performer’s movements. The source or ‘seed’ comes from using an Xsens MVN – Inertial Motion Capture system to capture spontaneous dance movements, with the visual generation conducted through a customised dynamics simulation. In my presentation I will be displaying and discussing a selected creative works from the project along with the process and considerations behind the work.

Visual performance with lenses correcting peripheral refractive errors

Purpose To design and manufacture lenses to correct peripheral refraction along the horizontal meridian and to determine whether these resulted in noticeable improvements in visual performance. Method Subjective refraction of a low myope was determined on the basis of best peripheral detection acuity along the horizontal visual field out to ±30° for both horizontal and vertical gratings. Subjective refraction was compared to objective refractions using a COAS-HD aberrometer. Special lenses were made to correct peripheral refraction, based on designs optimized with and without smoothing across a 3 mm diameter square aperture. Grating detection was retested with these lenses. Contrast thresholds of 1.25’ spots were determined across the field for the conditions of best correction, on-axis correction, and the special lenses. Results The participant had high relative peripheral hyperopia, particularly in the temporal visual field (maximum 2.9 D). There were differences > 0.5D between subjective and objective refractions at a few field angles. On-axis correction reduced peripheral detection acuity and increased peripheral contrast threshold in the peripheral visual field, relative to the best correction, by up to 0.4 and 0.5 log units, respectively. The special lenses restored most of the peripheral vision, although not all at angles to ±10°, and with the lens optimized with aperture-smoothing possibly giving better vision than the lens optimized without aperture-smoothing at some angles. Conclusion It is possible to design and manufacture lenses to give near optimum peripheral visual performance to at least ±30° along one visual field meridian. The benefit of such lenses is likely to be manifest only if a subject has a considerable relative peripheral refraction, for example of the order of 2 D.

Understanding the signature pedagogy of the design studio and the opportunities for its technological enhancement

This paper presents an analysis of the studio as the signature pedagogy of design education. A number of theoretical models of learning, pedagogy, and education are used to interrogate the studio for its advantages and shortcomings, and to identify opportunities for the integration of new technologies and to explore the affordances that they might offer. In particular the theoretical ideas of signature pedagogies, conversational frameworks, and pedagogical patterns are used to justify the ‘unique’ status of the studio as a dominant learning environment and mode of delivery within design education. Such analysis identifies the opportunities for technological intervention and enhancement of the design studio through a re-examining of its fundamental pedagogical signature. This paper maps the dimensions and qualities that define the signature pedagogy against a range of delivery modes and technological media forms. Through such investigation it seeks to identify appropriate opportunities for technology; in essence offering a structure or framework for the analysis of future enquiry and experimentation.

Towards persistent visual navigation using SMART

In this paper, we present SMART (Sequence Matching Across Route Traversals): a vision- based place recognition system that uses whole image matching techniques and odometry information to improve the precision-recall performance, latency and general applicability of the SeqSLAM algorithm. We evaluate the system’s performance on challenging day and night journeys over several kilometres at widely varying vehicle velocities from 0 to 60 km/h, compare performance to the current state-of- the-art SeqSLAM algorithm, and provide parameter studies that evaluate the effectiveness of each system component. Using 30-metre sequences, SMART achieves place recognition performance of 81% recall at 100% precision, outperforming SeqSLAM, and is robust to significant degradations in odometry.

Enabling aircraft emergency landings using active visual site detection

The ability to automate forced landings in an emergency such as engine failure is an essential ability to improve the safety of Unmanned Aerial Vehicles operating in General Aviation airspace. By using active vision to detect safe landing zones below the aircraft, the reliability and safety of such systems is vastly improved by gathering up-to-the-minute information about the ground environment. This paper presents the Site Detection System, a methodology utilising a downward facing camera to analyse the ground environment in both 2D and 3D, detect safe landing sites and characterise them according to size, shape, slope and nearby obstacles. A methodology is presented showing the fusion of landing site detection from 2D imagery with a coarse Digital Elevation Map and dense 3D reconstructions using INS-aided Structure-from-Motion to improve accuracy. Results are presented from an experimental flight showing the precision/recall of landing sites in comparison to a hand-classified ground truth, and improved performance with the integration of 3D analysis from visual Structure-from-Motion.

Eight weeks of episodic visual navigation inside a non-stationary environment using adaptive spherical views

This paper presents a long-term experiment where a mobile robot uses adaptive spherical views to localize itself and navigate inside a non-stationary office environment. The office contains seven members of staff and experiences a continuous change in its appearance over time due to their daily activities. The experiment runs as an episodic navigation task in the office over a period of eight weeks. The spherical views are stored in the nodes of a pose graph and they are updated in response to the changes in the environment. The updating mechanism is inspired by the concepts of long- and short-term memories. The experimental evaluation is done using three performance metrics which evaluate the quality of both the adaptive spherical views and the navigation over time.

Pupil shape as viewed along the horizontal visual field

Purpose: Changes in pupil size and shape are relevant for peripheral imagery by affecting aberrations and how much light enters and/or exits the eye. The purpose of this study is to model the pattern of pupil shape across the complete horizontal visual field and to show how the pattern is influenced by refractive error. Methods: Right eyes of thirty participants were dilated with 1% cyclopentolate and images were captured using a modified COAS-HD aberrometer alignment camera along the horizontal visual field to ±90°. A two lens relay system enabled fixation at targets mounted on the wall 3m from the eye. Participants placed their heads on a rotatable chin rest and eye rotations were kept to less than 30°. Best-fit elliptical dimensions of pupils were determined. Ratios of minimum to maximum axis diameters were plotted against visual field angle. Results: Participants’ data were well fitted by cosine functions, with maxima at (–)1° to (–)9° in the temporal visual field and widths 9% to 15% greater than predicted by the cosine of the field angle . Mean functions were 0.99cos[( + 5.3)/1.121], R2 0.99 for the whole group and 0.99cos[( + 6.2)/1.126], R2 0.99 for the 13 emmetropes. The function peak became less temporal, and the width became smaller, with increase in myopia. Conclusion: Off-axis pupil shape changes are well described by a cosine function which is both decentered by a few degrees and flatter by about 12% than the cosine of the viewing angle, with minor influences of refraction.

Improved line tracker using IMU and vision for visual servoing

This paper introduces an improved line tracker using IMU and vision data for visual servoing tasks. We utilize an Image Jacobian which describes motion of a line feature to corresponding camera movements. These camera motions are estimated using an IMU. We demonstrate impacts of the proposed method in challenging environments: maximum angular rate ~160 0/s, acceleration ~6m /s2 and in cluttered outdoor scenes. Simulation and quantitative tracking performance comparison with the Visual Servoing Platform (ViSP) are also presented.

Towards condition-invariant, top-down visual place recognition

In this paper we present a novel place recognition algorithm inspired by recent discoveries in human visual neuroscience. The algorithm combines intolerant but fast low resolution whole image matching with highly tolerant, sub-image patch matching processes. The approach does not require prior training and works on single images (although we use a cohort normalization score to exploit temporal frame information), alleviating the need for either a velocity signal or image sequence, differentiating it from current state of the art methods. We demonstrate the algorithm on the challenging Alderley sunny day – rainy night dataset, which has only been previously solved by integrating over 320 frame long image sequences. The system is able to achieve 21.24% recall at 100% precision, matching drastically different day and night-time images of places while successfully rejecting match hypotheses between highly aliased images of different places. The results provide a new benchmark for single image, condition-invariant place recognition.

Objectifying the Subjective : Assessment and Feedback in Creative Arts Studio Learning and Teaching

This paper examines the Assessment and Feedback aspects of Studio Teaching as Creative Arts pedagogy. Prompted by USQ’s newly offered Bachelor of Creative Arts (BCA), the author has developed an Assessment Matrix specifically designed to satisfy a number of imperatives, including: • ‘objectifying’ the subjective aspects of creative practice as assessable coursework/research • providing the means by which accurate, detailed, personalised and confidential feedback may be provided to students individually • providing consistent, accurate, meaningful assessment records for student, lecturer, and institution • ensuring consistency, continuity, and transparency of assessment processes and records to satisfy quality audits • minimising marking and assessment time, whilst maximising assessment integrity and depth • requiring only basic level skills and knowledge of a computer application already in common use (Microsoft Excel) • adaptability to a range of creative courses ‐ across disciplines This Assessment Matrix has been in development (and trialled) since January 2009.

Digital beings : an opportunity for Australian visual effects

Ongoing innovation in digital animation and visual effects technologies has provided new opportunities for stories to be visually rendered in ways never before possible. Films featuring animation and visual effects continue to perform well at the box office, proving to be highly profitable projects. The Avengers (Whedon, 2012) holds the current record for opening weekend sales, accruing as much as $207,438,708 USD and $623,357,910 USD gross at time of writing. Life of Pi (Lee, 2012) at time of writing has grossed as much as $608,791,063 USD (Box Office Mojo, 2013). With so much creative potential and a demonstrable ability to generate a large amount of revenue, the animation and visual effects industry – otherwise known as the Post, Digital and Visual Effects (PDV) industry – has become significant to the future growth and stability of the Australian film industry as a whole.

QUT Architecture Film Studio (Aus)

Research Statement: An urban film produced by Luke Harrison Mitchell Benham, Sharlene Anderson, Tristan Clark. RIVE NOIR explores the film noir tradition, shot on location in a dark urban space between high-rises and the river, sheltered by a highway. With an original score and striking cinematography, Rive Noir radically transforms the abandoned river’s edge through the production of an amplified reality ordinarily unseen in the Northbank. The work produced under my supervision was selected to appear in the Expanded Architecture Research Group’s International Architecture Film Festival and Panel Discussion in Sydney: The University of Sydney and Carriageworks Performance Space, 06 November 2011. QUT School of Design research submission was selected alongside exhibits by AA School of Architecture, London; The Bartlett School of Architecture, London; University of The Arts, London; Arrhaus School of Architecture, Denmark; Dublin as a Cinematic City, Ireland; Design Lab Screen Studio, Australia; and Sona Cinecity, The University of Melbourne. The exhibit included not only the screening of the film but the design project that derived from and extended the aesthetics of the urban film. The urban proposal and architectural intervention that followed the film was subsequently published in the Brisbane Times, after the urban proposal won first place in The Future of Brisbane architecture competition, which demonstrates the impact of the research project as a whole. EXPANDED ARCHITECTURE 2011 - 6th November Architecture Film Night + Panel Discussion @ Performance Space CarriageWorks was Sydney's first International Architectural Film Festival. With over 40 architectural films by local and international artists, film makers and architects. It was followed by Panel Discussion of esteemed academics and artists working in the field of architectural film.

The Brisbane Northbank Urban Film Studio : screening of shortlisted films

Research Statement: In this research project film groups of 4-5 students under my direction produced a 3-5 minute urban film that explored the Brisbane Northbank, and which would become the basis for an urban proposal and design of a small film studio for independent filmmakers in the site. The theoretical premise was that a film studio does not simply produce movies, it creates urban effects all around it and acts as a vortex of cultural activity and social life. For this modest facility where the cinema goes out into the street, the city itself becomes the studio. Students were called to observe the historical problematics of technique, image and effect that arise in the cinema, and to apply these to their own urban-film practice. A panel of judges working in film and architecture shortlisted the 12 best films in 2010 and a major public film screening event took place at the Tribal Cinema. The Shortlisted films today form a permanent "exhibit" in YouTube. The research project was funded by the Queensland University of Technology, School of Design and received accolades from film faculty in the Creative Industries Faculty. The diverse body of work that emanated from the screening contributed a unique analysis of the Northbank to Brisbane.

Selective combination of visual and thermal imaging for resilient localization in adverse conditions : day and night, smoke and fire

Long-term autonomy in robotics requires perception systems that are resilient to unusual but realistic conditions that will eventually occur during extended missions. For example, unmanned ground vehicles (UGVs) need to be capable of operating safely in adverse and low-visibility conditions, such as at night or in the presence of smoke. The key to a resilient UGV perception system lies in the use of multiple sensor modalities, e.g., operating at different frequencies of the electromagnetic spectrum, to compensate for the limitations of a single sensor type. In this paper, visual and infrared imaging are combined in a Visual-SLAM algorithm to achieve localization. We propose to evaluate the quality of data provided by each sensor modality prior to data combination. This evaluation is used to discard low-quality data, i.e., data most likely to induce large localization errors. In this way, perceptual failures are anticipated and mitigated. An extensive experimental evaluation is conducted on data sets collected with a UGV in a range of environments and adverse conditions, including the presence of smoke (obstructing the visual camera), fire, extreme heat (saturating the infrared camera), low-light conditions (dusk), and at night with sudden variations of artificial light. A total of 240 trajectory estimates are obtained using five different variations of data sources and data combination strategies in the localization method. In particular, the proposed approach for selective data combination is compared to methods using a single sensor type or combining both modalities without preselection. We show that the proposed framework allows for camera-based localization resilient to a large range of low-visibility conditions.

Perception Quality Evaluation with Visual and Infrared Cameras in Challenging Environmental Conditions

This work aims to contribute to the reliability and integrity of perceptual systems of unmanned ground vehicles (UGV). A method is proposed to evaluate the quality of sensor data prior to its use in a perception system by utilising a quality metric applied to heterogeneous sensor data such as visual and infrared camera images. The concept is illustrated specifically with sensor data that is evaluated prior to the use of the data in a standard SIFT feature extraction and matching technique. The method is then evaluated using various experimental data sets that were collected from a UGV in challenging environmental conditions, represented by the presence of airborne dust and smoke. In the first series of experiments, a motionless vehicle is observing a ’reference’ scene, then the method is extended to the case of a moving vehicle by compensating for its motion. This paper shows that it is possible to anticipate degradation of a perception algorithm by evaluating the input data prior to any actual execution of the algorithm.