Music selection using a touch screen interface : effect of auditory and visual feedback on driving and usability

This study aimed to examine the effects on driving, usability and subjective workload of performing music selection tasks using a touch screen interface. Additionally, to explore whether the provision of visual and/or auditory feedback offers any performance and usability benefits. Thirty participants performed music selection tasks with a touch screen interface while driving. The interface provided four forms of feedback: no feedback, auditory feedback, visual feedback, and a combination of auditory and visual feedback. Performance on the music selection tasks significantly increased subjective workload and degraded performance on a range of driving measures including lane keeping variation and number of lane excursions. The provision of any form of feedback on the touch screen interface did not significantly affect driving performance, usability or subjective workload, but was preferred by users over no feedback. Overall, the results suggest that touch screens may not be a suitable input device for navigating scrollable lists.

HIFiRE re-entry observation using an image-based visual servoing system

This paper presents an image based visual servoing system that is intended to be used for tracking and obtaining scientific observations of the HIFiRE vehicles. The primary aim of this tracking platform is to acquire and track the thermal signature emitted from the surface of the vehicle during the re-entry phase of the mission using an infra-red camera. The implemented visual servoing scheme uses a classical image based approach to identify and track the target using visual kinematic control. The paper utilizes simulation and experimental results to show the tracking performance of the system using visual feedback. Discussions on current implementation and control techniques to further improve the performance of the system are also explored.

MPC controlled multirotor with suspended slung load: System architecture and visual load detection

There is an increased interest in the use of Unmanned Aerial Vehicles for load transportation from environmental remote sensing to construction and parcel delivery. One of the main challenges is accurate control of the load position and trajectory. This paper presents an assessment of real flight trials for the control of an autonomous multi-rotor with a suspended slung load using only visual feedback to determine the load position. This method uses an onboard camera to take advantage of a common visual marker detection algorithm to robustly detect the load location. The load position is calculated using an onboard processor, and transmitted over a wireless network to a ground station integrating MATLAB/SIMULINK and Robotic Operating System (ROS) and a Model Predictive Controller (MPC) to control both the load and the UAV. To evaluate the system performance, the position of the load determined by the visual detection system in real flight is compared with data received by a motion tracking system. The multi-rotor position tracking performance is also analyzed by conducting flight trials using perfect load position data and data obtained only from the visual system. Results show very accurate estimation of the load position (~5% Offset) using only the visual system and demonstrate that the need for an external motion tracking system is not needed for this task.

Self and peer review in dance classes using personal video feedback

Many forms of formative feedback are used in dance training to refine the dancer’s spatial and kinaesthetic awareness in order that the dancer’s sensorimotor intentions and observable danced outcomes might converge. This paper documents the use of smartphones to record and playback movement sequences in ballet and contemporary technique classes. Peers in pairs took turns filming one another and then analysing the playback. This provided immediate visual feedback of the movement sequence as performed by each dancer. This immediacy facilitated the dancer’s capacity to associate what they felt as they were dancing with what they looked like during the dance. The often-dissonant realities of self-perception and perception by others were thus guided towards harmony, generating improved performance and knowledge relating to dance technique. An approach is offered for potential development of peer review activities to support summative progressive assessment in dance technique training.

Dopamine and visually regulated eye growth in chick

Retinal image properties such as contrast and spatial frequency play important roles in the development of normal vision. For example, visual environments comprised solely of low contrast and/or low spatial frequencies induce myopia. The visual image is processed by the retina and it then locally controls eye growth. In terms of the retinal neurotransmitters that link visual stimuli to eye growth, there is strong evidence to suggest involvement of the retinal dopamine (DA) system. For example, effectively increasing retinal DA levels by using DA agonists can suppress the development of form-deprivation myopia (FDM). However, whether visual feedback controls eye growth by modulating retinal DA release, and/or some other factors, is still being elucidated. This thesis is chiefly concerned with the relationship between the dopaminergic system and retinal image properties in eye growth control. More specifically, whether the amount of retinal DA release reduces as the complexity of the image degrades was determined. For example, we investigated whether the level of retinal DA release decreased as image contrast decreased. In addition, the effects of spatial frequency, spatial energy distribution slope, and spatial phase on retinal DA release and eye growth were examined. When chicks were 8-days-old, a cone-lens imaging system was applied monocularly (+30 D, 3.3 cm cone). A short-term treatment period (6 hr) and a longer-term treatment period (4.5 days) were used. The short-term treatment tests for the acute reduction in DA release by the visual stimulus, as is seen with diffusers and lenses, whereas the 4.5 day point tests for reduction in DA release after more prolonged exposure to the visual stimulus. In the contrast study, 1.35 cyc/deg square wave grating targets of 95%, 67%, 45%, 12% or 4.2% contrast were used. Blank (0% contrast) targets were included for comparison. In the spatial frequency study, both sine and square wave grating targets with either 0.017 cyc/deg and 0.13 cyc/deg fundamental spatial frequencies and 95% contrast were used. In the spectral slope study, 30% root-mean-squared (RMS) contrast fractal noise targets with spectral fall-off of 1/f0.5, 1/f and 1/f2 were used. In the spatial alignment study, a structured Maltese cross (MX) target, a structured circular patterned (C) target and the scrambled versions of these two targets (SMX and SC) were used. Each treatment group comprised 6 chicks for ocular biometry (refraction and ocular dimension measurement) and 4 for analysis of retinal DA release. Vitreal dihydroxyphenylacetic acid (DOPAC) was analysed through ion-paired reversed phase high performance liquid chromatography with electrochemical detection (HPLC-ED), as a measure of retinal DA release. For the comparison between retinal DA release and eye growth, large reductions in retinal DA release possibly due to the decreased light level inside the cone-lens imaging system were observed across all treated eyes while only those exposed to low contrast, low spatial frequency sine wave grating, 1/f2, C and SC targets had myopic shifts in refraction. Amongst these treatment groups, no acute effect was observed and longer-term effects were only found in the low contrast and 1/f2 groups. These findings suggest that retinal DA release does not causally link visual stimuli properties to eye growth, and these target induced changes in refractive development are not dependent on the level of retinal DA release. Retinal dopaminergic cells might be affected indirectly via other retinal cells that immediately respond to changes in the image contrast of the retinal image.

Improved sensorimotor adaptation after exhaustive exercise is accompanied by altered brain activity

Acute exercise has been shown to exhibit different effects on human sensorimotor behavior; however, the causes and mechanisms of the responses are often not clear. The primary aim of the present study was to determine the effects of incremental running until exhaustion on sensorimotor performance and adaptation in a tracking task. Subjects were randomly assigned to a running group (RG), a tracking group (TG), or a running followed by tracking group (RTG), with 10 subjects assigned to each group. Treadmill running velocity was initially set at 2.0 m s− 1, increasing by 0.5 m s− 1 every 5 min until exhaustion. Tracking consisted of 35 episodes (each 40 s) where the subjects' task was to track a visual target on a computer screen while the visual feedback was veridical (performance) or left-right reversed (adaptation). Resting electroencephalographic (EEG) activity was recorded before and after each experimental condition (running, tracking, rest). Tracking performance and the final amount of adaptation did not differ between groups. However, task adaptation was significantly faster in RTG compared to TG. In addition, increased alpha and beta power were observed following tracking in TG but not RTG although exhaustive running failed to induce significant changes in these frequency bands. Our results suggest that exhaustive running can facilitate adaptation processes in a manual tracking task. Attenuated cortical activation following tracking in the exercise condition was interpreted to indicate cortical efficiency and exercise-induced facilitation of selective central processes during actual task demands.

Ocular characteristics of anisometropia

Animal models of refractive error development have demonstrated that visual experience influences ocular growth. In a variety of species, axial anisometropia (i.e. a difference in the length of the two eyes) can be induced through unilateral occlusion, image degradation or optical manipulation. In humans, anisometropia may occur in isolation or in association with amblyopia, strabismus or unilateral pathology. Non-amblyopic myopic anisometropia represents an interesting anomaly of ocular growth, since the two eyes within one visual system have grown to different endpoints. These experiments have investigated a range of biometric, optical and mechanical properties of anisometropic eyes (with and without amblyopia) with the aim of improving our current understanding of asymmetric refractive error development. In the first experiment, the interocular symmetry in 34 non-amblyopic myopic anisometropes (31 Asian, 3 Caucasian) was examined during relaxed accommodation. A high degree of symmetry was observed between the fellow eyes for a range of optical, biometric and biomechanical measurements. When the magnitude of anisometropia exceeded 1.75 D, the more myopic eye was almost always the sighting dominant eye. Further analysis of the optical and biometric properties of the dominant and non-dominant eyes was conducted to determine any related factors but no significant interocular differences were observed with respect to best-corrected visual acuity, corneal or total ocular aberrations during relaxed accommodation. Given the high degree of symmetry observed between the fellow eyes during distance viewing in the first experiment and the strong association previously reported between near work and myopia development, the aim of the second experiment was to investigate the symmetry between the fellow eyes of the same 34 myopic anisometropes following a period of near work. Symmetrical changes in corneal and total ocular aberrations were observed following a short reading task (10 minutes, 2.5 D accommodation demand) which was attributed to the high degree of interocular symmetry for measures of anterior eye morphology, and corneal biomechanics. These changes were related to eyelid shape and position during downward gaze, but gave no clear indication of factors associated with near work that might cause asymmetric eye growth within an individual. Since the influence of near work on eye growth is likely to be most obvious during, rather than following near tasks, in the third experiment the interocular symmetry of the optical and biometric changes was examined during accommodation for 11 myopic anisometropes. The changes in anterior eye biometrics associated with accommodation were again similar between the eyes, resulting in symmetrical changes in the optical characteristics. However, the more myopic eyes exhibited slightly greater amounts of axial elongation during accommodation which may be related to the force exerted by the ciliary muscle. This small asymmetry in axial elongation we observed between the eyes may be due to interocular differences in posterior eye structure, given that the accommodative response was equal between eyes. Using ocular coherence tomography a reduced average choroidal thickness was observed in the more myopic eyes compared to the less myopic eyes of these subjects. The interocular difference in choroidal thickness was correlated with the magnitude of spherical equivalent and axial anisometropia. The symmetry in optics and biometrics between fellow eyes which have undergone significantly different visual development (i.e. anisometropic subjects with amblyopia) is also of interest with respect to refractive error development. In the final experiment the influence of altered visual experience upon corneal and ocular higher-order aberrations was investigated in 21 amblyopic subjects (8 refractive, 11 strabismic and 2 form deprivation). Significant differences in aberrations were observed between the fellow eyes, which varied according to the type of amblyopia. Refractive amblyopes displayed significantly higher levels of 4th order corneal aberrations (spherical aberration and secondary astigmatism) in the amblyopic eye compared to the fellow non-amblyopic eye. Strabismic amblyopes exhibited significantly higher levels of trefoil, a third order aberration, in the amblyopic eye for both corneal and total ocular aberrations. The results of this experiment suggest that asymmetric visual experience during development is associated with asymmetries in higher-order aberrations, proportional to the magnitude of anisometropia and dependent upon the amblyogenic factor. This suggests a direct link between the development of higher-order optical characteristics of the human eye and visual feedback. The results from these experiments have shown that a high degree of symmetry exists between the fellow eyes of non-amblyopic myopic anisometropes for a range of biomechanical, biometric and optical parameters for different levels of accommodation and following near work. While a single specific optical or biomechanical factor that is consistently associated with asymmetric refractive error development has not been identified, the findings from these studies suggest that further research into the association between ocular dominance, choroidal thickness and higher-order aberrations with anisometropia may improve our understanding of refractive error development.

Reactive reaching and grasping on a humanoid: Towards closing the action-perception loop on the iCub

We propose a system incorporating a tight integration between computer vision and robot control modules on a complex, high-DOF humanoid robot. Its functionality is showcased by having our iCub humanoid robot pick-up objects from a table in front of it. An important feature is that the system can avoid obstacles - other objects detected in the visual stream - while reaching for the intended target object. Our integration also allows for non-static environments, i.e. the reaching is adapted on-the-fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. Furthermore we show that this system can be used both in autonomous and tele-operation scenarios.

From vision to actions: Towards adaptive and autonomous humanoid robots

Although robotics research has seen advances over the last decades robots are still not in widespread use outside industrial applications. Yet a range of proposed scenarios have robots working together, helping and coexisting with humans in daily life. In all these a clear need to deal with a more unstructured, changing environment arises. I herein present a system that aims to overcome the limitations of highly complex robotic systems, in terms of autonomy and adaptation. The main focus of research is to investigate the use of visual feedback for improving reaching and grasping capabilities of complex robots. To facilitate this a combined integration of computer vision and machine learning techniques is employed. From a robot vision point of view the combination of domain knowledge from both imaging processing and machine learning techniques, can expand the capabilities of robots. I present a novel framework called Cartesian Genetic Programming for Image Processing (CGP-IP). CGP-IP can be trained to detect objects in the incoming camera streams and successfully demonstrated on many different problem domains. The approach requires only a few training images (it was tested with 5 to 10 images per experiment) is fast, scalable and robust yet requires very small training sets. Additionally, it can generate human readable programs that can be further customized and tuned. While CGP-IP is a supervised-learning technique, I show an integration on the iCub, that allows for the autonomous learning of object detection and identification. Finally this dissertation includes two proof-of-concepts that integrate the motion and action sides. First, reactive reaching and grasping is shown. It allows the robot to avoid obstacles detected in the visual stream, while reaching for the intended target object. Furthermore the integration enables us to use the robot in non-static environments, i.e. the reaching is adapted on-the- fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. The second integration highlights the capabilities of these frameworks, by improving the visual detection by performing object manipulation actions.

Visualizing digital media interactions : providing feedback on jam2jam AV performances

Instrumental music performance is a well-established case of real-time interaction with technology and, when extended to ensembles, of interaction with others. However, these interactions are fleeting and the opportunities to reflect on action is limited, even though audio and video recording has recently provided important opportunities in this regard. In this paper we report on research to further extend these reflective opportunities through the capture and visualization of gestural data collected during collaborative virtual performances; specifically using the digital media instrument Jam2jam AV and the specifically-developed visualization software Jam2jam AV Visualize. We discusses how such visualization may assist performance development and understanding. The discussion engages with issues of representation, authenticity of virtual experiences, intersubjectivity and wordless collaboration, and creativity support. Two usage scenarios are described showing that collaborative intent is evident in the data visualizations more clearly than in audio-visual recordings alone, indicating that the visualization of performance gestures can be an efficient way of identifying deliberate and co-operative performance behaviours.

Improving recall in appearance-based visual SLAM using visual expectation

In this paper, we present a new algorithm for boosting visual template recall performance through a process of visual expectation. Visual expectation dynamically modifies the recognition thresholds of learnt visual templates based on recently matched templates, improving the recall of sequences of familiar places while keeping precision high, without any feedback from a mapping backend. We demonstrate the performance benefits of visual expectation using two 17 kilometer datasets gathered in an outdoor environment at two times separated by three weeks. The visual expectation algorithm provides up to a 100% improvement in recall. We also combine the visual expectation algorithm with the RatSLAM SLAM system and show how the algorithm enables successful mapping

Image based visual servo control for fixed wing UAVs tracking linear infrastructure in wind

This paper presents an Image Based Visual Servo control design for Fixed Wing Unmanned Aerial Vehicles tracking locally linear infrastructure in the presence of wind using a body fixed imaging sensor. Visual servoing offers improved data collection by posing the tracking task as one of controlling a feature as viewed by the inspection sensor, although is complicated by the introduction of wind as aircraft heading and course angle no longer align. In this work it is shown that the effects of wind alter the desired line angle required for continuous tracking to equal the wind correction angle as would be calculated to set a desired course. A control solution is then sort by linearizing the interaction matrix about the new feature pose such that kinematics of the feature can be augmented with the lateral dynamics of the aircraft, from which a state feedback control design is developed. Simulation results are presented comparing no compensation, integral control and the proposed controller using the wind correction angle, followed by an assessment of response to atmospheric disturbances in the form of turbulence and wind gusts

Bombs Away: visual thinking and students’ engagement in design studios contexts

In design studio, sketching or visual thinking is part of processes that assist students to achieve final design solutions. At QUT’s First and Third Year industrial design studio classes we engage in a variety of teaching pedagogies from which we identify ‘Concept Bombs’ as an instrumental in the development of students’ visual thinking and reflective design process, and also as a vehicle to foster positive student engagement. Our ‘formula’: Concept Bombs are 20 minute design tasks focusing on rapid development of initial concept designs and free-hand sketching. Our experience and surveys tell us that students value intensive studio activities especially when combined with timely assessment and feedback. While conventional longer-duration design projects are essential for allowing students to engage with the full depth and complexity of the design process, short and intensive design activities introduce variety to the learning experience and enhance student engagement. This paper presents a comparative analysis of First and Third Year students’ Concept Bomb sketches to describe the types of design knowledge embedded in them, a discussion of limitations and opportunities of this pedagogical technique, as well as considerations for future development of studio based tasks of this kind as design pedagogies in the midst of current university education trends.

Mentor feedback : models, viewpoints and strategies

A mentor’s feedback can present professional insights to allow a mentee to reflect and develop practice. This paper positions two models for feedback that have emanated from empirical studies. It also demonstrates the diverse viewpoints of mentors and suggests strategies for providing quality feedback. In one qualitative study, 24 mentors observed a final-year preservice teacher through a professionally video-recorded lesson and wrote their observations towards giving feedback to the potential mentee. Tables illustrated in the paper, show that mentors’ positive feedback and constructive criticisms vary considerably on the same observed events. Data from this study were synthesised to posit a theoretical model for analysing mentor feedback in an interconnected, three-way Venn diagram, namely: visual, auditory and conceptual frames. Another study (n=28), which is a collection of mentor teachers’ work samples during the Mentoring for Effective Teaching (MET) program, provides strategies within six feedback practices, that is: (1) negotiated mentor-mentee expectations for providing feedback on practices, (2) reviewing teaching plans, (3) arranging for observations of practices, (4) providing oral feedback, (5) providing written feedback, and; (6) presenting opportunities for the mentee to evaluate teaching practices with consideration of the mentor’s feedback. For example, on the last mentioned practice (6) there were strategies such as “Plan a time for evaluation of practices (guided reflection)”, “Read the mentee’s reflection on practice and discuss how it aligns with your observations of their practices”, and “Highlight verbally and/or in writing where the mentee is perceptive about the reflection and how the reflection could be enhanced for future evaluations”. Developing a range of strategies that may assist the mentee in professional growth, include enlisting a community of mentors, ensuring mentors have a repertoire of strategies for articulating feedback, and using mentor feedback tools and models. This study has implications for the development of feedback models and strategies.