Sonification as Concurrent Augmented Feedback for Motor Skill Learning and the Importance of Mapping Design

In recent years, sonification of movement has emerged as a viable method for the provision of feedback in motor learning. Despite some experimental validation of its utility, controlled trials to test the usefulness of sonification in a motor learning context are still rare. As such, there are no accepted conventions for dealing with its implementation. This article addresses the question of how continuous movement information should be best presented as sound to be fed back to the learner. It is proposed that to establish effective approaches to using sonification in this context, consideration must be given to the processes that underlie motor learning, in particular the nature of the perceptual information available to the learner for performing the task at hand. Although sonification has much potential in movement performance enhancement, this potential is largely unrealised as of yet, in part due to the lack of a clear framework for sonification mapping: the relationship between movement and sound. By grounding mapping decisions in a firmer understanding of how perceptual information guides learning, and an embodied cognition stance in general, it is hoped that greater advances in use of sonification to enhance motor learning can be achieved.

Transposing Musical Skill: Sonification of movement as concurrent augmented feedback enhances learning in a bimanual task

Concurrent feedback provided during acquisition can enhance performance of novel tasks. The ‘guidance hypothesis’ predicts that feedback provision leads to dependence and poor performance in its absence. However, appropriately-structured feedback information provided through sound (‘sonification’) may not be subject to this effect. We test this directly using a rhythmic bimanual shape-tracing task in which participants learned to move at a 4:3 timing ratio. Sonification of movement and demonstration was compared to two other learning conditions: (1) sonification of task demonstration alone and (2) completely silent practice (control). Sonification of movement emerged as the most effective form of practice, reaching significantly lower error scores than control. Sonification of solely the demonstration, which was expected to benefit participants by perceptually unifying task requirements, did not lead to better performance than control. Good performance was maintained by participants in the sonification condition in an immediate retention test without feedback, indicating that the use of this feedback can overcome the guidance effect. On a 24-hour retention test, performance had declined and was equal between groups. We argue that this and similar findings in the feedback literature are best explained by an ecological approach to motor skill learning which places available perceptual information at the highest level of importance.

The effects of manipulated augmented sensory feedback on error detection when using a touch screen

The purpose of the present study was to determine which augmented sensory modality would best develop subjective error-detection capabilities of learners performing a spatial-temporal task when using a touch screen monitor. Participants were required to learn a 5-digit key-pressing task in a goal time of 2550 ms over 100 acquisition trials on a touch screen. Participants were randomized into 1 of 4 groups: 1) visual-feedback (colour change of button when selected), 2) auditory-feedback (click sound when button was selected), 3) visual-auditory feedback (both colour change and click sound when button was selected), and 4) no-feedback (no colour change or click sound when button was selected). Following each trial, participants were required to provide a subjective estimate regarding their performance time in relation to the actual time it took for them complete the 5-digit sequence. A no-KR retention test was conducted approximately 24-hours after the last completed acquisition trial. Results showed that practicing a timing task on a touch screen augmented with both visual and auditory information may have differentially impacted motor skill acquisition such that removal of one or both sources of augmented feedback did not result in a severe detriment to timing performance or error detection capabilities of the learner. The present study reflects the importance of multimodal augmented feedback conditions to maximize cognitive abilities for developing a stronger motor memory for subjective error-detection and correction capabilities.

Visual feedback alters the variations in corticospinal excitability that arise from rhythmic movements of the opposite limb

Augmented visual feedback can have a profound bearing on the stability of bimanual coordination. Indeed, this has been used to render tractable the study of patterns of coordination that cannot otherwise be produced in a stable fashion. In previous investigations (Carson et al. 1999), we have shown that rhythmic movements, brought about by the contraction of muscles on one side of the body, lead to phase-locked changes in the excitability of homologous motor pathways of the opposite limb. The present study was conducted to assess whether these changes are influenced by the presence of visual feedback of the moving limb. Eight participants performed rhythmic flexion-extension movements of the left wrist to the beat of a metronome (1.5 Hz). In 50% of trials, visual feedback of wrist displacement was provided in relation to a target amplitude, defined by the mean movement amplitude generated during the immediately preceding no feedback trial. Motor potentials (MEPs) were evoked in the quiescent muscles of the right limb by magnetic stimulation of the left motor cortex. Consistent with our previous observations, MEP amplitudes were modulated during the movement cycle of the opposite limb. The extent of this modulation was, however, smaller in the presence of visual feedback of the moving limb (FCR omega(2) =0.41; ECR omega(2)=0.29) than in trials in which there was no visual feedback (FCR omega(2)=0.51; ECR omega(2)=0.48). In addition, the relationship between the level of FCR activation and the excitability of the homologous corticospinal pathway of the opposite limb was sensitive to the vision condition; the degree of correlation between the two variables was larger when there was no visual feedback of the moving limb. The results of the present study support the view that increases in the stability of bimanual coordination brought about by augmented feedback may be mediated by changes in the crossed modulation of excitability in homologous motor pathways.

Riabilitazione e valutazione di pazienti con deficit neurologici mediante l'utilizzo di realtà virtuale, feedback aumentati, sensori inerziali e tecnologie video a basso costo.

La neuroriabilitazione è un processo attraverso cui individui affetti da patologie neurologiche mirano al conseguimento di un recupero completo o alla realizzazione del loro potenziale ottimale benessere fisico, mentale e sociale. Elementi essenziali per una riabilitazione efficace sono: una valutazione clinica da parte di un team multidisciplinare, un programma riabilitativo mirato e la valutazione dei risultati conseguiti mediante misure scientifiche e clinicamente appropriate. Obiettivo principale di questa tesi è stato sviluppare metodi e strumenti quantitativi per il trattamento e la valutazione motoria di pazienti neurologici. I trattamenti riabilitativi convenzionali richiedono a pazienti neurologici l’esecuzione di esercizi ripetitivi, diminuendo la loro motivazione. La realtà virtuale e i feedback sono in grado di coinvolgerli nel trattamento, permettendo ripetibilità e standardizzazione dei protocolli. È stato sviluppato e valutato uno strumento basato su feedback aumentati per il controllo del tronco. Inoltre, la realtà virtuale permette l’individualizzare il trattamento in base alle esigenze del paziente. Un’applicazione virtuale per la riabilitazione del cammino è stata sviluppata e testata durante un training su pazienti di sclerosi multipla, valutandone fattibilità e accettazione e dimostrando l'efficacia del trattamento. La valutazione quantitativa delle capacità motorie dei pazienti viene effettuata utilizzando sistemi di motion capture. Essendo il loro uso nella pratica clinica limitato, una metodologia per valutare l’oscillazione delle braccia in soggetti parkinsoniani basata su sensori inerziali è stata proposta. Questi sono piccoli, accurati e flessibili ma accumulano errori durante lunghe misurazioni. È stato affrontato questo problema e i risultati suggeriscono che, se il sensore è sul piede e le accelerazioni sono integrate iniziando dalla fase di mid stance, l’errore e le sue conseguenze nella determinazione dei parametri spaziali sono contenuti. Infine, è stata presentata una validazione del Kinect per il tracking del cammino in ambiente virtuale. Risultati preliminari consentono di definire il campo di utilizzo del sensore in riabilitazione.

Visual feedback alters the variations in corticospinal excitability that arise from rhythmic movements of the opposite limb

Augmented visual feedback can have a profound bearing on the stability of bimanual coordination. Indeed, this has been used to render tractable the study of patterns of coordination that cannot otherwise be produced in a stable fashion. In previous investigations (Carson et al. 1999), we have shown that rhythmic movements, brought about by the contraction of muscles on one side of the body, lead to phase-locked changes in the excitability of homologous motor pathways of the opposite limb. The present study was conducted to assess whether these changes are influenced by the presence of visual feedback of the moving limb. Eight participants performed rhythmic flexion-extension movements of the left wrist to the beat of a metronome (1.5 Hz). In 50% of trials, visual feedback of wrist displacement was provided in relation to a target amplitude, defined by the mean movement amplitude generated during the immediately preceding no feedback trial. Motor potentials (MEPs) were evoked in the quiescent muscles of the right limb by magnetic stimulation of the left motor cortex. Consistent with our previous observations, MEP amplitudes were modulated during the movement cycle of the opposite limb. The extent of this modulation was, however, smaller in the presence of visual feedback of the moving limb (FCR omega(2) =0.41; ECR omega(2)=0.29) than in trials in which there was no visual feedback (FCR omega(2)=0.51; ECR omega(2)=0.48). In addition, the relationship between the level of FCR activation and the excitability of the homologous corticospinal pathway of the opposite limb was sensitive to the vision condition; the degree of correlation between the two variables was larger when there was no visual feedback of the moving limb. The results of the present study support the view that increases in the stability of bimanual coordination brought about by augmented feedback may be mediated by changes in the crossed modulation of excitability in homologous motor pathways.

The effect of altering a single component of a rehabilitation programme on the functional recovery of stroke patients: a systematic review and meta-analysis

OBJECTIVE: To evaluate the effect of altering a single component of a rehabilitation programme (e.g. adding bilateral practice alone) on functional recovery after stroke, defined using a measure of activity.

DATA SOURCES: A search was conducted of Medline/Pubmed, CINAHL and Web of Science.

REVIEW METHODS: Two reviewers independently assessed eligibility. Randomized controlled trials were included if all participants received the same base intervention, and the experimental group experienced alteration of a single component of the training programme. This could be manipulation of an intrinsic component of training (e.g. intensity) or the addition of a discretionary component (e.g. augmented feedback). One reviewer extracted the data and another independently checked a subsample (20%). Quality was appraised according to the PEDro scale.

RESULTS: Thirty-six studies (n = 1724 participants) were included. These evaluated nine training components: mechanical degrees of freedom, intensity of practice, load, practice schedule, augmented feedback, bilateral movements, constraint of the unimpaired limb, mental practice and mirrored-visual feedback. Manipulation of the mechanical degrees of freedom of the trunk during reaching and the addition of mental practice during upper limb training were the only single components found to independently enhance recovery of function after stroke.

CONCLUSION: This review provides limited evidence to support the supposition that altering a single component of a rehabilitation programme realises greater functional recovery for stroke survivors. Further investigations are required to determine the most effective single components of rehabilitation programmes, and the combinations that may enhance functional recovery.

Cerebellar augmented joint control for a humanoid robot

The joints of a humanoid robot experience disturbances of markedly different magnitudes during the course of a walking gait. Consequently, simple feedback control techniques poorly track desired joint trajectories. This paper explores the addition of a control system inspired by the architecture of the cerebellum to improve system response. This system learns to compensate the changes in load that occur during a cycle of motion. The joint compensation scheme, called Trajectory Error Learning, augments the existing feedback control loop on a humanoid robot. The results from tests on the GuRoo platform show an improvement in system response for the system when augmented with the cerebellar compensator.

Assessing the strain experienced by managers and professional Australian footballers using an augmented job strain model

Generic models of job stress, such as the Job Strain Model (JSM), have recently been criticised for focusing on a small number of general work characteristics while ignoring those that are occupation-specific (Sparks & Cooper, 1999). However this criticism is based on limited research that has not examined the relative influence of all three dimensions of the JSM - job demand, job control and social support - and job-specific stressors. The JSM is the most commonly used model underpinning large-scale occupational stress research (Fox, Dwyer, & Ganster, 1993) and is regarded as the most influential model in the research on the psycho-social work environment, stress and disease in recent times (Kristensen, 1995). This thesis addresses the lack of information on the relative influence of the JSM and job-specific stressors by assessing the capacity of an augmented JSM to predict the strain experienced by managers and professional Australian footballers. The augmented JSM consisted of job-specific stressors in addition to the generic components of the model. Managers and professional Australian footballers represent two very different occupational groups. While the day-today roles of a manager include planning, organising, monitoring and controlling (Carroll & Gillen, 1987), the working life of a professional Australian footballer revolves around preparing for and playing football (Shanahan, 1998). It was expected that the large differences in the work undertaken by managers and professional Australian footballers would maximise the opportunities for identifying job-specific stressors and measuring the extent that these vary from one group to the next. The large disparity between managers and professional footballers was also used to assess the cross-occupational versatility of the JSM when it had been augmented by job-specific stressors. This thesis consisted of three major studies. Study One involved a survey of Australian managers, while studies Two and Three focused on professional Australian footballers. The latter group was under-represented in the literature, and as a result of the lack of information on the stressors commonly experienced by this group, an in-depth qualitative study was undertaken in Study Two. The results from Study Two then informed the survey of professional footballers that was conducted in Study Three. Contrary to previous research examining the relative influence of generic and job-specific stressors, the results only provided moderate support for augmenting the JSM with job-specific stressors. Instead of supporting the versatility of the augmented JSM, the overall findings reinforced the broad relevance of the original JSM. Of the four health outcomes measured in Studies One and Three, there was only one - the psychological health of professional Australian footballers - where the proportion of total variance explained by job-specific stressors exceeded 13%. Despite the generally strong performance of the JSM across the two occupational groups, the importance of demand, control and support diminished when examining the less conventional occupation of professional football. The generic model was too narrow to capture the highly specific work characteristics that are important for this occupational group and, as a result, the job-specific stressors explained significantly more of the strain over and above that already provided by the generic model. These findings indicate that when investigating the stressors experienced by conventional occupational groups such as managers, the large amount resources required to identify job-specific stressors are unlikely to be cost-effective. In contrast, the influence of the more situation specific stressors is significantly greater in unconventional occupations and thus the benefits of identifying these non-generic stressors are more likely to outweigh the costs. Studies One and Three identified strong connections between job-specific stressors and important characteristics of the occupation being studied. These connections were consistent with previous research and suggest that before attempting to identify job-specific stressors, researchers need to first become familiar with the nature and context of the occupation. The final issue addressed in this thesis was the role of work and non-work support. The findings indicate that the support provided by supervisors and colleagues was a significant predictor of wellbeing for both managers and professional footballers. In contrast, the level of explained strain accounted for by non-work support was not significant. These results indicate that when developing strategies to protect and enhance employee well-being, particular attention should be given to monitoring and, where necessary, boosting the effectiveness of work-based support. The findings from this thesis have been fed back to the management and sporting communities via conference presentations and peer-reviewed journals (refer pp 220-221). All three studies have been presented at national and international conferences and, overall, were well received by participants. Similarly, the methods, results and major findings arising from Studies One and Two have been critiqued by anonymous reviewers from two international journals. These papers have been accepted for publication in 2001 and 2002 and feedback from the reviewers indicates that the findings represent a significant and unique contribution to the literature. The results of Study Three are currently under review by a sports psychology journal.

Paradigm shift to enhanced water supply planning through augmented grids, scarcity pricing and adaptive factory water : a system dynamics approach

This paper details a system dynamics model developed to simulate proposed changes to water governance through the integration of supply, demand and asset management processes. To effectively accomplish this, interconnected feedback loops in tariff structures, demand levels and financing capacity are included in the model design, representing the first comprehensive life-cycle modelling of potable water systems. A number of scenarios were applied to Australia's populated South-east Queensland region, demonstrating that introducing temporary drought pricing (i.e. progressive water prices set inverse with availability), in conjunction with supply augmentation through rain-independent sources, is capable of efficiently providing water security in the future. Modelling demonstrated that this alternative tariff structure reduced demand in scarcity periods thereby preserving supply, whilst revenues are maintained to build new water supply infrastructure. In addition to exploring alternative tariffs, the potential benefits of using adaptive pressure-retarded osmosis desalination plants for both potable water and power generation was explored. This operation of these plants for power production, when they would otherwise be idle, shows promise in reducing their net energy and carbon footprints. Stakeholders in industry, government and academia were engaged in model development and validation. The constructed model displays how water resource systems can be reorganised to cope with systemic change and uncertainty.

Piloting the feasibility of head-mounted video technology to augment student feedback during simulated clinical decision-making: an observational design pilot study

BACKGROUND: Clinical decision-making is a complex activity that is critical to patient safety. Simulation, augmented by feedback, affords learners the opportunity to learn critical clinical decision-making skills. More detailed feedback following simulation exercises has the potential to further enhance student learning, particularly in relation to developing improved clinical decision-making skills. OBJECTIVE: To investigate the feasibility of head-mounted video camera recordings, to augment feedback, following acute patient deterioration simulations. DESIGN: Pilot study using an observational design. METHODS: Ten final-year nursing students participated in three simulation exercises, each focussed on detection and management of patient deterioration. Two observers collected behavioural data using an adapted version of Gaba's Clinical Simulation Tool, to provide verbal feedback to each participant, following each simulation exercise. Participants wore a head-mounted video camera during the second simulation exercise only. Video recordings were replayed to participants to augment feedback, following the second simulation exercise. Data were collected on: participant performance (observed and perceived); participant perceptions of feedback methods; and head-mounted video camera recording feasibility and capability for detailed audio-visual feedback. RESULTS: Management of patient deterioration improved for six participants (60%). Increased perceptions of confidence (70%) and competence (80%), were reported by the majority of participants. Few participants (20%) agreed that the video recording specifically enhanced their learning. The visual field of the head-mounted video camera was not always synchronised with the participant's field of vision, thus affecting the usefulness of some recordings. CONCLUSION: The usefulness of the video recordings, to enhance verbal feedback to participants on detection and management of simulated patient deterioration, was inconclusive. Modification of the video camera glasses, to improve visual-field synchronisation with participants' actual visual field, is recommended to further explore this technology for enhancing student performance.

SysteM of augmented reality for teaching kids: smartkids

As crianças do jardim de infância/pré-escola são um grupo de utilizadores muito especiais, uma vez que se encontram numa fase inicial das suas vidas onde têm de aprender a viver em sociedade, isto é, aprender a ouvir e respeitar as opiniões dos outros, partilhar os mesmos objectos e também a ajudar-se mutuamente. Este estudo mostra que através da utilização da tecnologia Realidade Aumentada, estas crianças são capazes de colaborar de uma forma espontânea suportada pela motivação, envolvimento e curiosidade. Descreve-se o design e a avaliação de um jogo de Realidade Aumentada, que consiste num sistema educativo para o ensino de crianças do pré-escolar. Este jogo permite que as crianças explorem conceitos tais como os animais e os meios onde vivem através de marcadores de Realidade Aumentada e de um tabuleiro de madeira. Estes marcadores consistem nas peças do jogo e através deles, as crianças podem manipular objectos virtuais em 3D. Foram realizados testes com diversas turmas de crianças em diferentes ambientes de aprendizagem, nomeadamente em escolas e num museu. Os resultados sugerem que o jogo é eficaz para obter níveis altos de concentração, motivação e colaboração entre as crianças, particularmente quando o feedback do jogo é fornecido de forma imediata. Os resultados mostram também que o jogo tem um impacto positivo sobre a experiência de aprendizagem das crianças.

The Effects of Systematic Self-Monitoring, Feedback, and Collaborative Assessment on Stress Reduction

Thesis (Ph.D.)--University of Washington, 2016-08