Transfer of dynamic learning across postures.

When learning a difficult motor task, we often decompose the task so that the control of individual body segments is practiced in isolation. But on re-composition, the combined movements can result in novel and possibly complex internal forces between the body segments that were not experienced (or did not need to be compensated for) during isolated practice. Here we investigate whether dynamics learned in isolation by one part of the body can be used by other parts of the body to immediately predict and compensate for novel forces between body segments. Subjects reached to targets while holding the handle of a robotic, force-generating manipulandum. One group of subjects was initially exposed to the novel robot dynamics while seated and was then tested in a standing position. A second group was tested in the reverse order: standing then sitting. Both groups adapted their arm dynamics to the novel environment, and this movement learning transferred between seated and standing postures and vice versa. Both groups also generated anticipatory postural adjustments when standing and exposed to the force field for several trials. In the group that had learned the dynamics while seated, the appropriate postural adjustments were observed on the very first reach on standing. These results suggest that the CNS can immediately anticipate the effect of learned movement dynamics on a novel whole-body posture. The results support the existence of separate mappings for posture and movement, which encode similar dynamics but can be adapted independently.

Context-dependent partitioning of motor learning in bimanual movements.

Human subjects easily adapt to single dynamic or visuomotor perturbations. In contrast, when two opposing dynamic or visuomotor perturbations are presented sequentially, interference is often observed. We examined the effect of bimanual movement context on interference between opposing perturbations using pairs of contexts, in which the relative direction of movement between the two arms was different across the pair. When each perturbation direction was associated with a different bimanual context, such as movement of the arms in the same direction versus movement in the opposite direction, interference was dramatically reduced. This occurred over a short period of training and was seen for both dynamic and visuomotor perturbations, suggesting a partitioning of motor learning for the different bimanual contexts. Further support for this was found in a series of transfer experiments. Having learned a single dynamic or visuomotor perturbation in one bimanual context, subjects showed incomplete transfer of this learning when the context changed, even though the perturbation remained the same. In addition, we examined a bimanual context in which one arm was moved passively and show that the reduction in interference requires active movement. The sensory consequences of movement are thus insufficient to allow opposing perturbations to be co-represented. Our results suggest different bimanual movement contexts engage at least partially separate representations of dynamics and kinematics in the motor system.

Effects of the user model on simulation-based learning of dialogue strategies

Over the past decade, a variety of user models have been proposed for user simulation-based reinforcement-learning of dialogue strategies. However, the strategies learned with these models are rarely evaluated in actual user trials and it remains unclear how the choice of user model affects the quality of the learned strategy. In particular, the degree to which strategies learned with a user model generalise to real user populations has not be investigated. This paper presents a series of experiments that qualitatively and quantitatively examine the effect of the user model on the learned strategy. Our results show that the performance and characteristics of the strategy are in fact highly dependent on the user model. Furthermore, a policy trained with a poor user model may appear to perform well when tested with the same model, but fail when tested with a more sophisticated user model. This raises significant doubts about the current practice of learning and evaluating strategies with the same user model. The paper further investigates a new technique for testing and comparing strategies directly on real human-machine dialogues, thereby avoiding any evaluation bias introduced by the user model. © 2005 IEEE.

The effect of contextual cues on the encoding of motor memories.

Several studies have shown that sensory contextual cues can reduce the interference observed during learning of opposing force fields. However, because each study examined a small set of cues, often in a unique paradigm, the relative efficacy of different sensory contextual cues is unclear. In the present study we quantify how seven contextual cues, some investigated previously and some novel, affect the formation and recall of motor memories. Subjects made movements in a velocity-dependent curl field, with direction varying randomly from trial to trial but always associated with a unique contextual cue. Linking field direction to the cursor or background color, or to peripheral visual motion cues, did not reduce interference. In contrast, the orientation of a visual object attached to the hand cursor significantly reduced interference, albeit by a small amount. When the fields were associated with movement in different locations in the workspace, a substantial reduction in interference was observed. We tested whether this reduction in interference was due to the different locations of the visual feedback (targets and cursor) or the movements (proprioceptive). When the fields were associated only with changes in visual display location (movements always made centrally) or only with changes in the movement location (visual feedback always displayed centrally), a substantial reduction in interference was observed. These results show that although some visual cues can lead to the formation and recall of distinct representations in motor memory, changes in spatial visual and proprioceptive states of the movement are far more effective than changes in simple visual contextual cues.

Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease.

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning.

Dopamine and performance in a reinforcement learning task: Evidence from Parkinson's disease

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning. © 2012 The Author.