Single trial learning of external dynamics: What can the brain teach us about learning mechanisms?

Humans have exceptional abilities to learn new skills, manipulate tools and objects, and interact with our environment. In order to be successful at these tasks, our brain has developed learning mechanisms to deal with and compensate for the constantly changing dynamics of the world. If this mechanism or mechanisms can be understood from a computational point of view, then they can also be used to drive the adaptability and learning of robots. In this paper, we will present a new technique for examining changes in the feedforward motor command due to adaptation. This technique can then be utilized for examining motor adaptation in humans and determining a computational algorithm which explains motor learning. © 2007.

Learning the dynamics of the external world: Brain inspired learning for robotic applications

Humans have exceptional abilities to learn new skills, manipulate tools and objects, and interact with our environment. In order to be successful at these tasks, our brain has become exceptionally well adapted to learning to deal not only with the complex dynamics of our own limbs but also with novel dynamics in the external world. While learning of these dynamics includes learning the complex time-varying forces at the end of limbs through the updating of internal models, it must also include learning the appropriate mechanical impedance in order to stabilize both the limb and any objects contacted in the environment. This article reviews the field of human learning by examining recent experimental evidence about adaptation to novel unstable dynamics and explores how this knowledge about the brain and neuro-muscular system can expand the learning capabilities of robotics and prosthetics. © 2006.

Learning feedforward commands to muscles using time-shifted sensory feedback

In adapting to changing forces in the mechanical environment, humans change the force being applied by the limb by reciprocal changes in the activation of antagonistic muscles. However, they also cocontract these muscles when interaction with the environment is mechanically unstable to increase the mechanical impedance of the limb. We have postulated that appropriate patterns of muscle activation could be learned using a simple scheme in which the naturally occurring stretch reflex is used as a template to adjust feedforward commands to muscles. Feedforward commands are modified iteratively by shifting a scaled version of the reflex response forward in time and adding it to the previous feedforward command. We show that such an algorithm can account for the principal features of changes in muscle activation observed when human subjects adapt to instabilities in the mechanical environment. © 2006.

How are internal models of unstable tasks formed?

The results of recent studies suggest that humans can form internal models that they use in a feedforward manner to compensate for both stable and unstable dynamics. To examine how internal models are formed, we performed adaptation experiments in novel dynamics, and measured the endpoint force, trajectory and EMG during learning. Analysis of reflex feedback and change of feedforward commands between consecutive trials suggested a unified model of motor learning, which can coherently unify the learning processes observed in stable and unstable dynamics and reproduce available data on motor learning. To our knowledge, this algorithm, based on the concurrent minimization of (reflex) feedback and muscle activation, is also the first nonlinear adaptive controller able to stabilize unstable dynamics.

Measuring stiffness during arm movements in various dynamic environments

The technique presented in this paper enables a simple, accurate and unbiased measurement of hand stiffness during human arm movements. Using a computer-controlled mechanical interface, the hand is shifted relative to a prediction of the undisturbed trajectory. Stiffness is then computed as the restoring force divided by the position amplitude of the perturbation. A precise prediction algorithm insures the measurement quality. We used this technique to measure stiffness in free movements and after adaptation to a linear velocity dependent force field. The subjects compensated for the external force by co-contracting muscles selectively. The stiffness geometry changed with learning and stiffness tended to increase in the direction of the external force.