Learning optimal adaptation strategies in unpredictable motor tasks.

Picking up an empty milk carton that we believe to be full is a familiar example of adaptive control, because the adaptation process of estimating the carton's weight must proceed simultaneously with the control process of moving the carton to a desired location. Here we show that the motor system initially generates highly variable behavior in such unpredictable tasks but eventually converges to stereotyped patterns of adaptive responses predicted by a simple optimality principle. These results suggest that adaptation can become specifically tuned to identify task-specific parameters in an optimal manner.

Intonation modelling and adaptation for emotional prosody generation

This paper proposes an HMM-based approach to generating emotional intonation patterns. A set of models were built to represent syllable-length intonation units. In a classification framework, the models were able to detect a sequence of intonation units from raw fundamental frequency values. Using the models in a generative framework, we were able to synthesize smooth and natural sounding pitch contours. As a case study for emotional intonation generation, Maximum Likelihood Linear Regression (MLLR) adaptation was used to transform the neutral model parameters with a small amount of happy and sad speech data. Perceptual tests showed that listeners could identify the speech with the sad intonation 80% of the time. On the other hand, listeners formed a bimodal distribution in their ability to detect the system generated happy intontation and on average listeners were able to detect happy intonation only 46% of the time. © Springer-Verlag Berlin Heidelberg 2005.

Bayesian adaptation and adaptively trained systems

As the use of found data increases, more systems are being built using adaptive training. Here transforms are used to represent unwanted acoustic variability, e.g. speaker and acoustic environment changes, allowing a canonical model that models only the "pure" variability of speech to be trained. Adaptive training may be described within a Bayesian framework. By using complexity control approaches to ensure robust parameter estimates, the standard point estimate adaptive training can be justified within this Bayesian framework. However during recognition there is usually no control over the amount of data available. It is therefore preferable to be able to use a full Bayesian approach to applying transforms during recognition rather than the standard point estimates. This paper discusses various approximations to Bayesian approaches including a new variational Bayes approximation. The application of these approaches to state-of-the-art adaptively trained systems using both CAT and MLLR transforms is then described and evaluated on a large vocabulary speech recognition task. © 2005 IEEE.

Constrained discriminative mapping transforms for unsupervised speaker adaptation

Discriminative mapping transforms (DMTs) is an approach to robustly adding discriminative training to unsupervised linear adaptation transforms. In unsupervised adaptation DMTs are more robust to unreliable transcriptions than directly estimating adaptation transforms in a discriminative fashion. They were previously proposed for use with MLLR transforms with the associated need to explicitly transform the model parameters. In this work the DMT is extended to CMLLR transforms. As these operate in the feature space, it is only necessary to apply a different linear transform at the front-end rather than modifying the model parameters. This is useful for rapidly changing speakers/environments. The performance of DMTs with CMLLR was evaluated on the WSJ 20k task. Experimental results show that DMTs based on constrained linear transforms yield 3% to 6% relative gain over MLE transforms in unsupervised speaker adaptation. © 2011 IEEE.

A single-rate context-dependent learning process underlies rapid adaptation to familiar object dynamics.

Motor learning has been extensively studied using dynamic (force-field) perturbations. These induce movement errors that result in adaptive changes to the motor commands. Several state-space models have been developed to explain how trial-by-trial errors drive the progressive adaptation observed in such studies. These models have been applied to adaptation involving novel dynamics, which typically occurs over tens to hundreds of trials, and which appears to be mediated by a dual-rate adaptation process. In contrast, when manipulating objects with familiar dynamics, subjects adapt rapidly within a few trials. Here, we apply state-space models to familiar dynamics, asking whether adaptation is mediated by a single-rate or dual-rate process. Previously, we reported a task in which subjects rotate an object with known dynamics. By presenting the object at different visual orientations, adaptation was shown to be context-specific, with limited generalization to novel orientations. Here we show that a multiple-context state-space model, with a generalization function tuned to visual object orientation, can reproduce the time-course of adaptation and de-adaptation as well as the observed context-dependent behavior. In contrast to the dual-rate process associated with novel dynamics, we show that a single-rate process mediates adaptation to familiar object dynamics. The model predicts that during exposure to the object across multiple orientations, there will be a degree of independence for adaptation and de-adaptation within each context, and that the states associated with all contexts will slowly de-adapt during exposure in one particular context. We confirm these predictions in two new experiments. Results of the current study thus highlight similarities and differences in the processes engaged during exposure to novel versus familiar dynamics. In both cases, adaptation is mediated by multiple context-specific representations. In the case of familiar object dynamics, however, the representations can be engaged based on visual context, and are updated by a single-rate process.

Functional significance of stiffness in adaptation of multijoint arm movements to stable and unstable dynamics.

This study compared the mechanisms of adaptation to stable and unstable dynamics from the perspective of changes in joint mechanics. Subjects were instructed to make point to point movements in force fields generated by a robotic manipulandum which interacted with the arm in either a stable or an unstable manner. After subjects adjusted to the initial disturbing effects of the force fields they were able to produce normal straight movements to the target. In the case of the stable interaction, subjects modified the joint torques in order to appropriately compensate for the force field. No change in joint torque or endpoint force was required or observed in the case of the unstable interaction. After adaptation, the endpoint stiffness of the arm was measured by applying displacements to the hand in eight different directions midway through the movements. This was compared to the stiffness measured similarly during movements in a null force field. After adaptation, the endpoint stiffness under both the stable and unstable dynamics was modified relative to the null field. Adaptation to unstable dynamics was achieved by selective modification of endpoint stiffness in the direction of the instability. To investigate whether the change in endpoint stiffness could be accounted for by change in joint torque or endpoint force, we estimated the change in stiffness on each trial based on the change in joint torque relative to the null field. For stable dynamics the change in endpoint stiffness was accurately predicted. However, for unstable dynamics the change in endpoint stiffness could not be reproduced. In fact, the predicted endpoint stiffness was similar to that in the null force field. Thus, the change in endpoint stiffness seen after adaptation to stable dynamics was directly related to changes in net joint torque necessary to compensate for the dynamics in contrast to adaptation to unstable dynamics, where a selective change in endpoint stiffness occurred without any modification of net joint torque.

Different mechanisms involved in adaptation to stable and unstable dynamics.

Recently, we demonstrated that humans can learn to make accurate movements in an unstable environment by controlling magnitude, shape, and orientation of the endpoint impedance. Although previous studies of human motor learning suggest that the brain acquires an inverse dynamics model of the novel environment, it is not known whether this control mechanism is operative in unstable environments. We compared learning of multijoint arm movements in a "velocity-dependent force field" (VF), which interacted with the arm in a stable manner, and learning in a "divergent force field" (DF), where the interaction was unstable. The characteristics of error evolution were markedly different in the 2 fields. The direction of trajectory error in the DF alternated to the left and right during the early stage of learning; that is, signed error was inconsistent from movement to movement and could not have guided learning of an inverse dynamics model. This contrasted sharply with trajectory error in the VF, which was initially biased and decayed in a manner that was consistent with rapid feedback error learning. EMG recorded before and after learning in the DF and VF are also consistent with different learning and control mechanisms for adapting to stable and unstable dynamics, that is, inverse dynamics model formation and impedance control. We also investigated adaptation to a rotated DF to examine the interplay between inverse dynamics model formation and impedance control. Our results suggest that an inverse dynamics model can function in parallel with an impedance controller to compensate for consistent perturbing force in unstable environments.

Adaptation to stable and unstable dynamics achieved by combined impedance control and inverse dynamics model.

This study compared adaptation in novel force fields where trajectories were initially either stable or unstable to elucidate the processes of learning novel skills and adapting to new environments. Subjects learned to move in a null force field (NF), which was unexpectedly changed either to a velocity-dependent force field (VF), which resulted in perturbed but stable hand trajectories, or a position-dependent divergent force field (DF), which resulted in unstable trajectories. With practice, subjects learned to compensate for the perturbations produced by both force fields. Adaptation was characterized by an initial increase in the activation of all muscles followed by a gradual reduction. The time course of the increase in activation was correlated with a reduction in hand-path error for the DF but not for the VF. Adaptation to the VF could have been achieved solely by formation of an inverse dynamics model and adaptation to the DF solely by impedance control. However, indices of learning, such as hand-path error, joint torque, and electromyographic activation and deactivation suggest that the CNS combined these processes during adaptation to both force fields. Our results suggest that during the early phase of learning there is an increase in endpoint stiffness that serves to reduce hand-path error and provides additional stability, regardless of whether the dynamics are stable or unstable. We suggest that the motor control system utilizes an inverse dynamics model to learn the mean dynamics and an impedance controller to assist in the formation of the inverse dynamics model and to generate needed stability.

Impedance control and internal model use during the initial stage of adaptation to novel dynamics in humans.

This study investigated the neuromuscular mechanisms underlying the initial stage of adaptation to novel dynamics. A destabilizing velocity-dependent force field (VF) was introduced for sets of three consecutive trials. Between sets a random number of 4-8 null field trials were interposed, where the VF was inactivated. This prevented subjects from learning the novel dynamics, making it possible to repeatedly recreate the initial adaptive response. We were able to investigate detailed changes in neural control between the first, second and third VF trials. We identified two feedforward control mechanisms, which were initiated on the second VF trial and resulted in a 50% reduction in the hand path error. Responses to disturbances encountered on the first VF trial were feedback in nature, i.e. reflexes and voluntary correction of errors. However, on the second VF trial, muscle activation patterns were modified in anticipation of the effects of the force field. Feedforward cocontraction of all muscles was used to increase the viscoelastic impedance of the arm. While stiffening the arm, subjects also exerted a lateral force to counteract the perturbing effect of the force field. These anticipatory actions indicate that the central nervous system responds rapidly to counteract hitherto unfamiliar disturbances by a combination of increased viscoelastic impedance and formation of a crude internal dynamics model.