Functional significance of stiffness in adaptation of multijoint arm movements to stable and unstable dynamics.

This study compared the mechanisms of adaptation to stable and unstable dynamics from the perspective of changes in joint mechanics. Subjects were instructed to make point to point movements in force fields generated by a robotic manipulandum which interacted with the arm in either a stable or an unstable manner. After subjects adjusted to the initial disturbing effects of the force fields they were able to produce normal straight movements to the target. In the case of the stable interaction, subjects modified the joint torques in order to appropriately compensate for the force field. No change in joint torque or endpoint force was required or observed in the case of the unstable interaction. After adaptation, the endpoint stiffness of the arm was measured by applying displacements to the hand in eight different directions midway through the movements. This was compared to the stiffness measured similarly during movements in a null force field. After adaptation, the endpoint stiffness under both the stable and unstable dynamics was modified relative to the null field. Adaptation to unstable dynamics was achieved by selective modification of endpoint stiffness in the direction of the instability. To investigate whether the change in endpoint stiffness could be accounted for by change in joint torque or endpoint force, we estimated the change in stiffness on each trial based on the change in joint torque relative to the null field. For stable dynamics the change in endpoint stiffness was accurately predicted. However, for unstable dynamics the change in endpoint stiffness could not be reproduced. In fact, the predicted endpoint stiffness was similar to that in the null force field. Thus, the change in endpoint stiffness seen after adaptation to stable dynamics was directly related to changes in net joint torque necessary to compensate for the dynamics in contrast to adaptation to unstable dynamics, where a selective change in endpoint stiffness occurred without any modification of net joint torque.

Different mechanisms involved in adaptation to stable and unstable dynamics.

Recently, we demonstrated that humans can learn to make accurate movements in an unstable environment by controlling magnitude, shape, and orientation of the endpoint impedance. Although previous studies of human motor learning suggest that the brain acquires an inverse dynamics model of the novel environment, it is not known whether this control mechanism is operative in unstable environments. We compared learning of multijoint arm movements in a "velocity-dependent force field" (VF), which interacted with the arm in a stable manner, and learning in a "divergent force field" (DF), where the interaction was unstable. The characteristics of error evolution were markedly different in the 2 fields. The direction of trajectory error in the DF alternated to the left and right during the early stage of learning; that is, signed error was inconsistent from movement to movement and could not have guided learning of an inverse dynamics model. This contrasted sharply with trajectory error in the VF, which was initially biased and decayed in a manner that was consistent with rapid feedback error learning. EMG recorded before and after learning in the DF and VF are also consistent with different learning and control mechanisms for adapting to stable and unstable dynamics, that is, inverse dynamics model formation and impedance control. We also investigated adaptation to a rotated DF to examine the interplay between inverse dynamics model formation and impedance control. Our results suggest that an inverse dynamics model can function in parallel with an impedance controller to compensate for consistent perturbing force in unstable environments.

Adaptation to stable and unstable dynamics achieved by combined impedance control and inverse dynamics model.

This study compared adaptation in novel force fields where trajectories were initially either stable or unstable to elucidate the processes of learning novel skills and adapting to new environments. Subjects learned to move in a null force field (NF), which was unexpectedly changed either to a velocity-dependent force field (VF), which resulted in perturbed but stable hand trajectories, or a position-dependent divergent force field (DF), which resulted in unstable trajectories. With practice, subjects learned to compensate for the perturbations produced by both force fields. Adaptation was characterized by an initial increase in the activation of all muscles followed by a gradual reduction. The time course of the increase in activation was correlated with a reduction in hand-path error for the DF but not for the VF. Adaptation to the VF could have been achieved solely by formation of an inverse dynamics model and adaptation to the DF solely by impedance control. However, indices of learning, such as hand-path error, joint torque, and electromyographic activation and deactivation suggest that the CNS combined these processes during adaptation to both force fields. Our results suggest that during the early phase of learning there is an increase in endpoint stiffness that serves to reduce hand-path error and provides additional stability, regardless of whether the dynamics are stable or unstable. We suggest that the motor control system utilizes an inverse dynamics model to learn the mean dynamics and an impedance controller to assist in the formation of the inverse dynamics model and to generate needed stability.

Generalization in Adaptation to Stable and Unstable Dynamics

Humans skillfully manipulate objects and tools despite the inherent instability. In order to succeed at these tasks, the sensorimotor control system must build an internal representation of both the force and mechanical impedance. As it is not practical to either learn or store motor commands for every possible future action, the sensorimotor control system generalizes a control strategy for a range of movements based on learning performed over a set of movements. Here, we introduce a computational model for this learning and generalization, which specifies how to learn feedforward muscle activity in a function of the state space. Specifically, by incorporating co-activation as a function of error into the feedback command, we are able to derive an algorithm from a gradient descent minimization of motion error and effort, subject to maintaining a stability margin. This algorithm can be used to learn to coordinate any of a variety of motor primitives such as force fields, muscle synergies, physical models or artificial neural networks. This model for human learning and generalization is able to adapt to both stable and unstable dynamics, and provides a controller for generating efficient adaptive motor behavior in robots. Simulation results exhibit predictions consistent with all experiments on learning of novel dynamics requiring adaptation of force and impedance, and enable us to re-examine some of the previous interpretations of experiments on generalization. © 2012 Kadiallah et al.

Learning the dynamics of the external world: Brain inspired learning for robotic applications

Humans have exceptional abilities to learn new skills, manipulate tools and objects, and interact with our environment. In order to be successful at these tasks, our brain has become exceptionally well adapted to learning to deal not only with the complex dynamics of our own limbs but also with novel dynamics in the external world. While learning of these dynamics includes learning the complex time-varying forces at the end of limbs through the updating of internal models, it must also include learning the appropriate mechanical impedance in order to stabilize both the limb and any objects contacted in the environment. This article reviews the field of human learning by examining recent experimental evidence about adaptation to novel unstable dynamics and explores how this knowledge about the brain and neuro-muscular system can expand the learning capabilities of robotics and prosthetics. © 2006.

How are internal models of unstable tasks formed?

The results of recent studies suggest that humans can form internal models that they use in a feedforward manner to compensate for both stable and unstable dynamics. To examine how internal models are formed, we performed adaptation experiments in novel dynamics, and measured the endpoint force, trajectory and EMG during learning. Analysis of reflex feedback and change of feedforward commands between consecutive trials suggested a unified model of motor learning, which can coherently unify the learning processes observed in stable and unstable dynamics and reproduce available data on motor learning. To our knowledge, this algorithm, based on the concurrent minimization of (reflex) feedback and muscle activation, is also the first nonlinear adaptive controller able to stabilize unstable dynamics.

Uncovering the dynamics of cardiac systems using stochastic pacing and frequency domain analyses

Alternans of cardiac action potential duration (APD) is a well-known arrhythmogenic mechanism which results from dynamical instabilities. The propensity to alternans is classically investigated by examining APD restitution and by deriving APD restitution slopes as predictive markers. However, experiments have shown that such markers are not always accurate for the prediction of alternans. Using a mathematical ventricular cell model known to exhibit unstable dynamics of both membrane potential and Ca2+ cycling, we demonstrate that an accurate marker can be obtained by pacing at cycle lengths (CLs) varying randomly around a basic CL (BCL) and by evaluating the transfer function between the time series of CLs and APDs using an autoregressive-moving-average (ARMA) model. The first pole of this transfer function corresponds to the eigenvalue (λalt) of the dominant eigenmode of the cardiac system, which predicts that alternans occurs when λalt≤−1. For different BCLs, control values of λalt were obtained using eigenmode analysis and compared to the first pole of the transfer function estimated using ARMA model fitting in simulations of random pacing protocols. In all versions of the cell model, this pole provided an accurate estimation of λalt. Furthermore, during slow ramp decreases of BCL or simulated drug application, this approach predicted the onset of alternans by extrapolating the time course of the estimated λalt. In conclusion, stochastic pacing and ARMA model identification represents a novel approach to predict alternans without making any assumptions about its ionic mechanisms. It should therefore be applicable experimentally for any type of myocardial cell.

Position information but not force information is used in adapting to changes in environmental dynamics

This study investigated how movement error is evaluated and used to change feedforward commands following a change in the environmental dynamics. In particular, we addressed the question of whether only position-error information is used or whether information about the force-field direction can also be used for rapid adaptation to changes in the environmental dynamics. Subjects learned to move in a position-dependent force field (PF) with a parabolic profile and the dynamics of a negative spring, which produced lateral force to the left of the target hand path. They adapted very rapidly, dramatically reducing lateral error after a single trial. Several times during training, the strength of the PF was unexpectedly doubled (PF2) for two trials. This again created a large leftward deviation, which was greatly reduced on the second PF2 trial, and an aftereffect when the force field subsequently returned to its original strength. The aftereffect was abolished if the second PF2 trial was replaced by an oppositely directed velocity-dependent force field (VF). During subsequent training in the VF, immediately after having adapted to the PF, subjects applied a force that assisted the force field for similar to 15 trials, indicating that they did not use information about the force-field direction. We concluded that the CNS uses only the position error for updating the internal model of the environmental dynamics and modifying feedforward commands. Although this strategy is not necessarily optimal, it may be the most reliable strategy for iterative improvement in performance.

Impedance control balances stability with metabolically costly muscle activation.

Humans are able to stabilize their movements in environments with unstable dynamics by selectively modifying arm impedance independently of force and torque. We further investigated adaptation to unstable dynamics to determine whether the CNS maintains a constant overall level of stability as the instability of the environmental dynamics is varied. Subjects performed reaching movements in unstable force fields of varying strength, generated by a robotic manipulator. Although the force fields disrupted the initial movements, subjects were able to adapt to the novel dynamics and learned to produce straight trajectories. After adaptation, the endpoint stiffness of the arm was measured at the midpoint of the movement. The stiffness had been selectively modified in the direction of the instability. The stiffness in the stable direction was relatively unchanged from that measured during movements in a null force field prior to exposure to the unstable force field. This impedance modification was achieved without changes in force and torque. The overall stiffness of the arm and environment in the direction of instability was adapted to the force field strength such that it remained equivalent to that of the null force field. This suggests that the CNS attempts both to maintain a minimum level of stability and minimize energy expenditure.

Impedance control is tuned to multiple directions of movement

Humans are able to learn tool-handling tasks, such as carving, demonstrating their competency to make and vary the direction of movements in unstable environments. It has been shown that when a single reaching movement is repeated in unstable dynamics, the central nervous system (CNS) learns an impedance internal model to compensate for the environment instability. However, there is still no explanation for how humans can learn to move in various directions in such environments. In this study, we investigated whether and how humans compensate for instability while learning two different reaching movements simultaneously. Results show that when performing movements in two different directions, separated by a 35° angle, the CNS was able to compensate for the unstable dynamics. After adaptation, the force was found to be similar to the free movement condition, but stiffness increased in the direction of instability, specifically for each direction of movement. Our findings suggest that the CNS either learned an internal model generalizing over different movements, or alternatively that it was able to switch between specific models acquired simultaneously. © 2008 IEEE.

In-Flight Collision Avoidance Controller Based Only on OS4 Embedded Sensors

The major goal of this research was the development and implementation of a control system able to avoid collisions during the flight for a mini-quadrotor helicopter, based only on its embedded sensors without changing the environment. However, it is important to highlight that the design aspects must be seriously considered in order to overcome hardware limitations and achieve control simplification. The controllers of a UAV (Unmanned Aerial Vehicle) robot deal with highly unstable dynamics and strong axes coupling. Furthermore, any additional embedded sensor increases the robot total weight and therefore, decreases its operating time. The best balance between embedded electronics and robot operating time is desired. This paper focuses not only on the development and implementation of a collision avoidance controller for a mini-robotic helicopter using only its embedded sensors, but also on the mathematical model that was essential for the controller developing phases. Based on this model we carried out the development of a simulation tool based on MatLab/Simulink that was fundamental for setting the controllers' parameters. This tool allowed us to simulate and improve the OS4 controllers in different modeled environments and test different approaches. After that, the controllers were embedded in the real robot and the results proved to be very robust and feasible. In addition to this, the controller has the advantage of being compatible with future path planners that we are developing.

Novel strategies in feedforward adaptation to a position-dependent perturbation

To investigate the control mechanisms used in adapting to position-dependent forces, subjects performed 150 horizontal reaching movements over 25 cm in the presence of a position-dependent parabolic force field (PF). The PF acted only over the first 10 cm of the movement. On every fifth trial, a virtual mechanical guide (double wall) constrained subjects to move along a straight-line path between the start and target positions. Its purpose was to register lateral force to track formation of an internal model of the force field, and to look for evidence of possible alternative adaptive strategies. The force field produced a force to the right, which initially caused subjects to deviate in that direction. They reacted by producing deviations to the left, into the force field, as early as the second trial. Further adaptation resulted in rapid exponential reduction of kinematic error in the latter portion of the movement, where the greatest perturbation to the handpath was initially observed, whereas there was little modification of the handpath in the region where the PF was active. Significant force directed to counteract the PF was measured on the first guided trial, and was modified during the first half of the learning set. The total force impulse in the region of the PF increased throughout the learning trials, but it always remained less than that produced by the PF. The force profile did not resemble a mirror image of the PF in that it tended to be more trapezoidal than parabolic in shape. As in previous studies of force-field adaptation, we found that changes in muscle activation involved a general increase in the activity of all muscles, which increased arm stiffness, and selectively-greater increases in the activation of muscles which counteracted the PF. With training, activation was exponentially reduced, albeit more slowly than kinematic error. Progressive changes in kinematics and EMG occurred predominantly in the region of the workspace beyond the force field. We suggest that constraints on muscle mechanics limit the ability of the central nervous system to employ an inverse dynamics model to nullify impulse-like forces by generating mirror-image forces. Consequently, subjects adopted a strategy of slightly overcompensating for the first half of the force field, then allowing the force field to push them in the opposite direction. Muscle activity patterns in the region beyond the boundary of the force field were subsequently adjusted because of the relatively-slow response of the second-order mechanics of muscle impedance to the force impulse.

Small-scale genetic structure of Cerastoderma glaucum in a lagoonal environment: potential significance of habitat discontinuity and unstable population dynamics

Environmental heterogeneity in coastal lagoons is expected to facilitate local adaptation in response to different ecological conditions, causing significant genetic structuring within lagoon populations at a small scale and also differentiation between lagoons. However, these patterns and processes of genetic structuring are still poorly understood. The aims of our study were (1) to seek genetic structure at a small scale in Cerastoderma glaucum inside the Mar Menor coastal lagoon using a mitochondrial DNA marker (COI) that has previously detected genetic differentiation inside the lagoon in other species and (2) to evaluate the influence of extreme environmental conditions and habitat discontinuity on its genetic composition. The results indicate high levels of haplotype diversity and low values of nucleotide diversity. COI data provide evidence of significant population differentiation among some localities within the lagoon. Limited gene flow and unstable population dynamics (i.e. fluctuations in population size caused by local extinction and recolonization), probably due to the high environmental heterogeneity, could generate the small-scale genetic divergence detected between populations within the lagoon.

Collective dynamics and control of a 3-D small-world network with time delays

The paper presents a detailed analysis on the collective dynamics and delayed state feedback control of a three-dimensional delayed small-world network. The trivial equilibrium of the model is first investigated, showing that the uncontrolled model exhibits complicated unbounded behavior. Then three control strategies, namely a position feedback control, a velocity feedback control, and a hybrid control combined velocity with acceleration feedback, are then introduced to stabilize this unstable system. It is shown in these three control schemes that only the hybrid control can easily stabilize the 3-D network system. And with properly chosen delay and gain in the delayed feedback path, the hybrid controlled model may have stable equilibrium, or periodic solutions resulting from the Hopf bifurcation, or complex stranger attractor from the period-doubling bifurcation. Moreover, the direction of Hopf bifurcation and stability of the bifurcation periodic solutions are analyzed. The results are further extended to any "d" dimensional network. It shows that to stabilize a "d" dimensional delayed small-world network, at least a "d – 1" order completed differential feedback is needed. This work provides a constructive suggestion for the high dimensional delayed systems.