Different mechanisms involved in adaptation to stable and unstable dynamics.

Recently, we demonstrated that humans can learn to make accurate movements in an unstable environment by controlling magnitude, shape, and orientation of the endpoint impedance. Although previous studies of human motor learning suggest that the brain acquires an inverse dynamics model of the novel environment, it is not known whether this control mechanism is operative in unstable environments. We compared learning of multijoint arm movements in a "velocity-dependent force field" (VF), which interacted with the arm in a stable manner, and learning in a "divergent force field" (DF), where the interaction was unstable. The characteristics of error evolution were markedly different in the 2 fields. The direction of trajectory error in the DF alternated to the left and right during the early stage of learning; that is, signed error was inconsistent from movement to movement and could not have guided learning of an inverse dynamics model. This contrasted sharply with trajectory error in the VF, which was initially biased and decayed in a manner that was consistent with rapid feedback error learning. EMG recorded before and after learning in the DF and VF are also consistent with different learning and control mechanisms for adapting to stable and unstable dynamics, that is, inverse dynamics model formation and impedance control. We also investigated adaptation to a rotated DF to examine the interplay between inverse dynamics model formation and impedance control. Our results suggest that an inverse dynamics model can function in parallel with an impedance controller to compensate for consistent perturbing force in unstable environments.

How is somatosensory information used to adapt to changes in the mechanical environment?

Recent studies examining adaptation to unexpected changes in the mechanical environment highlight the use of position error in the adaptation process. However, force information is also available. In this chapter, we examine adaptation processes in three separate studies where the mechanical environment was changed intermittently. We compare the expected consequences of using position error and force information in the changes to motor commands following a change in the mechanical environment. In general, our results support the use of position error over force information and are consistent with current computational models of motor learning. However, in situations where the change in the mechanical environment eliminates position error the central nervous system does not necessarily respond as would be predicted by these models. We suggest that it is necessary to take into account the statistics of prior experience to account for our observations. Another deficiency in these models is the absence of a mechanism for modulating limb mechanical impedance during adaptation. We propose a relatively simple computational model based on reflex responses to perturbations which is capable of accounting for iterative changes in temporal patterns of muscle co-activation.

Fast O2 measurement using modified UEGO sensors in the intake and exhaust of a diesel engine

Recent work has investigated the use of O2 concentration in the intake manifold as a control variable for diesel engines. It has been recognised as a very good indicator of NOX emissions especially during transient operation, however, much of the work is concentrated on estimating the O2 concentration as opposed to measuring it. This work investigates Universal Exhaust Gas Oxygen (UEGO) sensors and their potential to be used for such measurements. In previous work it was shown that these sensors can be operated in a controlled pressure environment such that their response time is of the order 10ms. In this paper, it is shown how the key causes of variation (and therefore potential sources of error) in sensor output, namely, pressure and temperature are largely mitigated by operating the sensors in such an environment. Experiments were undertaken on a representative light duty diesel engine using modified UEGO sensors in the intake and exhaust system. Results from other fast emissions measuring equipment are also shown and it is seen that the UEGO sensors are capable of giving an accurate measurement of O2 and EGR. Copyright © 2013 SAE International.

Application of a laminar flamelet model to confined explosion hazards

The majority of computational studies of confined explosion hazards apply simple and inaccurate combustion models, requiring ad hoc corrections to obtain realistic flame shapes and often predicting an order of magnitude error in the overpressures. This work describes the application of a laminar flamelet model to a series of two-dimensional test cases. The model is computationally efficient applying an algebraic expression to calculate the flame surface area, an empirical correlation for the laminar flame speed and a novel unstructured, solution adaptive numerical grid system which allows important features of the solution to be resolved close to the flame. Accurate flame shapes are predicted, the correct burning rate is predicted near the walls, and an improvement in the predicted overpressures is obtained. However, in these fully turbulent calculations the overpressures are still too high and the flame arrival times too low, indicating the need for a model for the early laminar burning phase. Due to the computational expense, it is unrealistic to model a laminar flame in the complex geometries involved and therefore a pragmatic approach is employed which constrains the flame to propagate at the laminar flame speed. Transition to turbulent burning occurs at a specified turbulent Reynolds number. With the laminar phase model included, the predicted flame arrival times increase significantly, but are still too low. However, this has no significant effect on the overpressures, which are predicted accurately for a baffled channel test case where rapid transition occurs once the flame reaches the first pair of baffles. In a channel with obstacles on the centreline, transition is more gradual and the accuracy of the predicted overpressures is reduced. However, although the accuracy is still less than desirable in some cases, it is much better than the order of magnitude error previously expected.

Approximate Bayesian Computation for Smoothing

We consider a method for approximate inference in hidden Markov models (HMMs). The method circumvents the need to evaluate conditional densities of observations given the hidden states. It may be considered an instance of Approximate Bayesian Computation (ABC) and it involves the introduction of auxiliary variables valued in the same space as the observations. The quality of the approximation may be controlled to arbitrary precision through a parameter ε > 0. We provide theoretical results which quantify, in terms of ε, the ABC error in approximation of expectations of additive functionals with respect to the smoothing distributions. Under regularity assumptions, this error is, where n is the number of time steps over which smoothing is performed. For numerical implementation, we adopt the forward-only sequential Monte Carlo (SMC) scheme of [14] and quantify the combined error from the ABC and SMC approximations. This forms some of the first quantitative results for ABC methods which jointly treat the ABC and simulation errors, with a finite number of data and simulated samples. © Taylor & Francis Group, LLC.

Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.

Collective motion, sensor networks, and ocean sampling

This paper addresses the design of mobile sensor networks for optimal data collection. The development is strongly motivated by the application to adaptive ocean sampling for an autonomous ocean observing and prediction system. A performance metric, used to derive optimal paths for the network of mobile sensors, defines the optimal data set as one which minimizes error in a model estimate of the sampled field. Feedback control laws are presented that stably coordinate sensors on structured tracks that have been optimized over a minimal set of parameters. Optimal, closed-loop solutions are computed in a number of low-dimensional cases to illustrate the methodology. Robustness of the performance to the influence of a steady flow field on relatively slow-moving mobile sensors is also explored © 2006 IEEE.