Dynamic learning with the EM algorithm for neural networks

In this paper, we derive an EM algorithm for nonlinear state space models. We use it to estimate jointly the neural network weights, the model uncertainty and the noise in the data. In the E-step we apply a forwardbackward Rauch-Tung-Striebel smoother to compute the network weights. For the M-step, we derive expressions to compute the model uncertainty and the measurement noise. We find that the method is intrinsically very powerful, simple and stable.

A single-rate context-dependent learning process underlies rapid adaptation to familiar object dynamics.

Motor learning has been extensively studied using dynamic (force-field) perturbations. These induce movement errors that result in adaptive changes to the motor commands. Several state-space models have been developed to explain how trial-by-trial errors drive the progressive adaptation observed in such studies. These models have been applied to adaptation involving novel dynamics, which typically occurs over tens to hundreds of trials, and which appears to be mediated by a dual-rate adaptation process. In contrast, when manipulating objects with familiar dynamics, subjects adapt rapidly within a few trials. Here, we apply state-space models to familiar dynamics, asking whether adaptation is mediated by a single-rate or dual-rate process. Previously, we reported a task in which subjects rotate an object with known dynamics. By presenting the object at different visual orientations, adaptation was shown to be context-specific, with limited generalization to novel orientations. Here we show that a multiple-context state-space model, with a generalization function tuned to visual object orientation, can reproduce the time-course of adaptation and de-adaptation as well as the observed context-dependent behavior. In contrast to the dual-rate process associated with novel dynamics, we show that a single-rate process mediates adaptation to familiar object dynamics. The model predicts that during exposure to the object across multiple orientations, there will be a degree of independence for adaptation and de-adaptation within each context, and that the states associated with all contexts will slowly de-adapt during exposure in one particular context. We confirm these predictions in two new experiments. Results of the current study thus highlight similarities and differences in the processes engaged during exposure to novel versus familiar dynamics. In both cases, adaptation is mediated by multiple context-specific representations. In the case of familiar object dynamics, however, the representations can be engaged based on visual context, and are updated by a single-rate process.

Facilitation of learning induced by both random and gradual visuomotor task variation.

Motor task variation has been shown to be a key ingredient in skill transfer, retention, and structural learning. However, many studies only compare training of randomly varying tasks to either blocked or null training, and it is not clear how experiencing different nonrandom temporal orderings of tasks might affect the learning process. Here we study learning in human subjects who experience the same set of visuomotor rotations, evenly spaced between -60° and +60°, either in a random order or in an order in which the rotation angle changed gradually. We compared subsequent learning of three test blocks of +30°→-30°→+30° rotations. The groups that underwent either random or gradual training showed significant (P < 0.01) facilitation of learning in the test blocks compared with a control group who had not experienced any visuomotor rotations before. We also found that movement initiation times in the random group during the test blocks were significantly (P < 0.05) lower than for the gradual or the control group. When we fit a state-space model with fast and slow learning processes to our data, we found that the differences in performance in the test block were consistent with the gradual or random task variation changing the learning and retention rates of only the fast learning process. Such adaptation of learning rates may be a key feature of ongoing meta-learning processes. Our results therefore suggest that both gradual and random task variation can induce meta-learning and that random learning has an advantage in terms of shorter initiation times, suggesting less reliance on cognitive processes.

Verifying stability of approximate explicit MPC

Several authors have proposed algorithms for approximate explicit MPC [1],[2],[3]. These algorithms have in common that they develop a stability criterion for approximate explicit MPC that require the approximate cost function to be within a certain distance from the optimal cost function. In this paper, stability is instead ascertained by considering only the cost function of the approximate MPC. If a region of the state space is found where the cost function is not decreasing, this indicates that an improved approximation (to the optimal control) is required in that region. If the approximate cost function is decreasing everywhere, no further refinement of the approximate MPC is necessary, since stability is guaranteed. ©2009 IEEE.

Generalization in Adaptation to Stable and Unstable Dynamics

Humans skillfully manipulate objects and tools despite the inherent instability. In order to succeed at these tasks, the sensorimotor control system must build an internal representation of both the force and mechanical impedance. As it is not practical to either learn or store motor commands for every possible future action, the sensorimotor control system generalizes a control strategy for a range of movements based on learning performed over a set of movements. Here, we introduce a computational model for this learning and generalization, which specifies how to learn feedforward muscle activity in a function of the state space. Specifically, by incorporating co-activation as a function of error into the feedback command, we are able to derive an algorithm from a gradient descent minimization of motion error and effort, subject to maintaining a stability margin. This algorithm can be used to learn to coordinate any of a variety of motor primitives such as force fields, muscle synergies, physical models or artificial neural networks. This model for human learning and generalization is able to adapt to both stable and unstable dynamics, and provides a controller for generating efficient adaptive motor behavior in robots. Simulation results exhibit predictions consistent with all experiments on learning of novel dynamics requiring adaptation of force and impedance, and enable us to re-examine some of the previous interpretations of experiments on generalization. © 2012 Kadiallah et al.

A Monte Carlo expectation maximisation algorithm for multiple target tracking

In this paper, we present an expectation-maximisation (EM) algorithm for maximum likelihood estimation in multiple target models (MTT) with Gaussian linear state-space dynamics. We show that estimation of sufficient statistics for EM in a single Gaussian linear state-space model can be extended to the MTT case along with a Monte Carlo approximation for inference of unknown associations of targets. The stochastic approximation EM algorithm that we present here can be used along with any Monte Carlo method which has been developed for tracking in MTT models, such as Markov chain Monte Carlo and sequential Monte Carlo methods. We demonstrate the performance of the algorithm with a simulation. © 2012 ISIF (Intl Society of Information Fusi).

Balanced realisations of all-pass systems using exact arithmetic with application to the approximation of delay systems

Numerically well-conditioned state-space realisations for all-pass systems, such as Padé approximations to exp(-s), are derived that can be computed using exact integer arithmetic. This is then applied to the a series of functions of exp(-s). It is also shown that the H-infinity norm of the transfer function from the input to the state of a balanced realisation of the Padé approximation of exp(-s) is unity. © 2012 IEEE.

Modelling and control of nonlinear systems using Gaussian processes with partial model information

Gaussian processes are gaining increasing popularity among the control community, in particular for the modelling of discrete time state space systems. However, it has not been clear how to incorporate model information, in the form of known state relationships, when using a Gaussian process as a predictive model. An obvious example of known prior information is position and velocity related states. Incorporation of such information would be beneficial both computationally and for faster dynamics learning. This paper introduces a method of achieving this, yielding faster dynamics learning and a reduction in computational effort from O(Dn2) to O((D - F)n2) in the prediction stage for a system with D states, F known state relationships and n observations. The effectiveness of the method is demonstrated through its inclusion in the PILCO learning algorithm with application to the swing-up and balance of a torque-limited pendulum and the balancing of a robotic unicycle in simulation. © 2012 IEEE.

A PDE viewpoint on basic properties of coordination algorithms with symmetries

Several recent control applications consider the coordination of subsystems through local interaction. Often the interaction has a symmetry in state space, e.g. invariance with respect to a uniform translation of all subsystem values. The present paper shows that in presence of such symmetry, fundamental properties can be highlighted by viewing the distributed system as the discrete approximation of a partial differential equation. An important fact is that the symmetry on the state space differs from the popular spatial invariance property, which is not necessary for the present results. The relevance of the viewpoint is illustrated on two examples: (i) ill-conditioning of interaction matrices in coordination/consensus problems and (ii) the string instability issue. ©2009 IEEE.

Synchronization in networks of identical linear systems

The paper investigates the synchronization of a network of identical linear state-space models under a possibly time-varying and directed interconnection structure. The main result is the construction of a dynamic output feedback coupling that achieves synchronization if the decoupled systems have no exponentially unstable mode and if the communication graph is uniformly connected. The result can be interpreted as a generalization of classical consensus algorithms. Stronger conditions are shown to be sufficient-but to some extent, also necessary-to ensure synchronization with the diffusive static output coupling often considered in the literature. © 2009 Elsevier Ltd. All rights reserved.

Synchronization in networks of identical linear systems

The paper investigates the synchronization of a network of identical linear time-invariant state-space models under a possibly time-varying and directed interconnection structure. The main result is the construction of a dynamic output feedback coupling that achieves synchronization if the decoupled systems have no exponentially unstable mode and if the communication graph is uniformly connected. Stronger conditions are shown to be sufficient - but to some extent, also necessary - to ensure synchronization with the diffusive static output coupling often considered in the literature. © 2008 IEEE.

A duality principle for homogeneous vectorfields with applications

A duality transformation principle was proposed for converting a positive order homogeneous vectorfield into a negative order homogeneous vectorfield. The principle also converted a uniformly locally asymptotically stable differential equation into a uniformly bounded differential equation. The duality transformations included the geometric framework for homogeneity and the removal of origin from the state space.

A differential lyapunov framework for contraction analysis

Lyapunov's second theorem is an essential tool for stability analysis of differential equations. The paper provides an analog theorem for incremental stability analysis by lifting the Lyapunov function to the tangent bundle. The Lyapunov function endows the state-space with a Finsler structure. Incremental stability is inferred from infinitesimal contraction of the Finsler metrics through integration along solutions curves. © 2013 IEEE.

基于神经网络的水泥生料配料多目标优化设计方法

为了降低生料成分的不确定性给水泥生料质量控制系统带来的影响,提出了率值补偿的控制策略.分别为三率值创建目标函数,并利用状态空间搜索策略解决多目标优化问题.针对初始样本空间不能覆盖所有样本的问题,提出了基于神经网络的估算模型,对初始样本空间进行拓扑.通过估价函数对状态空间中的状态量进行评价,得到最优的率值状态量;根据率值对原料配比进行调整,最后使率值偏差得到补偿,同时使给配比造成的波动最小.工业实验结果表明,生料的质量合格率由原来的30%提高到50%,该系统能有效地对配料过程进行优化控制.证明了基于神经网络的状态空间搜索策略为水泥生料配料多目标寻优问题提供了一种可行的方法。

多机器人动态编队的强化学习算法研究

在人工智能领域中 ,强化学习理论由于其自学习性和自适应性的优点而得到了广泛关注 随着分布式人工智能中多智能体理论的不断发展 ,分布式强化学习算法逐渐成为研究的重点 首先介绍了强化学习的研究状况 ,然后以多机器人动态编队为研究模型 ,阐述应用分布式强化学习实现多机器人行为控制的方法 应用SOM神经网络对状态空间进行自主划分 ,以加快学习速度 ;应用BP神经网络实现强化学习 ,以增强系统的泛化能力 ;并且采用内、外两个强化信号兼顾机器人的个体利益及整体利益 为了明确控制任务 ,系统使用黑板通信方式进行分层控制 最后由仿真实验证明该方法的有效性