Optical learning in a smart spatial light modulator

A synaptic plane rendered by an array of smart pixels was described regarding its application as a complementary component for neural network implementation. The smart spatial light modulator featured auto-modification abilities. Thus, an optical system incorporating this device can show self-reliant optical learning. Furthermore, the optical system design, in the area of its optical interconnection scheme, is highly flexible since the independent weight-plane pixels eliminated the difficulty between weight update calculation and weight representation.

Reinforcement learning with reference tracking control in continuous state spaces

The contribution described in this paper is an algorithm for learning nonlinear, reference tracking, control policies given no prior knowledge of the dynamical system and limited interaction with the system through the learning process. Concepts from the field of reinforcement learning, Bayesian statistics and classical control have been brought together in the formulation of this algorithm which can be viewed as a form of indirect self tuning regulator. On the task of reference tracking using a simulated inverted pendulum it was shown to yield generally improved performance on the best controller derived from the standard linear quadratic method using only 30 s of total interaction with the system. Finally, the algorithm was shown to work on the simulated double pendulum proving its ability to solve nontrivial control tasks. © 2011 IEEE.

Self-organization of spinal reflexes involving homonymous, antagonist and synergistic interactions

Recent results in spinal research are challenging the historical view that the spinal reflexes are mostly hardwired and fixed behaviours. In previous work we have shown that three of the simplest spinal reflexes could be self-organised in an agonist-antagonist pair of muscles. The simplicity of these reflexes is given from the fact that they entail at most one interneuron mediating the connectivity between afferent inputs and efferent outputs. These reflexes are: the Myotatic, the Reciprocal Inibition and the Reverse Myotatic reflexes. In this paper we apply our framework to a simulated 2D leg model actuated by six muscles (mono- and bi-articular). Our results show that the framework is successful in learning most of the spinal reflex circuitry as well as the corresponding behaviour in the more complicated muscle arrangement. © 2012 Springer-Verlag.

Learning legged locomotion

Legged locomotion of biological systems can be viewed as a self-organizing process of highly complex system-environment interactions. Walking behavior is, for example, generated from the interactions between many mechanical components (e.g., physical interactions between feet and ground, skeletons and muscle-tendon systems), and distributed informational processes (e.g., sensory information processing, sensory-motor control in central nervous system, and reflexes) [21]. An interesting aspect of legged locomotion study lies in the fact that there are multiple levels of self-organization processes (at the levels of mechanical dynamics, sensory-motor control, and learning). Previously, the self-organization of mechanical dynamics was nicely demonstrated by the so-called Passive Dynamic Walkers (PDWs; [18]). The PDW is a purely mechanical structure consisting of body, thigh, and shank limbs that are connected by passive joints. When placed on a shallow slope, it exhibits natural bipedal walking dynamics by converting potential to kinetic energy without any actuation. An important contribution of these case studies is that, if designed properly, mechanical dynamics can generate a relatively complex locomotion dynamics, on the one hand, and the mechanical dynamics induces self-stability against small disturbances without any explicit control of motors, on the other. The basic principle of the mechanical self-stability appears to be fairly general that there are several different physics models that exhibit similar characteristics in different kinds of behaviors (e.g., hopping, running, and swimming; [2, 4, 9, 16, 19]), and a number of robotic platforms have been developed based on them [1, 8, 13, 22]. © 2009 Springer London.