Reinforcement learning with reference tracking control in continuous state spaces

The contribution described in this paper is an algorithm for learning nonlinear, reference tracking, control policies given no prior knowledge of the dynamical system and limited interaction with the system through the learning process. Concepts from the field of reinforcement learning, Bayesian statistics and classical control have been brought together in the formulation of this algorithm which can be viewed as a form of indirect self tuning regulator. On the task of reference tracking using a simulated inverted pendulum it was shown to yield generally improved performance on the best controller derived from the standard linear quadratic method using only 30 s of total interaction with the system. Finally, the algorithm was shown to work on the simulated double pendulum proving its ability to solve nontrivial control tasks. © 2011 IEEE.

Design and control of a pendulum driven hopping robot

In this paper a new kind of hopping robot has been designed which uses inverse pendulum dynamics to induce bipedal hopping gaits. Its mechanical structure consists of a rigid inverted T-shape mounted on four compliant feet. An upright "T" structure is connected to this by a rotary joint. The horizontal beam of the upright "T" is connected to the vertical beam by a second rotary joint. Using this two degree of freedom mechanical structure, with simple reactive control, the robot is able to perform hopping, walking and running gaits. During walking, it is experimentally shown that the robot can move in a straight line, reverse direction and control its turning radius. The results show that such a simple but versatile robot displays stable locomotion and can be viable for practical applications on uneven terrain.