Reinforcement learning with reference tracking control in continuous state spaces

The contribution described in this paper is an algorithm for learning nonlinear, reference tracking, control policies given no prior knowledge of the dynamical system and limited interaction with the system through the learning process. Concepts from the field of reinforcement learning, Bayesian statistics and classical control have been brought together in the formulation of this algorithm which can be viewed as a form of indirect self tuning regulator. On the task of reference tracking using a simulated inverted pendulum it was shown to yield generally improved performance on the best controller derived from the standard linear quadratic method using only 30 s of total interaction with the system. Finally, the algorithm was shown to work on the simulated double pendulum proving its ability to solve nontrivial control tasks. © 2011 IEEE.

Design and control of a pendulum driven hopping robot

In this paper a new kind of hopping robot has been designed which uses inverse pendulum dynamics to induce bipedal hopping gaits. Its mechanical structure consists of a rigid inverted T-shape mounted on four compliant feet. An upright "T" structure is connected to this by a rotary joint. The horizontal beam of the upright "T" is connected to the vertical beam by a second rotary joint. Using this two degree of freedom mechanical structure, with simple reactive control, the robot is able to perform hopping, walking and running gaits. During walking, it is experimentally shown that the robot can move in a straight line, reverse direction and control its turning radius. The results show that such a simple but versatile robot displays stable locomotion and can be viable for practical applications on uneven terrain.

Enhanced reflection from arrays of silicon based inverted nanocones

We report enhanced reflection displayed by arrays of silicon based inverted nanocones. Theoretical studies suggest that such arrays display enhanced reflection and photonic band gaps within the optical and near infrared regions. Measured results show three to four fold enhancement in reflection and agree well with calculations. Such arrays can be used to enhance infrared reflection in photovoltaic devices which mostly contribute towards heating. © 2011 American Institute of Physics.

Vision-based global localization using a visual vocabular

This paper presents a novel coarse-to-fine global localization approach that is inspired by object recognition and text retrieval techniques. Harris-Laplace interest points characterized by SIFT descriptors are used as natural land-marks. These descriptors are indexed into two databases: an inverted index and a location database. The inverted index is built based on a visual vocabulary learned from the feature descriptors. In the location database, each location is directly represented by a set of scale invariant descriptors. The localization process consists of two stages: coarse localization and fine localization. Coarse localization from the inverted index is fast but not accurate enough; whereas localization from the location database using voting algorithm is relatively slow but more accurate. The combination of coarse and fine stages makes fast and reliable localization possible. In addition, if necessary, the localization result can be verified by epipolar geometry between the representative view in database and the view to be localized. Experimental results show that our approach is efficient and reliable. ©2005 IEEE.

GROOVE FORMATION AND INTERACTION DURING ABRASION OF COPPER.

A pin on cylinder wear rig has been built with precision stepping motor drives to both rotary and axial motions which enable accurate positional control to be achieved. Initial experiments using sapphire indenters running against copper substrates have investigated the build up of a single wear groove by repeated sliding along the same track. An approximate three dimensional ploughing analysis is also presented and the results of theory and experiment compared.

Laser slotting: a new laser machining technique

This work describes a new technique for the selective removal of steel using a conventional CO2 laser beam and a novel arrangement of inert and reactive gas jets to produce the gas equivalent of a rotary cutter.

An extended computational model of the circulatory system for designing ventricular assist devices.

An extended computational model of the circulatory system has been developed to predict blood flow in the presence of ventricular assist devices (VADs). A novel VAD, placed in the descending aorta, intended to offload the left ventricle (LV) and augment renal perfusion is being studied. For this application, a better understanding of the global hemodynamic response of the VAD, in essence an electrically driven pump, and the cardiovascular system is necessary. To meet this need, a model has been established as a nonlinear, lumped-parameter electrical analog, and simulated results under different states [healthy, congestive heart failure (CHF), and postinsertion of VAD] are presented. The systemic circulation is separated into five compartments and the descending aorta is composed of three components to accurately yield the system response of each section before and after the insertion of the VAD. Delays in valve closing time and blood inertia in the aorta were introduced to deliver a more realistic model. Pump governing equations and optimization are based on fundamental theories of turbomachines and can serve as a practical initial design point for rotary blood pumps. The model's results closely mimic established parameters for the circulatory system and confirm the feasibility of the intra-aortic VAD concept. This computational model can be linked with models of the pump motor to provide a valuable tool for innovative VAD design.