Reinforcement learning with reference tracking control in continuous state spaces

The contribution described in this paper is an algorithm for learning nonlinear, reference tracking, control policies given no prior knowledge of the dynamical system and limited interaction with the system through the learning process. Concepts from the field of reinforcement learning, Bayesian statistics and classical control have been brought together in the formulation of this algorithm which can be viewed as a form of indirect self tuning regulator. On the task of reference tracking using a simulated inverted pendulum it was shown to yield generally improved performance on the best controller derived from the standard linear quadratic method using only 30 s of total interaction with the system. Finally, the algorithm was shown to work on the simulated double pendulum proving its ability to solve nontrivial control tasks. © 2011 IEEE.

Closing the gap: Improved bounds on optimal POMDP solutions

POMDP algorithms have made significant progress in recent years by allowing practitioners to find good solutions to increasingly large problems. Most approaches (including point-based and policy iteration techniques) operate by refining a lower bound of the optimal value function. Several approaches (e.g., HSVI2, SARSOP, grid-based approaches and online forward search) also refine an upper bound. However, approximating the optimal value function by an upper bound is computationally expensive and therefore tightness is often sacrificed to improve efficiency (e.g., sawtooth approximation). In this paper, we describe a new approach to efficiently compute tighter bounds by i) conducting a prioritized breadth first search over the reachable beliefs, ii) propagating upper bound improvements with an augmented POMDP and iii) using exact linear programming (instead of the sawtooth approximation) for upper bound interpolation. As a result, we can represent the bounds more compactly and significantly reduce the gap between upper and lower bounds on several benchmark problems. Copyright © 2011, Association for the Advancement of Artificial Intelligence. All rights reserved.

Predictive control of a Boeing 747 aircraft using an FPGA

New embedded predictive control applications call for more eficient ways of solving quadratic programs (QPs) in order to meet demanding real-time, power and cost requirements. A single precision QP-on-a-chip controller is proposed, implemented in afield-programmable gate array (FPGA) with an iterative linear solver at its core. A novel offline scaling procedure is introduced to aid the convergence of the reduced precision solver. The feasibility of the proposed approach is demonstrated with a real-time hardware-in-the-loop (HIL) experimental setup where an ML605 FPGA board controls a nonlinear model of a Boeing 747 aircraft running on a desktop PC through an Ethernet link. Simulations show that the quality of the closed-loop control and accuracy of individual solutions is competitive with a conventional double precision controller solving linear systems using a Riccati recursion. © 2012 IFAC.

Comparison of stochastic methods for control in air traffic management

This paper provides a direct comparison of two stochastic optimisation techniques (Markov Chain Monte Carlo and Sequential Monte Carlo) when applied to the problem of conflict resolution and aircraft trajectory control in air traffic management. The two methods are then also compared to another existing technique of Mixed-Integer Linear Programming which is also popular in distributed control. © 2011 IFAC.

Channel-hopping model predictive control

In Multiplexed MPC, the control variables of a MIMO plant are moved asynchronously, following a pre-planned periodic sequence. The advantage of Multiplexed MPC lies in its reduced computational complexity, leading to faster response to disturbances, which may result in improved performance, despite finding sub-optimal solution to the original problem. This paper extends the original Multiplexed MPC in a way such that the control inputs are no longer restricted to a pre-planned periodic sequence. Instead, the most appropriate control input channel would be optimised and selected to counter the disturbances, hence the name 'Channel-Hopping'. In addition, the proposed algorithm is suitable for execution on modern computing platforms such as FPGA or GPU, exploits multi-core, parallel and pipeline computing techniques. The algorithm for the proposed Channel-hopping MPC (CH-MPC) will be described and its stability established. Illustrative examples are given to demonstrate the behaviour of the proposed Channel-Hopping MPC algorithm. © 2011 IFAC.

Robust predictive control with feasible contingencies for fault tolerance

This paper introduces the notion of M-step robust fault tolerance for discrete-time systems where finite-time completion of a control manoeuvre is desired. It considers a scenario with two distinct objectives; a primary and secondary target are specified as sets to be reached in finite-time, whilst satisfying operating constraints on the states and inputs. The primary target is switched to the secondary target when a fault affects the system. As it is unknown when or if the fault will occur, the trajectory to the primary target is constrained to ensure reachability of the secondary target within M steps. A variable-horizon linear MPC formulation is developed to illustrate the concept. The formulation is then extended to provide robustness to bounded disturbances by use of tightened constraints. Simulations demonstrate the efficacy of the controller formulation on a double-integrator model. © 2011 IFAC.

Linear multiscale analysis and finite element validation of stretching and bending dominated lattice materials

The paper presents a multiscale procedure for the linear analysis of components made of lattice materials. The method allows the analysis of both pin-jointed and rigid-jointed microtruss materials with arbitrary topology of the unit cell. At the macroscopic level, the procedure enables to determine the lattice stiffness, while at the microscopic level the internal forces in the lattice elements are expressed in terms of the macroscopic strain applied to the lattice component. A numeric validation of the method is described. The procedure is completely automated and can be easily used within an optimization framework to find the optimal geometric parameters of a given lattice material. © 2011 Elsevier Ltd. All rights reserved.

Finite horizon LQR control with limited controller-system communication

We consider finite-horizon LQR control with limited controller-system communication. Within a time-horizon T , the controller can only communicate with the system doptimal control data schedule for unstable first-order systems and a class of higher-order systems. We also discuss when such a control data schedule remains optimal (or is near optimal) for general systems. © 1963-2012 IEEE.

Predictive control using an FPGA with application to aircraft control

Alternative and more efficient computational methods can extend the applicability of model predictive control (MPC) to systems with tight real-time requirements. This paper presents a system-on-a-chip MPC system, implemented on a field-programmable gate array (FPGA), consisting of a sparse structure-exploiting primal dual interior point (PDIP) quadratic program (QP) solver for MPC reference tracking and a fast gradient QP solver for steady-state target calculation. A parallel reduced precision iterative solver is used to accelerate the solution of the set of linear equations forming the computational bottleneck of the PDIP algorithm. A numerical study of the effect of reducing the number of iterations highlights the effectiveness of the approach. The system is demonstrated with an FPGA-in-the-loop testbench controlling a nonlinear simulation of a large airliner. This paper considers many more manipulated inputs than any previous FPGA-based MPC implementation to date, yet the implementation comfortably fits into a midrange FPGA, and the controller compares well in terms of solution quality and latency to state-of-the-art QP solvers running on a standard PC. © 1993-2012 IEEE.

Model based control for closed loop testing of 1-D engine simulation models

1-D engine simulation models are widely used for the analysis and verification of air-path design concepts and prediction of the resulting engine transient response. The latter often requires closed loop control over the model to ensure operation within physical limits and tracking of reference signals. For this purpose, a particular implementation of Model Predictive Control (MPC) based on a corresponding Mean Value Engine Model (MVEM) is reported here. The MVEM is linearised on-line at each operating point to allow for the formulation of quadratic programming (QP) problems, which are solved as the part of the proposed MPC algorithm. The MPC output is used to control a 1-D engine model. The closed loop performance of such a system is benchmarked against the solution of a related optimal control problem (OCP). As an example this study is focused on the transient response of a light-duty car Diesel engine. For the cases examined the proposed controller implementation gives a more systematic procedure than other ad-hoc approaches that require considerable tuning effort. © 2012 IFAC.

Design trade-offs for energy regeneration and control in vehicle suspensions

This paper investigates the fundamental trade-offs involved in designing energy-regenerative suspensions, in particular, focusing on efficiency of power extraction and its effect on vehicle dynamics and control. It is shown that typical regenerative devices making use of linear-to-rotational elements can be modelled as a parallel arrangement of an inerter and a dissipative admittance. Taking account of typical adjustable parameters of the generator, it is shown, for a given suspension damping coefficient, that the power efficiency ratio scales with inertance. For a typical passenger vehicle, it is shown that there is a feasible compromise, namely that good efficiency is achievable with an inertance value that is not detrimental to vehicle performance. A prototype is designed and tested with a resistive termination and experimental results show good agreement between ideal and experimental admittances. The possibility to use dynamic (rather than purely resistive) loads to improve vehicle control without limiting the energy recovery is discussed. © 2013 Copyright Taylor and Francis Group, LLC.

A novel theoretical approach to passive control of thermo-acoustic oscillations: Application to ducted heat sources

In this paper, we develop a linear technique that predicts how the stability of a thermo-acoustic system changes due to the action of a generic passive feedback device or a generic change in the base state. From this, one can calculate the passive device or base state change that most stabilizes the system. This theoretical framework, based on adjoint equations, is applied to two types of Rijke tube. The first contains an electrically-heated hot wire and the second contains a diffusion flame. Both heat sources are assumed to be compact so that the acoustic and heat release models can be decoupled. We find that the most effective passive control device is an adiabatic mesh placed at the downstream end of the Rijke tube. We also investigate the effects of a second hot wire and a local variation of the cross-sectional area but find that both affect the frequency more than the growth rate. This application of adjoint sensitivity analysis opens up new possibilities for the passive control of thermo-acoustic oscillations. For example, the influence of base state changes can be combined with other constraints, such as that the total heat release rate remains constant, in order to show how an unstable thermo-acoustic system should be changed in order to make it stable. Copyright © 2013 by ASME.

Diameter and wall number control of carbon nanotubes by chemical vapor deposition

We analyze the relationship between the average wall number (N) and the diameter (d) for carbon nanotubes (CNTs) grown by chemical vapour deposition. It is found that N depends linearly on d for diameters in the range of 2.5-10 nm, while single wall nanotubes predominate for diameters under about 2.1 nm. The linear relationship is found to depend somewhat on the growth conditions. It is also verified that the mean diameter depends on the diameter of the originating catalyst nanoparticle, and thus on the initial catalyst thickness where a thin film catalyst is used. This simplifies the characterisation of CNTs by electron microscopy. We also find a linear relationship between nanotube diameter and initial catalyst film thickness. © 2013 AIP Publishing LLC.