Reinforcement learning with reference tracking control in continuous state spaces

The contribution described in this paper is an algorithm for learning nonlinear, reference tracking, control policies given no prior knowledge of the dynamical system and limited interaction with the system through the learning process. Concepts from the field of reinforcement learning, Bayesian statistics and classical control have been brought together in the formulation of this algorithm which can be viewed as a form of indirect self tuning regulator. On the task of reference tracking using a simulated inverted pendulum it was shown to yield generally improved performance on the best controller derived from the standard linear quadratic method using only 30 s of total interaction with the system. Finally, the algorithm was shown to work on the simulated double pendulum proving its ability to solve nontrivial control tasks. © 2011 IEEE.

Output disturbance rejection using parallel model predictive control

The solution time of the online optimization problems inherent to Model Predictive Control (MPC) can become a critical limitation when working in embedded systems. One proposed approach to reduce the solution time is to split the optimization problem into a number of reduced order problems, solve such reduced order problems in parallel and selecting the solution which minimises a global cost function. This approach is known as Parallel MPC. The potential capabilities of disturbance rejection are introduced using a simulation example. The algorithm is implemented in a linearised model of a Boeing 747-200 under nominal flight conditions and with an induced wind disturbance. Under significant output disturbances Parallel MPC provides a significant improvement in performance when compared to Multiplexed MPC (MMPC) and Linear Quadratic Synchronous MPC (SMPC). © 2013 IEEE.

The design of a navigation guidance and control system for an unmanned surface vehicle for environmental monitoring

Maintaining the ecosystem is one of the main concerns in this modern age. With the fear of ever-increasing global warming, the UK is one of the key players to participate actively in taking measures to slow down at least its phenomenal rate. As an ingredient to this process, the Springer vehicle was designed and developed for environmental monitoring and pollutant tracking. This special issue paper highlighted the Springer hardware and software architecture including various navigational sensors, a speed controller, and an environmental monitoring unit. In addition, details regarding the modelling of the vessel were outlined based mainly on experimental data. The formulation of a fault tolerant multi-sensor data fusion technique was also presented. Moreover, control strategy based on a linear quadratic Gaussian controller was developed and simulated on the Springer model.
Gaussian controller is developed and simulated on the Springer model.

An Autopilot Based on a Local Control Network Design for an Unmanned Surface Vehicle

Over recent years, a number of marine autopilots designed using linear techniques have underperformed owing to their inability to cope with nonlinear vessel dynamics. To this end, a new design framework for the development of nonlinear autopilots is proposed herein. Local control networks (LCNs) can be used in the design of nonlinear control systems. In this paper, a LCN approach is taken in the design of a nonlinear autopilot for controlling the nonlinear yaw dynamics of an unmanned surface vehicle known as Springer. It is considered the approach is the first of its kind to be used in marine control systems design. Simulation results are presented and the performance of the nonlinear autopilot is compared with that of an existing Springer linear quadratic Gaussian (LQG) autopilot using standard system performance criteria. From the results it can be concluded the LCN autopilot out performed that based on LQG techniques in terms of the selected criteria. Also it provided more energy saving control strategies and would thereby increase operational duration times for the vehicle during real-time missions.

Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems

An algorithm for solving nonlinear discrete time optimal control problems with model-reality differences is presented. The technique uses Dynamic Integrated System Optimization and Parameter Estimation (DISOPE), which achieves the correct optimal solution in spite of deficiencies in the mathematical model employed in the optimization procedure. A version of the algorithm with a linear-quadratic model-based problem, implemented in the C+ + programming language, is developed and applied to illustrative simulation examples. An analysis of the optimality and convergence properties of the algorithm is also presented.

Hierarchical control and optimisation of the Tennessee Eastman process

Based on integrated system optimisation and parameter estimation a method is described for on-line steady state optimisation which compensates for model-plant mismatch and solves a non-linear optimisation problem by iterating on a linear - quadratic representation. The method requires real process derivatives which are estimated using a dynamic identification technique. The utility of the method is demonstrated using a simulation of the Tennessee Eastman benchmark chemical process.

Optimal control of nonlinear differential algebraic equation systems

A novel iterative procedure is described for solving nonlinear optimal control problems subject to differential algebraic equations. The procedure iterates on an integrated modified linear quadratic model based problem with parameter updating in such a manner that the correct solution of the original non-linear problem is achieved. The resulting algorithm has a particular advantage in that the solution is achieved without the need to solve the differential algebraic equations . Convergence aspects are discussed and a simulation example is described which illustrates the performance of the technique. 1. Introduction When modelling industrial processes often the resulting equations consist of coupled differential and algebraic equations (DAEs). In many situations these equations are nonlinear and cannot readily be directly reduced to ordinary differential equations.

Body-centric modelling, identification, and acceleration tracking control of a quadrotor UAV

This paper presents the mathematical development of a body-centric nonlinear dynamic model of a quadrotor UAV that is suitable for the development of biologically inspired navigation strategies. Analytical approximations are used to find an initial guess of the parameters of the nonlinear model, then parameter estimation methods are used to refine the model parameters using the data obtained from onboard sensors during flight. Due to the unstable nature of the quadrotor model, the identification process is performed with the system in closed-loop control of attitude angles. The obtained model parameters are validated using real unseen experimental data. Based on the identified model, a Linear-Quadratic (LQ) optimal tracker is designed to stabilize the quadrotor and facilitate its translational control by tracking body accelerations. The LQ tracker is tested on an experimental quadrotor UAV and the obtained results are a further means to validate the quality of the estimated model. The unique formulation of the control problem in the body frame makes the controller better suited for bio-inspired navigation and guidance strategies than conventional attitude or position based control systems that can be found in the existing literature.

A control methodology for DFIG type wind turbines connected to distribution networks

This paper proposes a decentralised controller design for doubly-fed induction generators (DFIGs) to enhance dynamic performance of distribution networks. The change in the output power due to the variable nature of wind is considered as an uncertain term in the design algorithm. In addition, the interconnection effect of the other subsystems are considered in the design process. The H norm of the uncertain system is found out and simultaneous output-feedback linear controllers are designed based controller is verified on a 16 bus distribution test system for severe disturbances. Simulation results indicate that the designed controller is robust against uncertainties in operating conditions

D-STATCOM control in distribution networks with composite loads to ensure grid code compatible performance of photovoltaic generators

This paper proposes an optimal linear quadratic Gaussian (LQG) controller for D-STATCOM to improve the dynamic performance of distribution networks with photovoltaic generators. The controller is designed based on the H∞ norm of the uncertain system. The change in system model due to the variation of load compositions in the composite load is considered as an uncertain term in the design algorithm. The performance of the designed controller is demonstrated on a widely used test system. Simulation results indicate that the proposed controller can be a potential solution for improving the voltage stability of distribution networks.

Enhancement of transient stability limit and voltage regulation with dynamic loads using robust excitation control

In stressed power systems with large induction machine component, there exist undamped electromechanical modes and unstable montonic voltage modes. This article proposes a sequential design of an excitation controller and a power system stabiliser (PSS) to stabilise the system. The operating region, with induction machines in stressed power systems, is often not captured using a linearisation around an operating point, and to alleviate this situation a robust controller is designed which guaruntees stable operation in a large region of operation. A minimax linear quadratic Gaussian design is used for the design of the supplementary control to automatic voltage regulators, and a classical PSS structure is used to damp electromechanical oscillations. The novelty of this work is in proposing a method to capture the unmodelled nonlinear dynamics as uncertainty in the design of the robust controller. Tight bounds on the uncertainty are obtained using this method which enables high-performance controllers. An IEEE benchmark test system has been used to demonstrate the performance of the designed controller

Swing-up and stability control of Wheeled Acrobot (WAcrobot)

In this paper the Wheeled Acrobot (WAcrobot), a novel mechanical system consisting of an underactuated double inverted pendulum robot (Acrobot) equipped with actuated wheels, is described. This underactuated and highly nonlinear system has potential applications in mobile manipulators and leg-wheeled robots. It is also a testbed for researchers studying advanced methodologies in nonlinear control. The control system for swing-up of the WAcrobot based on collocated or non-collocated feedback linearisation to linearise the active or passive Degree Of Freedom (DOF) followed by Linear Quadratic Regulator (LQR) to stabilise the robot is discussed. The effectiveness of the proposed scheme is validated with numerical simulation. The numerical results are visualised by graphical simulation to demonstrate the correlation between the numerical results and the WAcrobot physical response.

System identification and active vibration control of a flexible structure

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Chaos control of a nonlinear oscillator with shape memory alloy using an optimal linear control: Part I: Ideal energy source

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Chaos control of a nonlinear oscillator with shape memory alloy using an optimal linear control: Part II: Nonideal energy source

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)