A unified framework for linear function approximation of value functions in stochastic control

This paper contributes with a unified formulation that merges previ- ous analysis on the prediction of the performance ( value function ) of certain sequence of actions ( policy ) when an agent operates a Markov decision process with large state-space. When the states are represented by features and the value function is linearly approxi- mated, our analysis reveals a new relationship between two common cost functions used to obtain the optimal approximation. In addition, this analysis allows us to propose an efficient adaptive algorithm that provides an unbiased linear estimate. The performance of the pro- posed algorithm is illustrated by simulation, showing competitive results when compared with the state-of-the-art solutions.

Neural network enhanced self tuning adaptive control application for non-linear control of dynamic systems

The main theme of research of this project concerns the study of neutral networks to control uncertain and non-linear control systems. This involves the control of continuous time, discrete time, hybrid and stochastic systems with input, state or output constraints by ensuring good performances. A great part of this project is devoted to the opening of frontiers between several mathematical and engineering approaches in order to tackle complex but very common non-linear control problems. The objectives are: 1. Design and develop procedures for neutral network enhanced self-tuning adaptive non-linear control systems; 2. To design, as a general procedure, neural network generalised minimum variance self-tuning controller for non-linear dynamic plants (Integration of neural network mapping with generalised minimum variance self-tuning controller strategies); 3. To develop a software package to evaluate control system performances using Matlab, Simulink and Neural Network toolbox. An adaptive control algorithm utilising a recurrent network as a model of a partial unknown non-linear plant with unmeasurable state is proposed. Appropriately, it appears that structured recurrent neural networks can provide conveniently parameterised dynamic models for many non-linear systems for use in adaptive control. Properties of static neural networks, which enabled successful design of stable adaptive control in the state feedback case, are also identified. A survey of the existing results is presented which puts them in a systematic framework showing their relation to classical self-tuning adaptive control application of neural control to a SISO/MIMO control. Simulation results demonstrate that the self-tuning design methods may be practically applicable to a reasonably large class of unknown linear and non-linear dynamic control systems.

Deep hole drilling with twist drills: aspects of the CNC process and its real time monitoring and adaptive control

Deep hole drilling is one of the most complicated metal cutting processes and one of the most difficult to perform on CNC machine-tools or machining centres under conditions of limited manpower or unmanned operation. This research work investigates aspects of the deep hole drilling process with small diameter twist drills and presents a prototype system for real time process monitoring and adaptive control; two main research objectives are fulfilled in particular : First objective is the experimental investigation of the mechanics of the deep hole drilling process, using twist drills without internal coolant supply, in the range of diarneters Ø 2.4 to Ø4.5 mm and working length up to 40 diameters. The definition of the problems associated with the low strength of these tools and the study of mechanisms of catastrophic failure which manifest themselves well before and along with the classic mechanism of tool wear. The relationships between drilling thrust and torque with the depth of penetration and the various machining conditions are also investigated and the experimental evidence suggests that the process is inherently unstable at depths beyond a few diameters. Second objective is the design and implementation of a system for intelligent CNC deep hole drilling, the main task of which is to ensure integrity of the process and the safety of the tool and the workpiece. This task is achieved by means of interfacing the CNC system of the machine tool to an external computer which performs the following functions: On-line monitoring of the drilling thrust and torque, adaptive control of feed rate, spindle speed and tool penetration (Z-axis), indirect monitoring of tool wear by pattern recognition of variations of the drilling thrust with cumulative cutting time and drilled depth, operation as a data base for tools and workpieces and finally issuing of alarms and diagnostic messages.

A probabilistic indirect adaptive control for systems with input-dependent noise

A probabilistic indirect adaptive controller is proposed for the general nonlinear multivariate class of discrete time system. The proposed probabilistic framework incorporates input–dependent noise prediction parameters in the derivation of the optimal control law. Moreover, because noise can be nonstationary in practice, the proposed adaptive control algorithm provides an elegant method for estimating and tracking the noise. For illustration purposes, the developed method is applied to the affine class of nonlinear multivariate discrete time systems and the desired result is obtained: the optimal control law is determined by solving a cubic equation and the distribution of the tracking error is shown to be Gaussian with zero mean. The efficiency of the proposed scheme is demonstrated numerically through the simulation of an affine nonlinear system.

System identification and control of an aerobot drive system

Fast thrust changes are important for authoritive control of VTOL micro air vehicles. Fixed-pitch rotors that alter thrust by varying rotor speed require high-bandwidth control systems to provide adequate performace. We develop a feedback compensator for a brushless hobby motor driving a custom rotor suitable for UAVs. The system plant is identified using step excitation experiments. The aerodynamic operating conditions of these rotors are unusual and so experiments are performed to characterise expected load disturbances. The plant and load models lead to a proportional controller design capable of significantly decreasing rise-time and propagation of disturbances, subject to bus voltage constraints.

Migrating to a sustainable energy system : distributed generation and storage, fuel cells and hypercars

This paper examines a number of issues in sustainable energy generation and distribution, and explores avenues that are available for integration of our society’s energy supplies. In particular, the paper presents a way in which transport vehicle energy supplies could be integrated with distributed generation schemes to achieve synergistic and beneficial outcomes. The worldwide energy system contains fundamental problems that result directly from the use of unsustainable fuels and a lack of energy system integration. There is a need to adopt an integrated, sustainable energy system for our society. The adoption of distributed generation could result in beneficial restructuring of the energy trade, and a change in the role of energy providers. Inherent benefits in distributed generation schemes would directly combat barriers to installation of renewable generation facilities, which might prove distributed renewable energy sources to be more feasible. The presence of fuel cells, batteries, power electronic inverters and intelligent controls in vehicles of the future provides many opportunities for the integration of vehicle energy supplies into a distributed generation scheme. In such a system, vehicles could play a major role in power generation and storage.

Migrating to a sustainable energy system : distributed generation and storage, fuel cells and hypercars

This paper examines a number of issues in sustainable energy generation and distribution, and explores avenues that are available for integration of our society’s energy supplies. In particular, the paper presents a way in which transport vehicle energy supplies could be integrated with distributed generation schemes to achieve synergistic and beneficial outcomes. The worldwide energy system contains fundamental problems that result directly from the use of unsustainable fuels and a lack of energy system integration. There is a need to adopt an integrated, sustainable energy system for our society. The adoption of distributed generation could result in beneficial restructuring of the energy trade, and a change in the role of energy providers. Inherent benefits in distributed generation schemes would directly combat barriers to installation of renewable generation facilities, which might prove distributed renewable energy sources to be more feasible. The presence of fuel cells, batteries, power electronic inverters and intelligent controls in vehicles of the future provides many opportunities for the integration of vehicle energy supplies into a distributed generation scheme. In such a system, vehicles could play a major role in power generation and storage.

Multi-rotor with suspended load: System dynamics and control toolbox

There is an increasing demand for Unmanned Aerial Systems (UAS) to carry suspended loads as this can provide significant benefits to several applications in agriculture, law enforcement and construction. The load impact on the underlying system dynamics should not be neglected as significant feedback forces may be induced on the vehicle during certain flight manoeuvres. The constant variation in operating point induced by the slung load also causes conventional controllers to demand increased control effort. Much research has focused on standard multi-rotor position and attitude control with and without a slung load. However, predictive control schemes, such as Nonlinear Model Predictive Control (NMPC), have not yet been fully explored. To this end, we present a novel controller for safe and precise operation of multi-rotors with heavy slung load in three dimensions. The paper describes a System Dynamics and Control Simulation Toolbox for use with MATLAB/SIMULINK which includes a detailed simulation of the multi-rotor and slung load as well as a predictive controller to manage the nonlinear dynamics whilst accounting for system constraints. It is demonstrated that the controller simultaneously tracks specified waypoints and actively damps large slung load oscillations. A linear-quadratic regulator (LQR) is derived and control performance is compared. Results show the improved performance of the predictive controller for a larger flight envelope, including aggressive manoeuvres and large slung load displacements. The computational cost remains relatively small, amenable to practical implementations.

On the Milito-Cruz adaptive control scheme for Markov chains

Milito and Cruz have introduced a novel adaptive control scheme for finite Markov chains when a finite parametrized family of possible transition matrices is available. The scheme involves the minimization of a composite functional of the observed history of the process incorporating both control and estimation aspects. We prove the a.s. optimality of a similar scheme when the state space is countable and the parameter space a compact subset ofR.

Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Flexible implementation of power system corrective topology control

This paper proposes a novel decision making framework for optimal transmission switching satisfying the AC feasibility, stability and circuit breaker (CB) reliability requirements needed for practical implementation. The proposed framework can be employed as a corrective tool in day to day operation planning scenarios in response to potential contingencies. The switching options are determined using an efficient heuristic algorithm based on DC optimal power flow, and are presented in a multi-branch tree structure. Then, the AC feasibility and stability checks are conducted and the CB condition monitoring data are employed to perform a CB reliability and line availability assessment. Ultimately, the operator will be offered multiple AC feasible and stable switching options with associated benefits. The operator can use this information, other operating conditions not explicitly considered in the optimization, and his/her own experience to implement the best and most reliable switching action(s). The effectiveness of the proposed approach is validated on the IEEE-118 bus test system. (C) 2015 Elsevier B.V. All rights reserved.