Bayesian Learning of Noisy Markov Decision Processes

We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we show how latent variables of the model can be estimated, and how predictions about actions can be made, in a unified framework. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from the posterior distribution. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.

Predictive control using an FPGA with application to aircraft control

Alternative and more efficient computational methods can extend the applicability of model predictive control (MPC) to systems with tight real-time requirements. This paper presents a system-on-a-chip MPC system, implemented on a field-programmable gate array (FPGA), consisting of a sparse structure-exploiting primal dual interior point (PDIP) quadratic program (QP) solver for MPC reference tracking and a fast gradient QP solver for steady-state target calculation. A parallel reduced precision iterative solver is used to accelerate the solution of the set of linear equations forming the computational bottleneck of the PDIP algorithm. A numerical study of the effect of reducing the number of iterations highlights the effectiveness of the approach. The system is demonstrated with an FPGA-in-the-loop testbench controlling a nonlinear simulation of a large airliner. This paper considers many more manipulated inputs than any previous FPGA-based MPC implementation to date, yet the implementation comfortably fits into a midrange FPGA, and the controller compares well in terms of solution quality and latency to state-of-the-art QP solvers running on a standard PC. © 1993-2012 IEEE.

Model based control for closed loop testing of 1-D engine simulation models

1-D engine simulation models are widely used for the analysis and verification of air-path design concepts and prediction of the resulting engine transient response. The latter often requires closed loop control over the model to ensure operation within physical limits and tracking of reference signals. For this purpose, a particular implementation of Model Predictive Control (MPC) based on a corresponding Mean Value Engine Model (MVEM) is reported here. The MVEM is linearised on-line at each operating point to allow for the formulation of quadratic programming (QP) problems, which are solved as the part of the proposed MPC algorithm. The MPC output is used to control a 1-D engine model. The closed loop performance of such a system is benchmarked against the solution of a related optimal control problem (OCP). As an example this study is focused on the transient response of a light-duty car Diesel engine. For the cases examined the proposed controller implementation gives a more systematic procedure than other ad-hoc approaches that require considerable tuning effort. © 2012 IFAC.

Design of diesel smoke feedback control using a combination of PI control algorithm and performance optimization

A novel smoke sensor was used to realize smoke feedback control on a diesel engine. The controller design based on a combination of PI control algorithm and the engine performance optimization is described. Experimental results demonstrate how this control system behave to meet both of the speed and smoke requirements during engine transients.

Engine modelling for diesel smoke feedback control system design

A novel smoke sensor was used to measure the smoke response to the fuel rack on a diesel engine. The conventional modelling methods used for engine control were investigated. The synchronization technique and Recursive Least Square method were applied to engine modelling and two models for controller design were derived.

Values and actions in aversion

Model-based and model-free controllers can, in principle, learn arbitrary actions to optimize their behavior, at least those actions that can be expressed and explored. Indeed, these are often referred to as instrumental controllers because their choices are learned to be instrumental for the delivery of desired outcomes. Although this flexibility is very powerful, it comes with an attendant cost of learning. Evolution appears to have endowed everything from the simplest organisms to us with powerful, pre-specified, but inflexible alternatives. These responses are termed Pavlovian, after the famous Russian physiologist and psychologist Pavlov. The responses of the Pavlovian controller are determined by evolutionary (phylogenetic) considerations rather than (ontogenetic) aspects of the contingent development or learning of an individual. These responses directly interact with instrumental choices arising from goal-directed and habitual controllers. This interaction has been studied in a wealth of animal paradigms, and can be helpful, neutral, or harmful, according to circumstance. Although there has been less careful or analytical study of it in humans, it can be interpreted as underpinning a wealth of behavioral aberrations. © 2009 Elsevier Inc. All rights reserved.

Active control of high frequency vibration in uncertain structures

The active suppression of structural vibration is normally achieved by either feedforward or feedback control. In the absence of a suitable reference signal feedforward control cannot be employed and feedback control is the only viable approach. Conventional feedback control algorithms (e.g. LQR and LQG) are designed on the basis of a mathematical model of the system and ideally the performance of the system should be robust against uncertainties in this model. The aim of this paper is to numerically investigate the robustness of LQR and LQG algorithms by designing the controller for a nominal system, and then assessing (via Monte Carlo simulation) the effects of uncertainties in the system. The ultimate concern is with the control of high frequency vibrations, where the short wavelength of the structural deformation induces a high sensitivity to imperfection. It is found that standard algorithms such as LQR and LQG are generally unfeasible for this case. This leads to a consideration of design strategies for the robust active control of high frequency vibrations. The system chosen for the numerical simulation concerns two coupled plates, which are randomized by the addition of point masses at random locations.

Designing and testing an advanced pneumatic braking system for heavy vehicles

Heavy goods vehicles exhibit poor braking performance in emergency situations when compared to other vehicles. Part of the problem is caused by sluggish pneumatic brake actuators, which limit the control bandwidth of their antilock braking systems. In addition, heuristic control algorithms are used that do not achieve the maximum braking force throughout the stop. In this article, a novel braking system is introduced for pneumatically braked heavy goods vehicles. The conventional brake actuators are improved by placing high-bandwidth, binary-actuated valves directly on the brake chambers. A made-for-purpose valve is described. It achieves a switching delay of 3-4 ms in tests, which is an order of magnitude faster than solenoids in conventional anti-lock braking systems. The heuristic braking control algorithms are replaced with a wheel slip regulator based on sliding mode control. The combined actuator and slip controller are shown to reduce stopping distances on smooth and rough, high friction (μ = 0.9) surfaces by 10% and 27% respectively in hardware-in-the-loop tests compared with conventional ABS. On smooth and rough, low friction (μ = 0.2) surfaces, stopping distances are reduced by 23% and 25%, respectively. Moreover, the overall air reservoir size required on a heavy goods vehicle is governed by its air usage during an anti-lock braking stop on a low friction, smooth surface. The 37% reduction in air usage observed in hardware-in-the-loop tests on this surface therefore represents the potential reduction in reservoir size that could be achieved by the new system. © 2012 IMechE.

Experimental LVRT performance of a 250 kW brushless DFIG

Nowadays, all new wind turbine generators have to meet strict grid codes, especially riding through certain grid faults, such as a low voltage caused by grid short circuits. The Low-Voltage Ride Through (LVRT) capability has become a key issue in assessing the performance of wind turbine generators. The mediumspeed Brushless DFIG in combination with a simplified two-stage gearbox shows commercial promise as a replacement for conventional DFIGs due to its lower cost and higher reliability. Furthermore, the Brushless DFIG has significantly improved LVRT performance when compared with the DFIG due to its inherent design characteristics. In this paper, the authors propose a control strategy for the Brushless DFIG to improve its LVRT performance. The controller has been implemented on a prototype 250 kW Brushless DFIG and test results show that LVRT is possible without a need for any external protective hardware such as a crowbar.

Designing output-feedback predictive controllers by reverse-engineering existing LTI controllers

An approach to designing a constrained output-feedback predictive controller that has the same small-signal properties as a pre-existing output-feedback linear time invariant controller is proposed. Systematic guidelines are proposed to select an appropriate (non-unique) realization of the resulting state observer. A method is proposed to transform a class of offset-free reference tracking controllers into the combination of an observer, steady-state target calculator and predictive controller. The procedure is demonstrated with a numerical example. © 2013 IEEE.

Model-based control of thermoacoustic instabilities in partially premixed lean combustion - A design case study

Self-excited oscillation is becoming a major issue in low-emission, lean partially premixed combustion systems, and active control has been shown to be a feasible method to suppress such instabilities. A number of robust control methods are employed to obtain a feedback controller and it is observed that the robustness to system uncertainty is significantly better for a low complexity controller in spite of the norms being similar. Moreover, we demonstrate that closed-loop stability for such a complex system can be proved via use of the integral quadratic constraint method. Open- and closed-loop nonlinear simulations are provided. © 2013 Copyright Taylor and Francis Group, LLC.

Identification of the steering control behaviour of five test subjects following a randomly curving path in a driving simulator

The paper is concerned with the identification of theoretical preview steering controllers using data obtained from five test subjects in a fixed-base driving simulator. An understanding of human steering control behaviour is relevant to the design of autonomous and semi-autonomous vehicle controls. The driving task involved steering a linear vehicle along a randomly curving path. The theoretical steering controllers identified from the data were based on optimal linear preview control. A direct-identification method was used, and the steering controllers were identified so that the predicted steering angle matched as closely as possible the measured steering angle of the test subjects. It was found that identification of the driver's time delay and noise is necessary to avoid bias in identification of the controller parameters. Most subjects' steering behaviour was predicted well by a theoretical controller based on the lateral/yaw dynamics of the vehicle. There was some evidence that an inexperienced driver's steering action was better represented by a controller based on a simpler model of the vehicle dynamics, perhaps reflecting incomplete learning by the driver. Copyright © 2014 Inderscience Enterprises Ltd.

Implementation of trailer steering control on a multi-unit vehicle at high speeds

A high-speed path-following controller for long combination vehicles (LCVs) was designed and implemented on a test vehicle consisting of a rigid truck towing a dolly and a semitrailer. The vehicle was driven through a 3.5 m wide lane change maneuver at 80 km/h. The axles of the dolly and trailer were steered actively by electrically-controlled hydraulic actuators. Substantial performance benefits were recorded compared with the unsteered vehicle. For the best controller weightings, performance improvements relative to unsteered case were: lateral tracking error 75% reduction, rearward amplification (RA) of lateral acceleration 18% reduction, and RA of yaw rate 37% reduction. This represents a substantial improvement in stability margins. The system was found to work well in conjunction with the braking-based stability control system of the towing vehicle with no negative interaction effects being observed. In all cases, the stability control system and the steering system improved the yaw stability of the combination. © 2014 by ASME.

800V lateral IGBT in bulk Si for low power compact SMPS applications

An 800V rated lateral IGBT for high frequency, low-cost off-line applications has been developed. The LIGBT features a new method of adjusting the bipolar gain, based on a floating N+ stripe in front of the P+ anode/drain region. The floating N+ layer enhances the carrier recombination at the anode/drain side of the drift region resulting in a very significant decrease in the turn-off speed and substantially lower overall losses. Switching speeds as low as 140ns at 25oC and 300ns at 125oC have been achieved with corresponding equivalent Rdson at 125oC below 90mω.cm2. A fully operational AC-DC converter using a controller with an integrated LIGBT+depletion mode MOSFET chip has been designed and qualified in plastic SOP8 packages and used in 5W, 65kHz SMPS applications. The device is fabricated in 0.6μm bulk silicon CMOS technology without any additional masking steps. © 2013 IEEE.

Predictive control for spacecraft rendezvous in an elliptical orbit using an FPGA

A field programmable gate array (FPGA)-based predictive controller for a spacecraft rendezvous manoeuvre is presented. A linear time varying prediction model is used to accommodate elliptical orbits, and a variable prediction horizon is used to facilitate finite time completion of manoeuvres. The resulting constrained optimisation problems are solved using a primal dual interior point algorithm. The majority of the computational demand is in solving a set of linear equations at each iteration of this algorithm. To accelerate this operation, a custom circuit is implemented, using a combination of Mathworks HDL Coder and Xilinx System Generator for DSP, and used as a peripheral to a MicroBlaze soft core processor. The system is demonstrated in closed loop by linking the FPGA with a simulation of the plant dynamics running in Simulink on a PC, using Ethernet. © 2013 EUCA.