A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems

Even though dynamic programming offers an optimal control solution in a state feedback form, the method is overwhelmed by computational and storage requirements. Approximate dynamic programming implemented with an Adaptive Critic (AC) neural network structure has evolved as a powerful alternative technique that obviates the need for excessive computations and storage requirements in solving optimal control problems. In this paper, an improvement to the AC architecture, called the �Single Network Adaptive Critic (SNAC)� is presented. This approach is applicable to a wide class of nonlinear systems where the optimal control (stationary) equation can be explicitly expressed in terms of the state and costate variables. The selection of this terminology is guided by the fact that it eliminates the use of one neural network (namely the action network) that is part of a typical dual network AC setup. As a consequence, the SNAC architecture offers three potential advantages: a simpler architecture, lesser computational load and elimination of the approximation error associated with the eliminated network. In order to demonstrate these benefits and the control synthesis technique using SNAC, two problems have been solved with the AC and SNAC approaches and their computational performances are compared. One of these problems is a real-life Micro-Electro-Mechanical-system (MEMS) problem, which demonstrates that the SNAC technique is applicable to complex engineering systems.

Clan-based evolutionary approach for solving control problems

This paper presents a dan-based evolutionary approach for solving control problems. Three selected control problems, viz. linear-quadratic, harvest, and push-cart problems, are solved using the proposed approach. Results are compared with those of the evolutionary programming (EP) approach. In most of the cases, the proposed approach is successful in obtaining (near) optimal solutions for these selected problems.

Optimal blood glucose regulation of diabetic patients using single network adaptive critics

Diabetes is a long-term disease during which the body's production and use of insulin are impaired, causing glucose concentration level to increase in the bloodstream. Regulating blood glucose levels as close to normal as possible leads to a substantial decrease in long-term complications of diabetes. In this paper, an intelligent online feedback-treatment strategy is presented for the control of blood glucose levels in diabetic patients using single network adaptive critic (SNAC) neural networks (which is based on nonlinear optimal control theory). A recently developed mathematical model of the nonlinear dynamics of glucose and insulin interaction in the blood system has been revised and considered for synthesizing the neural network for feedback control. The idea is to replicate the function of pancreatic insulin, i.e. to have a fairly continuous measurement of blood glucose and a situation-dependent insulin injection to the body using an external device. Detailed studies are carried out to analyze the effectiveness of this adaptive critic-based feedback medication strategy. A comparison study with linear quadratic regulator (LQR) theory shows that the proposed nonlinear approach offers some important advantages such as quicker response, avoidance of hypoglycemia problems, etc. Robustness of the proposed approach is also demonstrated from a large number of simulations considering random initial conditions and parametric uncertainties. Copyright (C) 2009 John Wiley & Sons, Ltd.

Optimal control of arrivals to queues with delayed queue length information

We consider discrete-time versions of two classical problems in the optimal control of admission to a queueing system: i) optimal routing of arrivals to two parallel queues and ii) optimal acceptance/rejection of arrivals to a single queue. We extend the formulation of these problems to permit a k step delay in the observation of the queue lengths by the controller. For geometric inter-arrival times and geometric service times the problems are formulated as controlled Markov chains with expected total discounted cost as the minimization objective. For problem i) we show that when k = 1, the optimal policy is to allocate an arrival to the queue with the smaller expected queue length (JSEQ: Join the Shortest Expected Queue). We also show that for this problem, for k greater than or equal to 2, JSEQ is not optimal. For problem ii) we show that when k = 1, the optimal policy is a threshold policy. There are, however, two thresholds m(0) greater than or equal to m(1) > 0, such that mo is used when the previous action was to reject, and mi is used when the previous action was to accept.

Optimal control of a stochastic hybrid system with discounted cost

We address the optimal control problem of a very general stochastic hybrid system with both autonomous and impulsive jumps. The planning horizon is infinite and we use the discounted-cost criterion for performance evaluation. Under certain assumptions, we show the existence of an optimal control. We then derive the quasivariational inequalities satisfied by the value function and establish well-posedness. Finally, we prove the usual verification theorem of dynamic programming.

Risk-sensitive optimal control for Markov decision processes with monotone cost

The existence of an optimal feedback law is established for the risk-sensitive optimal control problem with denumerable state space. The main assumptions imposed are irreducibility and a near monotonicity condition on the one-step cost function. A solution can be found constructively using either value iteration or policy iteration under suitable conditions on initial feedback law.

Decentralized control of linear multivariable systems

A strongly connected decentralized control system may be made single channel controllable and observable with respect to any channel by decentralized feedbacks. It is noted here that the system example considered by Corfmat and Morse to illustrate this fact is already single channel controllable and observable, with respect to one of the channels. An alternate example which fits into the situation is presented in this item.

Register Allocation and Optimal Spill code Scheduling in Software Pipelined Loops using 0-1 Integer Linear Programming Formulation

In achieving higher instruction level parallelism, software pipelining increases the register pressure in the loop. The usefulness of the generated schedule may be restricted to cases where the register pressure is less than the available number of registers. Spill instructions need to be introduced otherwise. But scheduling these spill instructions in the compact schedule is a difficult task. Several heuristics have been proposed to schedule spill code. These heuristics may generate more spill code than necessary, and scheduling them may necessitate increasing the initiation interval. We model the problem of register allocation with spill code generation and scheduling in software pipelined loops as a 0-1 integer linear program. The formulation minimizes the increase in initiation interval (II) by optimally placing spill code and simultaneously minimizes the amount of spill code produced. To the best of our knowledge, this is the first integrated formulation for register allocation, optimal spill code generation and scheduling for software pipelined loops. The proposed formulation performs better than the existing heuristics by preventing an increase in II in 11.11% of the loops and generating 18.48% less spill code on average among the loops extracted from Perfect Club and SPEC benchmarks with a moderate increase in compilation time.

Analysis of a linear one-dimensional active noise control system by means of block diagrams and transfer functions

In arriving at the ideal filter transfer function for an active noise control system in a duct, the effect of the auxiliary sources (generally loudspeakers) on the waves generated by the primary source has invariably been neglected in the existing literature, implying a rigid wall or infinite impedance. The present paper presents a fairly general analysis of a linear one-dimensional noise control system by means of block diagrams and transfer functions. It takes into account the passive as well as active role of a terminal primary source, wall-mounted auxiliary source, open duct radiation impedance, and the effects of mean flow and damping. It is proved that the pressure generated by a source against a load impedance can be looked upon as a sum of two pressure waves, one generated by the source against an anechoic termination and the other by reflecting the rearward wave (incident on the source) off the passive source impedance. Application of this concept is illustrated for both the types of sources. A concise closed-form expression for the ideal filter transfer function is thus derived and discussed. Finally, the dynamics of an adaptive noise control system is discussed briefly, relating its standing-wave variables and transfer functions with those of the progressive-wave model presented here.

Optimal control for the rolling pullout maneuver of a modern fighter aircraft

Optimal control laws are obtained for the elevator and the ailerons for a modern fighter aircraft in a rolling pullout maneuver. The problem is solved for three flight conditions using the conjugate gradient method.

Control technique for non-linear and unbalance compensation of an integrated magnetics-based three-phase series-shunt-compensated uninterruptible power supply

Handling unbalanced and non-linear loads in a three-phase AC power supply has always been a difficult issue. This has been addressed in the literature by either using fast controllers in the fundamental rotating reference frame or using separate controllers in reference frames specific to the harmonics. In the former case, the controller needs to be fast and in the latter case, besides the need for many controllers, negative-sequence components need to be extracted from the measured signal. This study proposes a control scheme for harmonic and unbalance compensation of a three-phase uninterruptible power supply wherein the problems mentioned above are addressed. The control takes place in the fundamental positive-sequence reference frame using only a set of feedback and feed-forward compensators. The harmonic components are extracted by a process of frame transformations and used as feed-forward compensation terms in the positive-sequence fundamental reference frame. This study uses a method wherein the measured signal itself is used for fundamental negative-sequence compensation. As the feed-forward compensator handles the high-bandwidth components, the feedback compensator can be a simple low-bandwidth one. This control algorithm is explained and validated experimentally.

Control technique for non-linear and unbalance compensation of an integrated magnetics-based three-phase series-shunt-compensated uninterruptible power supply

Handling unbalanced and non-linear loads in a three-phase AC power supply has always been a difficult issue. This has been addressed in the literature by either using fast controllers in the fundamental rotating reference frame or using separate controllers in reference frames specific to the harmonics. In the former case, the controller needs to be fast and in the latter case, besides the need for many controllers, negative-sequence components need to be extracted from the measured signal. This study proposes a control scheme for harmonic and unbalance compensation of a three-phase uninterruptible power supply wherein the problems mentioned above are addressed. The control takes place in the fundamental positive-sequence reference frame using only a set of feedback and feed-forward compensators. The harmonic components are extracted by a process of frame transformations and used as feed-forward compensation terms in the positive-sequence fundamental reference frame. This study uses a method wherein the measured signal itself is used for fundamental negative-sequence compensation. As the feed-forward compensator handles the high-bandwidth components, the feedback compensator can be a simple low-bandwidth one. This control algorithm is explained and validated experimentally.

Control technique for non-linear and unbalance compensation of an integrated magnetics-based three-phase series-shunt-compensated uninterruptible power supply

Handling unbalanced and non-linear loads in a three-phase AC power supply has always been a difficult issue. This has been addressed in the literature by either using fast controllers in the fundamental rotating reference frame or using separate controllers in reference frames specific to the harmonics. In the former case, the controller needs to be fast and in the lattercase, besides the need for many controllers, negative-sequence components need to be extracted from the measured signal.This study proposes a control scheme for harmonic and unbalance compensation of a three-phase uninterruptible power supply wherein the problems mentioned above are addressed. The control takes place in the fundamental positive-sequence reference frame using only a set of feedback and feed-forward compensators. The harmonic components are extracted by process of frame transformations and used as feed-forward compensation terms in the positive-sequence fundamental reference frame. This study uses a method wherein the measured signal itself is used for fundamental negative-sequence compensation. As the feed-forward compensator handles the high-bandwidth components, the feedback compensator can be a simple low-bandwidth one. This control algorithm is explained and validated experimentally.

Optimal Control of Markov Processes with Age-Dependent Transition Rates

We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite horizon average cost problems. Our approach is via the construction of an equivalent semi-Markov decision process. We characterise the value function and optimal controls for both discounted and average cost cases.