Pareto-optimal solutions for multi-objective optimization of fed-batch bioreactors using nondominated sorting genetic algorithm

Many optimal control problems are characterized by their multiple performance measures that are often noncommensurable and competing with each other. The presence of multiple objectives in a problem usually give rise to a set of optimal solutions, largely known as Pareto-optimal solutions. Evolutionary algorithms have been recognized to be well suited for multi-objective optimization because of their capability to evolve a set of nondominated solutions distributed along the Pareto front. This has led to the development of many evolutionary multi-objective optimization algorithms among which Nondominated Sorting Genetic Algorithm (NSGA and its enhanced version NSGA-II) has been found effective in solving a wide variety of problems. Recently, we reported a genetic algorithm based technique for solving dynamic single-objective optimization problems, with single as well as multiple control variables, that appear in fed-batch bioreactor applications. The purpose of this study is to extend this methodology for solution of multi-objective optimal control problems under the framework of NSGA-II. The applicability of the technique is illustrated by solving two optimal control problems, taken from literature, which have usually been solved by several methods as single-objective dynamic optimization problems. (C) 2004 Elsevier Ltd. All rights reserved.

Optimal threshold policies for admission control in communication networks via discrete parameter stochastic approximation

The problem of admission control of packets in communication networks is studied in the continuous time queueing framework under different classes of service and delayed information feedback. We develop and use a variant of a simulation based two timescale simultaneous perturbation stochastic approximation (SPSA) algorithm for finding an optimal feedback policy within the class of threshold type policies. Even though SPSA has originally been designed for continuous parameter optimization, its variant for the discrete parameter case is seen to work well. We give a proof of the hypothesis needed to show convergence of the algorithm on our setting along with a sketch of the convergence analysis. Extensive numerical experiments with the algorithm are illustrated for different parameter specifications. In particular, we study the effect of feedback delays on the system performance.

A reinforcement learning based algorithm for finite horizon Markov decision processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

A four-timescale algorithm for constrained stochastic optimization of RED

The overall performance of random early detection (RED) routers in the Internet is determined by the settings of their associated parameters. The non-availability of a functional relationship between the RED performance and its parameters makes it difficult to implement optimization techniques directly in order to optimize the RED parameters. In this paper, we formulate a generic optimization framework using a stochastically bounded delay metric to dynamically adapt the RED parameters. The constrained optimization problem thus formulated is solved using traditional nonlinear programming techniques. Here, we implement the barrier and penalty function approaches, respectively. We adopt a second-order nonlinear optimization framework and propose a novel four-timescale stochastic approximation algorithm to estimate the gradient and Hessian of the barrier and penalty objectives and update the RED parameters. A convergence analysis of the proposed algorithm is briefly sketched. We perform simulations to evaluate the performance of our algorithm with both barrier and penalty objectives and compare these with RED and a variant of it in the literature. We observe an improvement in performance using our proposed algorithm over RED, and the above variant of it.

Switching frequency variation control in hysteresis PWM controller for IM drives using variable parabolic bands for current error space phasor

Variation of switching frequency over the entire operating speed range of an induction motor (M drive is the major problem associated with conventional two-level three-phase hysteresis controller as well as the space phasor based PWM hysteresis controller. This paper describes a simple hysteresis current controller for controlling the switching frequency variation in the two-level PWM inverter fed IM drives for various operating speeds. A novel concept of continuously variable hysteresis boundary of current error space phasor with the varying speed of the IM drive is proposed in the present work. The variable parabolic boundary for the current error space phasor is suggested for the first time in this paper for getting the switching frequency pattern with the hysteresis controller, similar to that of the constant switching frequency voltage-controlled space vector PWM (VC-SVPWM) based inverter fed IM drive. A generalized algorithm is also developed to determine parabolic boundary for controlling the switching frequency variation, for any IM load. Only the adjacent inverter voltage vectors forming a triangular sector, in which tip of the machine voltage vector ties, are switched to keep current error space vector within the parabolic boundary. The controller uses a self-adaptive sector identification logic, which provides smooth transition between the sectors and is capable of taldng the inverter up to six-step mode of operation, if demanded by drive system. The proposed scheme is simulated and experimentally verified on a 3.7 kW IM drive.

Generation of parabolic trajectories for current error space phasor similar to an SVPWM controller for control of switching frequency variation in current hysteresis PWM controlled IM drives

Switching frequency variation over a fundamental period is a major problem associated with hysteresis controller based VSI fed IM drives. This paper describes a novel concept of generating parabolic trajectories for current error space phasor for controlling the switching frequency variation in the hysteresis controller based two-level inverter fed IM drives. A generalized algorithm is developed to determine unique set of parabolic trajectories for different speeds of operation for any given IM load. Proposed hysteresis controller provides the switching frequency spectrum of inverter output voltage, similar to that of the constant switching frequency VC-SVPWM based IM drive. The scheme is extensively simulated and experimentally verified on a 3.7 kW IM drive for steady state and transient performance.

Real-time image denoising algorithm in teleradiology systems - art.no.606304

Denoising of medical images in wavelet domain has potential application in transmission technologies such as teleradiology. This technique becomes all the more attractive when we consider the progressive transmission in a teleradiology system. The transmitted images are corrupted mainly due to noisy channels. In this paper, we present a new real time image denoising scheme based on limited restoration of bit-planes of wavelet coefficients. The proposed scheme exploits the fundamental property of wavelet transform - its ability to analyze the image at different resolution levels and the edge information associated with each sub-band. The desired bit-rate control is achieved by applying the restoration on a limited number of bit-planes subject to the optimal smoothing. The proposed method adapts itself to the preference of the medical expert; a single parameter can be used to balance the preservation of (expert-dependent) relevant details against the degree of noise reduction. The proposed scheme relies on the fact that noise commonly manifests itself as a fine-grained structure in image and wavelet transform allows the restoration strategy to adapt itself according to directional features of edges. The proposed approach shows promising results when compared with unrestored case, in context of error reduction. It also has capability to adapt to situations where noise level in the image varies and with the changing requirements of medical-experts. The applicability of the proposed approach has implications in restoration of medical images in teleradiology systems. The proposed scheme is computationally efficient.

Time Scale Separated Nonlinear Partial Integrated Guidance and Control of Interceptors in the Terminal Phase

The paper proposes a time scale separated partial integrated guidance and control of an interceptor for engaging high speed targets in the terminal phase. In this two loop design, the outer loop is an optimal control formulation based on nonlinear model predictive spread control philosophies. It gives the commanded pitch and yaw rates whereas necessary roll-rate command is generated from a roll-stabilization loop. The inner loop tracks the outer loop commands using the dynamicinversion philosophy. However, unlike conventional designs, in both the loops the Six degree of freedom (Six-DOF) interceptor model is used directly. This intelligent manipulation preserves the inherent time scale separation property between the translational and rotational dynamics, and hence overcomes the deficiency of current IGC designs, while preserving its benefits. Six-DOF simulation studies have been carried out accounting for three dimensional engagement geometry. Different comparison studies were also conducted to measure the performance of the algorithm.

Risk minimization in water quality control problems of a river system

Methodologies are presented for minimization of risk in a river water quality management problem. A risk minimization model is developed to minimize the risk of low water quality along a river in the face of conflict among various stake holders. The model consists of three parts: a water quality simulation model, a risk evaluation model with uncertainty analysis and an optimization model. Sensitivity analysis, First Order Reliability Analysis (FORA) and Monte-Carlo simulations are performed to evaluate the fuzzy risk of low water quality. Fuzzy multiobjective programming is used to formulate the multiobjective model. Probabilistic Global Search Laussane (PGSL), a global search algorithm developed recently, is used for solving the resulting non-linear optimization problem. The algorithm is based on the assumption that better sets of points are more likely to be found in the neighborhood of good sets of points, therefore intensifying the search in the regions that contain good solutions. Another model is developed for risk minimization, which deals with only the moments of the generated probability density functions of the water quality indicators. Suitable skewness values of water quality indicators, which lead to low fuzzy risk are identified. Results of the models are compared with the results of a deterministic fuzzy waste load allocation model (FWLAM), when methodologies are applied to the case study of Tunga-Bhadra river system in southern India, with a steady state BOD-DO model. The fractional removal levels resulting from the risk minimization model are slightly higher, but result in a significant reduction in risk of low water quality. (c) 2005 Elsevier Ltd. All rights reserved.

A Computationally Efficient PWM Algorithm for Multilevel Inverters

Centred space vector PWM (CSVPWM) technique is popularly used for three level voltage source inverters. The reference voltage vector is synthesized by time-averaging of the three nearest voltage vectors produced by the inverter. Identifying the three voltage vectors, and calculation of the dwelling time for each vector are both computationally intensive. This paper analyses the process of PWM generation in CSVPWM. This analysis breaks up a three-level inverter into six different conceptual two level inverters in different regions of the fundamental cycle. Control of 3-level inverter is viewed as the control of the appropriate 2-level inverter. The analysis leads to a systematic simplification of the computations involved, finally resulting in a computationally efficient PWM algorithm. This algorithm exploits the equivalence between triangle comparison and space vector approaches to PWM generation. This algorithm does not involve any 3-phase/2-phase or 2-phase/3-phase transformation. This also does not involve any transformation from rectangular to polar coordinates, and vice versa. Further no evaluation of trigonometric functions is necessary. This algorithm also provides for the mitigation of DC neutral point unbalance, and is well suited to digital implementation. Simulation and experimental results are presented.

An actor-critic algorithm with function approximation for discounted cost constrained Markov decision processes

We develop in this article the first actor-critic reinforcement learning algorithm with function approximation for a problem of control under multiple inequality constraints. We consider the infinite horizon discounted cost framework in which both the objective and the constraint functions are suitable expected policy-dependent discounted sums of certain sample path functions. We apply the Lagrange multiplier method to handle the inequality constraints. Our algorithm makes use of multi-timescale stochastic approximation and incorporates a temporal difference (TD) critic and an actor that makes a gradient search in the space of policy parameters using efficient simultaneous perturbation stochastic approximation (SPSA) gradient estimates. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal policy. (C) 2010 Elsevier B.V. All rights reserved.

Positional Consensus of Multi-Agent Systems Using Linear Programming Based Decentralized Control with Rectilinear Decision Domain

In this paper we develop a Linear Programming (LP) based decentralized algorithm for a group of multiple autonomous agents to achieve positional consensus. Each agent is capable of exchanging information about its position and orientation with other agents within their sensing region. The method is computationally feasible and easy to implement. Analytical results are presented. The effectiveness of the approach is illustrated with simulation results.

Convergence of Power Control in a Random Channel Environment

In this paper, we study the Foschini Miljanic algorithm, which was originally proposed in a static channel environment. We investigate the algorithm in a random channel environment, study its convergence properties and apply the Gerschgorin theorem to derive sufficient conditions for the convergence of the algorithm. We apply the Foschini and Miljanic algorithm to cellular networks and derive sufficient conditions for the convergence of the algorithm in distribution and validate the results with simulations. In cellular networks, the conditions which ensure convergence in distribution can be easily verified.

Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

A reinforcement learning approach to automatic generation control

This paper formulates the automatic generation control (AGC) problem as a stochastic multistage decision problem. A strategy for solving this new AGC problem formulation is presented by using a reinforcement learning (RL) approach This method of obtaining an AGC controller does not depend on any knowledge of the system model and more importantly it admits considerable flexibility in defining the control objective. Two specific RL based AGC algorithms are presented. The first algorithm uses the traditional control objective of limiting area control error (ACE) excursions, where as, in the second algorithm, the controller can restore the load-generation balance by only monitoring deviation in tie line flows and system frequency and it does not need to know or estimate the composite ACE signal as is done by all current approaches. The effectiveness and versatility of the approaches has been demonstrated using a two area AGC model. (C) 2002 Elsevier Science B.V. All rights reserved.