Optimal infinite-horizon feedback laws for a general class of constrained discrete-time systems: stability and moving-horizon approximations

Stability results are given for a class of feedback systems arising from the regulation of time-varying discrete-time systems using optimal infinite-horizon and moving-horizon feedback laws. The class is characterized by joint constraints on the state and the control, a general nonlinear cost function and nonlinear equations of motion possessing two special properties. It is shown that weak conditions on the cost function and the constraints are sufficient to guarantee uniform asymptotic stability of both the optimal infinite-horizon and movinghorizon feedback systems. The infinite-horizon cost associated with the moving-horizon feedback law approaches the optimal infinite-horizon cost as the moving horizon is extended.

Coupled electrostatic-elastic analysis for topology optimization using material interpolation

In this paper, we present a novel analytical formulation for the coupled partial differential equations governing electrostatically actuated constrained elastic structures of inhomogeneous material composition. We also present a computationally efficient numerical framework for solving the coupled equations over a reference domain with a fixed finite-element mesh. This serves two purposes: (i) a series of problems with varying geometries and piece-wise homogeneous and/or inhomogeneous material distribution can be solved with a single pre-processing step, (ii) topology optimization methods can be easily implemented by interpolating the material at each point in the reference domain from a void to a dielectric or a conductor. This is attained by considering the steady-state electrical current conduction equation with a `leaky capacitor' model instead of the usual electrostatic equation. This formulation is amenable for both static and transient problems in the elastic domain coupled with the quasi-electrostatic electric field. The procedure is numerically implemented on the COMSOL Multiphysics (R) platform using the weak variational form of the governing equations. Examples have been presented to show the accuracy and versatility of the scheme. The accuracy of the scheme is validated for the special case of piece-wise homogeneous material in the limit of the leaky-capacitor model approaching the ideal case.

Shape optimization of RC flexural members

A natural velocity field method for shape optimization of reinforced concrete (RC) flexural members has been demonstrated. The possibility of shape optimization by modifying the shape of an initially rectangular section, in addition to variation of breadth and depth along the length, has been explored. Necessary shape changes have been computed using the sequential quadratic programming (SQP) technique. Genetic algorithm (Goldberg and Samtani 1986) has been used to optimize the diameter and number of main reinforcement bars. A limit-state design approach has been adopted for the nonprismatic RC sections. Such relevant issues as formulation of optimization problem, finite-element modeling, and solution procedure have been described. Three design examples-a simply supported beam, a cantilever beam, and a two-span continuous beam, all under uniformly distributed loads-have been optimized. The results show a significant savings (40-56%) in material and cost and also result in aesthetically pleasing structures. This procedure will lead to considerable cost saving, particularly in cases of mass-produced precast members and a heavy cast-in-place member such as a bridge girder.

Topology Optimization of Electrostatically Actuated Micromechanical Structures with Accurate Electrostatic Modeling of the Interpolated Material Model

In this paper, we present a novel formulation for performing topology optimization of electrostatically actuated constrained elastic structures. We propose a new electrostatic-elastic formulation that uses the leaky capacitor model and material interpolation to define the material state at every point of a given design domain continuously between conductor and void states. The new formulation accurately captures the physical behavior when the material in between a conductor and a void is present during the iterative process of topology optimization. The method then uses the optimality criteria method to solve the optimization problem by iteratively pushing the state of the domain towards that of a conductor or a void in the appropriate regions. We present examples to illustrate the ability of the method in creating the stiffest structure under electrostatic force for different boundary conditions.

Swarm intelligence algorithms for integrated optimization of piezoelectric actuator and sensor placement and feedback gains

Swarm intelligence algorithms are applied for optimal control of flexible smart structures bonded with piezoelectric actuators and sensors. The optimal locations of actuators/sensors and feedback gain are obtained by maximizing the energy dissipated by the feedback control system. We provide a mathematical proof that this system is uncontrollable if the actuators and sensors are placed at the nodal points of the mode shapes. The optimal locations of actuators/sensors and feedback gain represent a constrained non-linear optimization problem. This problem is converted to an unconstrained optimization problem by using penalty functions. Two swarm intelligence algorithms, namely, Artificial bee colony (ABC) and glowworm swarm optimization (GSO) algorithms, are considered to obtain the optimal solution. In earlier published research, a cantilever beam with one and two collocated actuator(s)/sensor(s) was considered and the numerical results were obtained by using genetic algorithm and gradient based optimization methods. We consider the same problem and present the results obtained by using the swarm intelligence algorithms ABC and GSO. An extension of this cantilever beam problem with five collocated actuators/sensors is considered and the numerical results obtained by using the ABC and GSO algorithms are presented. The effect of increasing the number of design variables (locations of actuators and sensors and gain) on the optimization process is investigated. It is shown that the ABC and GSO algorithms are robust and are good choices for the optimization of smart structures.

An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

On maximum entropy and minimum KL-divergence optimization by Grobner basis methods

In this paper we study constrained maximum entropy and minimum divergence optimization problems, in the cases where integer valued sufficient statistics exists, using tools from computational commutative algebra. We show that the estimation of parametric statistical models in this case can be transformed to solving a system of polynomial equations. We give an implicit description of maximum entropy models by embedding them in algebraic varieties for which we give a Grobner basis method to compute it. In the cases of minimum KL-divergence models we show that implicitization preserves specialization of prior distribution. This result leads us to a Grobner basis method to embed minimum KL-divergence models in algebraic varieties. (C) 2012 Elsevier Inc. All rights reserved.

MPI-based parallel synchronous vector evaluated particle swarm optimization for multi-objective design optimization of composite structures

This paper presents a decentralized/peer-to-peer architecture-based parallel version of the vector evaluated particle swarm optimization (VEPSO) algorithm for multi-objective design optimization of laminated composite plates using message passing interface (MPI). The design optimization of laminated composite plates being a combinatorially explosive constrained non-linear optimization problem (CNOP), with many design variables and a vast solution space, warrants the use of non-parametric and heuristic optimization algorithms like PSO. Optimization requires minimizing both the weight and cost of these composite plates, simultaneously, which renders the problem multi-objective. Hence VEPSO, a multi-objective variant of the PSO algorithm, is used. Despite the use of such a heuristic, the application problem, being computationally intensive, suffers from long execution times due to sequential computation. Hence, a parallel version of the PSO algorithm for the problem has been developed to run on several nodes of an IBM P720 cluster. The proposed parallel algorithm, using MPI's collective communication directives, establishes a peer-to-peer relationship between the constituent parallel processes, deviating from the more common master-slave approach, in achieving reduction of computation time by factor of up to 10. Finally we show the effectiveness of the proposed parallel algorithm by comparing it with a serial implementation of VEPSO and a parallel implementation of the vector evaluated genetic algorithm (VEGA) for the same design problem. (c) 2012 Elsevier Ltd. All rights reserved.

Stochastic optimization for adaptive labor staffing in service systems

Service systems are labor intensive. Further, the workload tends to vary greatly with time. Adapting the staffing levels to the workloads in such systems is nontrivial due to a large number of parameters and operational variations, but crucial for business objectives such as minimal labor inventory. One of the central challenges is to optimize the staffing while maintaining system steady-state and compliance to aggregate SLA constraints. We formulate this problem as a parametrized constrained Markov process and propose a novel stochastic optimization algorithm for solving it. Our algorithm is a multi-timescale stochastic approximation scheme that incorporates a SPSA based algorithm for ‘primal descent' and couples it with a ‘dual ascent' scheme for the Lagrange multipliers. We validate this optimization scheme on five real-life service systems and compare it with a state-of-the-art optimization tool-kit OptQuest. Being two orders of magnitude faster than OptQuest, our scheme is particularly suitable for adaptive labor staffing. Also, we observe that it guarantees convergence and finds better solutions than OptQuest in many cases.

Generalized model predictive static programming and its application to 3D impact angle constrained guidance of air-to-surface missiles

A new `generalized model predictive static programming (G-MPSP)' technique is presented in this paper in the continuous time framework for rapidly solving a class of finite-horizon nonlinear optimal control problems with hard terminal constraints. A key feature of the technique is backward propagation of a small-dimensional weight matrix dynamics, using which the control history gets updated. This feature, as well as the fact that it leads to a static optimization problem, are the reasons for its high computational efficiency. It has been shown that under Euler integration, it is equivalent to the existing model predictive static programming technique, which operates on a discrete-time approximation of the problem. Performance of the proposed technique is demonstrated by solving a challenging three-dimensional impact angle constrained missile guidance problem. The problem demands that the missile must meet constraints on both azimuth and elevation angles in addition to achieving near zero miss distance, while minimizing the lateral acceleration demand throughout its flight path. Both stationary and maneuvering ground targets are considered in the simulation studies. Effectiveness of the proposed guidance has been verified by considering first order autopilot lag as well as various target maneuvers.

Multimodal therapy for complete regression of malignant melanoma using constrained nonlinear optimal dynamic inversion

Using a realistic nonlinear mathematical model for melanoma dynamics and the technique of optimal dynamic inversion (exact feedback linearization with static optimization), a multimodal automatic drug dosage strategy is proposed in this paper for complete regression of melanoma cancer in humans. The proposed strategy computes different drug dosages and gives a nonlinear state feedback solution for driving the number of cancer cells to zero. However, it is observed that when tumor is regressed to certain value, then there is no need of external drug dosages as immune system and other therapeutic states are able to regress tumor at a sufficiently fast rate which is more than exponential rate. As model has three different drug dosages, after applying dynamic inversion philosophy, drug dosages can be selected in optimized manner without crossing their toxicity limits. The combination of drug dosages is decided by appropriately selecting the control design parameter values based on physical constraints. The process is automated for all possible combinations of the chemotherapy and immunotherapy drug dosages with preferential emphasis of having maximum possible variety of drug inputs at any given point of time. Simulation study with a standard patient model shows that tumor cells are regressed from 2 x 107 to order of 105 cells because of external drug dosages in 36.93 days. After this no external drug dosages are required as immune system and other therapeutic states are able to regress tumor at greater than exponential rate and hence, tumor goes to zero (less than 0.01) in 48.77 days and healthy immune system of the patient is restored. Study with different chemotherapy drug resistance value is also carried out. (C) 2014 Elsevier Ltd. All rights reserved.

Smoothed Functional Algorithms for Stochastic Optimization Using q-Gaussian Distributions

Smoothed functional (SF) schemes for gradient estimation are known to be efficient in stochastic optimization algorithms, especially when the objective is to improve the performance of a stochastic system However, the performance of these methods depends on several parameters, such as the choice of a suitable smoothing kernel. Different kernels have been studied in the literature, which include Gaussian, Cauchy, and uniform distributions, among others. This article studies a new class of kernels based on the q-Gaussian distribution, which has gained popularity in statistical physics over the last decade. Though the importance of this family of distributions is attributed to its ability to generalize the Gaussian distribution, we observe that this class encompasses almost all existing smoothing kernels. This motivates us to study SF schemes for gradient estimation using the q-Gaussian distribution. Using the derived gradient estimates, we propose two-timescale algorithms for optimization of a stochastic objective function in a constrained setting with a projected gradient search approach. We prove the convergence of our algorithms to the set of stationary points of an associated ODE. We also demonstrate their performance numerically through simulations on a queuing model.

Generalized Model Predictive Static Programming and Angle-Constrained Guidance of Air-to-Ground Missiles

A new generalized model predictive static programming technique is presented for rapidly solving a class of finite-horizon nonlinear optimal control problems with hard terminal constraints. Two key features for its high computational efficiency include one-time backward integration of a small-dimensional weighting matrix dynamics, followed bya static optimization formulation that requires only a static Lagrange multiplier to update the control history. It turns out that under Euler integration and rectangular approximation of finite integrals it is equivalent to the existing model predictive static programming technique. In addition to the benchmark double integrator problem, usefulness of the proposed technique is demonstrated by solving a three-dimensional angle-constrained guidance problem for an air-to-ground missile, which demands that the missile must meet constraints on both azimuth and elevation angles at the impact point in addition to achieving near-zero miss distance, while minimizing the lateral acceleration demand throughout its flight path. Simulation studies include maneuvering ground targets along with a first-order autopilot lag. Comparison studies with classical augmented proportional navigation guidance and modern general explicit guidance lead to the conclusion that the proposed guidance is superior to both and has a larger capture region as well.

Necessary and sufficient conditions for optimality in constrained general sum stochastic games

In this paper we first derive a necessary and sufficient condition for a stationary strategy to be the Nash equilibrium of discounted constrained stochastic game under certain assumptions. In this process we also develop a nonlinear (non-convex) optimization problem for a discounted constrained stochastic game. We use the linear best response functions of every player and complementary slackness theorem for linear programs to derive both the optimization problem and the equivalent condition. We then extend this result to average reward constrained stochastic games. Finally, we present a heuristic algorithm motivated by our necessary and sufficient conditions for a discounted cost constrained stochastic game. We numerically observe the convergence of this algorithm to Nash equilibrium. (C) 2015 Elsevier B.V. All rights reserved.