Continuous-time Single Network Adaptive Critic for Regulator Design of Nonlinear Control Affine Systems

An optimal control law for a general nonlinear system can be obtained by solving Hamilton-Jacobi-Bellman equation. However, it is difficult to obtain an analytical solution of this equation even for a moderately complex system. In this paper, we propose a continuoustime single network adaptive critic scheme for nonlinear control affine systems where the optimal cost-to-go function is approximated using a parametric positive semi-definite function. Unlike earlier approaches, a continuous-time weight update law is derived from the HJB equation. The stability of the system is analysed during the evolution of weights using Lyapunov theory. The effectiveness of the scheme is demonstrated through simulation examples.

Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterize the value function via Hamilton Jacobi Bellman equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.

A mean-reverting stochastic model for the political business cycle

In this article, we look at the political business cycle problem through the lens of uncertainty. The feedback control used by us is the famous NKPC with stochasticity and wage rigidities. We extend the New Keynesian Phillips Curve model to the continuous time stochastic set up with an Ornstein-Uhlenbeck process. We minimize relevant expected quadratic cost by solving the corresponding Hamilton-Jacobi-Bellman equation. The basic intuition of the classical model is qualitatively carried forward in our set up but uncertainty also plays an important role in determining the optimal trajectory of the voter support function. The internal variability of the system acts as a base shifter for the support function in the risk neutral case. The role of uncertainty is even more prominent in the risk averse case where all the shape parameters are directly dependent on variability. Thus, in this case variability controls both the rates of change as well as the base shift parameters. To gain more insight we have also studied the model when the coefficients are time invariant and studied numerical solutions. The close relationship between the unemployment rate and the support function for the incumbent party is highlighted. The role of uncertainty in creating sampling fluctuation in this set up, possibly towards apparently anomalous results, is also explored.

Teoria de controle ótimo com aplicações a sistemas biológicos

Neste trabalho apresentamos as etapas para a utilização do método da Programação Dinâmica, ou Princípio de Otimização de Bellman, para aplicações de controle ótimo. Investigamos a noção de funções de controle de Lyapunov (FCL) e sua relação com a estabilidade de sistemas autônomos com controle. Uma função de controle de Lyapunov deverá satisfazer a equação de Hamilton-Jacobi-Bellman (H-J-B). Usando esse fato, se uma função de controle de Lyapunov é conhecida, será então possível determinar a lei de realimentação ótima; isto é, a lei de controle que torna o sistema globalmente assintóticamente controlável a um estado de equilíbrio. Como aplicação, apresentamos uma modelagem matemática adequada a um problema de controle ótimo de certos sistemas biológicos. Este trabalho conta também com um breve histórico sobre o desenvolvimento da Teoria de Controle de forma a ilustrar a importância, o progresso e a aplicação das técnicas de controle em diferentes áreas ao longo do tempo.

Dynamic programming solutions for decentralized state-feedback LQG problems with communication delays

This paper presents explicit solutions for a class of decentralized LQG problems in which players communicate their states with delays. A method for decomposing the Bellman equation into a hierarchy of independent subproblems is introduced. Using this decomposition, all of the gains for the optimal controller are computed from the solution of a single algebraic Riccati equation. © 2012 AACC American Automatic Control Council).

Dynamic programming for stochastic target problems, Viscosity solutions and hedging in markets with Portfolio constraints and large investors

The purpose of this expository arti le is to present a self- ontained overview of some results on the hara terization of the optimal value fun tion of a sto hasti target problem as (dis ontinuous) vis osity solution of a ertain dynami programming PDE and its appli ation to the problem of hedging ontingent laims in the presen e of portfolio onstraints and large investors

Dynamic hedging with stocastic differential utility

In this paper we study the dynamic hedging problem using three different utility specifications: stochastic differential utility, terminal wealth utility, and we propose a particular utility transformation connecting both previous approaches. In all cases, we assume Markovian prices. Stochastic differential utility, SDU, impacts the pure hedging demand ambiguously, but decreases the pure speculative demand, because risk aversion increases. We also show that consumption decision is, in some sense, independent of hedging decision. With terminal wealth utility, we derive a general and compact hedging formula, which nests as special all cases studied in Duffie and Jackson (1990). We then show how to obtain their formulas. With the third approach we find a compact formula for hedging, which makes the second-type utility framework a particular case, and show that the pure hedging demand is not impacted by this specification. In addition, with CRRA- and CARA-type utilities, the risk aversion increases and, consequently the pure speculative demand decreases. If futures price are martingales, then the transformation plays no role in determining the hedging allocation. We also derive the relevant Bellman equation for each case, using semigroup techniques.

Chaos control of a nonlinear oscillator with shape memory alloy using an optimal linear control: Part I: Ideal energy source

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Optimal pest control problem in population dynamics

One of the main goals of the pest control is to maintain the density of the pest population in the equilibrium level below economic damages. For reaching this goal, the optimal pest control problem was divided in two parts. In the first part, the two optimal control functions were considered. These functions move the ecosystem pest-natural enemy at an equilibrium state below the economic injury level. In the second part, the one optimal control function stabilizes the ecosystem in this level, minimizing the functional that characterizes quadratic deviations of this level. The first problem was resolved through the application of the Maximum Principle of Pontryagin. The Dynamic Programming was used for the resolution of the second optimal pest control problem.

Mathematical modeling and control of population systems: Applications in biological pest control

The aim of this paper is to apply methods from optimal control theory, and from the theory of dynamic systems to the mathematical modeling of biological pest control. The linear feedback control problem for nonlinear systems has been formulated in order to obtain the optimal pest control strategy only through the introduction of natural enemies. Asymptotic stability of the closed-loop nonlinear Kolmogorov system is guaranteed by means of a Lyapunov function which can clearly be seen to be the solution of the Hamilton-Jacobi-Bellman equation, thus guaranteeing both stability and optimality. Numerical simulations for three possible scenarios of biological pest control based on the Lotka-Volterra models are provided to show the effectiveness of this method. (c) 2007 Elsevier B.V. All rights reserved.

On control and synchronization in chaotic and hyperchaotic systems via linear feedback control

This paper presents the control and synchronization of chaos by designing linear feedback controllers. The linear feedback control problem for nonlinear systems has been formulated under optimal control theory viewpoint. Asymptotic stability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function which can clearly be seen to be the solution of the Hamilton-Jacobi-Bellman equation thus guaranteeing both stability and optimality. The formulated theorem expresses explicitly the form of minimized functional and gives the sufficient conditions that allow using the linear feedback control for nonlinear system. The numerical simulations were provided in order to show the effectiveness of this method for the control of the chaotic Rossler system and synchronization of the hyperchaotic Rossler system. (C) 2007 Elsevier B.V. All rights reserved.

An Optimal Linear Control Design for Nonlinear Systems

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

On an optimal control design for Rossler system

In this Letter, an optimal control strategy that directs the chaotic motion of the Rossler system to any desired fixed point is proposed. The chaos control problem is then formulated as being an infinite horizon optimal control nonlinear problem that was reduced to a solution of the associated Hamilton-Jacobi-Bellman equation. We obtained its solution among the correspondent Lyapunov functions of the considered dynamical system. (C) 2004 Elsevier B.V All rights reserved.

Optimal linear and nonlinear control design for chaotic systems

In this work, the linear and nonlinear feedback control techniques for chaotic systems were been considered. The optimal nonlinear control design problem has been resolved by using Dynamic Programming that reduced this problem to a solution of the Hamilton-Jacobi-Bellman equation. In present work the linear feedback control problem has been reformulated under optimal control theory viewpoint. The formulated Theorem expresses explicitly the form of minimized functional and gives the sufficient conditions that allow using the linear feedback control for nonlinear system. The numerical simulations for the Rössler system and the Duffing oscillator are provided to show the effectiveness of this method. Copyright © 2005 by ASME.

Sistemas de controle em domínios estratificados

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)