Optimality conditions for infinite horizon control problems with state constraints

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Necessary conditions of optimality for state constrained infinite horizon differential inclusions

This article presents and discusses necessary conditions of optimality for infinite horizon dynamic optimization problems with inequality state constraints and set inclusion constraints at both endpoints of the trajectory. The cost functional depends on the state variable at the final time, and the dynamics are given by a differential inclusion. Moreover, the optimization is carried out over asymptotically convergent state trajectories. The novelty of the proposed optimality conditions for this class of problems is that the boundary condition of the adjoint variable is given as a weak directional inclusion at infinity. This improves on the currently available necessary conditions of optimality for infinite horizon problems. © 2011 IEEE.

On the value function for nonautonomous optimal control problems with infinite horizon

In this paper we consider nonautonomous optimal control problems of infinite horizon type, whose control actions are given by L-1-functions. We verify that the value function is locally Lipschitz. The equivalence between dynamic programming inequalities and Hamilton-Jacobi-Bellman (HJB) inequalities for proximal sub (super) gradients is proven. Using this result we show that the value function is a Dini solution of the HJB equation. We obtain a verification result for the class of Dini sub-solutions of the HJB equation and also prove a minimax property of the value function with respect to the sets of Dini semi-solutions of the HJB equation. We introduce the concept of viscosity solutions of the HJB equation in infinite horizon and prove the equivalence between this and the concept of Dini solutions. In the Appendix we provide an existence theorem. (c) 2006 Elsevier B.V. All rights reserved.

A maximum principle for constrained infinite horizon dynamic control systems

This article presents and discusses a maximum principle for infinite horizon constrained optimal control problems with a cost functional depending on the state at the final time. The main feature of these optimality conditions is that, under reasonably weak assumptions, the multiplier is shown to satisfy a novel transversality condition at infinite time. It is also shown that these conditions can also be obtained for impulsive control problems whose dynamics are given by measure driven differential equations. © 2011 IFAC.

Infinite horizon MPC with non-minimal state space feedback

In the MPC literature, stability is usually assured under the assumption that the state is measured. Since the closed-loop system may be nonlinear because of the constraints, it is not possible to apply the separation principle to prove global stability for the Output feedback case. It is well known that, a nonlinear closed-loop system with the state estimated via an exponentially converging observer combined with a state feedback controller can be unstable even when the controller is stable. One alternative to overcome the state estimation problem is to adopt a non-minimal state space model, in which the states are represented by measured past inputs and outputs [P.C. Young, M.A. Behzadi, C.L. Wang, A. Chotai, Direct digital and adaptative control by input-output, state variable feedback pole assignment, International journal of Control 46 (1987) 1867-1881; C. Wang, P.C. Young, Direct digital control by input-output, state variable feedback: theoretical background, International journal of Control 47 (1988) 97-109]. In this case, no observer is needed since the state variables can be directly measured. However, an important disadvantage of this approach is that the realigned model is not of minimal order, which makes the infinite horizon approach to obtain nominal stability difficult to apply. Here, we propose a method to properly formulate an infinite horizon MPC based on the output-realigned model, which avoids the use of an observer and guarantees the closed loop stability. The simulation results show that, besides providing closed-loop stability for systems with integrating and stable modes, the proposed controller may have a better performance than those MPC controllers that make use of an observer to estimate the current states. (C) 2008 Elsevier Ltd. All rights reserved.

Collateral, default penalties and infinite horizon equilibrium

Pascoa and Seghir (2009) noticed that when collateralized promises become subject to utility penalties on default, Ponzi schemes may occur. However, equilibrium exists in some interesting cases. Under low penalties, equilibrium exists if the collateral does not yield utility (for example, when it is a productive asset or a security). Equilibrium exists also under more severe penalties and collateral utility gains, when the promise or the collateral are nominal assets and the margin requirements are endogenous: relative inflation rates and margin coefficients can make the income effects dominate the penalty effects. An equilibrium refinement avoids no-trade equilibria with unduly repayment beliefs. Our refinement differs from the one used by Dubey, Geanakoplos and Shubik (2005) as it does not eliminate no trade equilibria whose low delivery rates are consistent with the propensity to default of agents that are on the verge of selling.

Infinite-Horizon Choice Functions

We analyze infinite-horizon choice functions within the setting of a simple linear technology. Time consistency and efficiency are characterized by stationary consumption and inheritance functions, as well as a transversality condition. In addition, we consider the equity axioms Suppes-Sen, Pigou-Dalton, and resource monotonicity. We show that Suppes-Sen and Pigou-Dalton imply that the consumption and inheritance functions are monotone with respect to time—thus justifying sustainability—while resource monotonicity implies that the consumption and inheritance functions are monotone with respect to the resource. Examples illustrate the characterization results.

Coordinating Inventory Control and Pricing Strategies with Random Demand and Fixed Ordering Cost: the Infinite Horizon Case

We analyze an infinite horizon, single product, periodic review model in which pricing and production/inventory decisions are made simultaneously. Demands in different periods are identically distributed random variables that are independent of each other and their distributions depend on the product price. Pricing and ordering decisions are made at the beginning of each period and all shortages are backlogged. Ordering cost includes both a fixed cost and a variable cost proportional to the amount ordered. The objective is to maximize expected discounted, or expected average profit over the infinite planning horizon. We show that a stationary (s,S,p) policy is optimal for both the discounted and average profit models with general demand functions. In such a policy, the period inventory is managed based on the classical (s,S) policy and price is determined based on the inventory position at the beginning of each period.

Reservoir Management For Flood And Drought Protection Using Infinite Horizon Model Predictive Control

Model Predictive Control (MPC) is a control method that solves in real time an optimal control problem over a finite horizon. The finiteness of the horizon is both the reason of MPC's success and its main limitation. In operational water resources management, MPC has been in fact successfully employed for controlling systems with a relatively short memory, such as canals, where the horizon length is not an issue. For reservoirs, which have generally a longer memory, MPC applications are presently limited to short term management only. Short term reservoir management can be effectively used to deal with fast process, such as floods, but it is not capable of looking sufficiently ahead to handle long term issues, such as drought. To overcome this limitation, we propose an Infinite Horizon MPC (IH-MPC) solution that is particularly suitable for reservoir management. We propose to structure the input signal by use of orthogonal basis functions, therefore reducing the optimization argument to a finite number of variables, and making the control problem solvable in a reasonable time. We applied this solution for the management of the Manantali Reservoir. Manantali is a yearly reservoir located in Mali, on the Senegal river, affecting water systems of Mali, Senegal, and Mauritania. The long term horizon offered by IH-MPC is necessary to deal with the strongly seasonal climate of the region.

Competitive equilibria in infinite-horizon collateralized economies with default penalties

Araújo, Páscoa and Torres-Martinez (2002) have shown that, without imposing either debt constraints or transversality conditions, Ponzi schemes are ruled out in infinite horizon economies with default when collateral is the only mechanism that partially secures loans. Páscoa and Seghir (2008) subsequently show that Ponzi schemes may reappear if, additionally to the seizure of the collateral, there are sufficiently harsh default penalties assessed (directly in terms of utility) against the defaulters. They also claim that if default penalties are moderate then Ponzi schemes are ruled out and existence of a competitive equilibrium is ensured. The objective of this paper is two fold. First, contrary to what is claimed by Páscoa and Seghir (2008), we show that moderate default penalties do not always prevent agents to run a Ponzi scheme. Second, we provide an alternative condition on default penalties that is sufficient to rule out Ponzi schemes and ensure the existence of a competitive equilibrium.

On Ponzi schemes in infinite horizon collateralized economies with default penalties

Araujo, Páscoa and Torres-Martínez (2002) showed that, without imposing any debt constraint, Ponzi schemes are ruled out in infinite horizon economies with limited commitment when collateral is the only mechanism that partially secures loans. Páscoa and Seghir (2009) presented two examples in which they argued that Ponzi schemes may reappear if, additionally to the seizure of the collateral, there are sufficiently harsh default penalties assessed (directly in terms of utility) against the defaulters. Moreover, they claimed that if default penalties are moderate then Ponzi schemes are ruled out and existence of a competitive equilibrium is restored. This paper questions the validity of the claims made in Páscoa and Seghir (2009). First, we show that it is not true that harsh default penalties lead to Ponzi schemes in the examples they have proposed. A competitive equilibrium with no trade can be supported due to unduly pessimistic expectations on asset deliveries. We subsequently refine the equilibrium concept in the spirit of Dubey, Geanakoplos and Shubik (2005) in order to rule out spurious inactivity on asset markets due to irrational expectations. Our second contribution is to provide a specific example of an economy with moderate default penalties in which Ponzi schemes reappear when overpessimistic beliefs on asset deliveries are ruled out. Our finding shows that, contrary to what is claimed by Páscoa and Seghir (2009), moderate default penalties do not always prevent agents to run a Ponzi scheme.

Existence and Uniqueness of Equilibrium in Nonoptimal Unbounded Infinite Horizon Economies

In applied work in macroeconomics and finance, nonoptimal infinite horizon economies are often studied in the the state space is unbounded. Important examples of such economies are single vector growth models with production externalities, valued fiat money, monopolistic competition, and/or distortionary government taxation. Although sufficient conditions for existence and uniqueness of Markovian equilibrium are well known for the compact state space case, no similar sufficient conditions exist for unbounded growth. This paper provides such a set of sufficient conditions, and also present a computational algorithm that will prove asymptotically consistent when computing Markovian equilibrium.

Algebraic duality theorems for infinite LP problems

In this paper we consider a primal-dual infinite linear programming problem-pair, i.e. LPs on infinite dimensional spaces with infinitely many constraints. We present two duality theorems for the problem-pair: a weak and a strong duality theorem. We do not assume any topology on the vector spaces, therefore our results are algebraic duality theorems. As an application, we consider transferable utility cooperative games with arbitrarily many players.

Exact Solution of Bayes and Minimax Change-Detection Problems

The challenge of detecting a change in the distribution of data is a sequential decision problem that is relevant to many engineering solutions, including quality control and machine and process monitoring. This dissertation develops techniques for exact solution of change-detection problems with discrete time and discrete observations. Change-detection problems are classified as Bayes or minimax based on the availability of information on the change-time distribution. A Bayes optimal solution uses prior information about the distribution of the change time to minimize the expected cost, whereas a minimax optimal solution minimizes the cost under the worst-case change-time distribution. Both types of problems are addressed. The most important result of the dissertation is the development of a polynomial-time algorithm for the solution of important classes of Markov Bayes change-detection problems. Existing techniques for epsilon-exact solution of partially observable Markov decision processes have complexity exponential in the number of observation symbols. A new algorithm, called constellation induction, exploits the concavity and Lipschitz continuity of the value function, and has complexity polynomial in the number of observation symbols. It is shown that change-detection problems with a geometric change-time distribution and identically- and independently-distributed observations before and after the change are solvable in polynomial time. Also, change-detection problems on hidden Markov models with a fixed number of recurrent states are solvable in polynomial time. A detailed implementation and analysis of the constellation-induction algorithm are provided. Exact solution methods are also established for several types of minimax change-detection problems. Finite-horizon problems with arbitrary observation distributions are modeled as extensive-form games and solved using linear programs. Infinite-horizon problems with linear penalty for detection delay and identically- and independently-distributed observations can be solved in polynomial time via epsilon-optimal parameterization of a cumulative-sum procedure. Finally, the properties of policies for change-detection problems are described and analyzed. Simple classes of formal languages are shown to be sufficient for epsilon-exact solution of change-detection problems, and methods for finding minimally sized policy representations are described.