Existence and Uniqueness of Equilibrium in Nonoptimal Unbounded Infinite Horizon Economies

In applied work in macroeconomics and finance, nonoptimal infinite horizon economies are often studied in the the state space is unbounded. Important examples of such economies are single vector growth models with production externalities, valued fiat money, monopolistic competition, and/or distortionary government taxation. Although sufficient conditions for existence and uniqueness of Markovian equilibrium are well known for the compact state space case, no similar sufficient conditions exist for unbounded growth. This paper provides such a set of sufficient conditions, and also present a computational algorithm that will prove asymptotically consistent when computing Markovian equilibrium.

Competitive equilibria in infinite-horizon collateralized economies with default penalties

Araújo, Páscoa and Torres-Martinez (2002) have shown that, without imposing either debt constraints or transversality conditions, Ponzi schemes are ruled out in infinite horizon economies with default when collateral is the only mechanism that partially secures loans. Páscoa and Seghir (2008) subsequently show that Ponzi schemes may reappear if, additionally to the seizure of the collateral, there are sufficiently harsh default penalties assessed (directly in terms of utility) against the defaulters. They also claim that if default penalties are moderate then Ponzi schemes are ruled out and existence of a competitive equilibrium is ensured. The objective of this paper is two fold. First, contrary to what is claimed by Páscoa and Seghir (2008), we show that moderate default penalties do not always prevent agents to run a Ponzi scheme. Second, we provide an alternative condition on default penalties that is sufficient to rule out Ponzi schemes and ensure the existence of a competitive equilibrium.

On Ponzi schemes in infinite horizon collateralized economies with default penalties

Araujo, Páscoa and Torres-Martínez (2002) showed that, without imposing any debt constraint, Ponzi schemes are ruled out in infinite horizon economies with limited commitment when collateral is the only mechanism that partially secures loans. Páscoa and Seghir (2009) presented two examples in which they argued that Ponzi schemes may reappear if, additionally to the seizure of the collateral, there are sufficiently harsh default penalties assessed (directly in terms of utility) against the defaulters. Moreover, they claimed that if default penalties are moderate then Ponzi schemes are ruled out and existence of a competitive equilibrium is restored. This paper questions the validity of the claims made in Páscoa and Seghir (2009). First, we show that it is not true that harsh default penalties lead to Ponzi schemes in the examples they have proposed. A competitive equilibrium with no trade can be supported due to unduly pessimistic expectations on asset deliveries. We subsequently refine the equilibrium concept in the spirit of Dubey, Geanakoplos and Shubik (2005) in order to rule out spurious inactivity on asset markets due to irrational expectations. Our second contribution is to provide a specific example of an economy with moderate default penalties in which Ponzi schemes reappear when overpessimistic beliefs on asset deliveries are ruled out. Our finding shows that, contrary to what is claimed by Páscoa and Seghir (2009), moderate default penalties do not always prevent agents to run a Ponzi scheme.

Infinite-horizon policy-gradient estimation

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter β ∈ [0,1) (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter β is related to the mixing time of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward. ©2001 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.

An infinite-horizon robust filter for uncertain hidden Markov Models with conditional relative entropy constraints

We consider a robust filtering problem for uncertain discrete-time, homogeneous, first-order, finite-state hidden Markov models (HMMs). The class of uncertain HMMs considered is described by a conditional relative entropy constraint on measures perturbed from a nominal regular conditional probability distribution given the previous posterior state distribution and the latest measurement. Under this class of perturbations, a robust infinite horizon filtering problem is first formulated as a constrained optimization problem before being transformed via variational results into an unconstrained optimization problem; the latter can be elegantly solved using a risk-sensitive information-state based filtering.

Optimal infinite-horizon feedback laws for a general class of constrained discrete-time systems: stability and moving-horizon approximations

Stability results are given for a class of feedback systems arising from the regulation of time-varying discrete-time systems using optimal infinite-horizon and moving-horizon feedback laws. The class is characterized by joint constraints on the state and the control, a general nonlinear cost function and nonlinear equations of motion possessing two special properties. It is shown that weak conditions on the cost function and the constraints are sufficient to guarantee uniform asymptotic stability of both the optimal infinite-horizon and movinghorizon feedback systems. The infinite-horizon cost associated with the moving-horizon feedback law approaches the optimal infinite-horizon cost as the moving horizon is extended.

Robust Dynamic Pricing Over Infinite Horizon in The Presence of Model Uncertainty( 2* Journal)

This paper studies a problem of dynamic pricing faced by a retailer with limited inventory, uncertain about the demand rate model, aiming to maximize expected discounted revenue over an infinite time horizon. The retailer doubts his demand model which is generated by historical data and views it as an approximation. Uncertainty in the demand rate model is represented by a notion of generalized relative entropy process, and the robust pricing problem is formulated as a two-player zero-sum stochastic differential game. The pricing policy is obtained through the Hamilton-Jacobi-Isaacs (HJI) equation. The existence and uniqueness of the solution of the HJI equation is shown and a verification theorem is proved to show that the solution of the HJI equation is indeed the value function of the pricing problem. The results are illustrated by an example with exponential nominal demand rate.

Infinite-Horizon Choice Functions

We analyze infinite-horizon choice functions within the setting of a simple linear technology. Time consistency and efficiency are characterized by stationary consumption and inheritance functions, as well as a transversality condition. In addition, we consider the equity axioms Suppes-Sen, Pigou-Dalton, and resource monotonicity. We show that Suppes-Sen and Pigou-Dalton imply that the consumption and inheritance functions are monotone with respect to time—thus justifying sustainability—while resource monotonicity implies that the consumption and inheritance functions are monotone with respect to the resource. Examples illustrate the characterization results.

Coordinating Inventory Control and Pricing Strategies with Random Demand and Fixed Ordering Cost: the Infinite Horizon Case

We analyze an infinite horizon, single product, periodic review model in which pricing and production/inventory decisions are made simultaneously. Demands in different periods are identically distributed random variables that are independent of each other and their distributions depend on the product price. Pricing and ordering decisions are made at the beginning of each period and all shortages are backlogged. Ordering cost includes both a fixed cost and a variable cost proportional to the amount ordered. The objective is to maximize expected discounted, or expected average profit over the infinite planning horizon. We show that a stationary (s,S,p) policy is optimal for both the discounted and average profit models with general demand functions. In such a policy, the period inventory is managed based on the classical (s,S) policy and price is determined based on the inventory position at the beginning of each period.

Reservoir Management For Flood And Drought Protection Using Infinite Horizon Model Predictive Control

Model Predictive Control (MPC) is a control method that solves in real time an optimal control problem over a finite horizon. The finiteness of the horizon is both the reason of MPC's success and its main limitation. In operational water resources management, MPC has been in fact successfully employed for controlling systems with a relatively short memory, such as canals, where the horizon length is not an issue. For reservoirs, which have generally a longer memory, MPC applications are presently limited to short term management only. Short term reservoir management can be effectively used to deal with fast process, such as floods, but it is not capable of looking sufficiently ahead to handle long term issues, such as drought. To overcome this limitation, we propose an Infinite Horizon MPC (IH-MPC) solution that is particularly suitable for reservoir management. We propose to structure the input signal by use of orthogonal basis functions, therefore reducing the optimization argument to a finite number of variables, and making the control problem solvable in a reasonable time. We applied this solution for the management of the Manantali Reservoir. Manantali is a yearly reservoir located in Mali, on the Senegal river, affecting water systems of Mali, Senegal, and Mauritania. The long term horizon offered by IH-MPC is necessary to deal with the strongly seasonal climate of the region.

Endogenous debt constraints in collateralized economies with default penalties

In infinite horizon financial markets economies, competitive equilibria fail to exist if one does not impose restrictions on agents' trades that rule out Ponzi schemes. When there is limited commitment and collateral repossession is the unique default punishment, Araujo, Páscoa and Torres-Martínez (2002) proved that Ponzi schemes are ruled out without imposing any exogenous/endogenous debt constraints on agents' trades. Recently Páscoa and Seghir (2009) have shown that this positive result is not robust to the presence of additional default punishments. They provide several examples showing that, in the absence of debt constraints, harsh default penalties may induce agents to run Ponzi schemes that jeopardize equilibrium existence. The objective of this paper is to close a theoretical gap in the literature by identifying endogenous borrowing constraints that rule out Ponzi schemes and ensure existence of equilibria in a model with limited commitment and (possible) default. We appropriately modify the definition of finitely effective debt constraints, introduced by Levine and Zame (1996) (see also Levine and Zame (2002)), to encompass models with limited commitment, default penalties and collateral. Along this line, we introduce in the setting of Araujo, Páscoa and Torres-Martínez (2002), Kubler and Schmedders (2003) and Páscoa and Seghir (2009) the concept of actions with finite equivalent payoffs. We show that, independently of the level of default penalties, restricting plans to have finite equivalent payoffs rules out Ponzi schemes and guarantees the existence of an equilibrium that is compatible with the minimal ability to borrow and lend that we expect in our model. An interesting feature of our debt constraints is that they give rise to budget sets that coincide with the standard budget sets of economies having a collateral structure but no penalties (as defined in Araujo, Páscoa and Torres-Martínez (2002)). This illustrates the hidden relation between finitely effective debt constraints and collateral requirements.

Optimality conditions for infinite horizon control problems with state constraints

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

On the value function for nonautonomous optimal control problems with infinite horizon

In this paper we consider nonautonomous optimal control problems of infinite horizon type, whose control actions are given by L-1-functions. We verify that the value function is locally Lipschitz. The equivalence between dynamic programming inequalities and Hamilton-Jacobi-Bellman (HJB) inequalities for proximal sub (super) gradients is proven. Using this result we show that the value function is a Dini solution of the HJB equation. We obtain a verification result for the class of Dini sub-solutions of the HJB equation and also prove a minimax property of the value function with respect to the sets of Dini semi-solutions of the HJB equation. We introduce the concept of viscosity solutions of the HJB equation in infinite horizon and prove the equivalence between this and the concept of Dini solutions. In the Appendix we provide an existence theorem. (c) 2006 Elsevier B.V. All rights reserved.