Existence and Uniqueness of Equilibrium in Nonoptimal Unbounded Infinite Horizon Economies

In applied work in macroeconomics and finance, nonoptimal infinite horizon economies are often studied in the the state space is unbounded. Important examples of such economies are single vector growth models with production externalities, valued fiat money, monopolistic competition, and/or distortionary government taxation. Although sufficient conditions for existence and uniqueness of Markovian equilibrium are well known for the compact state space case, no similar sufficient conditions exist for unbounded growth. This paper provides such a set of sufficient conditions, and also present a computational algorithm that will prove asymptotically consistent when computing Markovian equilibrium.

Algebraic duality theorems for infinite LP problems

In this paper we consider a primal-dual infinite linear programming problem-pair, i.e. LPs on infinite dimensional spaces with infinitely many constraints. We present two duality theorems for the problem-pair: a weak and a strong duality theorem. We do not assume any topology on the vector spaces, therefore our results are algebraic duality theorems. As an application, we consider transferable utility cooperative games with arbitrarily many players.

Exact Solution of Bayes and Minimax Change-Detection Problems

The challenge of detecting a change in the distribution of data is a sequential decision problem that is relevant to many engineering solutions, including quality control and machine and process monitoring. This dissertation develops techniques for exact solution of change-detection problems with discrete time and discrete observations. Change-detection problems are classified as Bayes or minimax based on the availability of information on the change-time distribution. A Bayes optimal solution uses prior information about the distribution of the change time to minimize the expected cost, whereas a minimax optimal solution minimizes the cost under the worst-case change-time distribution. Both types of problems are addressed. The most important result of the dissertation is the development of a polynomial-time algorithm for the solution of important classes of Markov Bayes change-detection problems. Existing techniques for epsilon-exact solution of partially observable Markov decision processes have complexity exponential in the number of observation symbols. A new algorithm, called constellation induction, exploits the concavity and Lipschitz continuity of the value function, and has complexity polynomial in the number of observation symbols. It is shown that change-detection problems with a geometric change-time distribution and identically- and independently-distributed observations before and after the change are solvable in polynomial time. Also, change-detection problems on hidden Markov models with a fixed number of recurrent states are solvable in polynomial time. A detailed implementation and analysis of the constellation-induction algorithm are provided. Exact solution methods are also established for several types of minimax change-detection problems. Finite-horizon problems with arbitrary observation distributions are modeled as extensive-form games and solved using linear programs. Infinite-horizon problems with linear penalty for detection delay and identically- and independently-distributed observations can be solved in polynomial time via epsilon-optimal parameterization of a cumulative-sum procedure. Finally, the properties of policies for change-detection problems are described and analyzed. Simple classes of formal languages are shown to be sufficient for epsilon-exact solution of change-detection problems, and methods for finding minimally sized policy representations are described.

Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterize the value function via Hamilton Jacobi Bellman equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.

Dynamic pricing for used products in remanufacturing with back orders(ABS 3* Journal, Accepted with Revision on 2nd December 2011).

In remanufacturing, the supply of used products and the demand for remanufactured products are usually mismatched because of the great uncertainties on both sides. In this paper, we propose a dynamic pricing policy to balance this uncertain supply and demand. Specifically, we study a remanufacturer’s problem of pricing a single class of cores with random price-dependent returns and random demand for the remanufactured products with backlogs. We model this pricing task as a continuous-time Markov decision process, which addresses both the finite and infinite horizon problems, and provide managerial insights by analyzing the structural properties of the optimal policy. We then use several computational examples to illustrate the impacts of particular system parameters on pricing policy.

Calculus of variations on time scales and applications to economics

We consider some problems of the calculus of variations on time scales. On the beginning our attention is paid on two inverse extremal problems on arbitrary time scales. Firstly, using the Euler-Lagrange equation and the strengthened Legendre condition, we derive a general form for a variation functional that attains a local minimum at a given point of the vector space. Furthermore, we prove a necessary condition for a dynamic integro-differential equation to be an Euler-Lagrange equation. New and interesting results for the discrete and quantum calculus are obtained as particular cases. Afterwards, we prove Euler-Lagrange type equations and transversality conditions for generalized infinite horizon problems. Next we investigate the composition of a certain scalar function with delta and nabla integrals of a vector valued field. Euler-Lagrange equations in integral form, transversality conditions, and necessary optimality conditions for isoperimetric problems, on an arbitrary time scale, are proved. In the end, two main issues of application of time scales in economic, with interesting results, are presented. In the former case we consider a firm that wants to program its production and investment policies to reach a given production rate and to maximize its future market competitiveness. The model which describes firm activities is studied in two different ways: using classical discretizations; and applying discrete versions of our result on time scales. In the end we compare the cost functional values obtained from those two approaches. The latter problem is more complex and relates to rate of inflation, p, and rate of unemployment, u, which inflict a social loss. Using known relations between p, u, and the expected rate of inflation π, we rewrite the social loss function as a function of π. We present this model in the time scale framework and find an optimal path π that minimizes the total social loss over a given time interval.

Stochastic stability for Markovian jump linear systems associated with a finite number of jump times

This paper deals with a stochastic stability concept for discrete-time Markovian jump linear systems. The random jump parameter is associated to changes between the system operation modes due to failures or repairs, which can be well described by an underlying finite-state Markov chain. In the model studied, a fixed number of failures or repairs is allowed, after which, the system is brought to a halt for maintenance or for replacement. The usual concepts of stochastic stability are related to pure infinite horizon problems, and are not appropriate in this scenario. A new stability concept is introduced, named stochastic tau-stability that is tailored to the present setting. Necessary and sufficient conditions to ensure the stochastic tau-stability are provided, and the almost sure stability concept associated with this class of processes is also addressed. The paper also develops equivalences among second order concepts that parallels the results for infinite horizon problems. (C) 2003 Elsevier B.V. All rights reserved.

Stochastic Stability for Markovian Jump Linear Systems Subject to a Crucial Failure Event

This paper is concerned with the stability of discrete-time linear systems subject to random jumps in the parameters, described by an underlying finite-state Markov chain. In the model studied, a stopping time τ Δ is associated with the occurrence of a crucial failure after which the system is brought to a halt for maintenance. The usual stochastic stability concepts and associated results are not indicated, since they are tailored to pure infinite horizon problems. Using the concept named stochastic τ-stability, equivalent conditions to ensure the stochastic stability of the system until the occurrence of τ Δ is obtained. In addition, an intermediary and mixed case for which τ represents the minimum between the occurrence of a fix number N of failures and the occurrence of a crucial failure τ Δ is also considered. Necessary and sufficient conditions to ensure the stochastic τ-stability are provided in this setting that are auxiliary to the main result.

A maximum principle for infinite time asymptotically stable impulsive dynamic control systems

We consider an infinite horizon optimal impulsive control problems for which a given cost function is minimized by choosing control strategies driving the state to a point in a given closed set C ∞. We present necessary conditions of optimality in the form of a maximum principle for which the boundary condition of the adjoint variable is such that non-degeneracy due to the fact that the time horizon is infinite is ensured. These conditions are given for conventional systems in a first instance and then for impulsive control problems. They are proved by considering a family of approximating auxiliary interval conventional (without impulses) optimal control problems defined on an increasing sequence of finite time intervals. As far as we know, results of this kind have not been derived previously. © 2010 IFAC.

Notes on the Solution of Dynamic Optimization Problems with Explicit Controls in Discrete-Time Economic Models

Йордан Йорданов, Андрей Василев - В работата се изследват методи за решаването на задачи на оптималното управление в дискретно време с безкраен хоризонт и явни управления. Дадена е обосновка на една процедура за решаване на такива задачи, базирана на множители на Лагранж, коята често се употребява в икономическата литература. Извеждени са необходимите условия за оптималност на базата на уравнения на Белман и са приведени достатъчни условия за оптималност при допускания, които често се използват в икономиката.

Parametrized actor-critic algorithms for finite-horizon MDPs

Due to their non-stationarity, finite-horizon Markov decision processes (FH-MDPs) have one probability transition matrix per stage. Thus the curse of dimensionality affects FH-MDPs more severely than infinite-horizon MDPs. We propose two parametrized 'actor-critic' algorithms to compute optimal policies for FH-MDPs. Both algorithms use the two-timescale stochastic approximation technique, thus simultaneously performing gradient search in the parametrized policy space (the 'actor') on a slower timescale and learning the policy gradient (the 'critic') via a faster recursion. This is in contrast to methods where critic recursions learn the cost-to-go proper. We show w.p 1 convergence to a set with the necessary condition for constrained optima. The proposed parameterization is for FHMDPs with compact action sets, although certain exceptions can be handled. Further, a third algorithm for stochastic control of stopping time processes is presented. We explain why current policy evaluation methods do not work as critic to the proposed actor recursion. Simulation results from flow-control in communication networks attest to the performance advantages of all three algorithms.

Optimal Control of Markov Processes with Age-Dependent Transition Rates

We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite horizon average cost problems. Our approach is via the construction of an equivalent semi-Markov decision process. We characterise the value function and optimal controls for both discounted and average cost cases.

Time-dependent monoenergetic neutron transport in two adjacent semi-infinite media

An exact solution to the monoenergetic Boltzmann equation is obtained for the case of a plane isotropic burst of neutrons introduced at the interface separating two adjacent, dissimilar, semi-infinite media. The method of solution used is to remove the time dependence by a Laplace transformation, solve the transformed equation by the normal mode expansion method, and then invert to recover the time dependence.

The general result is expressed as a sum of definite, multiple integrals, one of which contains the uncollided wave of neutrons originating at the source plane. It is possible to obtain a simplified form for the solution at the interface, and certain numerical calculations are made there.

The interface flux in two adjacent moderators is calculated and plotted as a function of time for several moderator materials. For each case it is found that the flux decay curve has an asymptotic slope given accurately by diffusion theory. Furthermore, the interface current is observed to change directions when the scattering and absorption cross sections of the two moderator materials are related in a certain manner. More specifically, the reflection process in two adjacent moderators appears to depend initially on the scattering properties and for long times on the absorption properties of the media.

This analysis contains both the single infinite and semi-infinite medium problems as special cases. The results in these two special cases provide a check on the accuracy of the general solution since they agree with solutions of these problems obtained by separate analyses.

The Possibility of Ordering Infinite Utility Streams

This paper revisits Diamond’s classical impossibility result regarding the ordering of infinite utility streams. We show that if no representability condition is imposed, there do exist strongly Paretian and finitely anonymous orderings of intertemporal utility streams with attractive additional properties. We extend a possibility theorem due to Svensson to a characterization theorem and we provide characterizations of all strongly Paretian and finitely anonymous rankings satisfying the strict transfer principle. In addition, infinite horizon extensions of leximin and of utilitarianism are characterized by adding an equity preference axiom and finite translation-scale measurability, respectively, to strong Pareto and finite anonymity.

An Optimal Control Framework for Resources Management in Agriculture

An optimal control framework to support the management and control of resources in a wide range of problems arising in agriculture is discussed. Lessons extracted from past research on the weed control problem and a survey of a vast body of pertinent literature led to the specification of key requirements to be met by a suitable optimization framework. The proposed layered control structure—including planning, coordination, and execution layers—relies on a set of nested optimization processes of which an “infinite horizon” Model Predictive Control scheme plays a key role in planning and coordination. Some challenges and recent results on the Pontryagin Maximum Principle for infinite horizon optimal control are also discussed.