Computing Markov-Perfect Optimal Policies in Business-Cycle Models

Time-inconsistency is an essential feature of many policy problems (Kydland and Prescott, 1977). This paper presents and compares three methods for computing Markov-perfect optimal policies in stochastic nonlinear business cycle models. The methods considered include value function iteration, generalized Euler-equations, and parameterized shadow prices. In the context of a business cycle model in which a scal authority chooses government spending and income taxation optimally, while lacking the ability to commit, we show that the solutions obtained using value function iteration and generalized Euler equations are somewhat more accurate than that obtained using parameterized shadow prices. Among these three methods, we show that value function iteration can be applied easily, even to environments that include a risk-sensitive scal authority and/or inequality constraints on government spending. We show that the risk-sensitive scal authority lowers government spending and income-taxation, reducing the disincentive households face to accumulate wealth.

Twenty questions with noise: Bayes optimal policies for entropy loss

We consider the problem of twenty questions with noisy answers, in which we seek to find a target by repeatedly choosing a set, asking an oracle whether the target lies in this set, and obtaining an answer corrupted by noise. Starting with a prior distribution on the target's location, we seek to minimize the expected entropy of the posterior distribution. We formulate this problem as a dynamic program and show that any policy optimizing the one-step expected reduction in entropy is also optimal over the full horizon. Two such Bayes optimal policies are presented: one generalizes the probabilistic bisection policy due to Horstein and the other asks a deterministic set of questions. We study the structural properties of the latter, and illustrate its use in a computer vision application.

Education, preferences for leisure, and the optimal income tax schedule

Recent advances in dynamic Mirrlees economies have incorporated the treatment of human capital investments as an important dimension of government policy. This paper adds to this literature by considering a two period economy where agents are di erentiated by their preferences for leisure and their productivity, both private information. The fact that productivity is only learnt later in an agent's life introduces uncertainty to agent's savings and human capital choices and makes optimal the use of multi-period tie-ins in the mechanism that characterizes the government policy. We show that optimal policies are often interim ine cient and that the introduction of these ine ciencies may take the form of marginal tax rates on labor income of varying sign and educational policies that include the discouragement of human capital acquisition. With regards to implementation, state-dependent linear taxes implement optimal savings, while human capital policies may require labor income taxes that depend directly on agents' schooling.

Optimal monetary policy under administered prices

This work aims to analyze the interaction and the effects of administered prices in the economy, through a DSGE model and the derivation of optimal monetary policies. The model used is a standard New Keynesian DSGE model of a closed economy with two sectors companies. In the first sector, free prices, there is a continuum of firms, and in the second sector of administered prices, there is a single firm. In addition, the model has positive trend inflation in the steady state. The model results suggest that price movements in any sector will impact on both sectors, for two reasons. Firstly, the price dispersion causes productivity to be lower. As the dispersion of prices is a change in the relative price of any sector, relative to general prices in the economy, when a movement in the price of a sector is not followed by another, their relative weights will change, leading to an impact on productivity in both sectors. Second, the path followed by the administered price sector is considered in future inflation expectations, which is used by companies in the free sector to adjust its optimal price. When this path leads to an expectation of higher inflation, the free sector companies will choose a higher mark-up to accommodate this expectation, thus leading to higher inflation trend when there is imperfect competition in the free sector. Finally, the analysis of optimal policies proved inconclusive, certainly indicating that there is influence of the adjustment model of administered prices in the definition of optimal monetary policy, but a quantitative study is needed to define the degree of impact.

AVERAGE CONTINUOUS CONTROL OF PIECEWISE DETERMINISTIC MARKOV PROCESSES

This paper deals with the long run average continuous control problem of piecewise deterministic Markov processes (PDMPs) taking values in a general Borel space and with compact action space depending on the state variable. The control variable acts on the jump rate and transition measure of the PDMP, and the running and boundary costs are assumed to be positive but not necessarily bounded. Our first main result is to obtain an optimality equation for the long run average cost in terms of a discrete-time optimality equation related to the embedded Markov chain given by the postjump location of the PDMP. Our second main result guarantees the existence of a feedback measurable selector for the discrete-time optimality equation by establishing a connection between this equation and an integro-differential equation. Our final main result is to obtain some sufficient conditions for the existence of a solution for a discrete-time optimality inequality and an ordinary optimal feedback control for the long run average cost using the so-called vanishing discount approach. Two examples are presented illustrating the possible applications of the results developed in the paper.

Should fiscal authorities cooperate in a monetary union with public deficit targets?

In this paper we analyse the setting of optimal policies in a monetary union with one monetary authority and various fiscal authorities that have a public deficit target. We will show that fiscal cooperation among the fiscal authorities, in the presence of positive supply shocks, ends up producing higher public deficits than in a non-cooperative regime. JEL No. E61, E63, F33, H0. Keywords: monetary union, fiscal policy coordination.

Forecasting the European Carbon Market

In an effort to meet its obligations under the Kyoto Protocol, in 2005 the European Union introduced a cap-and-trade scheme where mandated installations are allocated permits to emit CO2. Financial markets have developed that allow companies to trade these carbon permits. For the EU to achieve reductions in CO2 emissions at a minimum cost, it is necessary that companies make appropriate investments and policymakers design optimal policies. In an effort to clarify the workings of the carbon market, several recent papers have attempted to statistically model it. However, the European carbon market (EU ETS) has many institutional features that potentially impact on daily carbon prices (and associated nancial futures). As a consequence, the carbon market has properties that are quite different from conventional financial assets traded in mature markets. In this paper, we use dynamic model averaging (DMA) in order to forecast in this newly-developing market. DMA is a recently-developed statistical method which has three advantages over conventional approaches. First, it allows the coefficients on the predictors in a forecasting model to change over time. Second, it allows for the entire fore- casting model to change over time. Third, it surmounts statistical problems which arise from the large number of potential predictors that can explain carbon prices. Our empirical results indicate that there are both important policy and statistical bene ts with our approach. Statistically, we present strong evidence that there is substantial turbulence and change in the EU ETS market, and that DMA can model these features and forecast accurately compared to conventional approaches. From a policy perspective, we discuss the relative and changing role of different price drivers in the EU ETS. Finally, we document the forecast performance of DMA and discuss how this relates to the efficiency and maturity of this market.

A General and Intuitive Envelope Theorem

We present an envelope theorem for establishing first-order conditions in decision problems involving continuous and discrete choices. Our theorem accommodates general dynamic programming problems, even with unbounded marginal utilities. And, unlike classical envelope theorems that focus only on differentiating value functions, we accommodate other endogenous functions such as default probabilities and interest rates. Our main technical ingredient is how we establish the differentiability of a function at a point: we sandwich the function between two differentiable functions from above and below. Our theory is widely applicable. In unsecured credit models, neither interest rates nor continuation values are globally differentiable. Nevertheless, we establish an Euler equation involving marginal prices and values. In adjustment cost models, we show that first-order conditions apply universally, even if optimal policies are not (S,s). Finally, we incorporate indivisible choices into a classic dynamic insurance analysis.

Uniform versus discriminatory tariffs

We analyze the non-cooperative interaction between two exporting countries producing differentiated products and one importing country when governments use optimal policies to maximize welfare. The analysis includes product differentiation, asymmetric costs, and Bertrand competition. For identical exporting countries we demonstrate that the importing country always prefers a uniform tariff regime while both exporting countries prefer a discriminatory tariff regime for any degree of product differentiation. If countries are asymmetric in terms of production cost then the higher-cost exporter always prefers the discriminatory regime but the lower-cost exporter prefers the uniform regime if there is a significant cost differential. With cost asymmetry the announcement of a uniform tariff regime by the importer is not a credible strategy since there is an incentive to deviate to discrimination. This implies an international body can play a role in ensuring that tariff agreements are respected.

Sustained growth, government expenditure and inflation

We construct and simulate a theoretical model in order to explain particular historical experiences in which inflation acceleration apparently helped to spur a period of economic growth. Government financed expenditures affect positively the produtivity growth in this model so that the distortionary effect of inflation tax is compensated by the productive effect of public expenditures. We show that for some interval of money creation rates there is an equilibrium where money is valued and where steady state physica1 capital grows with inflation. It is a1so shown that zero inflation and growth maximization are never the optimal policies.

Inflationary financing of public investment and economic growth

a theoretical model is constructed in order to explain particular historical experiences in which inflation acceleration apparently helped to spur a period of economic growth. Government financed expenditures affect positively the productivity growth in this model so that the distortionary effect of inflation tax is compensated by the productive effect of public expenditures. We show that for some interval of money creation rates there is an equilibrium where money is valued and where steady state physical capital grows with inflation. It is also shown that zero inflation and growth maximization are never the optimal policies.

QoS management in smart grids: a Markovian approach

In Smart Grids, a variety of new applications are available to users of the electrical system (from consumers to the electric system operators and market operators). Some applications such as the SCADA systems, which control generators or substations, have consequences, for example, with a communication delay. The result of a failure to deliver a control message due to noncompliance of the time constraint can be catastrophic. On the other hand, applications such as smart metering of consumption have fewer restrictions. Since each type of application has different quality of service requirements (importance, delay, and amount of data to transmit) to transmit its messages, the policy to control and share the resources of the data communication network must consider them. In this paper Markov Decision Process Theory is employed to determine optimal policies to explore as much as possible the availability of throughput in order to transmit all kinds of messages, considering the quality of service requirements defined to each kind of message. First a non-preemptive model is formulated and after that a preemptive model is derived. Numerical results are used to compare FIFO, non-preemptive and preemptive policies.

Compatibilidad del principio de óptimo de Bellman con los métodos secuenciales de optimización de redes de tuberías

En el proceso de cálculo de redes de tuberías se maneja un conjunto de variables con unas características muy peculiares, ya que son discretas y estandarizadas. Por lo tanto su evolución se produce por escalones (la presión nominal, el diámetro y el costo de los tubos). Por otro lado la presión de diseño de la red es una función directa de la presión de cabecera. En el proceso de optimización mediante programación dinámica la presión de cabecera se va reduciendo gradualmente en cada secuencia del proceso, haciendo que evolucione a la par la presión de diseño, lo que genera a su vez saltos discriminados en la presión nominal de los tramos, y con ello en su costo y en su gradiente de cambio. En esta tesis doctoral se analiza si estos cambios discriminados que se producen en el gradiente de cambio de algunos tramos en el curso de una secuencia, ocasionados por la evolución de la presión de cabecera de la red, generan interferencias que alteran el proceso secuencial de la programación dinámica. La modificación del gradiente de cambio durante el transcurso de una secuencia se conoce con el nombre de mutación, la cual puede ser activa cuando involucra a un tramo optimo modificando las condiciones de la transacción o pasiva si no crea afección alguna. En el análisis realizado se distingue entre la mutación del gradiente de cambio de los tramos óptimos (que puede generarse exclusivamente en el conjunto de los trayectos que los albergan), y entre los efectos que el cambio de timbraje produce en el resto de los tramos de la red (incluso los situados aguas abajo de los nudos con holgura de presión nula) sobre el mecanismo iterativo, estudiando la compatibilidad de este fenómeno con el principio de óptimo de Bellman. En el proceso de investigación llevado a cabo se destaca la fortaleza que da al proceso secuencial del método Granados el hecho de que el gradiente de cambio siempre sea creciente en el avance hacia el óptimo, es decir que el costo marginal de la reducción de las pérdidas de carga de la red que se consigue en una iteración siempre sea más caro que el de la iteración precedente. Asimismo, en el estudio realizado se revisan los condicionantes impuestos al proceso de optimización, incluyendo algunos que hasta ahora no se han tenido en cuenta en los estudios de investigación, pero que están totalmente integrados en la ingeniería práctica, como es la disposición telescópica de las redes (reordenación de los diámetros de mayor a menor de cabeza a cola de la red), y la disposición de un único diámetro por tramo, en lugar de que estén compartidos por dos diámetros contiguos (con sus salvedades en caso de tramos de gran longitud, o en otras situaciones muy específicas). Finalmente se incluye un capítulo con las conclusiones, aportaciones y recomendaciones, las cuales se consideran de gran utilidad para la ingeniería práctica, entre las que se destaca la perfección del método secuencial, la escasa transcendencia de las mutaciones del gradiente de cambio y la forma en que pueden obviarse, la inocuidad de las mutaciones pasivas y el cumplimiento del principio de Bellman en todo el proceso de optimización. The sizing process of a water distribution network is based on several variables, being some of them special, as they are discrete and their values are standardized: pipe pressure rating, pipe diameter and pipe cost. On another note, the sizing process is directly related with the pressure at the network head. Given that during the optimization by means of the Granados’ Method (based on dynamic programming) the pressure at the network head is being gradually reduced, a jump from one pipe pressure rating to another may arise during the sequential process, leading to changes on the pipe cost and on the gradient change (unitary cost for reducing the head losses). This chain of changes may, in turn, affect the sequential process diverting it from an optimal policies path. This thesis analyses how the abovementioned alterations could influence the results of the dynamic programming algorithm, that is to say the compatibility with the Bellman’s Principle of Optimality, which states that the sequence has to follow a route of optimal policies, and that past decisions should not influence the remaining ones. The modification of the gradient change is known as mutation. Mutations are active when they affect the optimal link (the one which was selected to be changed during iteration) or passive when they do not alter the selection of the optimal link. The thesis analysed the potential mutations processes along the network, both on the optimal paths and also on the rest of the network, and its influence on the final results. Moreover, the investigation analysed the practical restrictions of the sizing process that are fully integrated in the applied engineering, but not always taken into account by the optimization tools. As the telescopic distribution of the diameters (i.e. larger diameters are placed at the network head) and the use of a unique diameter per link (with the exception of very large links, where two consecutive diameters may be placed). Conclusions regarding robustness of the dynamic programming algorithm are given. The sequence of the Granados Method is quite robust and it has been shown capable to auto-correct the mutations that could arise during the optimization process, and to achieve an optimal distribution even when the Bellman’s Principle of Optimality is not fully accomplished. The fact that the gradient change is always increasing during the optimization (that is to say, the marginal cost of reducing head losses is always increasing), provides robustness to the algorithm, as looping are avoided in the optimization sequence. Additionally, insight into the causes of the mutation process is provided and practical rules to avoid it are given, improving the current definition and utilization of the Granados’ Method.

Optimal Climate Change Policies When Governments Cannot Commit

This paper examines the optimal design of climate change policies in the context where governments want to encourage the private sector to undertake significant immediate investment in developing cleaner technologies, but the carbon taxes and other environmental policies that could in principle stimulate such investment will be imposed over a very long future. The conventional claim by environmental economists is that environmental policies alone are sufficient to induce firms to undertake optimal investment. However this argument requires governments to be able to commit to these future taxes, and it is far from clear that governments have this degree of commitment. We assume instead that governments cannot commit, and so both they and the private sector have to contemplate the possibility of there being governments in power in the future that give different (relative) weights to the environment. We show that this lack of commitment has a significant asymmetric effect. Compared to the situation where governments can commit it increases the incentive of the current government to have the investment undertaken, but reduces the incentive of the private sector to invest. Consequently governments may need to use additional policy instruments – such as R&D subsidies – to stimulate the required investment.

Time is Running Out: The 2°C Target and Optimal Climate Policies

The quintessence of recent natural science studies is that the 2 degrees C target can only be achieved with massive emission reductions in the next few years. The central twist of this paper is the addition of this limited time to act into a non-perpetual real options framework analysing optimal climate policy under uncertainty. The window-of-opportunity modelling setup shows that the limited time to act may spark a trend reversal in the direction of low-carbon alternatives. However, the implementation of a climate policy is evaded by high uncertainty about possible climate pathways.