Policy Improvement for POMDPs Using Normalized Importance Sampling

We present a new method for estimating the expected return of a POMDP from experience. The estimator does not assume any knowle ge of the POMDP and allows the experience to be gathered with an arbitrary set of policies. The return is estimated for any new policy of the POMDP. We motivate the estimator from function-approximation and importance sampling points-of-view and derive its theoretical properties. Although the estimator is biased, it has low variance and the bias is often irrelevant when the estimator is used for pair-wise comparisons.We conclude by extending the estimator to policies with memory and compare its performance in a greedy search algorithm to the REINFORCE algorithm showing an order of magnitude reduction in the number of trials required.

Modeling the Transport Sector: The Role of Existing Fuel Taxes in Climate Policy

Existing fuel taxes play a major role in determining the welfare effects of exempting the transportation sector from measures to control greenhouse gases. To study this phenomenon we modify the MIT Emissions Prediction and Policy Analysis (EPPA) model to disaggregate the household transportation sector. This improvement requires an extension of the GTAP data set that underlies the model. The revised and extended facility is then used to compare economic costs of cap-and-trade systems differentiated by sector, focusing on two regions: the USA where the fuel taxes are low, and Europe where the fuel taxes are high. We find that the interplay between carbon policies and pre-existing taxes leads to different results in these regions: in the USA exemption of transport from such a system would increase the welfare cost of achieving a national emissions target, while in Europe such exemptions will correct pre-existing distortions and reduce the cost.

Memory Usage Inference for Object-Oriented Programs

We present a type-based approach to statically derive symbolic closed-form formulae that characterize the bounds of heap memory usages of programs written in object-oriented languages. Given a program with size and alias annotations, our inference system will compute the amount of memory required by the methods to execute successfully as well as the amount of memory released when methods return. The obtained analysis results are useful for networked devices with limited computational resources as well as embedded software.

Effects of Air Pollution Control on Climate

Urban air pollution and climate are closely connected due to shared generating processes (e.g., combustion) for emissions of the driving gases and aerosols. They are also connected because the atmospheric lifecycles of common air pollutants such as CO, NOx and VOCs, and of the climatically important methane gas (CH4) and sulfate aerosols, both involve the fast photochemistry of the hydroxyl free radical (OH). Thus policies designed to address air pollution may impact climate and vice versa. We present calculations using a model coupling economics, atmospheric chemistry, climate and ecosystems to illustrate some effects of air pollution policy alone on global warming. We consider caps on emissions of NOx, CO, volatile organic carbon, and SOx both individually and combined in two ways. These caps can lower ozone causing less warming, lower sulfate aerosols yielding more warming, lower OH and thus increase CH4 giving more warming, and finally, allow more carbon uptake by ecosystems leading to less warming. Overall, these effects significantly offset each other suggesting that air pollution policy has a relatively small net effect on the global mean surface temperature and sea level rise. However, our study does not account for the effects of air pollution policies on overall demand for fossil fuels and on the choice of fuels (coal, oil, gas), nor have we considered the effects of caps on black carbon or organic carbon aerosols on climate. These effects, if included, could lead to more substantial impacts of capping pollutant emissions on global temperature and sea level than concluded here. Caps on aerosols in general could also yield impacts on other important aspects of climate beyond those addressed here, such as the regional patterns of cloudiness and precipitation.