Linear Bayes policy for learning in contextual-bandits

Machine and Statistical Learning techniques are used in almost all online advertisement systems. The problem of discovering which content is more demanded (e.g. receive more clicks) can be modeled as a multi-armed bandit problem. Contextual bandits (i.e., bandits with covariates, side information or associative reinforcement learning) associate, to each specific content, several features that define the “context” in which it appears (e.g. user, web page, time, region). This problem can be studied in the stochastic/statistical setting by means of the conditional probability paradigm using the Bayes’ theorem. However, for very large contextual information and/or real-time constraints, the exact calculation of the Bayes’ rule is computationally infeasible. In this article, we present a method that is able to handle large contextual information for learning in contextual-bandits problems. This method was tested in the Challenge on Yahoo! dataset at ICML2012’s Workshop “new Challenges for Exploration & Exploitation 3”, obtaining the second place. Its basic exploration policy is deterministic in the sense that for the same input data (as a time-series) the same results are obtained. We address the deterministic exploration vs. exploitation issue, explaining the way in which the proposed method deterministically finds an effective dynamic trade-off based solely in the input-data, in contrast to other methods that use a random number generator.

A unified framework for linear function approximation of value functions in stochastic control

This paper contributes with a unified formulation that merges previ- ous analysis on the prediction of the performance ( value function ) of certain sequence of actions ( policy ) when an agent operates a Markov decision process with large state-space. When the states are represented by features and the value function is linearly approxi- mated, our analysis reveals a new relationship between two common cost functions used to obtain the optimal approximation. In addition, this analysis allows us to propose an efficient adaptive algorithm that provides an unbiased linear estimate. The performance of the pro- posed algorithm is illustrated by simulation, showing competitive results when compared with the state-of-the-art solutions.

Assessing the impact of adaptive accessibility on the optimal transport policy implementation by using an integrated land-use/transport model for Madrid

Accessibility is an essential concept widely used to evaluate the impact of land-use and transport strategies in transport and urban planning. Accessibility is typically evaluated by using a transport model or a land-use model independently or successively without a feedback loop, thus neglecting the interaction effects between the two systems and the induced competition effects among opportunities due to accessibility improvements. More than a mere methodological curiosity, failure to account for land- use/transport interactions and the competition effect may result in large underestimation of the policy effects. With the recent development of land-use and transport interaction (LUTI) models, there is a growing interest in using these models to adequately measure accessibility and evaluate its impact. The current study joins this research stream by embedding an accessibility measure in a LUTI model with two main aims. The first aim is to account for adaptive accessibility, namely the adjustment of the potential accessibility due to the effect of competition among opportunities (e.g., workplaces) as a result of improved accessibility. LUTI models are particularly suitable for assessing adaptive accessibility because the competition factor is a function of the number of jobs, which is related to land-use attractiveness and the number of workers which is related, among other factors, to the transport demand. The second aim is to identify the optimal implementation scenario of policy measures on the basis of the potential and adaptive accessibility and analyse the results in terms of social welfare and accessibility. The metropolitan area of Madrid is used as a case-study and two transport policy instruments, namely a cordon toll and bus frequency increase, have been chosen for the simulation study in order to present the usefulness of the approach to urban planners and policy makers. The MARS model (Metropolitan Activity Relocation Simulator) calibrated for Madrid was employed as the analysis tool. The impact of accessibility is embedded in the model through a social welfare function that includes not only costs and benefits to both road users and transport operators, but also costs and benefits for the government and society in general (external costs). An optimisation procedure is performed by the MARS model for maximizing the value of objective function in order to find the best (optimal) policy imp lementations intensity (i.e., price, frequency). Last, the two policy strategies are evaluated in terms of their accessibility. Results show that the accessibility with competition factor influences the optimal policy implementation level and also generates different results in terms of social welfare. In addition, mapping the difference between the potential and the adaptive accessibility indicators shows that the main changes occur in areas where there is a strong competition among land-use opportunities.

Assessing the accessibility impact of transport policy by a land-use and transport interaction model - The case of Madrid

Accessibility is an essential concept widely used to evaluate the impact of transport and land-use strategies in urban planning and policy making. Accessibility is typically evaluated by using separately a transport model or a land-use model. This paper embeds two accessibility indicators (i.e., potential and adaptive accessibility) in a land use and transport interaction (LUTI) model in order to assess transport policies implementation. The first aim is to define the adaptive accessibility, considering the competition factor at territorial level (e.g. workplaces and workers). The second aim is to identify the optimal implementation scenario of policy measures using potential and adaptive accessibility indicators. The analysis of the results in terms of social welfare and accessibility changes closes the paper. Two transport policy measures are applied in Madrid region: a cordon toll and increase bus frequency. They have been simulated through the MARS model (Metropolitan Activity Relocation Simulator, i.e. LUTI model). An optimisation procedure is performed by MARS for maximizing the value of the objective function in order to find the optimal policy implementation (first best). Both policy measures are evaluated in terms of accessibility. Results show that the introduction of the accessibility indicators (potential and adaptive) influence the optimal value of the toll price and bus frequency level, generating different results in terms of social welfare. Mapping the difference between potential and adaptive accessibility indicator shows that the main changes occur in areas where there is a strong competition among different land-use opportunities.

Modeling stochastic gene expression: Implications for haploinsufficiency

There is increasing recognition that stochastic processes regulate highly predictable patterns of gene expression in developing organisms, but the implications of stochastic gene expression for understanding haploinsufficiency remain largely unexplored. We have used simulations of stochastic gene expression to illustrate that gene copy number and expression deactivation rates are important variables in achieving predictable outcomes. In gene expression systems with non-zero expression deactivation rates, diploid systems had a higher probability of uninterrupted gene expression than haploid systems and were more successful at maintaining gene product above a very low threshold. Systems with relatively rapid expression deactivation rates (unstable gene expression) had more predictable responses to a gradient of inducer than systems with slow or zero expression deactivation rates (stable gene expression), and diploid systems were more predictable than haploid, with or without dosage compensation. We suggest that null mutations of a single allele in a diploid organism could decrease the probability of gene expression and present the hypothesis that some haploinsufficiency syndromes might result from an increased susceptibility to stochastic delays of gene initiation or interruptions of gene expression.

Was the ECB’s Comprehensive Assessment up to standard? CEPS Policy Brief No. 325, 10 November 2014

The Comprehensive Assessment conducted by the European Central Bank (ECB) represents a considerable step forward in enhancing transparency in euro-area banks’ balance sheets. The most notable progress since the previous European stress test has been the harmonisation of the definition of non-performing loans and other concepts as well as uncovering hidden losses, which resulted in a €34 billion aggregate capital-charge net of taxes. Despite this tightening, most banks were able to meet the 5.5% common equity tier 1 (CET1) threshold applied in the test, which suggests that the large majority of the euro-area banks have improved their financial position sufficiently that they should no longer be constrained in financing the economy. As shown in this CEPS Policy Brief by Willem Pieter de Groen, however, the detailed results provide a more nuanced picture: there remain a large number of the banks in the euro area that are still highly leveraged and in many cases unable to meet the regulatory capital requirements that will be introduced in the coming years under the adverse stress test scenario.

Investment behaviour of EU arable crop farms in selected EU countries and the impact of policy reforms. Factor Markets Working Document No. 42, May 2013

This deliverable provides a comparative analysis, among selected EU member states, of the investment demand of a sample of specialised field crop farms for farm buildings, machinery and equipment as determined by different types and levels of Common Agricultural Policy support. It allows for the existence of uncertainty in the price of output farmers receive and for both long- and short-run determinants of investment levels, as well as for the presence of irregularities in the cost adjustment function due to the existence of threshold-type behaviours. The empirical estimation reveals that three investment regimes are consistently identified in Germany and Hungary, across asset and support types, and in France for machinery and equipment. More traditional disinvestment-investment type behaviours characterise investment in farm building in France and the UK, across support types, and Italy for both asset classes under coupled payments. The long-run dynamic adjustment of capital stocks is consistently and significantly estimated to be towards a – mostly non-stationary – lower level of capitalisation of the farm analysed. By contrast, the expected largely positive short-run effects of an increase in output prices are often not significant. The effect of CAP support on both types of investment is positive, although seldom significant, while the proxy for uncertainty employed fails to be significant yet, in most cases, has the expected effect of reducing the investment levels.

The out-of-sample performance of stochastic methods in forecasting age-specific mortality rates /

"July 2008."

Dynamic, convex, and robust optimization with Bayesian learning for response-guided dosing

Thesis (Ph.D.)--University of Washington, 2016-08

Drought policy: a graphical analysis

The standard approach to modelling production under uncertainty has relied on the concept of the stochastic production function. In the present paper, it is argued that a state-contingent production model is more flexible and realistic. The model is applied to the problem of drought policy.

A genetic estimation algorithm for parameters of stochastic ordinary differential equations

A generic method for the estimation of parameters for Stochastic Ordinary Differential Equations (SODEs) is introduced and developed. This algorithm, called the GePERs method, utilises a genetic optimisation algorithm to minimise a stochastic objective function based on the Kolmogorov-Smirnov statistic. Numerical simulations are utilised to form the KS statistic. Further, the examination of some of the factors that improve the precision of the estimates is conducted. This method is used to estimate parameters of diffusion equations and jump-diffusion equations. It is also applied to the problem of model selection for the Queensland electricity market. (C) 2003 Elsevier B.V. All rights reserved.

Implications of seasonal climate forecasts on world wheat trade: a stochastic, dynamic analysis

Improvements in seasonal climate forecasts have potential economic implications for international agriculture. A stochastic, dynamic simulation model of the international wheat economy is developed to estimate the potential effects of seasonal climate forecasts for various countries' wheat production, exports and world trade. Previous studies have generally ignored the stochastic and dynamic aspects of the effects associated with the use of climate forecasts. This study shows the importance of these aspects. In particular with free trade, the use of seasonal forecasts results in increased producer surplus across all exporting countries. In fact, producers appear to capture a large share of the economic surplus created by using the forecasts. Further, the stochastic dimensions suggest that while the expected long-run benefits of seasonal forecasts are positive, considerable year-to-year variation in the distribution of benefits between producers and consumers should be expected. The possibility exists for an economic measure to increase or decrease over a 20-year horizon, depending on the particular sequence of years.

Issues in boys' education: A question of teacher threshold knowledges?

This paper explores the effects of specific teacher threshold knowledges about boys and gender on the implementation of a so-called 'boy friendly' curriculum at one junior secondary high school in Australia. Through semi-structured inter-views with selected staff at the school, it examines the normalizing assumptions and 'truth claims' about boys, as gendered subjects, which drive the pedagogical impetus for such a curriculum initiative. This research raises crucial questions about the need for the formulation of both school and governmental policy grounded in sound research-based knowledge about the social construction of gender and its impact on the lives of both boys and girls and their experiences of schooling. This is crucial, we argue, in light of the recent parliamentary report on boys' education in Australia which rejects gender theorizing and given the failure of key staff in the research school to interrogate the binary ways in which masculinity and femininity are socially constructed and institutionalized in schools through a particular 'gender regime'. While some good things are happening in the research school, the failure to acknowledge the social construction of gender means that ultimately the school's programs cannot be successful.

A way forward for land-use planning to achieve policy goals by using spatial modelling scenarios

This paper investigates how demographic (socioeconomic) and land-use (physical and environmental) data can be integrated within a decision support framework to formulate and evaluate land-use planning scenarios. A case-study approach is undertaken with land-use planning scenarios for a rapidly growing coastal area in Australia, the Shire of Hervey Bay. The town and surrounding area require careful planning of the future urban growth between competing land uses. Three potential urban growth scenarios are put forth to address this issue. Scenario A ('continued growth') is based on existing socioeconomic trends. Scenario B ('maximising rates base') is derived using optimisation modelling of land-valuation data. Scenario C ('sustainable development') is derived using a number of social, economic, and environmental factors and assigning weightings of importance to each factor using a multiple criteria analysis approach. The land-use planning scenarios are presented through the use of maps and tables within a geographical information system, which delineate future possible land-use allocations up until 2021. The planning scenarios are evaluated by using a goal-achievement matrix approach. The matrix is constructed with a number of criteria derived from key policy objectives outlined in the regional growth management framework and town planning schemes. The authors of this paper examine the final efficiency scores calculated for each of the three planning scenarios and discuss the advantages and disadvantages of the three land-use modelling approaches used to formulate the final scenarios.

Optimal strategies in political elections

In the Majoritarian Parliamentary System, the government has a constitutional right to call an early election. This right provides the government a control to achieve its objective to remain in power for as long as possible. We model the early election problem mathematically using opinion polls data as a stochastic process to proxy the government's probability of re-election. These data measure the difference in popularity between the government and the opposition. We fit a mean reverting Stochastic Differential Equation to describe the behaviour of the process and consider the possibility for the government to use other control tools, which are termed 'boosts' to induce shocks to the opinion polls by making timely policy announcements or economic actions. These actions improve the government's popularity and have some impact upon the early-election exercise boundary. © Austral. Mathematical Soc. 2005.