Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Plasmodium falciparum gametocyte carriage, sex ratios and asexual parasite rates in Nigerian children before and after a treatment protocol policy change instituting the use of artemisinin-based combination therapies

The effects of artemisinin-based combination therapies (ACTs) on transmission of Plasmodium falciparum were evaluated after a policy change instituting the use of ACTs in an endemic area. P. falciparum gametocyte carriage, sex ratios and inbreeding rates were examined in 2,585 children at presentation with acute falciparum malaria during a 10-year period from 2001-2010. Asexual parasite rates were also evaluated from 2003-2010 in 10,615 children before and after the policy change. Gametocyte carriage declined significantly from 12.4% in 2001 to 3.6% in 2010 (@@χ² for trend = 44.3, p < 0.0001), but sex ratios and inbreeding rates remained unchanged. Additionally, overall parasite rates remained unchanged before and after the policy change (47.2% vs. 45.4%), but these rates declined significantly from 2003-2010 (@@χ² for trend 35.4, p < 0.0001). Chloroquine (CQ) and artemether-lumefantrine (AL) were used as prototype drugs before and after the policy change, respectively. AL significantly shortened the duration of male gametocyte carriage in individual patients after treatment began compared with CQ (log rank statistic = 7.92, p = 0.005). ACTs reduced the rate of gametocyte carriage in children with acute falciparum infections at presentation and shortened the duration of male gametocyte carriage after treatment. However, parasite population sex ratios, inbreeding rates and overall parasite rate were unaffected.

Seeds of distrust: conflict in Uganda

We study the effect of civil conflict on social capital, focusing on Uganda's experience during the last decade. Using individual and county-level data, we document large causal effects on trust and ethnic identity of an exogenous outburst of ethnic conflicts in 2002-2005. We exploit two waves of survey data from Afrobarometer (Round 4 Afrobarometer Survey in Uganda, 2000, 2008), including information on socioeconomic characteristics at the individual level, and geo-referenced measures of fighting events from ACLED. Our identification strategy exploits variations in the both the spatial and ethnic intensity of fighting. We find that more intense fighting decreases generalized trust and increases ethnic identity. The effects are quantitatively large and robust to a number of control variables, alternative measures of violence, and different statistical techniques involving ethnic and spatial fixed effects and instrumental variables. Controlling for the intensity of violence during the conflict, we also document that post-conflict economic recovery is slower in ethnically fractionalized counties. Our findings are consistent with the existence of a self-reinforcing process between conflicts and ethnic cleavages.

Conceiving Comparative Policy Analysis : Implementing Air Quality Improvement Programs

The comparative analysis of air quality control policies provides an interesting field for studies of comparative policy analysis including program formulation and implementation processes. In European countries, the problem is comparable, whereas implementation structures, programs and policy impacts vary to a considerable extent. Analysis testing possibilities and constraints of air control policies under varying conditions are likely to contribute to a further development of a theory of policy analysis. This paper presents the analytical framework applied in a continuing empirical study explaining program formulation and implementation processes with respect to the different actors involved. Concrete emitter behavior can be explained by interaction processes at the very local level, by program elements of national legislation, and by structural constraints under which such programs are produced.

Agriculture, technological change and environmental sustainability: Looking for a win-win water policy strategy

This paper analyses the effects that technological changes in agriculture would have on environmental, social and economic indicators. Specifically, our study is focused on two alternative technological improvements: the modernization of water transportation systems versus the increase in the total factor productivity of agriculture. Using a computable general equilibrium model for the Catalan economy, our results suggest that a water policy that leads to greater economic efficiency is not necessarily optimal if we consider social or environmental criteria. Moreover, improving environmental sustainability depends less on the type of technological change than on the institutional framework in which technological change occurs. Keywords: agricultural technological changes, computable general equilibrium model, economic impact, water policy

Adverse selection, moral hazard, and outlier payment policy

In this article, we analyze the rationale for introducing outlier payments into a prospective payment system for hospitals under adverse selection and moral hazard. The payer has only two instruments: a fixed price for patients whose treatment cost is below a threshold and a cost-sharing rule for outlier patients. We show that a fixed-price policy is optimal when the hospital is sufficiently benevolent. When the hospital is weakly benevolent, a mixed policy solving a trade-off between rent extraction, efficiency, and dumping deterrence must be preferred. We show how the optimal combination of fixed price and partially cost-based payment depends on the degree of benevolence of the hospital, the social cost of public funds, and the distribution of patients severity. [Authors]

Ideas, Possibilities and Limitations of Sustainable Economic Policy – the Example of Ecological Economics

A mesura que el suport del creixement econòmic constitueix un objectiu fonamental de la formulació de polítiques econòmiques, cal assenyalar que aquest tipus de creixement està limitat naturalment per un planeta finit. Aquest article argumenta que, des del punt de vista de la justícia intergeneracional, la realització d'un concepte de desmaterialització i, com a efecte, d'una economia que no creix (en el sentit de dissociació absoluta del creixement econòmic i consum d'energia i materials) es pot justificar. Per tant, el creixement pot ser també entesa com la millora de la qualitat de vida sobretot en comptes d'ampliar quantitats escarpats de sortida. Per tant, una dràstica reducció del cabal de material es necessita, sobretot en els països d'alts ingressos. Després de presentar alguns crítica de les propostes, en el focus d'aquest article es dibuixen en els arguments de per què la política econòmica en el futur han de ser etiquetats com "ecològic" i, a continuació, les opcions de posar en acció les idees del teòric presentat marc en tasques manejables polítiques seran discutides. En aquest cas, s'argumentarà que l'enfocament clàssic de internalització d'efectes externs sovint seguides de decisions de política econòmica ortodoxa no és completament capaç de reflectir canvis ecològics en les estructures de preus dels mercats. Per tant, formal (industrial i l'establiment de la política de consum) i institucions informals (llars) representen punts clau de la política econòmica sostenible, assenyalant l'individu com així com la responsabilitat col · lectiva per omplir aquest buit substancial.