Prototype of a Configurable Web-Based Assessment System

The MIT Prototype Educational Assessment System provides subjects and courses at MIT with the ability to perform online assessment. The system includes polices to handle harassment and electronic "flaming" while protecting privacy. Within these frameworks, individual courses and subjects can make their own policy decisions about such matters as to when assessments can occur, who can submit assessments, and how anonymous assessments are. By allowing assessment to take place continually and allowing both students and staff to participate, the system can provide a forum for the online discussion of subjects. Even in the case of scheduled assessments, the system can provide advantages over end-of-term assessment, since the scheduled assessments can occur several times during the semester, allowing subjects to identify and adjust those areas that could use improvement. Subjects can also develop customized questionnaires, perhaps in response to previous assessments, to suit their needs.

Reinforcement Learning by Policy Search

One objective of artificial intelligence is to model the behavior of an intelligent agent interacting with its environment. The environment's transformations can be modeled as a Markov chain, whose state is partially observable to the agent and affected by its actions; such processes are known as partially observable Markov decision processes (POMDPs). While the environment's dynamics are assumed to obey certain rules, the agent does not know them and must learn. In this dissertation we focus on the agent's adaptation as captured by the reinforcement learning framework. This means learning a policy---a mapping of observations into actions---based on feedback from the environment. The learning can be viewed as browsing a set of policies while evaluating them by trial through interaction with the environment. The set of policies is constrained by the architecture of the agent's controller. POMDPs require a controller to have a memory. We investigate controllers with memory, including controllers with external memory, finite state controllers and distributed controllers for multi-agent systems. For these various controllers we work out the details of the algorithms which learn by ascending the gradient of expected cumulative reinforcement. Building on statistical learning theory and experiment design theory, a policy evaluation algorithm is developed for the case of experience re-use. We address the question of sufficient experience for uniform convergence of policy evaluation and obtain sample complexity bounds for various estimators. Finally, we demonstrate the performance of the proposed algorithms on several domains, the most complex of which is simulated adaptive packet routing in a telecommunication network.

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Policy Skills for Web Scientists

Practical group based work on reading and interrogating evidence and writing policy briefings/critiques

La reintegración social y económica de los grupos armados ilegales en Colombia: reflexiones a partir de la trayectoria de nueve excombatientes

Esta investigación toma como marco general la Política de Reintegración Social y económica de personas y grupos alzados en armas en Colombia, en donde tras el estudio de las trayectorias en el conflicto de un grupo de 9 excombatientes, se aborda la relación existente entre los beneficios otorgados por dicha política y aquello que facilitó y motivó el ingreso, la permanencia y desmovilización de los grupos armados. Se presenta una caracterización e interpretación conceptual de las denominadas trayectorias en el conflicto, son establecidas relaciones y diferencias entre las organizaciones ilegales FARC y las AUC, se revisan las percepciones que frente a los beneficios del programa de reintegración tienen excombatientes y profesionales de la entidad que lidera dicho proceso y a partir de ello, es argumentada la incidencia que sobre el éxito de esta política tienen las características individuales y particulares, tanto de los excombatientes como de las organizaciones armadas ilegales.

"La sostenibilidad de los proyectos de desarrollo alternativo en Colombia: el caso del proyecto B-91 de UNODC"

Este artículo busca evaluar la sostenibilidad de un proyecto de desarrollo alternativo realizado por la Oficina de las Naciones Unidas Contra la Droga y el Delito en Colombia (UNODC) una vez que esta institución se retiró de la zona de implementación del proyecto. El proyecto analizado se llevó a cabo en 30 municipios entre los años 1997 y 2003. Para determinar si este proyecto fue sostenible, se realizó una breve introducción al tema de la evaluación de proyectos de inversión pública, se delimitó el concepto de desarrollo y se contextualizó el concepto de desarrollo alternativo en Colombia. En el estudio empírico se escogieron cinco indicadores para evaluar la sostenibilidad del proyecto: número de familias asociadas, número de hectáreas erradicadas, ingresos de la organización campesina, cantidad de producción lícita y número de hectáreas sembradas. Estos indicadores fueron contrastados entre el año 2003 – fecha de finalización del proyecto – y el año 2006 para determinar su comportamiento en el mediano plazo, luego de 3 años de haber finalizado las actividades de UNODC en el proyecto. Se encontró finalmente, con base en la comparación de dichos indicadores, que este proyecto es sostenible, con base en los indicadores utilizados, pero que debido a la forma como está planteada la política anti-drogas en Colombia el impacto de estas experiencias exitosas en el problema general de los cultivos ilícitos en el país es muy reducido.

A prediction model for establishing the disability degree in adults with spinal cord injury: results based on WHO-DAS II perspective

Objective: To establish a prediction model of the degree of disability in adults with Spinal CordInjury (SCI ) based on the use of the WHO-DAS II . Methods: The disability degree was correlatedwith three variable groups: clinical, sociodemographic and those related with rehabilitation services.A model of multiple linear regression was built to predict disability. 45 people with sci exhibitingdiverse etiology, neurological level and completeness participated. Patients were older than 18 andthey had more than a six-month post-injury. The WHO-DAS II and the ASIA impairment scale(AIS ) were used. Results: Variables that evidenced a significant relationship with disability were thefollowing: occupational situation, type of affiliation to the public health care system, injury evolutiontime, neurological level, partial preservation zone, ais motor and sensory scores and number ofclinical complications during the last year. Complications significantly associated to disability werejoint pain, urinary infections, intestinal problems and autonomic disreflexia. None of the variablesrelated to rehabilitation services showed significant association with disability. The disability degreeexhibited significant differences in favor of the groups that received the following services: assistivedevices supply and vocational, job or educational counseling. Conclusions: The best predictiondisability model in adults with sci with more than six months post-injury was built with variablesof injury evolution time, AIS sensory score and injury-related unemployment.

An artificial economy based on reinforcement learning and agent based modeling

In this paper, we employ techniques from artificial intelligence such as reinforcement learning and agent based modeling as building blocks of a computational model for an economy based on conventions. First we model the interaction among firms in the private sector. These firms behave in an information environment based on conventions, meaning that a firm is likely to behave as its neighbors if it observes that their actions lead to a good pay off. On the other hand, we propose the use of reinforcement learning as a computational model for the role of the government in the economy, as the agent that determines the fiscal policy, and whose objective is to maximize the growth of the economy. We present the implementation of a simulator of the proposed model based on SWARM, that employs the SARSA(λ) algorithm combined with a multilayer perceptron as the function approximation for the action value function.

Industrial Organization: a Contract Based approach

Industrial Organization, a Contract Based approach (aka IOCB) offers an extensive and an up-to-date panorama of Industrial Organization. It is aimed at advanced undergraduates, graduates, academics and practitioners with an interest in the field. The analysis of market interactions, business strategies and public policy is performed using the standard framework of game theory and the recent advances of contract theory and information economics

Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

Establishing a community of practice of researchers, practitioners, policy-makers and communities to sustainably manage environmental health risks in Ecuador

The Sustainably Managing Environmental Health Risk in Ecuador project was launched in 2004 as a partnership linking a large Canadian university with leading Cuban and Mexican institutes to strengthen the capacities of four Ecuadorian universities for leading community-based learning and research in areas as diverse as pesticide poisoning, dengue control, water and sanitation, and disaster preparedness. By 2009, train-the-trainer project initiation involved 27 participatory action research Master’s theses in 15 communities where 1200 community learners participated in the implementation of associated interventions. This led to establishment of innovative Ecuadorian-led master’s and doctoral programs, and a Population Health Observatory on Collective Health, Environment and Society for the Andean region based at the Universidad Andina Simon Bolivar. Building on this network, numerous initiatives were begun, such as an internationally funded research project to strengthen dengue control in the coastal community of Machala, and establishment of a local community eco-health centre focusing on determinants of health near Cuenca. Alliances of academic and non-academic partners from the South and North provide a promising orientation for learning together about ways of addressing negative trends of development. Assessing the impacts and sustainability of such processes, however, requires longer term monitoring of results and related challenges.

Compositional effects on productivity, labour cost and export adjustments. Bruegel Policy Contribution 2012/11, June 2012

Sectoral shifts, such as shrinkage of low labour productivity and the low-wage construction sector, can lead to apparent increased aggregate average labour productivity and average wages, especially when capital intensity differs across sectors. For 11 main sectors and 13 manufacturing sub-sectors, we quantify the compositional effects on productivity, wages and unit labour costs (ULCs) based and real effective exchange rates (REER), for 24 EU countries. Compositional effects are greatest in Ireland, where the pharmaceutical sector drives the growth of output and productivity, but other sectors have suffered greatly and have not yet recovered. Our new ULC-REER measurements, which are free from compositional effects, correlate well with export performance. Among the countries facing the most severe external adjustment challenges, Lithuania, Portugal and Ireland have been the most successful based on five indicators, and Latvia, Estonia and Greece the least successful. There is evidence of downward wage flexibility in some countries, but wage cuts have corrected just a small fraction of pre-crisis wage rises and came with massive reductions in employment even in the business sector excluding construction and real estate, highlighting the difficulty of adjusting wages downward.

Pre-empting technical barriers in the Single Market. CEPS Policy Brief No. 277, 11 July 2012

Effective enforcement and compliance with EU law is not just a legal necessity, it is also of economic interest since the potential of the Single Market will be fully exploited. Enforcement barriers generate unjustified costs and hindrances or uncertainty for cross-border business and might deprive consumers from receiving the full benefit of greater choice and/or cheaper offers. The EU has developed several types of enforcement efforts (preventive initiatives, pre-infringement initiatives and formal infringement procedures). More recently, the emphasis has been placed on effective prevention. This CEPS Policy Brief analyses the functioning of one preventive mechanism (the 98/34 Directive) and assesses its potential to detect and prevent technical or other barriers in the course of the last 25 years. Based on an empirical approach, it shows that this amazing mechanism has successfully prevented thousands of new technical barriers from arising in the internal goods market.

Can Italy and Spain survive rates of 6-7%? CEPS Policy Brief No. 279, 27 July 2012

The sentiment that the euro is now in real danger is based in large part on the widespread conviction that interest rates of 6-7% are simply unsustainable for both Italy and Spain., After taking a closer look at the fundamentals, however, Daniel Gros concludes in this new Policy Brief that both countries should be able to live with this level of interest rates for quite some time, but only if they mobilize domestic savings, which remain strong in both countries. For Spain, some debt/equity swaps are also needed.

A sovereign wealth fund to lift Germany's curse of excess savings. CEPS Policy Brief No. 280, 28 August 2012

As an alternative to the present system of intermediation of the German savings surplus, this paper suggests that the risk-adjusted rate of return could be improved by creating a sovereign wealth fund for Germany (designated DESWF), which could invest excess German savings globally. Such a DESWF would offer German savers a secure vehicle paying a guaranteed positive minimum real interest rate, with a top-up when real investment returns allowed. The vehicle would invest the funds in a portfolio that is highly diversified by geography and asset classes. Positive real returns can be expected in the long run based on positive real global growth. Since, in this case, a significant amount of funds would flow outside the euro area, the euro would depreciate, which would help crisis countries presently struggling to revive growth through exports and to close their external deficits so as to recoup their international credit-worthiness. Target imbalances would gradually disappear and German claims abroad would move from nominal claims on the ECB to diversified real and nominal claims on various private and public foreign entities in a variety of asset classes.