Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

Learning and adaptation in physical heterogeneous teams of robots

In this paper we present a novel approach to assigning roles to robots in a team of physical heterogeneous robots. Its members compete for these roles and get rewards for them. The rewards are used to determine each agent’s preferences and which agents are better adapted to the environment. These aspects are included in the decision making process. Agent interactions are modelled using the concept of an ecosystem in which each robot is a species, resulting in emergent behaviour of the whole set of agents. One of the most important features of this approach is its high adaptability. Unlike some other learning techniques, this approach does not need to start a whole exploitation process when the environment changes. All this is exemplified by means of experiments run on a simulator. In addition, the algorithm developed was applied as applied to several teams of robots in order to analyse the impact of heterogeneity in these systems

Multidimensional Adaptations for Open Learning Management Systems

Our work is focused on alleviating the workload for designers of adaptive courses on the complexity task of authoring adaptive learning designs adjusted to specific user characteristics and the user context. We propose an adaptation platform that consists in a set of intelligent agents where each agent carries out an independent adaptation task. The agents apply machine learning techniques to support the user modelling for the adaptation process

Enhancing learning beyond the classroom: blogging and other social networking tools

Blogging has become one of the key ingredients of the so-called socials networks. This phenomenon has indeed invaded the world of education. Connections between people, comments on each other posts, and assessment of innovation are usually interesting characteristics of blogs related to students and scholars. Blogs have become a kind of new form of authority, bringing about (divergent) discussions which lead to creation of knowledge. The use of blogs as an innovative, educational tool is not at all new. However, their use in universities is not very widespread yet. Blogging for personal affairs is rather commonplace, but blogging for professional affairs – teaching, research and service, is scarce, despite the availability of ready-to-use, free tools. Unfortunately, Information Society has not reached yet enough some universities: not only are (student) blogs scarcely used as an educational tool, but it is quite rare to find a blog written by University professors. The Institute of Computational Chemistry of the University of Girona and the Department of Chemistry of the Universitat Autònoma de Barcelona has joined forces to create “InnoCiència”, a new Group on Digital Science Communitation. This group, formed by ca. ten researchers, has promoted the use of blogs, twitters. wikis and other tools of Web 2.0 in activities in Catalonia concerning the dissemination of Science, like Science Week, Open Day or Researchers’ Night. Likewise, its members promote use of social networking tools in chemistry- and communication-related courses. This communication explains the outcome of social-network experiences with teaching undergraduate students and organizing research communication events. We provide live, hands-on examples and interactive ground to show how blogs and twitters can be used to enhance the yield of teaching and research. Impact of blogging and other social networking tools on the outcome of the learning process is very depending on the target audience and the environmental conditions. A few examples are provided and some proposals to use these techniques efficiently to help students are hinted

Curriculum sequencing for an e-learning system based on learning styles

This work shows the use of adaptation techniques involved in an e-learning system that considers students' learning styles and students' knowledge states. The mentioned e-learning system is built on a multiagent framework designed to examine opportunities to improve the teaching and to motivate the students to learn what they want in a user-friendly and assisted environment