Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Graphical simulators for AUV development

A long development time is needed from the design to the implementation of an AUV. During the first steps, simulation plays an important role, since it allows for the development of preliminary versions of the control system to be integrated. Once the robot is ready, the control systems are implemented, tuned and tested. The use of a real-time simulator can help closing the gap between off-line simulation and real testing using the already implemented robot. When properly interfaced with the robot hardware, a real-time graphical simulation with a "hardware in the loop" configuration, can allow for the testing of the implemented control system running in the actual robot hardware. Hence, the development time is drastically reduced. These paper overviews the field of graphical simulators used for AUV development proposing a classification. It also presents NEPTUNE, a multi-vehicle, real-time, graphical simulator based on OpenGL that allows hardware in the loop simulations

Martin Heidegger

Nota biogràfica sobre el filòsof Martin Heidegger on es fa, per una banda, un breu repàs dels principals fets de la seva vida com a filòsof i, per una altra banda, un breu repàs de la seva obra

Aprenred: red interdisciplinar para el estudio y aplicación del abp en la universidad de zaragoza

AprenRED (http://aprenred.unizar.es) es una red interdisciplinar, formada por 29 profesores de la Universidad de Zaragoza de 14 Departamentos, que imparten su docencia en las Facultades de Veterinaria, Derecho, Ciencias de la Salud y el Deporte, Ciencias Económicas y Empresariales, Centro Politécnico Superior y EUITIZ. El objetivo principal es desarrollar y consolidar la metodología del ABP como herramienta docente de uso cotidiano en la docencia en la Universidad. Partiendo de las experiencias individuales, este grupo pretende servir de foro para favorecer la colaboración y el intercambio de experiencias y discutir sobre aspectos como la elaboración del problema o la gestión del método

Evaluación de la utilización de un modelo inanimado para la extracción de sangre por parte de los estudiantes de patología general de veterinaria

Debido a las restricciones que impone la legislación de experimentación y bienestar animal referente al uso de animales vivos en las prácticas veterinarias, durante el curso 2009-2010 se desarrolló la fabricación de un modelo inanimado para la obtención de muestras de sangre en las prácticas de Patología General y Propedéutica Clínica. En el curso académico 2010-11 se ha puesto en funcionamiento estos modelos en las prácticas de Patología General y se ha evaluado el grado de satisfacción de los estudiantes. Los resultados han sido altamente satisfactorios, tanto para los estudiantes como para el profesorado que ha impartido dichas sesiones prácticas

Propuesta para la intervención psicopedagógica con vista a la mejora de la orientación en el contexto del Centro Universitario de Sancti Spíritus "José Martí Pérez"

La investigación se propuso identificar las posibilidades que ofrece el marco universitario actual para la práctica de la orientación y la tutoría en Cuba y asesorar el diseño e implementación de un programa de orientación a ser aplicado en el Centro Universitario de Sancti Spíritus. El abordaje de la problemática desde una perspectiva interpretativa, con un marco metodológico cualitativo, basado en la investigación - acción como estrategia de intervención y el asesoramiento colaborativo como modelo de interrelación de las partes implicadas, nos posibilitó conocer que las modificaciones en las concepciones y políticas educativas cubanas están creando un espacio en el que la tutoría se convierte en un instrumento capaz de facilitar el autoaprendizaje desarrollador, la cual desarrollada sobre la base de un asesoramiento psicopedagógico colaborativo propicia un alto grado de participación e implicación de los profesores en sus decisiones y les concede un importante nivel de autonomía en sus prácticas.