Estudi de comportaments socials d'aixams robòtics amb aplicació a la neteja d'espais no estructurats

La intel·ligència d’eixams és una branca de la intel·ligència artificial que està agafant molta força en els últims temps, especialment en el camp de la robòtica. En aquest projecte estudiarem el comportament social sorgit de les interaccions entre un nombre determinat de robots autònoms en el camp de la neteja de grans superfícies. Un cop triat un escenari i un robot que s’ajustin als requeriments del projecte, realitzarem una sèrie de simulacions a partir de diferents polítiques de cerca que ens permetran avaluar el comportament dels robots per unes condicions inicials de distribució dels robots i zones a netejar. A partir dels resultats obtinguts serem capaços de determinar quina configuració genera millors resultats.

An empirical analysis of a merger between a network and low-cost airlines

We use a difference-in-difference estimator to examine the effects of a merger involving three airlines. The novelty lies in the examination of this operation in two distinct scenarios: (1) on routes where two low-cost carriers and (2) on routes where a network and one of the low-cost airlines had previously been competing. We report a reduction in frequencies but no substantial effect on prices in the first scenario, while in the second we report an increase in prices but no substantial effect on frequencies. These results may be attributed to the differences in passenger types flying on these routes.

Diseño y puesta en marcha de un laboratorio remoto de robótica

Este proyecto surge de la iniciativa de mejorar la calidad docente de las prácticas en la asignatura Robótica y Automatización Industrial impartida en la ETSE (Escola Tècnica Superior d’Enginyeria) de la UAB, mediante un sistema innovador. El objetivo es sustituir las actuales prácticas, basadas en la realización de simulaciones en entorno MATLAB para verificar las ecuaciones que gobiernan a los robots manipuladores, por un entorno de prácticas más atractivo consistente en un robot manipulador real, que podrá ser programado para la realización de tareas de PPO (Pick and Place Operation).

Disseny d'un sistema de gestió per a una cambra de congelació robotitzada de masses congelades.

Alguns dels grans fabricants del sector de les masses congelades fan servir robots per a l'emmagatzematge, servei i gestió dels productes de les cambres de congelació. Aquests robots només emmagatzemen i serveixen palets de caixes d'un mateix tipus de producte; així, doncs, només són útils quan el client és un gran consumidor, com un distribuïdor o una delegació, però no un petit comerç, com un restaurant o una fleca, els quals també són clients potencials del fabricant. Per tant, el problema es planteja en el moment de servir els clients que tenen problemes d'emmagatzematge, els quals es veuen obligats a fer comandes de diversos productes, que, a més, en total no fan ni de bon tros un palet sencer. A partir d'aquí la idea és dissenyar un sistema, emprant la tècnica de l'orientació a objectes (O-O), que gestioni un conjunt de robots que, a més d'emmagatzemar i gestionar una cambra de congelació, siguin capaços de servir les comandes fent una selecció de caixes de productes concrets, és a dir, les que sol·licita el client que ha fet la comanda per caixes i no per palets. Per qüestions d'organització i d'estocs, aquests robots han de ser independents dels altres, és a dir, han de treballar en una cambra independent de la dels robots que treballen per palets (actualment aquesta feina la fan treballadors dins una cambra independent de la dels robots).

An hybrid methodology for RL-based behavior coordination in a target following mission with an AUV

Proposes a behavior-based scheme for high-level control of autonomous underwater vehicles (AUVs). Two main characteristics can be highlighted in the control scheme. Behavior coordination is done through a hybrid methodology, which takes in advantages of the robustness and modularity in competitive approaches, as well as optimized trajectories

Differential epipolar constraint in mobile robot egomotion estimation

The estimation of camera egomotion is a well established problem in computer vision. Many approaches have been proposed based on both the discrete and the differential epipolar constraint. The discrete case is mainly used in self-calibrated stereoscopic systems, whereas the differential case deals with a unique moving camera. The article surveys several methods for mobile robot egomotion estimation covering more than 0.5 million samples using synthetic data. Results from real data are also given

Hybrid coordination of reinforcement learning-based behaviors for AUV control

This paper proposes a hybrid coordination method for behavior-based control architectures. The hybrid method takes advantages of the robustness and modularity in competitive approaches as well as optimized trajectories in cooperative ones. This paper shows the feasibility of applying this hybrid method with a 3D-navigation to an autonomous underwater vehicle (AUV). The behaviors are learnt online by means of reinforcement learning. A continuous Q-learning implemented with a feed-forward neural network is employed. Realistic simulations were carried out. The results obtained show the good performance of the hybrid method on behavior coordination as well as the convergence of the behaviors

Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Vision-based localization of an underwater robot in a structured environment

This paper presents a vision-based localization approach for an underwater robot in a structured environment. The system is based on a coded pattern placed on the bottom of a water tank and an onboard down looking camera. Main features are, absolute and map-based localization, landmark detection and tracking, and real-time computation (12.5 Hz). The proposed system provides three-dimensional position and orientation of the vehicle along with its velocity. Accuracy of the drift-free estimates is very high, allowing them to be used as feedback measures of a velocity-based low-level controller. The paper details the localization algorithm, by showing some graphical results, and the accuracy of the system

A behavior-based scheme using reinforcement learning for autonomous underwater vehicles

This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q_learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q_learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs

An approach to vision-based station keeping for an unmanned underwater vehicle

This paper presents an automatic vision-based system for UUV station keeping. The vehicle is equipped with a down-looking camera, which provides images of the sea-floor. The station keeping system is based on a feature-based motion detection algorithm, which exploits standard correlation and explicit textural analysis to solve the correspondence problem. A visual map of the area surveyed by the vehicle is constructed to increase the flexibility of the system, allowing the vehicle to position itself when it has lost the reference image. The testing platform is the URIS underwater vehicle. Experimental results demonstrating the behavior of the system on a real environment are presented

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task