Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

A behavior-based scheme using reinforcement learning for autonomous underwater vehicles

This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q_learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q_learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs

Infohabitants: laboratori per la simulació d’ infohabitants com agents recomanadors (SIRA)

L’ objectiu del projecte és la implementació d’un simulador de sistema de recomanació que permeti estudiar algoritmes de dissociació entre agent-recomanador i usuari, combinant-los amb diverses tècniques de recomanació i fent servir infohabitants com Agents Recomanadors i veure com treballen en un sistema recomanador

Sistemas inteligentes para mundos virtuales

El desarrollo de mundos virtuales inteligentes requiere el conocimiento de áreas tan diversas como la realidad virtual, la inteligencia artificial, la psicología, la sociología y la física. El documento que se desarrolla a continuación recoge las nociones básicas para entender lo que representa dicho desarrollo, el estado del arte de varias de las técnicas y modelos utilizados en algunas de estas áreas y sus posibles aplicaciones, además de una posible solución para su implementación.

Desenvolupament d'un agent intel·ligent dins el marc de la competició TAC SCM

Aquest projecte està emmarcat dins el grup eXiT d’Intel•lig`encia Artificial del Departament d’Electrònica i Automàtica (EIA) de la Universitat de Girona. Pertany a l’àmbit de la Intel•ligència Artificial i, concretament, en l’apartat d’agents intel•ligents. En el nostre cas, tractarem el desenvolupament d’un agent intel•ligent en un entorn determinat, el de la gestió d’una cadena de producció. Amb l’objectiu de proporcionar un marc experimental on provar diferents tecnologies de suport a la gestió de la cadena de producció, la comunitat d’investigadors va proposar una competició internacional: la Trading Agent Competiton (TAC). En aquesta competició existeixen diferents modalitats. En particular, la Swedish Institution of Computer Science (SICS), juntament amb la Carnegie Mellon University de Pittsburg, Minnesotta, van proposar al 2003 un escenari de muntatge de PC’s basat en el proveïment de recursos, l’embalatge de PC’s i les ventes a clients. Aquesta modalitat és coneguda com aTAC-SCM (Supply Chain Management)

A Negotiation Style Recommender Based on Computational Ecology in Open Negotiation Environments

The system described herein represents the first example of a recommender system in digital ecosystems where agents negotiate services on behalf of small companies. The small companies compete not only with price or quality, but with a wider service-by-service composition by subcontracting with other companies. The final result of these offerings depends on negotiations at the scale of millions of small companies. This scale requires new platforms for supporting digital business ecosystems, as well as related services like open-id, trust management, monitors and recommenders. This is done in the Open Negotiation Environment (ONE), which is an open-source platform that allows agents, on behalf of small companies, to negotiate and use the ecosystem services, and enables the development of new agent technologies. The methods and tools of cyber engineering are necessary to build up Open Negotiation Environments that are stable, a basic condition for predictable business and reliable business environments. Aiming to build stable digital business ecosystems by means of improved collective intelligence, we introduce a model of negotiation style dynamics from the point of view of computational ecology. This model inspires an ecosystem monitor as well as a novel negotiation style recommender. The ecosystem monitor provides hints to the negotiation style recommender to achieve greater stability of an open negotiation environment in a digital business ecosystem. The greater stability provides the small companies with higher predictability, and therefore better business results. The negotiation style recommender is implemented with a simulated annealing algorithm at a constant temperature, and its impact is shown by applying it to a real case of an open negotiation environment populated by Italian companies

Towards smart user models in open environments

Our work is concerned with user modelling in open environments. Our proposal then is the line of contributions to the advances on user modelling in open environments thanks so the Agent Technology, in what has been called Smart User Model. Our research contains a holistic study of User Modelling in several research areas related to users. We have developed a conceptualization of User Modelling by means of examples from a broad range of research areas with the aim of improving our understanding of user modelling and its role in the next generation of open and distributed service environments. This report is organized as follow: In chapter 1 we introduce our motivation and objectives. Then in chapters 2, 3, 4 and 5 we provide the state-of-the-art on user modelling. In chapter 2, we give the main definitions of elements described in the report. In chapter 3, we present an historical perspective on user models. In chapter 4 we provide a review of user models from the perspective of different research areas, with special emphasis on the give-and-take relationship between Agent Technology and user modelling. In chapter 5, we describe the main challenges that, from our point of view, need to be tackled by researchers wanting to contribute to advances in user modelling. From the study of the state-of-the-art follows an exploratory work in chapter 6. We define a SUM and a methodology to deal with it. We also present some cases study in order to illustrate the methodology. Finally, we present the thesis proposal to continue the work, together with its corresponding work scheduling and temporalisation

Intelligent task-level grasp mapping for robot control

In the future, robots will enter our everyday lives to help us with various tasks.For a complete integration and cooperation with humans, these robots needto be able to acquire new skills. Sensor capabilities for navigation in real humanenvironments and intelligent interaction with humans are some of the keychallenges.Learning by demonstration systems focus on the problem of human robotinteraction, and let the human teach the robot by demonstrating the task usinghis own hands. In this thesis, we present a solution to a subproblem within thelearning by demonstration field, namely human-robot grasp mapping. Robotgrasping of objects in a home or office environment is challenging problem.Programming by demonstration systems, can give important skills for aidingthe robot in the grasping task.The thesis presents two techniques for human-robot grasp mapping, directrobot imitation from human demonstrator and intelligent grasp imitation. Inintelligent grasp mapping, the robot takes the size and shape of the object intoconsideration, while for direct mapping, only the pose of the human hand isavailable.These are evaluated in a simulated environment on several robot platforms.The results show that knowing the object shape and size for a grasping taskimproves the robot precision and performance

Desarrollo y gestión del videojuego The Condemned

The Condemned es un juego de lucha en dos dimensiones desarrollado en Flash CS4 y ActionScript 3. El juego consta de cuatro pantallas, en cada una de ellas el jugador se enfrenta a un enemigo controlado por el ordenador a través de una inteligencia artificial. En la creación de este videojuego se ha pasado por todas las fases de desarrollo: diseño gráfico de personajes y escenarios, programación y control de errores.

Embedding objects into Matlab/Simulink for process supervision

This paper introduces how artificial intelligence technologies can be integrated into a known computer aided control system design (CACSD) framework, Matlab/Simulink, using an object oriented approach. The aim is to build a framework to aid supervisory systems analysis, design and implementation. The idea is to take advantage of an existing CACSD framework, Matlab/Simulink, so that engineers can proceed: first to design a control system, and then to design a straightforward supervisory system of the control system in the same framework. Thus, expert systems and qualitative reasoning tools are incorporated into this popular CACSD framework to develop a computer aided supervisory system design (CASSD) framework. Object-variables an introduced into Matlab/Simulink for sharing information between tools

Sistemas heterogéneos adaptativos basados en el contexto

Delivery context-aware adaptative heterogenous systems. Currently, many types of devices that have gained access to the network is large and diverse. The different capabilities and characteristics of them, in addition to the different characteristics and preferences of users, have generated a new goal to overcome: how to adapt the contents taking into account this heterogeneity, known as the “delivery context.” The concepts of adaptation and accessibility have been widely discussed and have resulted in many proposals, standards and techniques designed to solve the problem, making it necessary to refine the analysis of the issue to be considered in the process of adaptation. We present a tour of the various proposals and standards that have marked the area of heterogeneous systems works, and others who have worked since the real-time interaction through agents based platforms. All targeted to solve a common goal: the delivery context

Detecció de quadres des d'un dispositiu mòbil

En aquest treball es presenta una aplicació mòbil que, mitjançant l'ús de classificadors prèviament entrenats a un ordinador emprant l'algorisme Random Ferns, és capaç de detectar en temps real, i mitjançant la càmera del dispositiu mòbil, quadres i diferents parts dels quadres detectats. La informació dels elements detectats es presenta per pantalla, identificant el nom i autor de l'obra d'art, i assenyalant quines parts s'han detectat. L'usuari pot polsar sobre una de les parts assenyalades per tal de veure la informació relacionada.

Goal recognition over POMDPs: inferring the intention of a POMDP agent

Plan recognition is the problem of inferring the goals and plans of an agent from partial observations of her behavior. Recently, it has been shown that the problem can be formulated and solved usingplanners, reducing plan recognition to plan generation.In this work, we extend this model-basedapproach to plan recognition to the POMDP setting, where actions are stochastic and states are partially observable. The task is to infer a probability distribution over the possible goals of an agent whose behavior results from a POMDP model. The POMDP model is shared between agent and observer except for the true goal of the agent that is hidden to the observer. The observations are action sequences O that may contain gaps as some or even most of the actions done by the agent may not be observed. We show that the posterior goal distribution P(GjO) can be computed from the value function VG(b) over beliefs b generated by the POMDPplanner for each possible goal G. Some extensionsof the basic framework are discussed, and a numberof experiments are reported.

Computing infinite plans for LTL goals using a classical planner

Classical planning has been notably successful in synthesizing finite plans to achieve states where propositional goals hold. In the last few years, classical planning has also been extended to incorporate temporally extended goals, expressed in temporal logics such as LTL, to impose restrictions on the state sequences generated by finite plans. In this work, we take the next step and consider the computation of infinite plans for achieving arbitrary LTL goals. We show that infinite plans can also be obtained efficiently by calling a classical planner once over a classical planning encoding that represents and extends the composition of the planningdomain and the B¨uchi automaton representingthe goal. This compilation scheme has been implemented and a number of experiments are reported.

Spacial debris collector robot : An AI-based and autonomous debris collector satellite simulation

In this paper the core functions of an artificial intelligence (AI) for controlling a debris collector robot are designed and implemented. Using the robot operating system (ROS) as the base of this work a multi-agent system is built with abilities for task planning.