Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

A behavior-based scheme using reinforcement learning for autonomous underwater vehicles

This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q_learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q_learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs

Adaptive intelligent agent approach to guide the web navigation on the PLAN-G distance learning platform

Hypermedia systems based on the Web for open distance education are becoming increasinglypopular as tools for user-driven access learning information. Adaptive hypermedia is a new direction in research within the area of user-adaptive systems, to increase its functionality by making it personalized [Eklu 961. This paper sketches a general agents architecture to include navigationaladaptability and user-friendly processes which would guide and accompany the student during hislher learning on the PLAN-G hypermedia system (New Generation Telematics Platform to Support Open and Distance Learning), with the aid of computer networks and specifically WWW technology [Marz 98-1] [Marz 98-2]. The PLAN-G actual prototype is successfully used with some informatics courses (the current version has no agents yet). The propased multi-agent system, contains two different types of adaptive autonomous software agents: Personal Digital Agents {Interface), to interacl directly with the student when necessary; and Information Agents (Intermediaries), to filtrate and discover information to learn and to adapt navigation space to a specific student

Dissolution of Dynamic Electronic Institutions, a first approach: Relevant factors and causes

In the context of the digital business ecosystems, small organizations cooperate between them in order to achieve common goals or offer new services for expanding their markets. There are different approaches for these cooperation models such as virtual enterprises, virtual organizations or dynamic electronic institutions which in their lifecycle have in common a dissolution phase. However this phase has not been studied deeply in the current literature and it lacks formalization. In this paper a first approach for achieving and managing the dissolution phase is proposed, as well as a CBR process in order to support it in a multi-agent system

Infohabitants: laboratori per la simulació d’ infohabitants com agents recomanadors (SIRA)

L’ objectiu del projecte és la implementació d’un simulador de sistema de recomanació que permeti estudiar algoritmes de dissociació entre agent-recomanador i usuari, combinant-los amb diverses tècniques de recomanació i fent servir infohabitants com Agents Recomanadors i veure com treballen en un sistema recomanador

Sistemas inteligentes para mundos virtuales

El desarrollo de mundos virtuales inteligentes requiere el conocimiento de áreas tan diversas como la realidad virtual, la inteligencia artificial, la psicología, la sociología y la física. El documento que se desarrolla a continuación recoge las nociones básicas para entender lo que representa dicho desarrollo, el estado del arte de varias de las técnicas y modelos utilizados en algunas de estas áreas y sus posibles aplicaciones, además de una posible solución para su implementación.

Pattern recognition of sleep in rodents using piezoelectric signals generated by gross body movements.

Current research on sleep using experimental animals is limited by the expense and time-consuming nature of traditional EEG/EMG recordings. We present here an alternative, noninvasive approach utilizing piezoelectric films configured as highly sensitive motion detectors. These film strips attached to the floor of the rodent cage produce an electrical output in direct proportion to the distortion of the material. During sleep, movement associated with breathing is the predominant gross body movement and, thus, output from the piezoelectric transducer provided an accurate respiratory trace during sleep. During wake, respiratory movements are masked by other motor activities. An automatic pattern recognition system was developed to identify periods of sleep and wake using the piezoelectric generated signal. Due to the complex and highly variable waveforms that result from subtle postural adjustments in the animals, traditional signal analysis techniques were not sufficient for accurate classification of sleep versus wake. Therefore, a novel pattern recognition algorithm was developed that successfully distinguished sleep from wake in approximately 95% of all epochs. This algorithm may have general utility for a variety of signals in biomedical and engineering applications. This automated system for monitoring sleep is noninvasive, inexpensive, and may be useful for large-scale sleep studies including genetic approaches towards understanding sleep and sleep disorders, and the rapid screening of the efficacy of sleep or wake promoting drugs.

A multidimensional segmentation evaluation for medical image data.

Evaluation of segmentation methods is a crucial aspect in image processing, especially in the medical imaging field, where small differences between segmented regions in the anatomy can be of paramount importance. Usually, segmentation evaluation is based on a measure that depends on the number of segmented voxels inside and outside of some reference regions that are called gold standards. Although some other measures have been also used, in this work we propose a set of new similarity measures, based on different features, such as the location and intensity values of the misclassified voxels, and the connectivity and the boundaries of the segmented data. Using the multidimensional information provided by these measures, we propose a new evaluation method whose results are visualized applying a Principal Component Analysis of the data, obtaining a simplified graphical method to compare different segmentation results. We have carried out an intensive study using several classic segmentation methods applied to a set of MRI simulated data of the brain with several noise and RF inhomogeneity levels, and also to real data, showing that the new measures proposed here and the results that we have obtained from the multidimensional evaluation, improve the robustness of the evaluation and provides better understanding about the difference between segmentation methods.

Desenvolupament d'un agent intel·ligent dins el marc de la competició TAC SCM

Aquest projecte està emmarcat dins el grup eXiT d’Intel•lig`encia Artificial del Departament d’Electrònica i Automàtica (EIA) de la Universitat de Girona. Pertany a l’àmbit de la Intel•ligència Artificial i, concretament, en l’apartat d’agents intel•ligents. En el nostre cas, tractarem el desenvolupament d’un agent intel•ligent en un entorn determinat, el de la gestió d’una cadena de producció. Amb l’objectiu de proporcionar un marc experimental on provar diferents tecnologies de suport a la gestió de la cadena de producció, la comunitat d’investigadors va proposar una competició internacional: la Trading Agent Competiton (TAC). En aquesta competició existeixen diferents modalitats. En particular, la Swedish Institution of Computer Science (SICS), juntament amb la Carnegie Mellon University de Pittsburg, Minnesotta, van proposar al 2003 un escenari de muntatge de PC’s basat en el proveïment de recursos, l’embalatge de PC’s i les ventes a clients. Aquesta modalitat és coneguda com aTAC-SCM (Supply Chain Management)

Curriculum sequencing for an e-learning system based on learning styles

This work shows the use of adaptation techniques involved in an e-learning system that considers students' learning styles and students' knowledge states. The mentioned e-learning system is built on a multiagent framework designed to examine opportunities to improve the teaching and to motivate the students to learn what they want in a user-friendly and assisted environment

Desarrollo y gestión del videojuego The Condemned

The Condemned es un juego de lucha en dos dimensiones desarrollado en Flash CS4 y ActionScript 3. El juego consta de cuatro pantallas, en cada una de ellas el jugador se enfrenta a un enemigo controlado por el ordenador a través de una inteligencia artificial. En la creación de este videojuego se ha pasado por todas las fases de desarrollo: diseño gráfico de personajes y escenarios, programación y control de errores.

Ambulatory monitoring of physical activities in patients with Parkinson's disease.

A new ambulatory method of monitoring physical activities in Parkinson's disease (PD) patients is proposed based on a portable data-logger with three body-fixed inertial sensors. A group of ten PD patients treated with subthalamic nucleus deep brain stimulation (STN-DBS) and ten normal control subjects followed a protocol of typical daily activities and the whole period of the measurement was recorded by video. Walking periods were recognized using two sensors on shanks and lying periods were detected using a sensor on trunk. By calculating kinematics features of the trunk movements during the transitions between sitting and standing postures and using a statistical classifier, sit-to-stand (SiSt) and stand-to-sit (StSi) transitions were detected and separated from other body movements. Finally, a fuzzy classifier used this information to detect periods of sitting and standing. The proposed method showed a high sensitivity and specificity for the detection of basic body postures allocations: sitting, standing, lying, and walking periods, both in PD patients and healthy subjects. We found significant differences in parameters related to SiSt and StSi transitions between PD patients and controls and also between PD patients with and without STN-DBS turned on. We concluded that our method provides a simple, accurate, and effective means to objectively quantify physical activities in both normal and PD patients and may prove useful to assess the level of motor functions in the latter.

Detecció de quadres des d'un dispositiu mòbil

En aquest treball es presenta una aplicació mòbil que, mitjançant l'ús de classificadors prèviament entrenats a un ordinador emprant l'algorisme Random Ferns, és capaç de detectar en temps real, i mitjançant la càmera del dispositiu mòbil, quadres i diferents parts dels quadres detectats. La informació dels elements detectats es presenta per pantalla, identificant el nom i autor de l'obra d'art, i assenyalant quines parts s'han detectat. L'usuari pot polsar sobre una de les parts assenyalades per tal de veure la informació relacionada.

Goal recognition over POMDPs: inferring the intention of a POMDP agent

Plan recognition is the problem of inferring the goals and plans of an agent from partial observations of her behavior. Recently, it has been shown that the problem can be formulated and solved usingplanners, reducing plan recognition to plan generation.In this work, we extend this model-basedapproach to plan recognition to the POMDP setting, where actions are stochastic and states are partially observable. The task is to infer a probability distribution over the possible goals of an agent whose behavior results from a POMDP model. The POMDP model is shared between agent and observer except for the true goal of the agent that is hidden to the observer. The observations are action sequences O that may contain gaps as some or even most of the actions done by the agent may not be observed. We show that the posterior goal distribution P(GjO) can be computed from the value function VG(b) over beliefs b generated by the POMDPplanner for each possible goal G. Some extensionsof the basic framework are discussed, and a numberof experiments are reported.