Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Towards a real-time vision-based navigation system for a small-class UUV

This paper deals with the problem of navigation for an unmanned underwater vehicle (UUV) through image mosaicking. It represents a first step towards a real-time vision-based navigation system for a small-class low-cost UUV. We propose a navigation system composed by: (i) an image mosaicking module which provides velocity estimates; and (ii) an extended Kalman filter based on the hydrodynamic equation of motion, previously identified for this particular UUV. The obtained system is able to estimate the position and velocity of the robot. Moreover, it is able to deal with visual occlusions that usually appear when the sea bottom does not have enough visual features to solve the correspondence problem in a certain area of the trajectory

A new approach to real time traffic sign recognition based on colour information

The automatic interpretation of conventional traffic signs is very complex and time consuming. The paper concerns an automatic warning system for driving assistance. It does not interpret the standard traffic signs on the roadside; the proposal is to incorporate into the existing signs another type of traffic sign whose information will be more easily interpreted by a processor. The type of information to be added is profuse and therefore the most important object is the robustness of the system. The basic proposal of this new philosophy is that the co-pilot system for automatic warning and driving assistance can interpret with greater ease the information contained in the new sign, whilst the human driver only has to interpret the "classic" sign. One of the codings that has been tested with good results and which seems to us easy to implement is that which has a rectangular shape and 4 vertical bars of different colours. The size of these signs is equivalent to the size of the conventional signs (approximately 0.4 m2). The colour information from the sign can be easily interpreted by the proposed processor and the interpretation is much easier and quicker than the information shown by the pictographs of the classic signs

Disseny i construcció d'un motor 3D en temps real per la Gameboy Advance

El projecte desenvolupat ha tractat l’estudi i disseny d’un motor 3D interactiu a la consolaGame Boy Advance (GBA). La GBA disposa d’un processador ARM7TDMI a 16’78Mhz i no disposa de operacions 3D per-hardware, és una consola lenta en comparació lesque podem trobar al mercat d’avui en dia. Aquest treball, va partir de la construcció d’un prototipus ray-casting per-columna. Després,vàrem adaptar-lo a una estructura de portals i sectors. Més tard,es va introduir el mapeig de sostre/terra i de paisatges. Per últim,vàrem introduir efectes a la renderització per donar més realismeal recorregut del món, com il·luminació, objectes, etc.Tot i que es va estudiar l’arquitectura d’un motor eficient, no estenia prou per arribar a tenir un motor interactiu. Una de lestasques més difícils va ser la part de optimització. Peraconseguir-ho s’ha hagut de substituir operacions a temps realcostoses a temps de execució, replantejar parts de l’algorisme per fer-lo més eficient, entre altres

Visualització en temps real de multituds

Avui en dia, hi ha moltes pel.lícules o videojocs on apareixen grans quantitats de persones. Sovint és molt costós aconseguir grans quantitats de persones per realitzar les escenes, amb els problemes que això comporta (ja siguin econòmics o d’infraestructura) i la majoria de les vegades resulta inviable. Amb l’arribada de les noves targes gràfiques, és possible calcular aquesta visualització en temps real, donades una sèrie de simplificacions respecte de l’estructuració dels models a visualitzar.El que es pretén amb el GdM no és oferir una visualització molt realista de multituds (com ho faria el Massive Software, per exemple), sinó que el que es vol és la simulació de multituds usant molt menys temps per obtenir uns resultats prou bons per a simular el comportament i el moviment d' una multitud humana.Aquest projecte consta de dues parts: la primera, és la creació del model de persones que formaran la multitud. La segona, és la creació del GdM i la integració d’aquest model per la generació de la multitud

Route optimization and customization using real geographical data in Android mobile devices

Nowadays, there are several services and applications that allow users to locate and move to different tourist areas using a mobile device. These systems can be used either by internet or downloading an application in concrete places like a visitors centre. Although such applications are able to facilitate the location and the search for points of interest, in most cases, these services and applications do not meet the needs of each user. This paper aims to provide a solution by studying the main projects, services and applications, their routing algorithms and their treatment of the real geographical data in Android mobile devices, focusing on the data acquisition and treatment to improve the routing searches in off-line environments.

Real-Time implementation of a blind authentication method using self-synchronous speech watermarking

A blind speech watermarking scheme that meets hard real-time deadlines is presented and implemented. In addition, one of the key issues in these block-oriented watermarking techniques is to preserve the synchronization. Namely, to recover the exact position of each block in the mark extract process. In fact, the presented scheme can be split up into two distinguished parts, the synchronization and the information mark methods. The former is embedded into the time domain and it is fast enough to be run meeting real-time requirements. The latter contains the authentication information and it is embedded into the wavelet domain. The synchronization and information mark techniques are both tunable in order to allow a con gurable method. Thus, capacity, transparency and robustness can be con gured depending on the needs. It makes the scheme useful for professional applications, such telephony authentication or even sending information throw radio applications.

Tractament de la malaltia i mort al budisme tibetà : relació que estableix el practicant budista amb la malaltia en un cas real treballat amb el mètode biogràfic

Tractament de la malaltia i la mort a través del budisme tibetà i de l'experiència d'una practicant budista malalta de càncer. Anàlisis del relat de vida amb l'ajut del mètode biogràfic.

Análisis del proceso de migración de aplicación embebida en tiempo real para estación del segmento de tierra del sistema EGNOS

El TFC describe el proceso de migración HW para una aplicación embebida crítica en tiempo real. El TFC resume todos los pasos del proceso: toma de requisitos, diseño, implementación, test y certificación.

Visualització de pluja realista en temps real

L’objectiu d’aquest PFC és desenvolupar un sistema de pluja per a videojocs i aplicacions de realitat virtual que sigui acurat, tant en el sentit del realisme visual com del seu comportament. El projecte permetrà al desenvolupador de videojocs incorporar a les seves aplicacions, zones de pluja amb diferents intensitats utilitzant el hardware gràfic més modern, per així evitar que aquesta pluja sigui processada per la CPU i per tant pugui alentir el videojoc que està creant. S’han desenvolupat dos sistemes, el sistema d’edició de pluja i el de visualització en temps real

Bandwidth allocation based on real time calculations using the convolution approach

This paper focuses on one of the methods for bandwidth allocation in an ATM network: the convolution approach. The convolution approach permits an accurate study of the system load in statistical terms by accumulated calculations, since probabilistic results of the bandwidth allocation can be obtained. Nevertheless, the convolution approach has a high cost in terms of calculation and storage requirements. This aspect makes real-time calculations difficult, so many authors do not consider this approach. With the aim of reducing the cost we propose to use the multinomial distribution function: the enhanced convolution approach (ECA). This permits direct computation of the associated probabilities of the instantaneous bandwidth requirements and makes a simple deconvolution process possible. The ECA is used in connection acceptance control, and some results are presented

The Coherence of the European Foreign Policy: A Real Barrier or an Academic Term?

The purpose of the present study is to break down, in a general and summarised way, the concept of the coherence of the European Foreign Policy, in order to analyse its meaning, its scope and its implications. This focus on coherence will be specifically dealt with upon the analysis of the impact of this phenomenon on single institutional framework, as defined by the Treaty on European Union.

A distributed environment for virtual and/or real experiments for underwater robots

This paper presents the distributed environment for virtual and/or real experiments for underwater robots (DEVRE). This environment is composed of a set of processes running on a local area network composed of three sites: 1) the onboard AUV computer; 2) a surface computer used as human-machine interface (HMI); and 3) a computer used for simulating the vehicle dynamics and representing the virtual world. The HMI can be transparently linked to the real sensors and actuators dealing with a real mission. It can also be linked with virtual sensors and virtual actuators, dealing with a virtual mission. The aim of DEVRE is to assist engineers during the software development and testing in the lab prior to real experiments