Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

QoS online routing and MPLS multilevel protection: a survey

A survey of MPLS protection methods and their utilization in combination with online routing methods is presented in this article. Usually, fault management methods pre-establish backup paths to recover traffic after a failure. In addition, MPLS allows the creation of different backup types, and hence MPLS is a suitable method to support traffic-engineered networks. In this article, an introduction of several label switch path backup types and their pros and cons are pointed out. The creation of an LSP involves a routing phase, which should include QoS aspects. In a similar way, to achieve a reliable network the LSP backups must also be routed by a QoS routing method. When LSP creation requests arrive one by one (a dynamic network scenario), online routing methods are applied. The relationship between MPLS fault management and QoS online routing methods is unavoidable, in particular during the creation of LSP backups. Both aspects are discussed in this article. Several ideas on how these actual technologies could be applied together are presented and compared

De la Presencialitat i de l'online, on acaba una i comença l´altra? Propostes per a plantejar-les, disfrutar-les i valorar-les'

Hi ha dos tipus d'interrogants que ens plantegem en aquesta presentació: En primer lloc exposem un seguit de realitats que afecten directament al disseny de la nostra assignatura: els professors assumim habilitats apreses en tècniques de treball dels nostres estudiants: treball en equip, presentacions, debats.... Els uns i els altres entenem el mateix, del què i el com s'ha de treballar? Ho comprovarem. Alhora, ens preguntarem si els estudiants assumim habilitats tecnològiques del professorat? Moodle, wikis, Twitter, xats..... tots entenem el mateix i els fem servir de la mateixa manera? Fixem-nos en les dues perspectives, la del professor i la de l'estudiant, tant referent a dinamiques com a recursos, i ens adonarem que cal tenir en compte tant la una com l'altra. Aquestes reflexions afecten al plantejament de l'assignatura i ens deixen clar que hi ha molta feina a fer per trobar les respostes (conèixer als estudiants que tenim a les aules, explicar convenientment el que esperem dels estudiants, com plantegem l'assignatura i com l'avaluarem... ) No ens referim exclusivament a tenir un pla docent detallat a la maxima expresió, sinó que ens referim a reconèixer el com, el quan i el que expliquem als nostres estudiants, de manera que les regles del joc quedin enteses de la mateixa manera per una part i per l'altra. A partir d'aquí, i com a segon interrogant, ens plantejarem l'estructura que combina la part presencial i la part no presencial de l'assignatura: Quines són les activitats de les que puc treure més rendiment en la part presencial de l'assignatura i quines en la part no presencial? Quant temps i com calcular-lo, cal invertir en les activitats, tant per part dels estudiants com per part del professorat? Com podem avaluar una activitat que ha estat realitzada en part presencial i en part no presencial