Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Designing context-aware adaptive units of learning based on IMS-LD standard

Learning contents adaptation has been a subject of interest in the research area of the adaptive hypermedia systems. Defining which variables and which standards can be considered to model adaptive content delivery processes is one of the main challenges in pedagogical design over e-learning environments. In this paper some specifications, architectures and technologies that can be used in contents adaptation processes considering characteristics of the context are described and a proposal to integrate some of these characteristics in the design of units of learning using adaptation conditions in a structure of IMS-Learning Design (IMS-LD) is presented. The key contribution of this work is the generation of instructional designs considering the context, which can be used in Learning Management Systems (LMSs) and diverse mobile devices

The value of using short-format podcasts to enhance learning and teaching

This paper presents the findings of a podcasting trial held in 2007-2008 within the Faculty of Economics and Business at the University of Sydney, Australia. The trial investigates the value of using short-format podcasts to support assessment for postgraduate and undergraduate students. A multi-method approach is taken in investigating perceptions of the benefits of podcasting, incorporating surveys, focus groups and interviews. The results show that a majority of students believe they gained learning benefits from the podcasts and appreciated the flexibility of the medium to support their learning, and the lecturers felt the innovation helped diversify their pedagogical approach and support a diverse student population. Three primary conclusions are presented: (1) most students reject the mobile potential of podcasting in favour of their traditional study space at home; (2) what students and lecturers value about this podcasting design overlap; (3) the assessment-focussed, short-format podcast design may be considered a successful podcasting model. The paper finishes by identifying areas for future research on the effective use of podcasting in learning and teaching.

A User-Centric Approach to eCertificate for Electronic Identities (eIDs) Management in Mobile Environment

The use of electronic documents is constantly growing and the necessity to implement an ad-hoc eCertificate which manages access to private information is not only required but also necessary. This paper presents a protocol for the management of electronic identities (eIDs), meant as a substitute for the paper-based IDs, in a mobile environment with a user-centric approach. Mobile devices have been chosen because they provide mobility, personal use and high computational complexity. The inherent user-centricity also allows the user to personally manage the ID information and to display only what is required. The chosen path to develop the protocol is to migrate the existing eCert technologies implemented by the Learning Societies Laboratory in Southampton. By comparing this protocol with the analysis of the eID problem domain, a new solution has been derived which is compatible with both systems without loss of features.

Assessment and Feedback for the CIP: learning technologies

This 11-minute video provides some guidance on the learning technologies available at Southampton that can be used to support assessment and feedback. It was produced using the Panopto lecture capture system. The first link is to the native Panopto podcast, which requires the Silverlight player to be installed. The second link is the MP4 video version of the podcast, which should play on all PCs, Macs and suitable mobile devices. This share also provides a link to the video's script.

A proposal of a behavior-based control architecture with reinforcement learning for an autonomous underwater robot

Aquesta tesi proposa l'ús d'un seguit de tècniques pel control a alt nivell d'un robot autònom i també per l'aprenentatge automàtic de comportaments. L'objectiu principal de la tesis fou el de dotar d'intel·ligència als robots autònoms que han d'acomplir unes missions determinades en entorns desconeguts i no estructurats. Una de les premisses tingudes en compte en tots els passos d'aquesta tesis va ser la selecció d'aquelles tècniques que poguessin ésser aplicades en temps real, i demostrar-ne el seu funcionament amb experiments reals. El camp d'aplicació de tots els experiments es la robòtica submarina. En una primera part, la tesis es centra en el disseny d'una arquitectura de control que ha de permetre l'assoliment d'una missió prèviament definida. En particular, la tesis proposa l'ús de les arquitectures de control basades en comportaments per a l'assoliment de cada una de les tasques que composen la totalitat de la missió. Una arquitectura d'aquest tipus està formada per un conjunt independent de comportaments, els quals representen diferents intencions del robot (ex.: "anar a una posició", "evitar obstacles",...). Es presenta una recerca bibliogràfica sobre aquest camp i alhora es mostren els resultats d'aplicar quatre de les arquitectures basades en comportaments més representatives a una tasca concreta. De l'anàlisi dels resultats se'n deriva que un dels factors que més influeixen en el rendiment d'aquestes arquitectures, és la metodologia emprada per coordinar les respostes dels comportaments. Per una banda, la coordinació competitiva és aquella en que només un dels comportaments controla el robot. Per altra banda, en la coordinació cooperativa el control del robot és realitza a partir d'una fusió de totes les respostes dels comportaments actius. La tesis, proposa un esquema híbrid d'arquitectura capaç de beneficiar-se dels principals avantatges d'ambdues metodologies. En una segona part, la tesis proposa la utilització de l'aprenentatge per reforç per aprendre l'estructura interna dels comportaments. Aquest tipus d'aprenentatge és adequat per entorns desconeguts i el procés d'aprenentatge es realitza al mateix temps que el robot està explorant l'entorn. La tesis presenta també un estat de l'art d'aquest camp, en el que es detallen els principals problemes que apareixen en utilitzar els algoritmes d'aprenentatge per reforç en aplicacions reals, com la robòtica. El problema de la generalització és un dels que més influeix i consisteix en permetre l'ús de variables continues sense augmentar substancialment el temps de convergència. Després de descriure breument les principals metodologies per generalitzar, la tesis proposa l'ús d'una xarxa neural combinada amb l'algoritme d'aprenentatge per reforç Q_learning. Aquesta combinació proporciona una gran capacitat de generalització i una molt bona disposició per aprendre en tasques de robòtica amb exigències de temps real. No obstant, les xarxes neurals són aproximadors de funcions no-locals, el que significa que en treballar amb un conjunt de dades no homogeni es produeix una interferència: aprendre en un subconjunt de l'espai significa desaprendre en la resta de l'espai. El problema de la interferència afecta de manera directa en robòtica, ja que l'exploració de l'espai es realitza sempre localment. L'algoritme proposat en la tesi té en compte aquest problema i manté una base de dades representativa de totes les zones explorades. Així doncs, totes les mostres de la base de dades s'utilitzen per actualitzar la xarxa neural, i per tant, l'aprenentatge és homogeni. Finalment, la tesi presenta els resultats obtinguts amb la arquitectura de control basada en comportaments i l'algoritme d'aprenentatge per reforç. Els experiments es realitzen amb el robot URIS, desenvolupat a la Universitat de Girona, i el comportament après és el seguiment d'un objecte mitjançant visió per computador. La tesi detalla tots els dispositius desenvolupats pels experiments així com les característiques del propi robot submarí. Els resultats obtinguts demostren la idoneïtat de les propostes en permetre l'aprenentatge del comportament en temps real. En un segon apartat de resultats es demostra la capacitat de generalització de l'algoritme d'aprenentatge mitjançant el "benchmark" del "cotxe i la muntanya". Els resultats obtinguts en aquest problema milloren els resultats d'altres metodologies, demostrant la millor capacitat de generalització de les xarxes neurals.

Architecture for Living Neuronal Network Control of a mobile robot

The intelligent controlling mechanism of a typical mobile robot is usually a computer system. Research is however now ongoing in which biological neural networks are being cultured and trained to act as the brain of an interactive real world robot – thereby either completely replacing or operating in a cooperative fashion with a computer system. Studying such neural systems can give a distinct insight into biological neural structures and therefore such research has immediate medical implications. The principal aims of the present research are to assess the computational and learning capacity of dissociated cultured neuronal networks with a view to advancing network level processing of artificial neural networks. This will be approached by the creation of an artificial hybrid system (animat) involving closed loop control of a mobile robot by a dissociated culture of rat neurons. This paper details the components of the overall animat closed loop system architecture and reports on the evaluation of the results from preliminary real-life and simulated robot experiments.

Controlling a mobile robot with a biological brain

The intelligent controlling mechanism of a typical mobile robot is usually a computer system. Some recent research is ongoing in which biological neurons are being cultured and trained to act as the brain of an interactive real world robot�thereby either completely replacing, or operating in a cooperative fashion with, a computer system. Studying such hybrid systems can provide distinct insights into the operation of biological neural structures, and therefore, such research has immediate medical implications as well as enormous potential in robotics. The main aim of the research is to assess the computational and learning capacity of dissociated cultured neuronal networks. A hybrid system incorporating closed-loop control of a mobile robot by a dissociated culture of neurons has been created. The system is flexible and allows for closed-loop operation, either with hardware robot or its software simulation. The paper provides an overview of the problem area, gives an idea of the breadth of present ongoing research, establises a new system architecture and, as an example, reports on the results of conducted experiments with real-life robots.

Uses of mobile phones in higher education

This brief paper explores the current and potential usage of mobile phones within higher education. It reports on the outcomes of a brain storming session regarding this subject undertaken with a cohort of final year Computer Science students undertaking a Social, Legal and Ethical Aspects of Information Technology course at The University of Reading. Subsequent analysis was undertaken as a result of online discussion using a Managed Learning Environment and a web based survey completed by over 250 undergraduates from around the UK.

Architecture for neuronal cell control of a mobile robot

It is usually expected that the intelligent controlling mechanism of a robot is a computer system. Research is however now ongoing in which biological neural networks are being cultured and trained to act as the brain of an interactive real world robot - thereby either completely replacing or operating in a cooperative fashion with a computer system. Studying such neural systems can give a distinct insight into biological neural structures and therefore such research has immediate medical implications. In particular, the use of rodent primary dissociated cultured neuronal networks for the control of mobile `animals' (artificial animals, a contraction of animal and materials) is a novel approach to discovering the computational capabilities of networks of biological neurones. A dissociated culture of this nature requires appropriate embodiment in some form, to enable appropriate development in a controlled environment within which appropriate stimuli may be received via sensory data but ultimate influence over motor actions retained. The principal aims of the present research are to assess the computational and learning capacity of dissociated cultured neuronal networks with a view to advancing network level processing of artificial neural networks. This will be approached by the creation of an artificial hybrid system (animal) involving closed loop control of a mobile robot by a dissociated culture of rat neurons. This 'closed loop' interaction with the environment through both sensing and effecting will enable investigation of its learning capacity This paper details the components of the overall animat closed loop system and reports on the evaluation of the results from the experiments being carried out with regard to robot behaviour.

Using mobile devices to enhance undergraduate field research

Mobile devices can enhance undergraduate research projects and students’ research capabilities. The use of mobile devices such as tablet computers will not automatically make undergraduates better researchers, but their use should make investigations, writing, and publishing more effective and may even save students time. We have explored some of the possibilities of using “tablets” and “smartphones” to aid the research and inquiry process in geography and bioscience fieldwork. We provide two case studies as illustration of how students working in small research groups use mobile devices to gather and analyze primary data in field-based inquiry. Since April 2010, Apple’s iPad has changed the way people behave in the digital world and how they access their music, watch videos, or read their email much as the entrepreneurs Steve Jobs and Jonathan Ive intended. Now with “apps” and “the cloud” and the ubiquitous references to them appearing in the press and on TV, academics’ use of tablets is also having an impact on education and research. In our discussion we will refer to use of smartphones such as the iPhone, iPod, and Android devices under the term “tablet”. Android and Microsoft devices may not offer the same facilities as the iPad/iphone, but many app producers now provide versions for several operating systems. Smartphones are becoming more affordable and ubiquitous (Melhuish and Falloon 2010), but a recent study of undergraduate students (Woodcock et al. 2012, 1) found that many students who own smartphones are “largely unaware of their potential to support learning”. Importantly, however, students were found to be “interested in and open to the potential as they become familiar with the possibilities” (Woodcock et al. 2012). Smartphones and iPads could be better utilized than laptops when conducting research in the field because of their portability (Welsh and France 2012). It is imperative for faculty to provide their students with opportunities to discover and employ the potential uses of mobile devices in their learning. However, it is not only the convenience of the iPad or tablet devices or smartphones we wish to promote, but also a way of thinking and behaving digitally. We essentially suggest that making a tablet the center of research increases the connections between related research activities.

Effectiveness of mobile apps in teaching field-based identification skills

It has been suggested that few students graduate with the skills required for many ecological careers, as field-based learning is said to be in decline in academic institutions. Here, we asked if mobile technology could improve field-based learning, using ability to identify birds as the study metric. We divided a class of ninety-one undergraduate students into two groups for field-based sessions where they were taught bird identification skills. The first group has access to a traditional identification book and the second group were provided with an identification app. We found no difference between the groups in the ability of students to identify birds after three field sessions. Furthermore, we found that students using the traditional book were significantly more likely to identify novel species. Therefore, we find no evidence that mobile technology improved students’ ability to retain what they experienced in the field; indeed, there is evidence that traditional field guides were more useful to students as they attempted to identify new species. Nevertheless, students felt positively about using their own smartphone devices for learning, highlighting that while apps did not lead to an improvement in bird identification ability, they gave greater accessibility to relevant information outside allocated teaching times.

Semantic mapping using mobile robots

Robotic mapping is the process of automatically constructing an environment representation using mobile robots. We address the problem of semantic mapping, which consists of using mobile robots to create maps that represent not only metric occupancy but also other properties of the environment. Specifically, we develop techniques to build maps that represent activity and navigability of the environment. Our approach to semantic mapping is to combine machine learning techniques with standard mapping algorithms. Supervised learning methods are used to automatically associate properties of space to the desired classification patterns. We present two methods, the first based on hidden Markov models and the second on support vector machines. Both approaches have been tested and experimentally validated in two problem domains: terrain mapping and activity-based mapping.