Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

A behavior-based scheme using reinforcement learning for autonomous underwater vehicles

This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q_learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q_learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

The interactional space in CLIL (Content and Language Integrated Learning) and EFL (English as a Foreign Language) classrooms

Aquest estudi pretén investigar els intercanvis verbals mestre/a – aprenent(s) en dos contextos d'instrucció diferents: classes amb un enfocament AICLE (Aprenentatge Integrat de Continguts Curriculars i Llengua Estrangera) on s’aprenen continguts no lingüístics a través de l’anglès, per una banda, i classes 'tradicionals' d'anglès com a llengua estrangera, on l’anglès és alhora objecte d’estudi i vehicle de comunicació, per una altra banda. Més concretament, les preguntes que formula el/la mestre/a, la producció oral dels aprenents i el 'feedback' del/de la mestre/a en els episodis d’atenció a la forma s’han estudiat a la llum de les principals teories provinents del camp de l’Adquisició de Segones Llengües (SLA) per tal de demostrar el seu paper en l’aprenentatge de l’anglès. El corpus de dades prové de l’enregistrament de 7 sessions AICLE i d'11 sessions EFL enregistrades en format àudio i vídeo en dos centres públics d’Educació Primària (EP) de Catalunya. A cadascuna de les escoles, el/la mateix/a mestre/a és l’encarregat/da dels dos tipus d’instrucció amb el mateix grup d’aprenents (10-11 anys d’edat), fet que permet eliminar variables individuals com l'aptitud dels aprenents o l'estil del/de la mestre/a.Els resultats mostren un cert nombre de similituds discursives entre AICLE i EFL donat que ambdós enfocaments tenen lloc en el context-classe amb unes característiques ben definides. Tal com apunta la recerca realitzada en aquest camp, la instrucció AICLE reuneix un seguit de condicions idònies per un major desenvolupament dels nivells de llengua anglesa més enllà de les classes ‘tradicionals’ d’anglès. Malgrat això, aquest estudi sembla indicar que el potencial d'AICLE pel que fa a facilitar una exposició rica a l’anglès i una producció oral significativa no s’explota degudament. En aquest sentit, els resultats d’aquest estudi poden contribuir a la formació dels futurs professors d'AICLE si es busca l’assoliment d’una complementarietat d’ambdós contextos amb l’objectiu últim de millorar els nivells de domini de la llengua anglesa.

Decoding sequence learning from single-trial intracranial EEG in humans.

We propose and validate a multivariate classification algorithm for characterizing changes in human intracranial electroencephalographic data (iEEG) after learning motor sequences. The algorithm is based on a Hidden Markov Model (HMM) that captures spatio-temporal properties of the iEEG at the level of single trials. Continuous intracranial iEEG was acquired during two sessions (one before and one after a night of sleep) in two patients with depth electrodes implanted in several brain areas. They performed a visuomotor sequence (serial reaction time task, SRTT) using the fingers of their non-dominant hand. Our results show that the decoding algorithm correctly classified single iEEG trials from the trained sequence as belonging to either the initial training phase (day 1, before sleep) or a later consolidated phase (day 2, after sleep), whereas it failed to do so for trials belonging to a control condition (pseudo-random sequence). Accurate single-trial classification was achieved by taking advantage of the distributed pattern of neural activity. However, across all the contacts the hippocampus contributed most significantly to the classification accuracy for both patients, and one fronto-striatal contact for one patient. Together, these human intracranial findings demonstrate that a multivariate decoding approach can detect learning-related changes at the level of single-trial iEEG. Because it allows an unbiased identification of brain sites contributing to a behavioral effect (or experimental condition) at the level of single subject, this approach could be usefully applied to assess the neural correlates of other complex cognitive functions in patients implanted with multiple electrodes.

A context-based methodology for integrating Web 2.0 services in learning scenarios

The emergence of the Web 2.0 technologies in the last years havechanged the way people interact with knowledge. Services for cooperation andcollaboration have placed the user in the centre of a new knowledge buildingspace. The development of new second generation learning environments canbenefit from the potential of these Web 2.0 services when applied to aneducational context. We propose a methodology for designing learningenvironments that relates Web 2.0 services with the functional requirements ofthese environments. In particular, we concentrate on the design of the KRSMsystem to discuss the components of this methodology and its application.

Discovering the campus together: a mobile and computer-based learning experience

One of the most relevant difficulties faced by first-year undergraduate students is to settle into the educational environment of universities. This paper presents a case study that proposes a computer-assisted collaborative experience designed to help students in their transition from high school to university. This is done by facilitating their first contact with the campus and its services, the university community, methodologies and activities. The experience combines individual and collaborative activities, conducted in and out of the classroom, structured following the Jigsaw Collaborative Learning Flow Pattern. A specific environment including portable technologies with network and computer applications has been developed to support and facilitate the orchestration of a flow of learning activities into a single integrated learning setting. The result is a Computer-Supported Collaborative Blended Learning scenario, which has been evaluated with first-year university students of the degrees of Software and Audiovisual Engineering within the subject Introduction to Information and Communications Technologies. The findings reveal that the scenario improves significantly students’ interest in their studies and their understanding about the campus and services provided. The environment is also an innovative approach to successfully support the heterogeneous activities conducted by both teachers and students during the scenario. This paper introduces the goals and context of the case study, describes how the technology was employed to conduct the learning scenario, the evaluation methods and the main results of the experience.

A multicase study for the evaluation of a pattern-based visual design process for collaborative learning

Collage is a pattern-based visual design authoring tool for the creation of collaborative learning scripts computationally modelled with IMS Learning Design (LD). The pattern-based visual approach aims to provide teachers with design ideas that are based on broadly accepted practices. Besides, it seeks hiding the LD notation so that teachers can easily create their own designs. The use of visual representations supports both the understanding of the design ideas and the usability of the authoring tool. This paper presents a multicase study comprising three different cases that evaluate the approach from different perspectives. The first case includes workshops where teachers use Collage. A second case implies the design of a scenario proposed by a third-party using related approaches. The third case analyzes a situation where students follow a design created with Collage. The cross-case analysis provides a global understanding of the possibilities and limitations of the pattern-based visual design approach.

Free-and open source software for a course on network management: authoring and enactment of scripts based on collaborative learning strategies

This paper describes a Computer-Supported Collaborative Learning (CSCL) case study in engineering education carried out within the context of a network management course. The case study shows that the use of two computing tools developed by the authors and based on Free- and Open-Source Software (FOSS) provide significant educational benefits over traditional engineering pedagogical approaches in terms of both concepts and engineering competencies acquisition. First, the Collage authoring tool guides and supports the course teacher in the process of authoring computer-interpretable representations (using the IMS Learning Design standard notation) of effective collaborative pedagogical designs. Besides, the Gridcole system supports the enactment of that design by guiding the students throughout the prescribed sequence of learning activities. The paper introduces the goals and context of the case study, elaborates onhow Collage and Gridcole were employed, describes the applied evaluation methodology, anddiscusses the most significant findings derived from the case study.

Reusing IMS-LD formalized best practices in collaborative learning structuring

Designs of CSCL (Computer Supported Collaborative Learning)activities should be flexible, effective and customizable toparticular learning situations. On the other hand, structureddesigns aim to create favourable conditions for learning. Thus,this paper proposes the collection of representative and broadlyaccepted (best practices) structuring techniques in collaborative learning. With the aim of establishing a conceptual common ground among collaborative learning practitioners and softwaredevelopers, and reusing the expertise that best practicesrepresent, the paper also proposes the formulation of these techniques as patterns: the so-called CLFPs (CollaborativeLearning Flow Patterns). To formalize these patterns, we havechosen the educational modelling language IMS Learning Design (IMS-LD). IMS-LD has the capability to specify many of the collaborative characteristics of the CLFPs. Nevertheless, the language bears limited capability for describing the services that mediate interactions within a learning activity and the specification of temporal or rotated roles. This analysis isdiscussed in the paper, as well as our approaches towards thedevelopment of a system capable of integrating tools using IMSLDscripts and a CLFP-based Learning Design authoring tool.

Towards a User-Friendly Platform for Building Language Resources based on Web Services

This paper presents the platform developed in the PANACEA project, a distributed factory that automates the stages involved in the acquisition, production, updating and maintenance of Language Resources required by Machine Translation and other Language Technologies. We adopt a set of tools that have been successfully used in the Bioinformatics field, they are adapted to the needs of our field and used to deploy web services, which can be combined to build more complex processing chains (workflows). This paper describes the platform and its different components (web services, registry, workflows, social network and interoperability). We demonstrate the scalability of the platform by carrying out a set of massive data experiments. Finally, a validation of the platform across a set of required criteria proves its usability for different types of users (non-technical users and providers).

CLFP intrinsic constraints-based group management of blended learning situations

When applying a Collaborative Learning Flow Pattern (CLFP) to structure sequences of activities in real contexts, one of the tasks is to organize groups of students according to the constraints imposed by the pattern. Sometimes,unexpected events occurring at runtime force this pre-defined distribution to be changed. In such situations, an adjustment of the group structures to be adapted to the new context is needed. If the collaborative pattern is complex, this group redefinitionmight be difficult and time consuming to be carried out in real time. In this context, technology can help on notifying the teacher which incompatibilitiesbetween the actual context and the constraints imposed by the pattern. This chapter presents a flexible solution for supporting teachers in the group organization profiting from the intrinsic constraints defined by a CLFPs codified in IMS Learning Design. A prototype of a web-based tool for the TAPPS and Jigsaw CLFPs and the preliminary results of a controlled user study are alsopresented as a first step towards flexible technological systems to support grouping tasks in this context.

From a pattern-language to a pattern ontology approach for CSCL script design

Collaborative activities, in which students actively interact with each other, have proved to provide significant learning benefits. In Computer-Supported Collaborative Learning (CSCL), these collaborative activities are assisted by technologies. However, the use of computers does not guarantee collaboration, as free collaboration does not necessary lead to fruitful learning. Therefore, practitioners need to design CSCL scripts that structure the collaborative settings so that they promote learning. However, not all teachers have the technical and pedagogical background needed to design such scripts. With the aim of assisting teachers in designing effective CSCL scripts, we propose a model to support the selection of reusable good practices (formulated as patterns) so that they can be used as a starting point for their own designs. This model is based on a pattern ontology that computationally represents the knowledge captured on a pattern language for the design of CSCL scripts. A preliminary evaluation of the proposed approach is provided with two examples based on a set of meaningful interrelated patters computationally represented with the pattern ontology, and a paper prototyping experience carried out with two teaches. The results offer interesting insights towards the implementation of the pattern ontology in software tools.

VIKI : a semiotic-based system for multilingual knowledge retrieval

Abstract Since its creation, the Internet has permeated our daily life. The web is omnipresent for communication, research and organization. This exploitation has resulted in the rapid development of the Internet. Nowadays, the Internet is the biggest container of resources. Information databases such as Wikipedia, Dmoz and the open data available on the net are a great informational potentiality for mankind. The easy and free web access is one of the major feature characterizing the Internet culture. Ten years earlier, the web was completely dominated by English. Today, the web community is no longer only English speaking but it is becoming a genuinely multilingual community. The availability of content is intertwined with the availability of logical organizations (ontologies) for which multilinguality plays a fundamental role. In this work we introduce a very high-level logical organization fully based on semiotic assumptions. We thus present the theoretical foundations as well as the ontology itself, named Linguistic Meta-Model. The most important feature of Linguistic Meta-Model is its ability to support the representation of different knowledge sources developed according to different underlying semiotic theories. This is possible because mast knowledge representation schemata, either formal or informal, can be put into the context of the so-called semiotic triangle. In order to show the main characteristics of Linguistic Meta-Model from a practical paint of view, we developed VIKI (Virtual Intelligence for Knowledge Induction). VIKI is a work-in-progress system aiming at exploiting the Linguistic Meta-Model structure for knowledge expansion. It is a modular system in which each module accomplishes a natural language processing task, from terminology extraction to knowledge retrieval. VIKI is a supporting system to Linguistic Meta-Model and its main task is to give some empirical evidence regarding the use of Linguistic Meta-Model without claiming to be thorough.