Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Supporting the reuse of effective CSCL learning designs through social structure representations

Distance and blended collaborative learning settings are usually characterized by different social structures defined in terms of groups' number, dimension, and composition; these structures are variable and can change within the same activity. This variability poses additional complexity to instructional designers, when they are trying to develop successful experiences from existing designs. This complexity is greatly associated with the fact that learning designs do not render explicit how social structures influenced the decisions of the original designer, and thus whether the social structures of the new setting could preclude the effectiveness of the reused design. This article proposes the usage of new representations (social structure representations, SSRs) able to support unskilled designers in reusing existing learning designs, through the explicit characterization of the social structures and constraints embedded either by the original designers or the reusing teachers, according to well-known principles of good collaborative learning practice. The article also describes an evaluation process that involved university professors, as well as the main findings derived from it. This process supported the initial assumptions about the effectiveness of SSRs, with significant evidence from both qualitative and qualitative data.

Aplicaciones educativas de la teoría vygotskiana. El programa 'Key to learning' = Applications of vigotsky's theory to education. The 'Key to learning' proggram

En ciencias de la educación, las últimas décadas han estado marcadas por un interés en las ideas de Lev S. Vygotski. De hecho, a partir de esas ideas se han propuesto varias aplicaciones educativas. Una de ellas es el “Key to learning”. El artículo propone una visión general de este programa educativo desarrollado a partir de algunos trabajos e ideas de autores rusos contemporáneos. Primero, desarrollamos algunas ideas en torno a la noción de zona de desarrollo próximo (ZpD). Después, sugerimos la teoría de las habilidades de aprendizaje. En este sentido, el objetivo principal de “Key to learning” es mejorar las habilidades de aprendizaje cognitivas, comunicativas y directivas de niños de entre 3 a 7 años de edad. Para este propósito son creadas 12 unidades curriculares que componen el programa. Para concluir se enfatiza la creación de zonas de desarrollo próximo estructuradas como parte de un sistema de enseñanza y aprendizaje que vincula la actividad, la asistencia y la agencia

Sequential Machine learning Approaches for Portfolio Management

Cette thèse envisage un ensemble de méthodes permettant aux algorithmes d'apprentissage statistique de mieux traiter la nature séquentielle des problèmes de gestion de portefeuilles financiers. Nous débutons par une considération du problème général de la composition d'algorithmes d'apprentissage devant gérer des tâches séquentielles, en particulier celui de la mise-à-jour efficace des ensembles d'apprentissage dans un cadre de validation séquentielle. Nous énumérons les desiderata que des primitives de composition doivent satisfaire, et faisons ressortir la difficulté de les atteindre de façon rigoureuse et efficace. Nous poursuivons en présentant un ensemble d'algorithmes qui atteignent ces objectifs et présentons une étude de cas d'un système complexe de prise de décision financière utilisant ces techniques. Nous décrivons ensuite une méthode générale permettant de transformer un problème de décision séquentielle non-Markovien en un problème d'apprentissage supervisé en employant un algorithme de recherche basé sur les K meilleurs chemins. Nous traitons d'une application en gestion de portefeuille où nous entraînons un algorithme d'apprentissage à optimiser directement un ratio de Sharpe (ou autre critère non-additif incorporant une aversion au risque). Nous illustrons l'approche par une étude expérimentale approfondie, proposant une architecture de réseaux de neurones spécialisée à la gestion de portefeuille et la comparant à plusieurs alternatives. Finalement, nous introduisons une représentation fonctionnelle de séries chronologiques permettant à des prévisions d'être effectuées sur un horizon variable, tout en utilisant un ensemble informationnel révélé de manière progressive. L'approche est basée sur l'utilisation des processus Gaussiens, lesquels fournissent une matrice de covariance complète entre tous les points pour lesquels une prévision est demandée. Cette information est utilisée à bon escient par un algorithme qui transige activement des écarts de cours (price spreads) entre des contrats à terme sur commodités. L'approche proposée produit, hors échantillon, un rendement ajusté pour le risque significatif, après frais de transactions, sur un portefeuille de 30 actifs.

A Reinforcement Learning Algorithm to Economic Dispatch Considering Transmission Losses

Reinforcement Learning (RL) refers to a class of learning algorithms in which learning system learns which action to take in different situations by using a scalar evaluation received from the environment on performing an action. RL has been successfully applied to many multi stage decision making problem (MDP) where in each stage the learning systems decides which action has to be taken. Economic Dispatch (ED) problem is an important scheduling problem in power systems, which decides the amount of generation to be allocated to each generating unit so that the total cost of generation is minimized without violating system constraints. In this paper we formulate economic dispatch problem as a multi stage decision making problem. In this paper, we also develop RL based algorithm to solve the ED problem. The performance of our algorithm is compared with other recent methods. The main advantage of our method is it can learn the schedule for all possible demands simultaneously.

Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

The use of computer-based learning tools for teaching and clinical purposes: Interactive computing strategy for Iraq

Medical universities and teaching hospitals in Iraq are facing a lack of professional staff due to the ongoing violence that forces them to flee the country. The professionals are now distributed outside the country which reduces the chances for the staff and students to be physically in one place to continue the teaching and limits the efficiency of the consultations in hospitals. A survey was done among students and professional staff in Iraq to find the problems in the learning and clinical systems and how Information and Communication Technology could improve it. The survey has shown that 86% of the participants use the Internet as a learning resource and 25% for clinical purposes while less than 11% of them uses it for collaboration between different institutions. A web-based collaborative tool is proposed to improve the teaching and clinical system. The tool helps the users to collaborate remotely to increase the quality of the learning system as well as it can be used for remote medical consultation in hospitals.

Semantic organization to enhance active learning based on distributed multimedia educational resources

In order to organize distributed educational resources efficiently, to provide active learners an integrated, extendible and cohesive interface to share the dynamically growing multimedia learning materials on the Internet, this paper proposes a generic resource organization model with semantic structures to improve expressiveness, scalability and cohesiveness. We developed an active learning system with semantic support for learners to access and navigate through efficient and flexible manner. We learning resources in an efficient and flexible manner. We provide facilities for instructors to manipulate the structured educational resources via a convenient visual interface. We also developed a resource discovering and gathering engine based on complex semantic associations for several specific topics.

Integrating constructive feedback in personalised e-learning

When using e-learning material some students progress readily, others have difficulties. In a traditional classroom the teacher would identify those with difficulties and direct them to additional resources. This support is not easily available within e-learning. A new approach to providing constructive feedback is developed that will enable an e-learning system to identify areas of weakness and provide guidance on further study. The approach is based on the tagging of learning material with appropriate keywords that indicate the contents. Thus if a student performs poorly on an assessment on topic X, there is a need to suggest further study of X and participation in activities related to X such as forums. As well as supporting the learner this type of constructive feedback can also inform other stakeholders. For example a tutor can monitor the progress of a cohort; an instructional designer can monitor the quality of learning objects in facilitating the appropriate knowledge across many learners.

Review of monitoring tools for e-learning platforms

The advancement of e-learning technologies has made it viable for developments in education and technology to be combined in order to fulfil educational needs worldwide. E-learning consists of informal learning approaches and emerging technologies to support the delivery of learning skills, materials, collaboration and knowledge sharing. E-learning is a holistic approach that covers a wide range of courses, technologies and infrastructures to provide an effective learning environment. The Learning Management System (LMS) is the core of the entire e-learning process along with technology, content, and services. This paper investigates the role of model-driven personalisation support modalities in providing enhanced levels of learning and trusted assimilation in an e-learning delivery context. We present an analysis of the impact of an integrated learning path that an e-learning system may employ to track activities and evaluate the performance of learners.

Evaluating e-learning systems: an empirical investigation on students' perception in higher education area

In search of better, traditional learning universities have expanded their ways to deliver knowledge and integrate cost effective e-learning systems. Universities’ use of information and communication technologies has grown tremendously over the last decade. To ensure efficient use of the e-learning system, the Arab Open University (AOU) in Bahrain was the first to use e-learning system there, aimed to evaluate the good and bad practices, detect errors and determine areas for further improvements in usage. This study critically evaluated the students’ perception of the elearning system in Bahrain and recommended changes to improve students’ e-learning usage. Results of the study indicated that, in general, students have favourable perceptions toward using the e-learning system. This study has shown that technology acceptance is the most variable, factor that contributes to students’ perception and satisfaction of the e-learning system.

Machine learning and small robot navigation

Developing successful navigation and mapping strategies is an essential part of autonomous robot research. However, hardware limitations often make for inaccurate systems. This project serves to investigate efficient alternatives to mapping an environment, by first creating a mobile robot, and then applying machine learning to the robot and controlling systems to increase the robustness of the robot system. My mapping system consists of a semi-autonomous robot drone in communication with a stationary Linux computer system. There are learning systems running on both the robot and the more powerful Linux system. The first stage of this project was devoted to designing and building an inexpensive robot. Utilizing my prior experience from independent studies in robotics, I designed a small mobile robot that was well suited for simple navigation and mapping research. When the major components of the robot base were designed, I began to implement my design. This involved physically constructing the base of the robot, as well as researching and acquiring components such as sensors. Implementing the more complex sensors became a time-consuming task, involving much research and assistance from a variety of sources. A concurrent stage of the project involved researching and experimenting with different types of machine learning systems. I finally settled on using neural networks as the machine learning system to incorporate into my project. Neural nets can be thought of as a structure of interconnected nodes, through which information filters. The type of neural net that I chose to use is a type that requires a known set of data that serves to train the net to produce the desired output. Neural nets are particularly well suited for use with robotic systems as they can handle cases that lie at the extreme edges of the training set, such as may be produced by "noisy" sensor data. Through experimenting with available neural net code, I became familiar with the code and its function, and modified it to be more generic and reusable for multiple applications of neural nets.

Proposal of convergence of e-Learning systems for t-Learning

This article presents considerations about viability on reutilize existing web based e-Learning systems on Interactive Digital TV environment according to Digital TV standard adopted in Brazil. Considering the popularity of Moodle system in academic and corporative area, such system was chosen as a foundation for a survey into its properties to create a specification of an Application Programming Interface (API) for convergence to t-Learning characteristics that demands efforts in interface design area due the fact that computer and TV concepts are totally different. This work aims to present studies concerning user interface design during two stages: survey and detail of functionalities from an e-Learning system and how to adapt them for the Interactive TV regarding usability context and Information Architecture concepts.