Teaching an Old Robot New Tricks: Learning Novel Tasks via Interaction with People and Things

As AI has begun to reach out beyond its symbolic, objectivist roots into the embodied, experientialist realm, many projects are exploring different aspects of creating machines which interact with and respond to the world as humans do. Techniques for visual processing, object recognition, emotional response, gesture production and recognition, etc., are necessary components of a complete humanoid robot. However, most projects invariably concentrate on developing a few of these individual components, neglecting the issue of how all of these pieces would eventually fit together. The focus of the work in this dissertation is on creating a framework into which such specific competencies can be embedded, in a way that they can interact with each other and build layers of new functionality. To be of any practical value, such a framework must satisfy the real-world constraints of functioning in real-time with noisy sensors and actuators. The humanoid robot Cog provides an unapologetically adequate platform from which to take on such a challenge. This work makes three contributions to embodied AI. First, it offers a general-purpose architecture for developing behavior-based systems distributed over networks of PC's. Second, it provides a motor-control system that simulates several biological features which impact the development of motor behavior. Third, it develops a framework for a system which enables a robot to learn new behaviors via interacting with itself and the outside world. A few basic functional modules are built into this framework, enough to demonstrate the robot learning some very simple behaviors taught by a human trainer. A primary motivation for this project is the notion that it is practically impossible to build an "intelligent" machine unless it is designed partly to build itself. This work is a proof-of-concept of such an approach to integrating multiple perceptual and motor systems into a complete learning agent.

Learning Commonsense Categorical Knowledge in a Thread Memory System

If we are to understand how we can build machines capable of broad purpose learning and reasoning, we must first aim to build systems that can represent, acquire, and reason about the kinds of commonsense knowledge that we humans have about the world. This endeavor suggests steps such as identifying the kinds of knowledge people commonly have about the world, constructing suitable knowledge representations, and exploring the mechanisms that people use to make judgments about the everyday world. In this work, I contribute to these goals by proposing an architecture for a system that can learn commonsense knowledge about the properties and behavior of objects in the world. The architecture described here augments previous machine learning systems in four ways: (1) it relies on a seven dimensional notion of context, built from information recently given to the system, to learn and reason about objects' properties; (2) it has multiple methods that it can use to reason about objects, so that when one method fails, it can fall back on others; (3) it illustrates the usefulness of reasoning about objects by thinking about their similarity to other, better known objects, and by inferring properties of objects from the categories that they belong to; and (4) it represents an attempt to build an autonomous learner and reasoner, that sets its own goals for learning about the world and deduces new facts by reflecting on its acquired knowledge. This thesis describes this architecture, as well as a first implementation, that can learn from sentences such as ``A blue bird flew to the tree'' and ``The small bird flew to the cage'' that birds can fly. One of the main contributions of this work lies in suggesting a further set of salient ideas about how we can build broader purpose commonsense artificial learners and reasoners.

Learning Three-Dimensional Shape Models for Sketch Recognition

Artifacts made by humans, such as items of furniture and houses, exhibit an enormous amount of variability in shape. In this paper, we concentrate on models of the shapes of objects that are made up of fixed collections of sub-parts whose dimensions and spatial arrangement exhibit variation. Our goals are: to learn these models from data and to use them for recognition. Our emphasis is on learning and recognition from three-dimensional data, to test the basic shape-modeling methodology. In this paper we also demonstrate how to use models learned in three dimensions for recognition of two-dimensional sketches of objects.

Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

A behavior-based scheme using reinforcement learning for autonomous underwater vehicles

This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q_learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q_learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs

Curriculum sequencing for an e-learning system based on learning styles

This work shows the use of adaptation techniques involved in an e-learning system that considers students' learning styles and students' knowledge states. The mentioned e-learning system is built on a multiagent framework designed to examine opportunities to improve the teaching and to motivate the students to learn what they want in a user-friendly and assisted environment

Aula de música : desarrollo de contenidos en una herramienta de e-learning para la asignatura de música en Educación Primaria.

Aula de música es una herramienta e-learning para el desarrollo del aprendizaje de la música para niños con edades comprendidas entre los 6 y 12 años, edades correspondientes a las de los alumnos de la etapa de la Educación Primaria. En esta herramienta destaca el uso de estándares y especificaciones como LOM, IMS, etc. que van a facilitar la tarea de reutilizar la documentación incluida para compartir conocimiento. El proceso de elaboración del contenido ha sido fundamental y en relación con el entorno de trabajo debe mencionarse que se ha primado la construcción de una GUI que sirva para aprender y que motive a los alumnos a aprender música de una forma diferente, en contraposición a realizar una diseño estético que fuera incapaz de adaptarse a las capacidades de cada tipo de usuario, para lo que se han tenido en cuenta criterios de usabilidad y accesibilidad (WAI).

Example General Feedback on Routes to Success Portfolios

This document provides example feedback which has been generated following the marking of a class set of portfolios. It is used as a part of the Routes to Success Module, specifically on the section titled Sustaining Success. Students can read the feedback prior to completing the portfolio to alert them to the possible shortfalls which may occur when they undertake this type of task. The feedback is introduced in the context that the task of completing the portfolio is a developmental one, and that students can expect to learn and improve their performance for this type of task as they develop and refine their skills.

Dialogue Systems for Language Learning

En este artículo se presenta el caso de Milao, un entorno virtual que ofrece a los estudiantes de idiomas extranjeros la oportunidad de desarrollar y mejorar sus habilidades comunicativas dialogando en escenarios de conversación predefinidos que simulan la interacción con un nativo. Esta tecnología propone una solución a uno de los mayores retos en el aprendizaje de lenguas extranjeras: la falta de oportunidades para poner en práctica la gramática y el vocabulario recién adquiridos. Combinando la investigación sobre la lingüística y el aprendizaje de lenguas con los avances tecnológicos en el campo del Procesamiento del Lenguaje Natural (NPL), particularmente sobre sistemas de diálogo, hemos creado oportunidades en la demanda de los estudiantes a conversar en la lengua que tratan de aprender.

La Autogestión del aprendizaje y la autonomía e iniciativa personal = The self-regulation of learning and autonomy and personal Initiative

En esta investigación se trata de vincular la autogestión del aprendizaje con el desarrollo de la autonomía personal, la cual favorece el aprendizaje a lo largo de la vida. Partimos, en este proyecto de investigación, de la hipótesis de que la autogestión del aprendizaje es un elemento multidimensional cuya mejora revierte de forma positiva en el desarrollo de la autonomía e iniciativa personal. En esta comunicación se presentan los resultados obtenidos en la aplicación de la fase piloto de un proyecto más amplio, en el que se ha recogido información a través de autoinforme sobre las distintas estrategias y aspectos de la autogestión del aprendizaje y sobre la autonomía e iniciativa personal. Los resultados indican que, en general, hay una relación significativa positiva moderada entre las estrategias de aprendizaje y la autonomía, lo que confirma la importancia de ambos aspectos para favorecer el desarrollo integral de los estudiantes. El fomento de las estrategias de aprendizaje hace que los estudiantes desarrollen la autonomía. A su vez, exponer a los estudiantes a situaciones de aprendizaje que fomenten su autonomía mejora su competencia para aprender

'Word-learning wizardry' at I;6

This article explores whether infants are able to learn words as rapidly as has been reported for preschoolers. Sixty-four infants aged 1;6 were taught labels for either two moving images or two still images. Each image-label pair was presented three times, after which comprehension was assessed using an adaptation of the intermodal preferential looking paradigm. Three repetitions of each label were found to be sufficient for learning to occur, fewer than has previously been reported for infants under two years. Moreover, contrary to a previous finding, learning was equally rapid for infants who were taught labels for moving versus still images. The findings indicate that infants in the early stages of acquiring a vocabulary learn new word-referent associations with ease, and that the learning conditions that allow such learning are less restricted that was previously believed.

The use of social and salience cues in early word learning

This article explores young infants' ability to learn new words in situations providing tightly controlled social and salience cues to their reference. Four experiments investigated whether, given two potential referents, 15-month-olds would attach novel labels to (a) an image toward which a digital recording of a face turned and gazed, (b) a moving image versus a stationary image, (c) a moving image toward which the face gazed, and (d) a gazed-on image versus a moving image. Infants successfully used the recorded gaze cue to form new word-referent associations and also showed learning in the salience condition. However, their behavior in the salience condition and in the experiments that followed suggests that, rather than basing their judgments of the words' reference on the mere presence or absence of the referent's motion, infants were strongly biased to attend to the consistency with which potential referents moved when a word was heard. (c) 2006 Elsevier Inc. All rights reserved.

The role of perirhinal cortex in visual discrimination learning for visual secondary reinforcement in rats

Perirhinal cortex in monkeys has been thought to be involved in visual associative learning. The authors examined rats' ability to make associations between visual stimuli in a visual secondary reinforcement task. Rats learned 2-choice visual discriminations for secondary visual reinforcement. They showed significant learning of discriminations before any primary reinforcement. Following bilateral perirhinal cortex lesions, rats continued to learn visual discriminations for visual secondary reinforcement at the same rate as before surgery. Thus, this study does not support a critical role of perirhinal cortex in learning for visual secondary reinforcement. Contrasting this result with other positive results, the authors suggest that the role of perirhinal cortex is in "within-object" associations and that it plays a much lesser role in stimulus-stimulus associations between objects.

Dynamic automata for mobile robot learning

Researchers at the University of Reading have developed over many years some simple mobile robots that explore an environment they perceive through simple ultrasonic sensors. Information from these sensors has allowed the robots to learn the simple task of moving around while avoiding dynamic obstacles using a static set of fuzzy automata, the choice of which has been criticised, due to its arbitrary nature. This paper considers how a dynamic set of automata can overcome this criticism. In addition, a new reinforcement learning function is outlined which is both scalable to different numbers and types of sensors. The innovations compare successfully with earlier work.