Embodiment and Manipulation Learning Process for a Humanoid Hand

Babies are born with simple manipulation capabilities such as reflexes to perceived stimuli. Initial discoveries by babies are accidental until they become coordinated and curious enough to actively investigate their surroundings. This thesis explores the development of such primitive learning systems using an embodied light-weight hand with three fingers and a thumb. It is self-contained having four motors and 36 exteroceptor and proprioceptor sensors controlled by an on-palm microcontroller. Primitive manipulation is learned from sensory inputs using competitive learning, back-propagation algorithm and reinforcement learning strategies. This hand will be used for a humanoid being developed at the MIT Artificial Intelligence Laboratory.

Alignment by Maximization of Mutual Information

A new information-theoretic approach is presented for finding the pose of an object in an image. The technique does not require information about the surface properties of the object, besides its shape, and is robust with respect to variations of illumination. In our derivation, few assumptions are made about the nature of the imaging process. As a result the algorithms are quite general and can foreseeably be used in a wide variety of imaging situations. Experiments are presented that demonstrate the approach registering magnetic resonance (MR) images with computed tomography (CT) images, aligning a complex 3D object model to real scenes including clutter and occlusion, tracking a human head in a video sequence and aligning a view-based 2D object model to real images. The method is based on a formulation of the mutual information between the model and the image called EMMA. As applied here the technique is intensity-based, rather than feature-based. It works well in domains where edge or gradient-magnitude based methods have difficulty, yet it is more robust than traditional correlation. Additionally, it has an efficient implementation that is based on stochastic approximation. Finally, we will describe a number of additional real-world applications that can be solved efficiently and reliably using EMMA. EMMA can be used in machine learning to find maximally informative projections of high-dimensional data. EMMA can also be used to detect and correct corruption in magnetic resonance images (MRI).

Modeling, Estimation, and Control of Robot-Soil Interactions

This thesis presents the development of hardware, theory, and experimental methods to enable a robotic manipulator arm to interact with soils and estimate soil properties from interaction forces. Unlike the majority of robotic systems interacting with soil, our objective is parameter estimation, not excavation. To this end, we design our manipulator with a flat plate for easy modeling of interactions. By using a flat plate, we take advantage of the wealth of research on the similar problem of earth pressure on retaining walls. There are a number of existing earth pressure models. These models typically provide estimates of force which are in uncertain relation to the true force. A recent technique, known as numerical limit analysis, provides upper and lower bounds on the true force. Predictions from the numerical limit analysis technique are shown to be in good agreement with other accepted models. Experimental methods for plate insertion, soil-tool interface friction estimation, and control of applied forces on the soil are presented. In addition, a novel graphical technique for inverting the soil models is developed, which is an improvement over standard nonlinear optimization. This graphical technique utilizes the uncertainties associated with each set of force measurements to obtain all possible parameters which could have produced the measured forces. The system is tested on three cohesionless soils, two in a loose state and one in a loose and dense state. The results are compared with friction angles obtained from direct shear tests. The results highlight a number of key points. Common assumptions are made in soil modeling. Most notably, the Mohr-Coulomb failure law and perfectly plastic behavior. In the direct shear tests, a marked dependence of friction angle on the normal stress at low stresses is found. This has ramifications for any study of friction done at low stresses. In addition, gradual failures are often observed for vertical tools and tools inclined away from the direction of motion. After accounting for the change in friction angle at low stresses, the results show good agreement with the direct shear values.

A Reinforcement-Learning Approach to Power Management

We describe an adaptive, mid-level approach to the wireless device power management problem. Our approach is based on reinforcement learning, a machine learning framework for autonomous agents. We describe how our framework can be applied to the power management problem in both infrastructure and ad~hoc wireless networks. From this thesis we conclude that mid-level power management policies can outperform low-level policies and are more convenient to implement than high-level policies. We also conclude that power management policies need to adapt to the user and network, and that a mid-level power management framework based on reinforcement learning fulfills these requirements.

Learning Commonsense Categorical Knowledge in a Thread Memory System

If we are to understand how we can build machines capable of broad purpose learning and reasoning, we must first aim to build systems that can represent, acquire, and reason about the kinds of commonsense knowledge that we humans have about the world. This endeavor suggests steps such as identifying the kinds of knowledge people commonly have about the world, constructing suitable knowledge representations, and exploring the mechanisms that people use to make judgments about the everyday world. In this work, I contribute to these goals by proposing an architecture for a system that can learn commonsense knowledge about the properties and behavior of objects in the world. The architecture described here augments previous machine learning systems in four ways: (1) it relies on a seven dimensional notion of context, built from information recently given to the system, to learn and reason about objects' properties; (2) it has multiple methods that it can use to reason about objects, so that when one method fails, it can fall back on others; (3) it illustrates the usefulness of reasoning about objects by thinking about their similarity to other, better known objects, and by inferring properties of objects from the categories that they belong to; and (4) it represents an attempt to build an autonomous learner and reasoner, that sets its own goals for learning about the world and deduces new facts by reflecting on its acquired knowledge. This thesis describes this architecture, as well as a first implementation, that can learn from sentences such as ``A blue bird flew to the tree'' and ``The small bird flew to the cage'' that birds can fly. One of the main contributions of this work lies in suggesting a further set of salient ideas about how we can build broader purpose commonsense artificial learners and reasoners.

Comparing Support Vector Machines with Gaussian Kernels to Radial Basis Function Classifiers

The Support Vector (SV) machine is a novel type of learning machine, based on statistical learning theory, which contains polynomial classifiers, neural networks, and radial basis function (RBF) networks as special cases. In the RBF case, the SV algorithm automatically determines centers, weights and threshold such as to minimize an upper bound on the expected test error. The present study is devoted to an experimental comparison of these machines with a classical approach, where the centers are determined by $k$--means clustering and the weights are found using error backpropagation. We consider three machines, namely a classical RBF machine, an SV machine with Gaussian kernel, and a hybrid system with the centers determined by the SV method and the weights trained by error backpropagation. Our results show that on the US postal service database of handwritten digits, the SV machine achieves the highest test accuracy, followed by the hybrid approach. The SV approach is thus not only theoretically well--founded, but also superior in a practical application.

Multidimensional Adaptations for Open Learning Management Systems

Our work is focused on alleviating the workload for designers of adaptive courses on the complexity task of authoring adaptive learning designs adjusted to specific user characteristics and the user context. We propose an adaptation platform that consists in a set of intelligent agents where each agent carries out an independent adaptation task. The agents apply machine learning techniques to support the user modelling for the adaptation process

Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Understanding and production of prosody in children with Williams syndrome: A developmental trajectory approach

This study investigated the development of three aspects of linguistic prosody in a group of children with Williams syndrome compared to typically developing children. The prosodic abilities investigated were: (1) the ability to understand and use prosody to make specific words or syllables stand out in an utterance (focus); (2) the ability to understand and use prosody to disambiguate complex noun phrases (chunking); (3) the ability to understand and use prosody to regulate conversational behaviour (turn-end). The data were analysed using a cross-sectional developmental trajectory approach. The results showed that, relative to chronological age, there was a delayed onset in the development of the ability of children with WS to use prosody to signal the most important word in an utterance (the focus function). Delayed rate of development was found for all the other aspects of expressive and receptive prosody under investigation. However, when non-verbal mental age was taken into consideration, there were no differences between the children with WS and the controls neither with the onset nor with the rate of development for any of the prosodic skills under investigation apart from the ability to use prosody in order to regulate conversational behaviour. We conclude that prosody is not a ‘preserved’ cognitive skill in WS. The genetic factors, development in other cognitive domains and environmental influences affect developmental pathways and as a result, development proceeds along an atypical trajectory.

Do children with Williams syndrome have unusual vocabularies?

Aims: The present study investigated whether children with Williams syndrome (WS) produced a higher number of different word roots and low-frequency words in spontaneous speech in a topic controlled setting. Method: A group of children with WS was compared to a group of typically developing children matched for chronological age (CA), and a group of typically developing children matched for receptive language abilities (LA). A further comparison was made between the WS group and a group of children matched for non-verbal abilities (NA). Spontaneous speech was elicited using a narrative task. The data were analysed using three different measures of lexical diversity. The results revealed that the children with WS neither produce a higher number of different word roots nor significantly more low-frequency items in comparison to the CA, LA and NA matched participants. Furthermore, language and non-verbal abilities did not predict the number of different and low frequency words used by the typically developing children, however in the WS group non-verbal abilities predicted the number of low-frequency words and receptive language skills predicted the number of different words produced. It is concluded that individuals with WS do not have unusual vocabularies and that the subdomain of language, lexical semantics, does not seem to be an independent cognitive skill. (c) 2007 Elsevier Ltd. All rights reserved.

Divided attention, selective attention and drawing: processing preferences in Williams syndrome are dependent on the task administered

The visuo-spatial abilities of individuals with Williams syndrome (WS) have consistently been shown to be generally weak. These poor visuo-spatial abilities have been ascribed to a local processing bias by some [R. Rossen, E.S. Klima, U. Bellugi, A. Bihrle, W. Jones, Interaction between language and cognition: evidence from Williams syndrome, in: J. Beitchman, N. Cohen, M. Konstantareas, R. Tannock (Eds.), Language, Learning and Behaviour disorders: Developmental, Behavioural and Clinical Perspectives, Cambridge University Press, New York, 1996, pp. 367-392] and conversely, to a global processing bias by others [Psychol. Sci. 10 (1999) 453]. In this study, two identification versions and one drawing version of the Navon hierarchical processing task, a non-verbal task, were employed to investigate this apparent contradiction. The two identification tasks were administered to 21 individuals with WS, 21 typically developing individuals, matched by non-verbal ability, and 21 adult participants matched to the WS group by mean chronological age (CA). The third, drawing task was administered to the WS group and the typically developing (TD) controls only. It was hypothesised that the WS group would show differential processing biases depending on the type of processing the task was measuring. Results from two identification versions of the Navon task measuring divided and selective attention showed that the WS group experienced equal interference from global to local as from local to global levels, and did not show an advantage of one level over another. This pattern of performance was broadly comparable to that of the control groups. The third task, a drawing version of the Navon task, revealed that individuals with WS were significantly better at drawing the local form in comparison to the global figure, whereas the typically developing control group did not show a bias towards either level. In summary, this study demonstrates that individuals with WS do not have a local or a global processing bias when asked to identify stimuli, but do show a local bias in their drawing abilities. This contrast may explain the apparently contrasting findings from previous studies. (C) 2002 Elsevier Science Ltd. All rights reserved.

Integrating constructive feedback in personalised e-learning

When using e-learning material some students progress readily, others have difficulties. In a traditional classroom the teacher would identify those with difficulties and direct them to additional resources. This support is not easily available within e-learning. A new approach to providing constructive feedback is developed that will enable an e-learning system to identify areas of weakness and provide guidance on further study. The approach is based on the tagging of learning material with appropriate keywords that indicate the contents. Thus if a student performs poorly on an assessment on topic X, there is a need to suggest further study of X and participation in activities related to X such as forums. As well as supporting the learner this type of constructive feedback can also inform other stakeholders. For example a tutor can monitor the progress of a cohort; an instructional designer can monitor the quality of learning objects in facilitating the appropriate knowledge across many learners.

Single trial BCI operation via Wackermann parameters

Accurate single trial P300 classification lends itself to fast and accurate control of Brain Computer Interfaces (BCIs). Highly accurate classification of single trial P300 ERPs is achieved by characterizing the EEG via corresponding stationary and time-varying Wackermann parameters. Subsets of maximally discriminating parameters are then selected using the Network Clustering feature selection algorithm and classified with Naive-Bayes and Linear Discriminant Analysis classifiers. Hence the method is assessed on two different data-sets from BCI competitions and is shown to produce accuracies of between approximately 70% and 85%. This is promising for the use of Wackermann parameters as features in the classification of single-trial ERP responses.

Self-organising neural networks for adaptive control

Self-organizing neural networks have been implemented in a wide range of application areas such as speech processing, image processing, optimization and robotics. Recent variations to the basic model proposed by the authors enable it to order state space using a subset of the input vector and to apply a local adaptation procedure that does not rely on a predefined test duration limit. Both these variations have been incorporated into a new feature map architecture that forms an integral part of an Hybrid Learning System (HLS) based on a genetic-based classifier system. Problems are represented within HLS as objects characterized by environmental features. Objects controlled by the system have preset targets set against a subset of their features. The system's objective is to achieve these targets by evolving a behavioural repertoire that efficiently explores and exploits the problem environment. Feature maps encode two types of knowledge within HLS — long-term memory traces of useful regularities within the environment and the classifier performance data calibrated against an object's feature states and targets. Self-organization of these networks constitutes non-genetic-based (experience-driven) learning within HLS. This paper presents a description of the HLS architecture and an analysis of the modified feature map implementing associative memory. Initial results are presented that demonstrate the behaviour of the system on a simple control task.

Blueberry supplementation induces spatial memory improvements and region-specific regulation of hippocampal BDNF mRNA expression in young rats

Rationale: Flavonoid-rich foods have been shown to be able to reverse age-related cognitive deficits in memory and learning in both animals and humans. However, to date, there have been only a limited number of studies investigating the effects of flavonoid-rich foods on cognition in young/healthy animals. Objectives: The aim of this study was to investigate the effects of a blueberry-rich diet in young animals using a spatial working memory paradigm, the delayed non-match task, using an eight-arm radial maze. Furthermore, the mechanisms underlying such behavioural effects were investigated. Results: We show that a 7-week supplementation with a blueberry diet (2 % w/w) improves the spatial memory performance of young rats (2 months old). Blueberry-fed animals also exhibited a faster rate of learning compared to those on the control diet. These behavioural outputs were accompanied by the activation of extracellular signal-related kinase (ERK1/2), increases in total cAMP-response element binding protein (CREB) and elevated levels of pro- and mature brain-derived neurotrophic factor (BDNF) in the hippocampus. Changes in hippocampal CREB correlated well with memory performance. Further regional analysis of BDNF gene expression in the hippocampus revealed a specific increase in BDNF mRNA in the dentate gyrus and CA1 areas of hippocampi of blueberry-fed animals. Conclusions: The present study suggests that consumption of flavonoid-rich blueberries has a positive impact on spatial learning performance in young healthy animals, and these improvements are linked to the activation of ERK–CREB– BDNF pathway in the hippocampus.