Novelty and Habituation: the Driving Forces in Early Stage Learning for Developmental Robotics

Q. Meng and M. H. Lee, Novelty and Habituation: the Driving Forces in Early Stage Learning for Developmental Robotics, AI-Workshop on NeuroBotics, University of Ulm, Germany. September 2004.

Staged Competence Learning in Developmental Robotics

M.H. Lee, Q. Meng and F. Chao, 'Staged Competence Learning in Developmental Robotics', Adaptive Behavior, 15(3), pp 241-255, 2007. the full text will be available in September 2008

Staged development of Robot Motor Coordination

M.H. Lee and Q. Meng, 'Staged development of Robot Motor Coordination', IEEE International Conference on Systems, Man and Cybernetics, (IEEE SMC 05), Hawaii, USA, v3, 2917-2922, 2005.

A Quantitative Evaluation of the AVITEWRITE Model of Handwriting Learning

Much sensory-motor behavior develops through imitation, as during the learning of handwriting by children. Such complex sequential acts are broken down into distinct motor control synergies, or muscle groups, whose activities overlap in time to generate continuous, curved movements that obey an intense relation between curvature and speed. The Adaptive Vector Integration to Endpoint (AVITEWRITE) model of Grossberg and Paine (2000) proposed how such complex movements may be learned through attentive imitation. The model suggest how frontal, parietal, and motor cortical mechanisms, such as difference vector encoding, under volitional control from the basal ganglia, interact with adaptively-timed, predictive cerebellar learning during movement imitation and predictive performance. Key psycophysical and neural data about learning to make curved movements were simulated, including a decrease in writing time as learning progresses; generation of unimodal, bell-shaped velocity profiles for each movement synergy; size scaling with isochrony, and speed scaling with preservation of the letter shape and the shapes of the velocity profiles; an inverse relation between curvature and tangential velocity; and a Two-Thirds Power Law relation between angular velocity and curvature. However, the model learned from letter trajectories of only one subject, and only qualitative kinematic comparisons were made with previously published human data. The present work describes a quantitative test of AVITEWRITE through direct comparison of a corpus of human handwriting data with the model's performance when it learns by tracing human trajectories. The results show that model performance was variable across subjects, with an average correlation between the model and human data of 89+/-10%. The present data from simulations using the AVITEWRITE model highlight some of its strengths while focusing attention on areas, such as novel shape learning in children, where all models of handwriting and learning of other complex sensory-motor skills would benefit from further research.

The Hippocampus and Cerebellum in Adaptively Timed Learning, Recognition, and Movement

The concepts of declarative memory and procedural memory have been used to distinguish two basic types of learning. A neural network model suggests how such memory processes work together as recognition learning, reinforcement learning, and sensory-motor learning take place during adaptive behaviors. To coordinate these processes, the hippocampal formation and cerebellum each contain circuits that learn to adaptively time their outputs. Within the model, hippocampal timing helps to maintain attention on motivationally salient goal objects during variable task-related delays, and cerebellar timing controls the release of conditioned responses. This property is part of the model's description of how cognitive-emotional interactions focus attention on motivationally valued cues, and how this process breaks down due to hippocampal ablation. The model suggests that the hippocampal mechanisms that help to rapidly draw attention to salient cues could prematurely release motor commands were not the release of these commands adaptively timed by the cerebellum. The model hippocampal system modulates cortical recognition learning without actually encoding the representational information that the cortex encodes. These properties avoid the difficulties faced by several models that propose a direct hippocampal role in recognition learning. Learning within the model hippocampal system controls adaptive timing and spatial orientation. Model properties hereby clarify how hippocampal ablations cause amnesic symptoms and difficulties with tasks which combine task delays, novelty detection, and attention towards goal objects amid distractions. When these model recognition, reinforcement, sensory-motor, and timing processes work together, they suggest how the brain can accomplish conditioning of multiple sensory events to delayed rewards, as during serial compound conditioning.

A Neural Network of Adaptively Timed Reinforcement Learning and Hippocampal Dynamics

A neural model is described of how adaptively timed reinforcement learning occurs. The adaptive timing circuit is suggested to exist in the hippocampus, and to involve convergence of dentate granule cells on CA3 pyramidal cells, and NMDA receptors. This circuit forms part of a model neural system for the coordinated control of recognition learning, reinforcement learning, and motor learning, whose properties clarify how an animal can learn to acquire a delayed reward. Behavioral and neural data are summarized in support of each processing stage of the system. The relevant anatomical sites are in thalamus, neocortex, hippocampus, hypothalamus, amygdala, and cerebellum. Cerebellar influences on motor learning are distinguished from hippocampal influences on adaptive timing of reinforcement learning. The model simulates how damage to the hippocampal formation disrupts adaptive timing, eliminates attentional blocking, and causes symptoms of medial temporal amnesia. It suggests how normal acquisition of subcortical emotional conditioning can occur after cortical ablation, even though extinction of emotional conditioning is retarded by cortical ablation. The model simulates how increasing the duration of an unconditioned stimulus increases the amplitude of emotional conditioning, but does not change adaptive timing; and how an increase in the intensity of a conditioned stimulus "speeds up the clock", but an increase in the intensity of an unconditioned stimulus does not. Computer simulations of the model fit parametric conditioning data, including a Weber law property and an inverted U property. Both primary and secondary adaptively timed conditioning are simulated, as are data concerning conditioning using multiple interstimulus intervals (ISIs), gradually or abruptly changing ISis, partial reinforcement, and multiple stimuli that lead to time-averaging of responses. Neurobiologically testable predictions are made to facilitate further tests of the model.

Neural pathways mediating cross education of motor function

Cross education is the process whereby training of one limb gives rise to enhancements in the performance of the opposite, untrained limb. Despite interest in this phenomenon having been sustained for more than a century, a comprehensive explanation of the mediating neural mechanisms remains elusive. With new evidence emerging that cross education may have therapeutic utility, the need to provide a principled evidential basis upon which to design interventions becomes ever more pressing. Generally, mechanistic accounts of cross education align with one of two explanatory frameworks. Models of the 'cross activation' variety encapsulate the observation that unilateral execution of a movement task gives rise to bilateral increases in corticospinal excitability. The related conjecture is that such distributed activity, when present during unilateral practice, leads to simultaneous adaptations in neural circuits that project to the muscles of the untrained limb, thus facilitating subsequent performance of the task. Alternatively, 'bilateral access' models entail that motor engrams formed during unilateral practise, may subsequently be utilised bilaterally - that is, by the neural circuitry that constitutes the control centres for movements of both limbs. At present there is a paucity of direct evidence that allows the corresponding neural processes to be delineated, or their relative contributions in different task contexts to be ascertained. In the current review we seek to synthesise and assimilate the fragmentary information that is available, including consideration of knowledge that has emerged as a result of technological advances in structural and functional brain imaging. An emphasis upon task dependency is maintained throughout, the conviction being that the neural mechanisms that mediate cross education may only be understood in this context. © 2013 Ruddy and Carson.

An examination of the effects on learning seen in children afforded the opportunity to control the order of repetitions for three novel spatiotemporal sequences

Children were afforded the opportunity to control the order of repetitions for three novel spatiotemporal sequences. The following was predicted: a) children and adults in the self-regulated (SELF) groups would produce faster movement (MT) and reaction times (R T) and greater recall success (RS) during retention compared to the age-matched yoked (YOKE) groups; b) children would choose to switch sequences less often than adults; c) adults would produce faster MT and RT and greater RS than the children during acquisition and retention, independent of experimental group. During acquisition, no effects were seen for RS, however for MT and RT there was a main effect for age as well as block. During retention a main effect for practice condition was seen for RS and failed to reach statistical significance for MT and RT, thus partially supporting our first and second hypotheses. The third hypothesis was not supported.

Examining peer-controlled KR schedules during the learning of a movement-timing task as a function of task experience.

Learners can be provided with feedback in the form of knowledge of results (KR), under self-controlled and peer-controlled schedules. Recently, McRae, Hansen, and Patterson (2015), identified that inexperienced peers can provide KR that can facilitate motor skill acquisition. However, it is currently unknown whether previous task experience differentially impacts how peers present learners with KR and whether this KR impacts motor skill acquisition. In the present study, participants were randomly assigned to become inexperienced peer facilitators, learners with an inexperienced peer, learners with self-control who later became experienced peers, learners with an experienced peer, or learners in a control group. During acquisition learners completed a serial-timing task with a goal of 2500ms and returned approximately twenty four hours later for a delayed retention, time transfer, and pattern transfer test. We predicted that during the delayed tests, learners with self-control would outperform all other groups. Furthermore, we predicted that learners who received KR from experienced peers would outperform learners who received KR from inexperienced peers. However, our results indicated that participants who received peer-controlled and self-controlled KR schedules learned the task in an equivalent manner. Thus, our results are novel as they identify that inexperienced peers can provide KR that is as effective as KR provided by experienced peers and KR requested under self-controlled conditions.

Neural substrates and functional connectivity associated with sleep-dependent and independent consolidation of new motor skills

La mémoire n’est pas un processus unitaire et est souvent divisée en deux catégories majeures: la mémoire déclarative (pour les faits) et procédurale (pour les habitudes et habiletés motrices). Pour perdurer, une trace mnésique doit passer par la consolidation, un processus par lequel elle devient plus robuste et moins susceptible à l’interférence. Le sommeil est connu comme jouant un rôle clé pour permettre le processus de consolidation, particulièrement pour la mémoire déclarative. Depuis plusieurs années cependant, son rôle est aussi reconnu pour la mémoire procédurale. Il est par contre intéressant de noter que ce ne sont pas tous les types de mémoire procédurale qui requiert le sommeil afin d’être consolidée. Entre autres, le sommeil semble nécessaire pour consolider un apprentissage de séquences motrices (s’apparentant à l’apprentissage du piano), mais pas un apprentissage d’adaptation visuomotrice (tel qu’apprendre à rouler à bicyclette). Parallèlement, l’apprentissage à long terme de ces deux types d’habiletés semble également sous-tendu par des circuits neuronaux distincts; c’est-à-dire un réseau cortico-striatal et cortico-cérébelleux respectivement. Toutefois, l’implication de ces réseaux dans le processus de consolidation comme tel demeure incertain. Le but de cette thèse est donc de mieux comprendre le rôle du sommeil, en contrôlant pour le simple passage du temps, dans la consolidation de ces deux types d’apprentissage, à l’aide de l’imagerie par résonnance magnétique fonctionnelle et d’analyses de connectivité cérébrale. Nos résultats comportementaux supportent l’idée que seul l’apprentissage séquentiel requiert le sommeil pour déclencher le processus de consolidation. Nous suggérons de plus que le putamen est fortement associé à ce processus. En revanche, les performances d’un apprentissage visuomoteur s’améliorent indépendamment du sommeil et sont de plus corrélées à une plus grande activation du cervelet. Finalement, en explorant l’effet du sommeil sur la connectivité cérébrale, nos résultats démontrent qu’en fait, un système cortico-striatal semble être plus intégré suite à la consolidation. C’est-à-dire que l’interaction au sein des régions du système est plus forte lorsque la consolidation a eu lieu, après une nuit de sommeil. En opposition, le simple passage du temps semble nuire à l’intégration de ce réseau cortico-striatal. En somme, nous avons pu élargir les connaissances quant au rôle du sommeil pour la mémoire procédurale, notamment en démontrant que ce ne sont pas tous les types d’apprentissages qui requièrent le sommeil pour amorcer le processus de consolidation. D’ailleurs, nous avons également démontré que cette dissociation de l’effet du sommeil est également reflétée par l’implication de deux réseaux cérébraux distincts. À savoir, un réseau cortico-striatal et un réseau cortico-cérébelleux pour la consolidation respective de l’apprentissage de séquence et d’adaptation visuomotrice. Enfin, nous suggérons que la consolidation durant le sommeil permet de protéger et favoriser une meilleure cohésion au sein du réseau cortico-striatal associé à notre tâche; un phénomène qui, s’il est retrouvé avec d’autres types d’apprentissage, pourrait être considéré comme un nouveau marqueur de la consolidation.

Estimulacion magnetica trascraneal en el desempeno motor en enfermedades del sistema nervioso central: Revision Sistematica

Introducción: El uso de la estimulación cerebral no invasiva en procesos de rehabilitación es de gran interés, por cuanto con mediación tecnológica se generan nuevas posibilidades de recuperación motora, a partir de la activación de la corteza cerebral. El objetivo del estudio es establecer la evidencia del uso terapéutico de la EMT, relacionado con el desempeño motor de pacientes con enfermedades del sistema nervioso central. Metodología: Se realizó una revisión sistemática de la literatura. Se incluyeron 10 estudios en el análisis cualitativo que incluyó la evaluación de calidad con la escala de Jadad y del riesgo de sesgo con la herramienta Cochrane. Fueron excluidos 1613 estudios. Se aplicó el protocolo del estudio para la extracción, revisión y validez de los estudios incluidos. Resultados: La evidencia disponible muestra resultados positivos del uso terapéutico de la EMT en el desempeño motor en aspectos como la aceleración, la fuerza de pinza y de agarre, la estabilidad y la fuerza muscular, así como una mejor velocidad de la marcha y una disminución en la frecuencia y severidad de los espasmos. Discusión: La EMT puede constituir una estrategia terapéutica para mejorar el desempeño motor en pacientes con ECV, Lesión Medular y enfermedad de Parkinson, que requiere más investigación por la heterogeneidad de los diseños y medidas de descenlace utilizados, así como por la alta variabilidad interindividual que hace complejo estandarizar los protocolos de su uso terapéutico.

Learning to minimize energy costs and maximize mechanical work in a bimanual coordination task

The authors addressed the hypothesis that economy in motor coordination is a learning phenomenon realized by both reduced energy cost for a given workload and more external work at the same prepractice metabolic and attentional energy expenditure. "Self-optimization" of movement parameters has been proposed to reflect learned motor adaptations that minimize energy costs. Twelve men aged 22.3 [+ or -] 3.9 years practiced a 90[degrees] relative phase, upper limb, independent ergometer cycling task at 60 rpm, followed by a transfer test of unpracticed (45 and 75 rpm) and self-paced cadences. Performance in all conditions was initially unstable, inaccurate, and relatively high in both metabolic and attentional energy costs. With practice, coordinative stability increased, more work was performed for the same metabolic and attentional costs, and the same work was done at a reduced energy cost. Self-paced cycling was initially below the metabolically optimal, but following practice at 60 rpm was closer to optimal cadence. Given the many behavioral options of the motor system in meeting a variety of everyday movement task goals, optimal metabolic and attentional energy criteria may provide a solution to the problem of selecting the most adaptive coordination and control parameters.

Procedural learning in children with developmental coordination disorder

Despite the fact that developmental coordination disorder (DCD) is characterised by a deficit in the ability to learn or automate motor skills, few studies have examined motor learning over repeated trials. In this study we examined procedural learning in a group of 10 children with DCD (aged 8–12 years) and age-matched controls without DCD. The learning task was modelled on that of Nissen and Bullemer [Cognitive Psychology 19 (1987) 1]. Children performed a serial reaction time (SRT) task in which they were required to learn a spatial sequence that repeated itself every 10 trials. Children were not aware of the repetition. Spatial targets were four (horizontal) locations presented on a computer monitor. Children responded using four response keys with the same horizontal mapping as the stimulus. They were tested over five blocks of 100 trials each. The first four blocks presented the same repeating sequence, while the fifth block was randomised. Procedural learning was indexed by the slope of the regression of RT on blocks 1–4. Results showed that most children displayed strong procedural learning of the sequence, despite having no explicit knowledge about it. Overall, there was no group difference in the magnitude of learning over blocks of trials – most children performed within the normal range. Procedural learning for simple sequential movements appears to be intact in children with DCD. This suggests that cortico-striatal circuits that are strongly implicated in the sequencing of simple movements appear to be function normally in DCD.

Effects of contextual interference on children learning forehand tennis groundstrokes

Previous research on contextual interference theory in controlled laboratory situations consistently illustrated that random practice was superior to blocked practice when learning motor skills. However, when considered in relation to physical education class settings the findings of the contextual interference experiments were not as uniform. Furthermore, the results of the contextual interference research were ambiguous when an open skill was used as the experimental task, with no definite trend evident. Random practice was found more effective for learning (del Rey, 1989) whereas French, Rink and Werner (1990) demonstrated blocked practice to be superior. In the present study, the influence of high and low contextual interference as practice schedules was investigated within an applied sports setting using an open sports skill as the experimental task. Two groups of boys and girls, 8-9 and 10-12 years of age, were taught a forehand tennis groundstroke using both their preferred and non-preferred hands over a ten week coaching and practice period. The findings showed that male subjects were significantly better at the experimental task than female subjects at the pre-test stage only. The result also demonstrated that the 10-12 year old subjects were significantly better than the 8-9 year olds at the task with the exception of the preferred hand at the post and retention test stage. The contextual interference effect was demonstrated in the retention phase on the preferred hand of the 10-12 year old subjects where the random practice group was significantly better than the blocked practice group in an applied sports setting. These findings were discussed in relation to the role of cognition in the learning of these motor skills and the influence of the subjects related background experiences.

The energetics of self-paced motor skills

In activities such as walking individuals can select an optimum speed that minimises energy expenditure. When learning to row, individuals initially selected fast inefficient stroke rates but learned to become more efficient by taking longer, slower strokes. The research showed, therefore, that optimum pacing depends on extensive practice and sensitivity to energy cost helps us to change movements in order to become more efficient.