Optical learning in a smart spatial light modulator

A synaptic plane rendered by an array of smart pixels was described regarding its application as a complementary component for neural network implementation. The smart spatial light modulator featured auto-modification abilities. Thus, an optical system incorporating this device can show self-reliant optical learning. Furthermore, the optical system design, in the area of its optical interconnection scheme, is highly flexible since the independent weight-plane pixels eliminated the difficulty between weight update calculation and weight representation.

Short- and long-term changes in joint co-contraction associated with motor learning as revealed from surface EMG.

In the field of motor control, two hypotheses have been controversial: whether the brain acquires internal models that generate accurate motor commands, or whether the brain avoids this by using the viscoelasticity of musculoskeletal system. Recent observations on relatively low stiffness during trained movements support the existence of internal models. However, no study has revealed the decrease in viscoelasticity associated with learning that would imply improvement of internal models as well as synergy between the two hypothetical mechanisms. Previously observed decreases in electromyogram (EMG) might have other explanations, such as trajectory modifications that reduce joint torques. To circumvent such complications, we required strict trajectory control and examined only successful trials having identical trajectory and torque profiles. Subjects were asked to perform a hand movement in unison with a target moving along a specified and unusual trajectory, with shoulder and elbow in the horizontal plane at the shoulder level. To evaluate joint viscoelasticity during the learning of this movement, we proposed an index of muscle co-contraction around the joint (IMCJ). The IMCJ was defined as the summation of the absolute values of antagonistic muscle torques around the joint and computed from the linear relation between surface EMG and joint torque. The IMCJ during isometric contraction, as well as during movements, was confirmed to correlate well with joint stiffness estimated using the conventional method, i.e., applying mechanical perturbations. Accordingly, the IMCJ during the learning of the movement was computed for each joint of each trial using estimated EMG-torque relationship. At the same time, the performance error for each trial was specified as the root mean square of the distance between the target and hand at each time step over the entire trajectory. The time-series data of IMCJ and performance error were decomposed into long-term components that showed decreases in IMCJ in accordance with learning with little change in the trajectory and short-term interactions between the IMCJ and performance error. A cross-correlation analysis and impulse responses both suggested that higher IMCJs follow poor performances, and lower IMCJs follow good performances within a few successive trials. Our results support the hypothesis that viscoelasticity contributes more when internal models are inaccurate, while internal models contribute more after the completion of learning. It is demonstrated that the CNS regulates viscoelasticity on a short- and long-term basis depending on performance error and finally acquires smooth and accurate movements while maintaining stability during the entire learning process.

Different mechanisms involved in adaptation to stable and unstable dynamics.

Recently, we demonstrated that humans can learn to make accurate movements in an unstable environment by controlling magnitude, shape, and orientation of the endpoint impedance. Although previous studies of human motor learning suggest that the brain acquires an inverse dynamics model of the novel environment, it is not known whether this control mechanism is operative in unstable environments. We compared learning of multijoint arm movements in a "velocity-dependent force field" (VF), which interacted with the arm in a stable manner, and learning in a "divergent force field" (DF), where the interaction was unstable. The characteristics of error evolution were markedly different in the 2 fields. The direction of trajectory error in the DF alternated to the left and right during the early stage of learning; that is, signed error was inconsistent from movement to movement and could not have guided learning of an inverse dynamics model. This contrasted sharply with trajectory error in the VF, which was initially biased and decayed in a manner that was consistent with rapid feedback error learning. EMG recorded before and after learning in the DF and VF are also consistent with different learning and control mechanisms for adapting to stable and unstable dynamics, that is, inverse dynamics model formation and impedance control. We also investigated adaptation to a rotated DF to examine the interplay between inverse dynamics model formation and impedance control. Our results suggest that an inverse dynamics model can function in parallel with an impedance controller to compensate for consistent perturbing force in unstable environments.

Mechanisms of H+ modulation of glycinergic response in rat sacral dorsal commissural neurons

Many ionotropic receptors are modulated by extracellular H+. So far, few studies have directly addressed the role of such modulation at synapses. In the present study, we investigated the effects of changes in extracellular pH on glycinergic miniature inhibitory postsynaptic currents (mIPSCs) as well as glycine-evoked currents (I-Gly) in mechanically dissociated spinal neurons with native synaptic boutons preserved. H+ modulated both the mIPSCs and I-Gly, biphasically, although it activated an amiloride-sensitive inward current by itself. Decreasing extracellular pH reversibly inhibited the amplitude of the mIPSCs and I-Gly, while increasing external pH reversibly potentiated these parameters. Blockade of acid-sensing ion channels (ASICs) with amiloride, the selective antagonist of ASICs, or decreasing intracellular pH did not alter the modulatory effect of H+ on either mIPSCs or I-Gly, H+ shifted the EC50 of the glycine concentration-response curve from 49.3 +/- 5.7 muM at external pH 7.4 to 131.5 +/- 8.1 muM at pH 5.5, without altering the Cl- selectivity of the glycine receptor (GlyR), the Hill coefficient and the maximal I-Gly, suggesting a competitive inhibition of I-Gly by H+. Both Zn2+ and H+ inhibited I-Gly. However, H+ induced no further inhibition of I-Gly in the presence of a saturating concentration of Zn2+. In addition, H+ significantly affected the kinetics of glycinergic mIPSCs and I-Gly. It is proposed that H+ and/or Zn2+ compete with glycine binding and inhibit the amplitude of glycinergic mIPSCs and I-Gly. Moreover, binding of H+ induces a global conformational change in GlyR, which closes the GlyR Cl- channel and results in the acceleration of the seeming desensitization of IGly as well as speeding up the decay time constant of glycinergic mIPSCs. However, the deprotonation rate is faster than the unbinding rate of glycine from the GlyR, leading to reactivation of the undesensitized GlyR after washout of agonist and the appearance of a rebound I-Gly. H+ also modulated the glycine cotransmitter, GABA-activated current (I-GABA). Taken together, the results support a 'conformational coupling' model for H+ modulation of the GlyR and suggest that W may act as a novel modulator for inhibitory neurotransmission in the mammalian spinal cord.

Effects of unconditioned and conditioned aversive stimuli in an intense fear conditioning paradigm on synaptic plasticity in the hippocampal CA1 area in vivo

Repeated vivid recalls or flashbacks of traumatic memories and memory deficits are the cardinal features of post-traumatic stress disorder (PTSD). The underlying mechanisms are not fully understood yet. Here, we examined the effects of very strong fear conditioning (20 pairings of a light with a 1.5-mA, 0.5-s foot shock) and subsequent reexposure to the conditioning context (chamber A), a similar context (chamber B), and/or to the fear conditioned stimulus (CS) (a light) on synaptic plasticity in the hippocampal CA1 area in anesthetized Sprague-Dawley rats. The conditioning procedure resulted in very strong conditioned fear, as reflected by high levels of persistent freezing, to both the contexts and to the CS, 24 h after fear conditioning. The induction of long-term potentiation ON was blocked immediately after fear conditioning. It was still markedly impaired 24 h after fear conditioning; reexposure to the conditioning chamber A (CA) or to a similar chamber 13 (CB) did not affect the impairment. However, presentation of the CS in the CA exacerbated the impairment of LTP, whereas the CS presentation in a CB ameliorated the impairment so that LTP induction did not differ from that of control groups. The induction of long-term depression (LTD) was facilitated immediately, but not 24 h, after fear conditioning. Only reexposure to the CS in the CA, but not reexposure to either chamber A or B alone, or the CS in chamber B, 24 h after conditioning, reinstated the facilitation of LTD induction. These data demonstrate that unconditioned and conditioned aversive stimuli in an intense fear conditioning paradigm can have profound effects on hippocampal synaptic plasticity, which may aid to understand the mechanisms underlying impairments of hippocampus-dependent memory by stress or in PTSD. (c) 2005 Wiley-Liss, Inc.

Enriched environment treatment restores impaired hippocampal synaptic plasticity and cognitive deficits induced by prenatal chronic stress

Prenatal stress can cause long-term effects on cognitive functions in offspring. Hippocampal synaptic plasticity, believed to be the mechanism underlying certain types of learning and memory, and known to be sensitive to behavioral stress, can be changed

Principles of sensorimotor learning.

The exploits of Martina Navratilova and Roger Federer represent the pinnacle of motor learning. However, when considering the range and complexity of the processes that are involved in motor learning, even the mere mortals among us exhibit abilities that are impressive. We exercise these abilities when taking up new activities - whether it is snowboarding or ballroom dancing - but also engage in substantial motor learning on a daily basis as we adapt to changes in our environment, manipulate new objects and refine existing skills. Here we review recent research in human motor learning with an emphasis on the computational mechanisms that are involved.

Computational mechanisms of sensorimotor control.

In order to generate skilled and efficient actions, the motor system must find solutions to several problems inherent in sensorimotor control, including nonlinearity, nonstationarity, delays, redundancy, uncertainty, and noise. We review these problems and five computational mechanisms that the brain may use to limit their deleterious effects: optimal feedback control, impedance control, predictive control, Bayesian decision theory, and sensorimotor learning. Together, these computational mechanisms allow skilled and fluent sensorimotor behavior.

Principles of sensorimotor learning

The exploits of Martina Navratilova and Roger Federer represent the pinnacle of motor learning. However, when considering the range and complexity of the processes that are involved in motor learning, even the mere mortals among us exhibit abilities that are impressive. We exercise these abilities when taking up new activities-whether it is snowboarding or ballroom dancing-but also engage in substantial motor learning on a daily basis as we adapt to changes in our environment, manipulate new objects and refine existing skills. Here we review recent research in human motor learning with an emphasis on the computational mechanisms that are involved. © 2011 Macmillan Publishers Limited. All rights reserved.

Perceptual learning, roving and the unsupervised bias

Perceptual learning improves perception through training. Perceptual learning improves with most stimulus types but fails when . certain stimulus types are mixed during training (roving). This result is surprising because classical supervised and unsupervised neural network models can cope easily with roving conditions. What makes humans so inferior compared to these models? As experimental and conceptual work has shown, human perceptual learning is neither supervised nor unsupervised but reward-based learning. Reward-based learning suffers from the so-called unsupervised bias, i.e., to prevent synaptic " drift" , the . average reward has to be exactly estimated. However, this is impossible when two or more stimulus types with different rewards are presented during training (and the reward is estimated by a running average). For this reason, we propose no learning occurs in roving conditions. However, roving hinders perceptual learning only for combinations of similar stimulus types but not for dissimilar ones. In this latter case, we propose that a critic can estimate the reward for each stimulus type separately. One implication of our analysis is that the critic cannot be located in the visual system. © 2011 Elsevier Ltd.

Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.

Growth of Motor Coordination in Early Robot Learning

M. H. Lee and Q. Meng, Growth of Motor Coordination in Early Robot Learning, IJCAI-05, 2005.

Psychologically Inspired Sensory-Motor Development in Early Robot Learning

Lee, M., Meng, Q. (2005). Psychologically Inspired Sensory-Motor Development in Early Robot Learning. International Journal of Advanced Robotic Systems, 325-334.

How Does the Cerebral Cortex Work? Developement, Learning, Attention, and 3D Vision by Laminar Circuits of Visual Cortex

A key goal of behavioral and cognitive neuroscience is to link brain mechanisms to behavioral functions. The present article describes recent progress towards explaining how the visual cortex sees. Visual cortex, like many parts of perceptual and cognitive neocortex, is organized into six main layers of cells, as well as characteristic sub-lamina. Here it is proposed how these layered circuits help to realize the processes of developement, learning, perceptual grouping, attention, and 3D vision through a combination of bottom-up, horizontal, and top-down interactions. A key theme is that the mechanisms which enable developement and learning to occur in a stable way imply properties of adult behavior. These results thus begin to unify three fields: infant cortical developement, adult cortical neurophysiology and anatomy, and adult visual perception. The identified cortical mechanisms promise to generalize to explain how other perceptual and cognitive processes work.

A Quantitative Evaluation of the AVITEWRITE Model of Handwriting Learning

Much sensory-motor behavior develops through imitation, as during the learning of handwriting by children. Such complex sequential acts are broken down into distinct motor control synergies, or muscle groups, whose activities overlap in time to generate continuous, curved movements that obey an intense relation between curvature and speed. The Adaptive Vector Integration to Endpoint (AVITEWRITE) model of Grossberg and Paine (2000) proposed how such complex movements may be learned through attentive imitation. The model suggest how frontal, parietal, and motor cortical mechanisms, such as difference vector encoding, under volitional control from the basal ganglia, interact with adaptively-timed, predictive cerebellar learning during movement imitation and predictive performance. Key psycophysical and neural data about learning to make curved movements were simulated, including a decrease in writing time as learning progresses; generation of unimodal, bell-shaped velocity profiles for each movement synergy; size scaling with isochrony, and speed scaling with preservation of the letter shape and the shapes of the velocity profiles; an inverse relation between curvature and tangential velocity; and a Two-Thirds Power Law relation between angular velocity and curvature. However, the model learned from letter trajectories of only one subject, and only qualitative kinematic comparisons were made with previously published human data. The present work describes a quantitative test of AVITEWRITE through direct comparison of a corpus of human handwriting data with the model's performance when it learns by tracing human trajectories. The results show that model performance was variable across subjects, with an average correlation between the model and human data of 89+/-10%. The present data from simulations using the AVITEWRITE model highlight some of its strengths while focusing attention on areas, such as novel shape learning in children, where all models of handwriting and learning of other complex sensory-motor skills would benefit from further research.