The Role of Chemical Mechanisms in Neural Computation and Learning

Most computational models of neurons assume that their electrical characteristics are of paramount importance. However, all long-term changes in synaptic efficacy, as well as many short-term effects, are mediated by chemical mechanisms. This technical report explores the interaction between electrical and chemical mechanisms in neural learning and development. Two neural systems that exemplify this interaction are described and modelled. The first is the mechanisms underlying habituation, sensitization, and associative learning in the gill withdrawal reflex circuit in Aplysia, a marine snail. The second is the formation of retinotopic projections in the early visual pathway during embryonic development.

Governance of learning mechanisms : evidence from construction firms

The focus of knowledge management (KM) in the construction industry is moving towards capability building for value creation. The study reported by this paper is motivated by recent assertions about the genesis and evolution of knowledge management capability (KMC) in the strategic management field. It attempts to shed light on the governance of learning mechanisms that develop KMC within the context of construction firms. A questionnaire survey was administered to a sample of construction contractors operating in the very dynamic Hong Kong market to elicit opinions on the learning mechanisms and business outcomes of targeted firms. On the basis of a total of 149 usable responses, structural equation modeling (SEM) analysis identified relationships among knowledge-governance mechanisms, knowledge processes, and business performance, thereby supporting the existence of strategic learning loops. The study findings provide evidence from the construction context for capability assertions that knowledge-governance mechanisms and processes form learning mechanisms that carry out strategic learning to create value, effect performance outcomes, and ultimately drive the evolution of KMC. The findings imply that it is feasible for managing construction firms to govern learning mechanisms through managing the capability-based holistic KM system, thereby reconfiguring KMC to match needs in the dynamic market environment over time.

A Calcium-Dependent Plasticity Rule for HCN Channels Maintains Activity Homeostasis and Stable Synaptic Learning

Theoretical and computational frameworks for synaptic plasticity and learning have a long and cherished history, with few parallels within the well-established literature for plasticity of voltage-gated ion channels. In this study, we derive rules for plasticity in the hyperpolarization-activated cyclic nucleotide-gated (HCN) channels, and assess the synergy between synaptic and HCN channel plasticity in establishing stability during synaptic learning. To do this, we employ a conductance-based model for the hippocampal pyramidal neuron, and incorporate synaptic plasticity through the well-established Bienenstock-Cooper-Munro (BCM)-like rule for synaptic plasticity, wherein the direction and strength of the plasticity is dependent on the concentration of calcium influx. Under this framework, we derive a rule for HCN channel plasticity to establish homeostasis in synaptically-driven firing rate, and incorporate such plasticity into our model. In demonstrating that this rule for HCN channel plasticity helps maintain firing rate homeostasis after bidirectional synaptic plasticity, we observe a linear relationship between synaptic plasticity and HCN channel plasticity for maintaining firing rate homeostasis. Motivated by this linear relationship, we derive a calcium-dependent rule for HCN-channel plasticity, and demonstrate that firing rate homeostasis is maintained in the face of synaptic plasticity when moderate and high levels of cytosolic calcium influx induced depression and potentiation of the HCN-channel conductance, respectively. Additionally, we show that such synergy between synaptic and HCN-channel plasticity enhances the stability of synaptic learning through metaplasticity in the BCM-like synaptic plasticity profile. Finally, we demonstrate that the synergistic interaction between synaptic and HCN-channel plasticity preserves robustness of information transfer across the neuron under a rate-coding schema. Our results establish specific physiological roles for experimentally observed plasticity in HCN channels accompanying synaptic plasticity in hippocampal neurons, and uncover potential links between HCN-channel plasticity and calcium influx, dynamic gain control and stable synaptic learning.

Single trial learning of external dynamics: What can the brain teach us about learning mechanisms?

Humans have exceptional abilities to learn new skills, manipulate tools and objects, and interact with our environment. In order to be successful at these tasks, our brain has developed learning mechanisms to deal with and compensate for the constantly changing dynamics of the world. If this mechanism or mechanisms can be understood from a computational point of view, then they can also be used to drive the adaptability and learning of robots. In this paper, we will present a new technique for examining changes in the feedforward motor command due to adaptation. This technique can then be utilized for examining motor adaptation in humans and determining a computational algorithm which explains motor learning. © 2007.

Formation of temporal-feature maps by axonal propagation of synaptic learning

Computational maps are of central importance to a neuronal representation of the outside world. In a map, neighboring neurons respond to similar sensory features. A well studied example is the computational map of interaural time differences (ITDs), which is essential to sound localization in a variety of species and allows resolution of ITDs of the order of 10 μs. Nevertheless, it is unclear how such an orderly representation of temporal features arises. We address this problem by modeling the ontogenetic development of an ITD map in the laminar nucleus of the barn owl. We show how the owl's ITD map can emerge from a combined action of homosynaptic spike-based Hebbian learning and its propagation along the presynaptic axon. In spike-based Hebbian learning, synaptic strengths are modified according to the timing of pre- and postsynaptic action potentials. In unspecific axonal learning, a synapse's modification gives rise to a factor that propagates along the presynaptic axon and affects the properties of synapses at neighboring neurons. Our results indicate that both Hebbian learning and its presynaptic propagation are necessary for map formation in the laminar nucleus, but the latter can be orders of magnitude weaker than the former. We argue that the algorithm is important for the formation of computational maps, when, in particular, time plays a key role.

Learning Mechanisms for Information Filtering Agents

Postprint

Using strategic movement to calibrate a neural compass : a spiking network for tracking head direction in rats and robots

The head direction (HD) system in mammals contains neurons that fire to represent the direction the animal is facing in its environment. The ability of these cells to reliably track head direction even after the removal of external sensory cues implies that the HD system is calibrated to function effectively using just internal (proprioceptive and vestibular) inputs. Rat pups and other infant mammals display stereotypical warm-up movements prior to locomotion in novel environments, and similar warm-up movements are seen in adult mammals with certain brain lesion-induced motor impairments. In this study we propose that synaptic learning mechanisms, in conjunction with appropriate movement strategies based on warm-up movements, can calibrate the HD system so that it functions effectively even in darkness. To examine the link between physical embodiment and neural control, and to determine that the system is robust to real-world phenomena, we implemented the synaptic mechanisms in a spiking neural network and tested it on a mobile robot platform. Results show that the combination of the synaptic learning mechanisms and warm-up movements are able to reliably calibrate the HD system so that it accurately tracks real-world head direction, and that calibration breaks down in systematic ways if certain movements are omitted. This work confirms that targeted, embodied behaviour can be used to calibrate neural systems, demonstrates that ‘grounding’ of modeled biological processes in the real world can reveal underlying functional principles (supporting the importance of robotics to biology), and proposes a functional role for stereotypical behaviours seen in infant mammals and those animals with certain motor deficits. We conjecture that these calibration principles may extend to the calibration of other neural systems involved in motion tracking and the representation of space, such as grid cells in entorhinal cortex.

On Neuron Mechanisms Used to Resolve Mental Problems of Identification and Learning in Sensorium

The paper considers some possible neuron mechanisms that do not contradict biological data. They are represented in terms of the notion of an elementary sensorium discussed in the previous authors’ works. Such mechanisms resolve problems of two large classes: when identification mechanisms are used and when sensory learning mechanisms are applied along with identification.

Sequential versus simultaneous viewing of an environment : effects of focal attention to individual object locations on visual spatial learning

We investigated memories of room-sized spatial layouts learned by sequentially or simultaneously viewing objects from a stationary position. In three experiments, sequential viewing (one or two objects at a time) yielded subsequent memory performance that was equivalent or superior to simultaneous viewing of all objects, even though sequential viewing lacked direct access to the entire layout. This finding was replicated by replacing sequential viewing with directed viewing in which all objects were presented simultaneously and participants’ attention was externally focused on each object sequentially, indicating that the advantage of sequential viewing over simultaneous viewing may have originated from focal attention to individual object locations. These results suggest that memory representation of object-to-object relations can be constructed efficiently by encoding each object location separately, when those locations are defined within a single spatial reference system. These findings highlight the importance of considering object presentation procedures when studying spatial learning mechanisms.

Code-specific policy gradient rules for spiking neurons

Although it is widely believed that reinforcement learning is a suitable tool for describing behavioral learning, the mechanisms by which it can be implemented in networks of spiking neurons are not fully understood. Here, we show that different learning rules emerge from a policy gradient approach depending on which features of the spike trains are assumed to influence the reward signals, i.e., depending on which neural code is in effect. We use the framework of Williams (1992) to derive learning rules for arbitrary neural codes. For illustration, we present policy-gradient rules for three different example codes - a spike count code, a spike timing code and the most general "full spike train" code - and test them on simple model problems. In addition to classical synaptic learning, we derive learning rules for intrinsic parameters that control the excitability of the neuron. The spike count learning rule has structural similarities with established Bienenstock-Cooper-Munro rules. If the distribution of the relevant spike train features belongs to the natural exponential family, the learning rules have a characteristic shape that raises interesting prediction problems.

Does temporal discounting explain unhealthy behavior? A systematic review and reinforcement learning perspective.

The tendency to make unhealthy choices is hypothesized to be related to an individual's temporal discount rate, the theoretical rate at which they devalue delayed rewards. Furthermore, a particular form of temporal discounting, hyperbolic discounting, has been proposed to explain why unhealthy behavior can occur despite healthy intentions. We examine these two hypotheses in turn. We first systematically review studies which investigate whether discount rates can predict unhealthy behavior. These studies reveal that high discount rates for money (and in some instances food or drug rewards) are associated with several unhealthy behaviors and markers of health status, establishing discounting as a promising predictive measure. We secondly examine whether intention-incongruent unhealthy actions are consistent with hyperbolic discounting. We conclude that intention-incongruent actions are often triggered by environmental cues or changes in motivational state, whose effects are not parameterized by hyperbolic discounting. We propose a framework for understanding these state-based effects in terms of the interplay of two distinct reinforcement learning mechanisms: a "model-based" (or goal-directed) system and a "model-free" (or habitual) system. Under this framework, while discounting of delayed health may contribute to the initiation of unhealthy behavior, with repetition, many unhealthy behaviors become habitual; if health goals then change, habitual behavior can still arise in response to environmental cues. We propose that the burgeoning development of computational models of these processes will permit further identification of health decision-making phenotypes.

Implication de Syngap1 dans la transmission GABAergique et la plasticité synaptique

La déficience intellectuelle affecte de 1 à 3% de la population mondiale, ce qui en fait le trouble cognitif le plus commun de l’enfance. Notre groupe à découvert que des mutations dans le gène SYNGAP1 sont une cause fréquente de déficience intellectuelle non-syndromique, qui compte pour 1-3% de l’ensemble des cas. À titre d’exemple, le syndrome du X fragile, qui est la cause monogénique la plus fréquente de déficience intellectuelle, compte pour environ 2% des cas. Plusieurs patients affectés au niveau de SYNGAP1 présentent également des symptômes de l’autisme et d’une forme d’épilepsie. Notre groupe a également montré que SYNGAP1 cause la déficience intellectuelle par un mécanisme d’haploinsuffisance. SYNGAP1 code pour une protéine exprimée exclusivement dans le cerveau qui interagit avec la sous-unité GluN2B des récepteurs glutamatergique de type NMDA (NMDAR). SYNGAP1 possède une activité activatrice de Ras-GTPase qui régule négativement Ras au niveau des synapses excitatrices. Les souris hétérozygotes pour Syngap1 (souris Syngap1+/-) présentent des anomalies de comportement et des déficits cognitifs, ce qui en fait un bon modèle d’étude. Plusieurs études rapportent que l’haploinsuffisance de Syngap1 affecte le développement cérébral en perturbant l’activité et la plasticité des neurones excitateurs. Le déséquilibre excitation/inhibition est une théorie émergente de l’origine de la déficience intellectuelle et de l’autisme. Cependant, plusieurs groupes y compris le nôtre ont rapporté que Syngap1 est également exprimé dans au moins une sous-population d’interneurones GABAergiques. Notre hypothèse était donc que l’haploinsuffisance de Syngap1 dans les interneurones contribuerait en partie aux déficits cognitifs et au déséquilibre d’excitation/inhibition observés chez les souris Syngap1+/-. Pour tester cette hypothèse, nous avons généré un modèle de souris transgéniques dont l’expression de Syngap1 a été diminuée uniquement dans les interneurones dérivés des éminences ganglionnaires médianes qui expriment le facteur de transcription Nkx2.1 (souris Tg(Nkx2,1-Cre);Syngap1). Nous avons observé une diminution des courants postsynaptiques inhibiteurs miniatures (mIPSCs) au niveau des cellules pyramidales des couches 2/3 du cortex somatosensoriel primaire (S1) et dans le CA1 de l’hippocampe des souris Tg(Nkx2,1-Cre);Syngap1. Ces résultats supportent donc l’hypothèse selon laquelle la perte de Syngap1 dans les interneurones contribue au déséquilibre d’excitation/inhibition. De manière intéressante, nous avons également observé que les courants postsynaptiques excitateurs miniatures (mEPSCs) étaient augmentés dans le cortex S1, mais diminués dans le CA1 de l’hippocampe. Par la suite, nous avons testé si les mécanismes de plasticité synaptique qui sous-tendraient l’apprentissage étaient affectés par l’haploinsuffisance de Syngap1 dans les interneurones. Nous avons pu montrer que la potentialisation à long terme (LTP) NMDAR-dépendante était diminuée chez les souris Tg(Nkx2,1-Cre);Syngap1, sans que la dépression à long terme (LTD) NMDAR-dépendante soit affectée. Nous avons également montré que l’application d’un bloqueur des récepteurs GABAA renversait en partie le déficit de LTP rapporté chez les souris Syngap1+/-, suggérant qu’un déficit de désinhibition serait présent chez ces souris. L’ensemble de ces résultats supporte un rôle de Syngap1 dans les interneurones qui contribue aux déficits observés chez les souris affectées par l’haploinsuffisance de Syngap1.

Hierarchical Bayesian models of subtask learning

 The current study used Bayesian hierarchical methods to challenge and extend previous work on subtask learning consistency. A general model of individual-level subtask learning was proposed focusing on power and exponential functions with constraints to test for inconsistency. To study subtask learning, we developed a novel computer-based booking task, which logged participant actions, enabling measurement of strategy use and subtask performance. Model comparison was performed using deviance information criterion (DIC), posterior predictive checks, plots of model fits, and model recovery simulations. Results showed that although learning tended to be monotonically decreasing and decelerating, and approaching an asymptote for all subtasks, there was substantial inconsistency in learning curves both at the group- and individual-levels. This inconsistency was most apparent when constraining both the rate and the ratio of learning to asymptote to be equal across subtasks, thereby giving learning curves only 1 parameter for scaling. The inclusion of 6 strategy covariates provided improved prediction of subtask performance capturing different subtask learning processes and subtask trade-offs. In addition, strategy use partially explained the inconsistency in subtask learning. Overall, the model provided a more nuanced representation of how complex tasks can be decomposed in terms of simpler learning mechanisms.

Multimodal Integration in Statistical Learning: Evidence from the McGurk Illusion

Recent advances in the field of statistical learning have established that learners are able to track regularities of multimodal stimuli, yet it is unknown whether the statistical computations are performed on integrated representations or on separate, unimodal representations. In the present study, we investigated the ability of adults to integrate audio and visual input during statistical learning. We presented learners with a speech stream synchronized with a video of a speaker's face. In the critical condition, the visual (e.g., /gi/) and auditory (e.g., /mi/) signals were occasionally incongruent, which we predicted would produce the McGurk illusion, resulting in the perception of an audiovisual syllable (e.g., /ni/). In this way, we used the McGurk illusion to manipulate the underlying statistical structure of the speech streams, such that perception of these illusory syllables facilitated participants' ability to segment the speech stream. Our results therefore demonstrate that participants can integrate audio and visual input to perceive the McGurk illusion during statistical learning. We interpret our findings as support for modality-interactive accounts of statistical learning.

Learning Software-Maintenance Tasks in the Transition Phase of Offshore Outsourcing Projects: Two Learning-Theoretical Perspectives

The increasing practice of offshore outsourcing software maintenance has posed the challenge of effectively transferring knowledge to individual software engineers of the vendor. In this theoretical paper, we discuss the implications of two learning theories, the model of work-based learning (MWBL) and cognitive load theory (CLT), for knowledge transfer during the transition phase. Taken together, the theories suggest that learning mechanisms need to be aligned with the type of knowledge (tacit versus explicit), task characteristics (complexity and recurrence), and the recipients’ expertise. The MWBL proposes that learning mechanisms need to include conceptual and practical activities based on the relative importance of explicit and tacit knowledge. CLT explains how effective portfolios of learning mechanisms change over time. While jobshadowing, completion tasks, and supportive information may prevail at the outset of transition, they may be replaced by the work on conventional tasks towards the end of transition.