Learning joint representations for order and timing of perceptual-motor sequences: a dynamic neural field approach

Many of our everyday tasks require the control of the serial order and the timing of component actions. Using the dynamic neural field (DNF) framework, we address the learning of representations that support the performance of precisely time action sequences. In continuation of previous modeling work and robotics implementations, we ask specifically the question how feedback about executed actions might be used by the learning system to fine tune a joint memory representation of the ordinal and the temporal structure which has been initially acquired by observation. The perceptual memory is represented by a self-stabilized, multi-bump activity pattern of neurons encoding instances of a sensory event (e.g., color, position or pitch) which guides sequence learning. The strength of the population representation of each event is a function of elapsed time since sequence onset. We propose and test in simulations a simple learning rule that detects a mismatch between the expected and realized timing of events and adapts the activation strengths in order to compensate for the movement time needed to achieve the desired effect. The simulation results show that the effector-specific memory representation can be robustly recalled. We discuss the impact of the fast, activation-based learning that the DNF framework provides for robotics applications.

Complementary function and integrated wiring of the evolutionarily distinct Drosophila olfactory subsystems.

To sense myriad environmental odors, animals have evolved multiple, large families of divergent olfactory receptors. How and why distinct receptor repertoires and their associated circuits are functionally and anatomically integrated is essentially unknown. We have addressed these questions through comprehensive comparative analysis of the Drosophila olfactory subsystems that express the ionotropic receptors (IRs) and odorant receptors (ORs). We identify ligands for most IR neuron classes, revealing their specificity for select amines and acids, which complements the broader tuning of ORs for esters and alcohols. IR and OR sensory neurons exhibit glomerular convergence in segregated, although interconnected, zones of the primary olfactory center, but these circuits are extensively interdigitated in higher brain regions. Consistently, behavioral responses to odors arise from an interplay between IR- and OR-dependent pathways. We integrate knowledge on the different phylogenetic and developmental properties of these receptors and circuits to propose models for the functional contributions and evolution of these distinct olfactory subsystems.

A Survey of Active Learning Algorithms for Supervised Remote Sensing Image Classification

Defining an efficient training set is one of the most delicate phases for the success of remote sensing image classification routines. The complexity of the problem, the limited temporal and financial resources, as well as the high intraclass variance can make an algorithm fail if it is trained with a suboptimal dataset. Active learning aims at building efficient training sets by iteratively improving the model performance through sampling. A user-defined heuristic ranks the unlabeled pixels according to a function of the uncertainty of their class membership and then the user is asked to provide labels for the most uncertain pixels. This paper reviews and tests the main families of active learning algorithms: committee, large margin, and posterior probability-based. For each of them, the most recent advances in the remote sensing community are discussed and some heuristics are detailed and tested. Several challenging remote sensing scenarios are considered, including very high spatial resolution and hyperspectral image classification. Finally, guidelines for choosing the good architecture are provided for new and/or unexperienced user.

Consequences of sleep deprivation on neurotransmitter receptor expression and function.

Several pieces of evidence suggest that sleep deprivation causes marked alterations in neurotransmitter receptor function in diverse neuronal cell types. To date, this has been studied mainly in wake- and sleep-promoting areas of the brain and in the hippocampus, which is implicated in learning and memory. This article reviews findings linking sleep deprivation to modifications in neurotransmitter receptor function, including changes in receptor subunit expression, ligand affinity and signal transduction mechanisms. We focus on studies using sleep deprivation procedures that control for side-effects such as stress. We classify the changes with respect to their functional consequences on the activity of wake-promoting and/or sleep-promoting systems. We suggest that elucidation of how sleep deprivation affects neurotransmitter receptor function will provide functional insight into the detrimental effects of sleep loss.

Unpacking the cognitive map: the parallel map theory of hippocampal function

In the parallel map theory, the hippocampus encodes space with 2 mapping systems. The bearing map is constructed primarily in the dentate gyrus from directional cues such as stimulus gradients. The sketch map is constructed within the hippocampus proper from positional cues. The integrated map emerges when data from the bearing and sketch maps are combined. Because the component maps work in parallel, the impairment of one can reveal residual learning by the other. Such parallel function may explain paradoxes of spatial learning, such as learning after partial hippocampal lesions, taxonomic and sex differences in spatial learning, and the function of hippocampal neurogenesis. By integrating evidence from physiology to phylogeny, the parallel map theory offers a unified explanation for hippocampal function.

Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

The influence of learning on evolution: a mathematical framework.

The Baldwin effect can be observed if phenotypic learning influences the evolutionary fitness of individuals, which can in turn accelerate or decelerate evolutionary change. Evidence for both learning-induced acceleration and deceleration can be found in the literature. Although the results for both outcomes were supported by specific mathematical or simulation models, no general predictions have been achieved so far. Here we propose a general framework to predict whether evolution benefits from learning or not. It is formulated in terms of the gain function, which quantifies the proportional change of fitness due to learning depending on the genotype value. With an inductive proof we show that a positive gain-function derivative implies that learning accelerates evolution, and a negative one implies deceleration under the condition that the population is distributed on a monotonic part of the fitness landscape. We show that the gain-function framework explains the results of several specific simulation models. We also use the gain-function framework to shed some light on the results of a recent biological experiment with fruit flies.

Subtype-specific Modulation of Acid-sensing Ion Channel (ASIC) Function by 2-Guanidine-4-methylquinazoline.

Acid-sensing ion channels (ASICs) are neuronal Na(+)-selective channels that are transiently activated by extracellular acidification. ASICs are involved in fear and anxiety, learning, neurodegeneration after ischemic stroke, and pain sensation. The small molecule 2-guanidine-4-methylquinazoline (GMQ) was recently shown to open ASIC3 at physiological pH. We have investigated the mechanisms underlying this effect and the possibility that GMQ may alter the function of other ASICs besides ASIC3. GMQ shifts the pH dependence of activation to more acidic pH in ASIC1a and ASIC1b, whereas in ASIC3 this shift goes in the opposite direction and is accompanied by a decrease in its steepness. GMQ also induces an acidic shift of the pH dependence of inactivation of ASIC1a, -1b, -2a, and -3. As a consequence, the activation and inactivation curves of ASIC3 but not other ASICs overlap in the presence of GMQ at pH 7.4, thereby creating a window current. At concentrations >1 mm, GMQ decreases maximal peak currents by reducing the unitary current amplitude. Mutation of residue Glu-79 in the palm domain of ASIC3, previously shown to be critical for channel opening by GMQ, disrupted the GMQ effects on inactivation but not activation. This suggests that this residue is involved in the consequences of GMQ binding rather than in the binding interaction itself. This study describes the mechanisms underlying the effects of a novel class of ligands that modulate the function of all ASICs as well as activate ASIC3 at physiological pH.

An Automatic Approach for Learning and Tuning Gaussian Interval Type-2 Fuzzy Membership Functions Applied to Lung CAD Classification System

The potential of type-2 fuzzy sets for managing high levels of uncertainty in the subjective knowledge of experts or of numerical information has focused on control and pattern classification systems in recent years. One of the main challenges in designing a type-2 fuzzy logic system is how to estimate the parameters of type-2 fuzzy membership function (T2MF) and the Footprint of Uncertainty (FOU) from imperfect and noisy datasets. This paper presents an automatic approach for learning and tuning Gaussian interval type-2 membership functions (IT2MFs) with application to multi-dimensional pattern classification problems. T2MFs and their FOUs are tuned according to the uncertainties in the training dataset by a combination of genetic algorithm (GA) and crossvalidation techniques. In our GA-based approach, the structure of the chromosome has fewer genes than other GA methods and chromosome initialization is more precise. The proposed approach addresses the application of the interval type-2 fuzzy logic system (IT2FLS) for the problem of nodule classification in a lung Computer Aided Detection (CAD) system. The designed IT2FLS is compared with its type-1 fuzzy logic system (T1FLS) counterpart. The results demonstrate that the IT2FLS outperforms the T1FLS by more than 30% in terms of classification accuracy.

Dynamic optimal law enforcement with learning

We incorporate the process of enforcement learning by assuming that the agency's current marginal cost is a decreasing function of its past experience of detecting and convicting. The agency accumulates data and information (on criminals, on opportunities of crime) enhancing the ability to apprehend in the future at a lower marginal cost.We focus on the impact of enforcement learning on optimal stationary compliance rules. In particular, we show that the optimal stationary fine could be less-than-maximal and the optimal stationary probability of detection could be higher-than-otherwise.

Learning in Digital Media: the legacy of McLuhan and his Impact on Formal Education

This paper describes a bibliographic analysis of the vision of Marshal McLuhan and the vision adopted by diverse current authors regarding the use of new interactive learning technologies. The paper also analyzes the transformation that will have to take place in the formal surroundings of education in order to improve their social function. The main points of view and contributions made by diverse authors are discussed. It is important that all actors involved in the educational process take in consideration these contributions in order to be ready for future changes.

Cognition and brain function in schizotypy : a selective review

Schizotypy refers to a set of personality traits thought to reflect the subclinical expression of the signs and symptoms of schizophrenia. Here, we review the cognitive and brain functional profile associated with high questionnaire scores in schizotypy. We discuss empirical evidence from the domains of perception, attention, memory, imagery and representation, language, and motor control. Perceptual deficits occur early and across various modalities. Whilst the neural mechanisms underlying visual impairments may be linked to magnocellular dysfunction, further effects may be seen downstream in higher cognitive functions. Cognitive deficits are observed in inhibitory control, selective and sustained attention, incidental learning and memory. In concordance with the cognitive nature of many of the aberrations of schizotypy, higher levels of schizotypy are associated with enhanced vividness and better performance on tasks of mental rotation. Language deficits seem most pronounced in higher-level processes. Finally, higher levels of schizotypy are associated with reduced performance on oculomotor tasks, resembling the impairments seen in schizophrenia. Some of these deficits are accompanied by reduced brain activation, akin to the pattern of hypoactivations in schizophrenia spectrum individuals. We conclude that schizotypy is a construct with apparent phenomenological overlap with schizophrenia and stable inter-individual differences that covary with performance on a wide range of perceptual, cognitive and motor tasks known to be impaired in schizophrenia. The importance of these findings lies not only in providing a fine-grained neurocognitive characterisation of a personality constellation known to be associated with real-life impairments, but also in generating hypotheses concerning the aetiology of schizophrenia.

Novel action of a wake-promoting neuropeptide in hippocampus : inhibition of nmda receptor function at excitatory synapses through orexin-a

One of the most intriguing functions of the brain is the ability to learn and memorize. The mechanism through which memory and learning are expressed requires the activation of NMDA receptors (NMDARs). These molecular entities are placed at the postsynaptic density of excitatory synapses and their function is tightly controlled by the actions of several modulators at the extracellular, intracellular and pore sites. A large part of the intracellular modulation comes from the action of G-protein coupled receptors (GPCRs). Through intracellular cascades typically involving kinases and phosphatases, GPCRs potentiate or inhibit NMDARs, controlling the conductive state but also the trafficking within the synapse. The GPCRs are involved in the modulation of a variety of brain functions. Many of them control cognition, memory and learning performance, therefore, their effects on NMDARs are extensively studied. The orexinergic system signals through GPCRs and it is well known for the regulation of waking, feeding, reward and autonomic functions. Moreover, it is involved in potentiating hippocampus-related cognitive tasks. Orexin receptors and fibers are present within the hippocampus, but whether these directly modulate hippocampal cells and synapses has not yet been determined. During my thesis, I studied orexinergic actions on excitatory synaptic transmission via whole-cell patch-clamp recordings in rat acute hippocampal slices. I observed that exogenously applied orexin-A (ox-A) exerted a strong inhibitory action on NMDAR-mediated synaptic potentials at mossy fiber (MF)-CA3 synapses, by postsynaptically activating orexin-2 receptors, a minor inhibition at Schaffer collateral-CAl synapses and did not affect other synapses with the CA3 area. Moreover, I demonstrated that the susceptibility of NMDARs to ox- A depends on the tone of endogenous orexin known to fluctuate during the day-night cycle. In fact, in slices prepared during the active period of the rats, when endogenous orexin levels are high, NMDAR-currents were not affected by exogenously applied ox-A. The inhibitory effect of ox-A was, however, reverted when interfering with the orexinergic system through intraperitoneal injections of almorexant, a dual orexin receptor antagonist, during the active phase prior to slice preparation. This thesis work suggests that the orexinergic system regulates NMDAR-dependent information flow through select hippocampal pathways depending on the time-of-day. The specific orexinergic modulation of NMDARs at MFs dampens the excitability of the hippocampal circuit and could impede the mechanisms related to memory formation, possibly also following extended periods of waking. -- La capacité d'apprentissage et de mémorisation est une des fonctions les plus intrigantes de notre cerveau. Il a été montré qu'elles requièrent l'activation des récepteurs NMDA (NMDARs). Ces entités moléculaires sont présentes au niveau de la densité post-synaptique des synapses excitatrices et leur fonction est étroitement contrôlée par l'action de nombreux modulateurs au niveau extracellulaire, intracellulaire et membranaire de ces récepteurs. Une grande partie de la modulation intracellulaire s'effectue via l'action de récepteurs couplés aux protéines G (GPCRs). Grace à leurs cascades intracellulaires typiquement impliquant des kinases et des phosphatases, les GPCRs favorisent l'activation ou l'inhibition des NMDARs, contrôlant ainsi leur perméabilité mais aussi leur mouvement à la synapse. Les GPCRs sont impliquées dans de nombreuses fonctions cérébrales telles que la cognition, la mémoire ainsi que la capacité d'apprentissage c'est pour cela que leurs effets sur les NMDARs sont très étudiés. Le système orexinergique fait intervenir ces GPCRs et est connu par son rôle dans la régulation de fonctions physiologiques telles que l'éveil, la prise alimentaire, la récompense ainsi que d'autres fonctions du système nerveux autonome. De plus, ce système est impliqué dans la régulation de tâches cognitives liées à l'hippocampe. Bien que les fibres et les récepteurs à l'orexine soient présents dans l'hippocampe, leur mécanisme d'action sur les cellules et les synapses de l'hippocampe n'a pas encore été élucidé. Durant ma thèse, je me suis intéressée aux effets de l'orexine sur la transmission synaptique excitatrice en utilisant la méthode d'enregistrement en patch-clamp en configuration cellule entière sur des tranches aiguës d'hippocampes de rats. J'ai observé que l'application exogène d'orexine A d'une part inhibe fortement les courants synaptiques dépendants de l'activation des NMDARs au niveau de la synapse entre les fibres moussues et CA3 via l'activation post-synaptique des orexine récepteurs 2 mais d'autre part n'inhibe que de façon mineure la synapse entre les collatérales de Schaffer et CAI et n'affecte pas les autres synapses impliquant CA3. J'ai également démontré que la sensibilité des NMDARs à l'orexine A dépend de sa concentration endogène qui fluctue durant le cycle éveil-sommeil. En effet, lorsque les coupes d'hippocampes sont préparées durant la période active de l'animal correspondant à un niveau endogène d'orexine élevé, l'application exogène d'orexine A n'a aucun effet sur les courants dépendants de l'activation des NMDARs. Cependant, l'injection dans le péritoine, durant la phase active de l'animal, d'un antagoniste des orexine récepteurs, l'almorexant, va supprimer l'effet inhibiteur de l'orexine A. Les résultats de ma thèse suggèrent donc que le système orexinergique module les informations véhiculées par les NMDARs via des voies de signalisation sélectives de l'hippocampe en fonction du moment de la journée. La modulation orexinergique des NMDARs au niveau des fibres moussues diminue ainsi l'excitabilité du circuit hippocampal et pourrait entraver les mécanismes liés à la formation de la mémoire, potentiellement après de longues périodes d'éveil.

Glucose-dependent insulinotropic polypeptide receptor knockout mice are impaired in learning, synaptic plasticity, and neurogenesis.

Glucose-dependent insulinotropic polypeptide (GIP) is a key incretin hormone, released from intestine after a meal, producing a glucose-dependent insulin secretion. The GIP receptor (GIPR) is expressed on pyramidal neurons in the cortex and hippocampus, and GIP is synthesized in a subset of neurons in the brain. However, the role of the GIPR in neuronal signaling is not clear. In this study, we used a mouse strain with GIPR gene deletion (GIPR KO) to elucidate the role of the GIPR in neuronal communication and brain function. Compared with C57BL/6 control mice, GIPR KO mice displayed higher locomotor activity in an open-field task. Impairment of recognition and spatial learning and memory of GIPR KO mice were found in the object recognition task and a spatial water maze task, respectively. In an object location task, no impairment was found. GIPR KO mice also showed impaired synaptic plasticity in paired-pulse facilitation and a block of long-term potentiation in area CA1 of the hippocampus. Moreover, a large decrease in the number of neuronal progenitor cells was found in the dentate gyrus of transgenic mice, although the numbers of young neurons was not changed. Together the results suggest that GIP receptors play an important role in cognition, neurotransmission, and cell proliferation.