Learning-induced gene expression in the heads of two Nasonia species that differ in long-term memory formation.

BACKGROUND: Cellular processes underlying memory formation are evolutionary conserved, but natural variation in memory dynamics between animal species or populations is common. The genetic basis of this fascinating phenomenon is poorly understood. Closely related species of Nasonia parasitic wasps differ in long-term memory (LTM) formation: N. vitripennis will form transcription-dependent LTM after a single conditioning trial, whereas the closely-related species N. giraulti will not. Genes that were differentially expressed (DE) after conditioning in N. vitripennis, but not in N. giraulti, were identified as candidate genes that may regulate LTM formation. RESULTS: RNA was collected from heads of both species before and immediately, 4 or 24 hours after conditioning, with 3 replicates per time point. It was sequenced strand-specifically, which allows distinguishing sense from antisense transcripts and improves the quality of expression analyses. We determined conditioning-induced DE compared to naïve controls for both species. These expression patterns were then analysed with GO enrichment analyses for each species and time point, which demonstrated an enrichment of signalling-related genes immediately after conditioning in N. vitripennis only. Analyses of known LTM genes and genes with an opposing expression pattern between the two species revealed additional candidate genes for the difference in LTM formation. These include genes from various signalling cascades, including several members of the Ras and PI3 kinase signalling pathways, and glutamate receptors. Interestingly, several other known LTM genes were exclusively differentially expressed in N. giraulti, which may indicate an LTM-inhibitory mechanism. Among the DE transcripts were also antisense transcripts. Furthermore, antisense transcripts aligning to a number of known memory genes were detected, which may have a role in regulating these genes. CONCLUSION: This study is the first to describe and compare expression patterns of both protein-coding and antisense transcripts, at different time points after conditioning, of two closely related animal species that differ in LTM formation. Several candidate genes that may regulate differences in LTM have been identified. This transcriptome analysis is a valuable resource for future in-depth studies to elucidate the role of candidate genes and antisense transcription in natural variation in LTM formation.

What can ecosystems learn? Expanding evolutionary ecology with learning theory.

BACKGROUND: The structure and organisation of ecological interactions within an ecosystem is modified by the evolution and coevolution of the individual species it contains. Understanding how historical conditions have shaped this architecture is vital for understanding system responses to change at scales from the microbial upwards. However, in the absence of a group selection process, the collective behaviours and ecosystem functions exhibited by the whole community cannot be organised or adapted in a Darwinian sense. A long-standing open question thus persists: Are there alternative organising principles that enable us to understand and predict how the coevolution of the component species creates and maintains complex collective behaviours exhibited by the ecosystem as a whole? RESULTS: Here we answer this question by incorporating principles from connectionist learning, a previously unrelated discipline already using well-developed theories on how emergent behaviours arise in simple networks. Specifically, we show conditions where natural selection on ecological interactions is functionally equivalent to a simple type of connectionist learning, 'unsupervised learning', well-known in neural-network models of cognitive systems to produce many non-trivial collective behaviours. Accordingly, we find that a community can self-organise in a well-defined and non-trivial sense without selection at the community level; its organisation can be conditioned by past experience in the same sense as connectionist learning models habituate to stimuli. This conditioning drives the community to form a distributed ecological memory of multiple past states, causing the community to: a) converge to these states from any random initial composition; b) accurately restore historical compositions from small fragments; c) recover a state composition following disturbance; and d) to correctly classify ambiguous initial compositions according to their similarity to learned compositions. We examine how the formation of alternative stable states alters the community's response to changing environmental forcing, and we identify conditions under which the ecosystem exhibits hysteresis with potential for catastrophic regime shifts. CONCLUSIONS: This work highlights the potential of connectionist theory to expand our understanding of evo-eco dynamics and collective ecological behaviours. Within this framework we find that, despite not being a Darwinian unit, ecological communities can behave like connectionist learning systems, creating internal conditions that habituate to past environmental conditions and actively recalling those conditions. REVIEWERS: This article was reviewed by Prof. Ricard V Solé, Universitat Pompeu Fabra, Barcelona and Prof. Rob Knight, University of Colorado, Boulder.

Brain dynamics sustaining rapid rule extraction from speech

Language acquisition is a complex process that requires the synergic involvement of different cognitive functions, which include extracting and storing the words of the language and their embedded rules for progressive acquisition of grammatical information. As has been shown in other fields that study learning processes, synchronization mechanisms between neuronal assemblies might have a key role during language learning. In particular, studying these dynamics may help uncover whether different oscillatory patterns sustain more item-based learning of words and rule-based learning from speech input. Therefore, we tracked the modulation of oscillatory neural activity during the initial exposure to an artificial language, which contained embedded rules. We analyzed both spectral power variations, as a measure of local neuronal ensemble synchronization, as well as phase coherence patterns, as an index of the long-range coordination of these local groups of neurons. Synchronized activity in the gamma band (2040 Hz), previously reported to be related to the engagement of selective attention, showed a clear dissociation of local power and phase coherence between distant regions. In this frequency range, local synchrony characterized the subjects who were focused on word identification and was accompanied by increased coherence in the theta band (48 Hz). Only those subjects who were able to learn the embedded rules showed increased gamma band phase coherence between frontal, temporal, and parietal regions.

The brain decade in debate: VI. Sensory and motor maps: dynamics and plasticity

This article is an edited transcription of a virtual symposium promoted by the Brazilian Society of Neuroscience and Behavior (SBNeC). Although the dynamics of sensory and motor representations have been one of the most studied features of the central nervous system, the actual mechanisms of brain plasticity that underlie the dynamic nature of sensory and motor maps are not entirely unraveled. Our discussion began with the notion that the processing of sensory information depends on many different cortical areas. Some of them are arranged topographically and others have non-topographic (analytical) properties. Besides a sensory component, every cortical area has an efferent output that can be mapped and can influence motor behavior. Although new behaviors might be related to modifications of the sensory or motor representations in a given cortical area, they can also be the result of the acquired ability to make new associations between specific sensory cues and certain movements, a type of learning known as conditioning motor learning. Many types of learning are directly related to the emotional or cognitive context in which a new behavior is acquired. This has been demonstrated by paradigms in which the receptive field properties of cortical neurons are modified when an animal is engaged in a given discrimination task or when a triggering feature is paired with an aversive stimulus. The role of the cholinergic input from the nucleus basalis to the neocortex was also highlighted as one important component of the circuits responsible for the context-dependent changes that can be induced in cortical maps.

JNK regulates dendrite and spine architecture in the central nervous system influencing spatial learning and motor tasks

JNK1 is a MAP-kinase that has proven a significant player in the central nervous system. It regulates brain development and the maintenance of dendrites and axons. Several novel phosphorylation targets of JNK1 were identified in a screen performed in the Coffey lab. These proteins were mainly involved in the regulation of neuronal cytoskeleton, influencing the dynamics and stability of microtubules and actin. These structural proteins form the dynamic backbone for the elaborate architecture of the dendritic tree of a neuron. The initiation and branching of the dendrites requires a dynamic interplay between the cytoskeletal building blocks. Both microtubules and actin are decorated by associated proteins which regulate their dynamics. The dendrite-specific, high molecular weight microtubule associated protein 2 (MAP2) is an abundant protein in the brain, the binding of which stabilizes microtubules and influences their bundling. Its expression in non-neuronal cells induces the formation of neurite-like processes from the cell body, and its function is highly regulated by phosphorylation. JNK1 was shown to phosphorylate the proline-rich domain of MAP2 in vivo in a previous study performed in the group. Here we verify three threonine residues (T1619, T1622 and T1625) as JNK1 targets, the phosphorylation of which increases the binding of MAP2 to microtubules. This binding stabilizes the microtubules and increases process formation in non-neuronal cells. Phosphorylation-site mutants were engineered in the lab. The non-phosphorylatable mutant of MAP2 (MAP2- T1619A, T1622A, T1625A) in these residues fails to bind microtubules, while the pseudo-phosphorylated form, MAP2- T1619D, T1622D, Thr1625D, efficiently binds and induces process formation even without the presence of active JNK1. Ectopic expression of the MAP2- T1619D, T1622D, Thr1625D in vivo in mouse brain led to a striking increase in the branching of cortical layer 2/3 (L2/3) pyramidal neurons, compared to MAP2-WT. The dendritic complexity defines the receptive field of a neuron and dictates the output to the postsynaptic cells. Previous studies in the group indicated altered dendrite architecture of the pyramidal neurons in the Jnk1-/- mouse motor cortex. Here, we used Lucifer Yellow loading and Sholl analysis of neurons in order to study the dendritic branching in more detail. We report a striking, opposing effect in the absence of Jnk1 in the cortical layers 2/3 and 5 of the primary motor cortex. The basal dendrites of pyramidal neurons close to the pial surface at L2/3 show a reduced complexity. In contrast, the L5 neurons, which receive massive input from the L2/3 neurons, show greatly increased branching. Another novel substrate identified for JNK1 was MARCKSL1, a protein that regulates actin dynamics. It is highly expressed in neurons, but also in various cancer tissues. Three phosphorylation target residues for JNK1 were identified, and it was demonstrated that their phosphorylation reduces actin turnover and retards migration of these cells. Actin is the main cytoskeletal component in dendritic spines, the site of most excitatory synapses in pyramidal neurons. The density and gross morphology of the Lucifer Yellow filled dendrites were characterized and we show reduced density and altered morphology of spines in the motor cortex and in the hippocampal area CA3. The dynamic dendritic spines are widely considered to function as the cellular correlate during learning. We used a Morris water maze to test spatial memory. Here, the wild-type mice outperformed the knock-out mice during the acquisition phase of the experiment indicating impaired special memory. The L5 pyramidal neurons of the motor cortex project to the spinal cord and regulate the movement of distinct muscle groups. Thus the altered dendrite morphology in the motor cortex was expected to have an effect on the input-output balance in the signaling from the cortex to the lower motor circuits. A battery of behavioral tests were conducted for the wild-type and Jnk1-/- mice, and the knock-outs performed poorly compared to wild-type mice in tests assessing balance and fine motor movements. This study expands our knowledge of JNK1 as an important regulator of the dendritic fields of neurons and their manifestations in behavior.

Dynamics AX -toiminnanohjausjärjestelmän jatkuvuudenhallinta ja toipumissuunnitelma - Simulaatiotestausmalli: case CGI Suomi Oy

Nyky-yhteiskunta nojautuu vahvasti tietojärjestelmiin luoden laajemman arkkitehtuurisen kokonaisuuden, kybertoimintaympäristön. Liiketoimintaa tukevat tietojärjestelmät tukevat myös organisaatioiden prosesseja kokonaisvaltaisesti. Jotta näitä tärkeitä kyberympäristön tietojärjestelmä- sekä liiketoimintaympäristöresursseja pystytään käyttämään, tulee järjestelmien olla luotettavia ja sovellusten saatavilla vuorokauden ympäri tai aina tarvittaessa. Tilanteet, joissa järjestelmän käytettävyys vaarantuu, voivat eskaloitua yllättäen suuremmiksi, jos poikkeustilanteisiin ei ole varauduttu. Poikkeustilanteisiin varautumiseen tarvitaan jatkuvuudenhallintaa, joka on kiinteästi osa kattavampaa IT-strategiaa ja koko yhteiskunta-/yrityskulttuuria. Työ on toteutettu yhdistäen empiriaa ja teoriaa eli tavoitteena on teoreettisen tietämyksen ja käytännön kokemuksellisen oppimisen ja tietämyksen kautta luoda konstruktiivisella otteella toipumissuunnitelman testauksessa käytettävä simulaatiotestausmalli. Dynamics AX -toiminnanohjausjärjestelmän toipumissuunnitelman simulaatiotestausmallista rakentui selkeä ja kevyt työkalu asiakasyritysten toipumissuunnitelman simulaatiotestauksiin. Diplomityössä kuvatuilla keinotekoisilla järjestelmän häiriötilanteilla pystytään simuloimaan Dynamics AX:n toipumissuunnitelman testauksissa oikeita häiriötilanteita suhteellisen kattavalla tasolla.

Assessing the Reproducibility of Clustering of Molecular Dynamics Conformations on Self-Organizing Maps

The goal of most clustering algorithms is to find the optimal number of clusters (i.e. fewest number of clusters). However, analysis of molecular conformations of biological macromolecules obtained from computer simulations may benefit from a larger array of clusters. The Self-Organizing Map (SOM) clustering method has the advantage of generating large numbers of clusters, but often gives ambiguous results. In this work, SOMs have been shown to be reproducible when the same conformational dataset is independently clustered multiple times (~100), with the help of the Cramérs V-index (C_v). The ability of C_v to determine which SOMs are reproduced is generalizable across different SOM source codes. The conformational ensembles produced from MD (molecular dynamics) and REMD (replica exchange molecular dynamics) simulations of the penta peptide Met-enkephalin (MET) and the 34 amino acid protein human Parathyroid Hormone (hPTH) were used to evaluate SOM reproducibility. The training length for the SOM has a huge impact on the reproducibility. Analysis of MET conformational data definitively determined that toroidal SOMs cluster data better than bordered maps due to the fact that toroidal maps do not have an edge effect. For the source code from MATLAB, it was determined that the learning rate function should be LINEAR with an initial learning rate factor of 0.05 and the SOM should be trained by a sequential algorithm. The trained SOMs can be used as a supervised classification for another dataset. The toroidal 10×10 hexagonal SOMs produced from the MATLAB program for hPTH conformational data produced three sets of reproducible clusters (27%, 15%, and 13% of 100 independent runs) which find similar partitionings to those of smaller 6×6 SOMs. The χ^2 values produced as part of the C_v calculation were used to locate clusters with identical conformational memberships on independently trained SOMs, even those with different dimensions. The χ^2 values could relate the different SOM partitionings to each other.

Reinforcement Learning by Policy Search

One objective of artificial intelligence is to model the behavior of an intelligent agent interacting with its environment. The environment's transformations can be modeled as a Markov chain, whose state is partially observable to the agent and affected by its actions; such processes are known as partially observable Markov decision processes (POMDPs). While the environment's dynamics are assumed to obey certain rules, the agent does not know them and must learn. In this dissertation we focus on the agent's adaptation as captured by the reinforcement learning framework. This means learning a policy---a mapping of observations into actions---based on feedback from the environment. The learning can be viewed as browsing a set of policies while evaluating them by trial through interaction with the environment. The set of policies is constrained by the architecture of the agent's controller. POMDPs require a controller to have a memory. We investigate controllers with memory, including controllers with external memory, finite state controllers and distributed controllers for multi-agent systems. For these various controllers we work out the details of the algorithms which learn by ascending the gradient of expected cumulative reinforcement. Building on statistical learning theory and experiment design theory, a policy evaluation algorithm is developed for the case of experience re-use. We address the question of sufficient experience for uniform convergence of policy evaluation and obtain sample complexity bounds for various estimators. Finally, we demonstrate the performance of the proposed algorithms on several domains, the most complex of which is simulated adaptive packet routing in a telecommunication network.

Online Learning of Non-stationary Sequences

We consider an online learning scenario in which the learner can make predictions on the basis of a fixed set of experts. The performance of each expert may change over time in a manner unknown to the learner. We formulate a class of universal learning algorithms for this problem by expressing them as simple Bayesian algorithms operating on models analogous to Hidden Markov Models (HMMs). We derive a new performance bound for such algorithms which is considerably simpler than existing bounds. The bound provides the basis for learning the rate at which the identity of the optimal expert switches over time. We find an analytic expression for the a priori resolution at which we need to learn the rate parameter. We extend our scalar switching-rate result to models of the switching-rate that are governed by a matrix of parameters, i.e. arbitrary homogeneous HMMs. We apply and examine our algorithm in the context of the problem of energy management in wireless networks. We analyze the new results in the framework of Information Theory.

Modeling Stock Order Flows and Learning Market-Making from Data

Stock markets employ specialized traders, market-makers, designed to provide liquidity and volume to the market by constantly supplying both supply and demand. In this paper, we demonstrate a novel method for modeling the market as a dynamic system and a reinforcement learning algorithm that learns profitable market-making strategies when run on this model. The sequence of buys and sells for a particular stock, the order flow, we model as an Input-Output Hidden Markov Model fit to historical data. When combined with the dynamics of the order book, this creates a highly non-linear and difficult dynamic system. Our reinforcement learning algorithm, based on likelihood ratios, is run on this partially-observable environment. We demonstrate learning results for two separate real stocks.

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Studies of dynamics of physical agent ecosystems

This thesis addresses the problem of learning in physical heterogeneous multi-agent systems (MAS) and the analysis of the benefits of using heterogeneous MAS with respect to homogeneous ones. An algorithm is developed for this task; building on a previous work on stability in distributed systems by Tad Hogg and Bernardo Huberman, and combining two phenomena observed in natural systems, task partition and hierarchical dominance. This algorithm is devised for allowing agents to learn which are the best tasks to perform on the basis of each agent's skills and the contribution to the team global performance. Agents learn by interacting with the environment and other teammates, and get rewards from the result of the actions they perform. This algorithm is specially designed for problems where all robots have to co-operate and work simultaneously towards the same goal. One example of such a problem is role distribution in a team of heterogeneous robots that form a soccer team, where all members take decisions and co-operate simultaneously. Soccer offers the possibility of conducting research in MAS, where co-operation plays a very important role in a dynamical and changing environment. For these reasons and the experience of the University of Girona in this domain, soccer has been selected as the test-bed for this research. In the case of soccer, tasks are grouped by means of roles. One of the most interesting features of this algorithm is that it endows MAS with a high adaptability to changes in the environment. It allows the team to perform their tasks, while adapting to the environment. This is studied in several cases, for changes in the environment and in the robot's body. Other features are also analysed, especially a parameter that defines the fitness (biological concept) of each agent in the system, which contributes to performance and team adaptability. The algorithm is applied later to allow agents to learn in teams of homogeneous and heterogeneous robots which roles they have to select, in order to maximise team performance. The teams are compared and the performance is evaluated in the games against three hand-coded teams and against the different homogeneous and heterogeneous teams built in this thesis. This section focuses on the analysis of performance and task partition, in order to study the benefits of heterogeneity in physical MAS. In order to study heterogeneity from a rigorous point of view, a diversity measure is developed building on the hierarchic social entropy defined by Tucker Balch. This is adapted to quantify physical diversity in robot teams. This tool presents very interesting features, as it can be used in the future to design heterogeneous teams on the basis of the knowledge on other teams.

Lesser Scaup Population Dynamics: What Can Be Learned from Available Data?

Populations of Lesser Scaup (Aythya affinis) have declined markedly in North America since the early 1980s. When considering alternatives for achieving population recovery, it would be useful to understand how the rate of population growth is functionally related to the underlying vital rates and which vital rates affect population growth rate the most if changed (which need not be those that influenced historical population declines). To establish a more quantitative basis for learning about life history and population dynamics of Lesser Scaup, we summarized published and unpublished estimates of vital rates recorded between 1934 and 2005, and developed matrix life-cycle models with these data for females breeding in the boreal forest, prairie-parklands, and both regions combined. We then used perturbation analysis to evaluate the effect of changes in a variety of vital-rate statistics on finite population growth rate and abundance. Similar to Greater Scaup (Aythya marila), our modeled population growth rate for Lesser Scaup was most sensitive to unit and proportional change in adult female survival during the breeding and non-breeding seasons, but much less so to changes in fecundity parameters. Interestingly, population growth rate was also highly sensitive to unit and proportional changes in the mean of nesting success, duckling survival, and juvenile survival. Given the small samples of data for key aspects of the Lesser Scaup life cycle, we recommend additional research on vital rates that demonstrate a strong effect on population growth and size (e.g., adult survival probabilities). Our life-cycle models should be tested and regularly updated in the future to simultaneously guide science and management of Lesser Scaup populations in an adaptive context.

e-learning in project management using simulation models: a case study based on the replication of an experiment

Current e-learning systems are increasing their importance in higher education. However, the state of the art of e-learning applications, besides the state of the practice, does not achieve the level of interactivity that current learning theories advocate. In this paper, the possibility of enhancing e-learning systems to achieve deep learning has been studied by replicating an experiment in which students had to learn basic software engineering principles. One group learned these principles using a static approach, while the other group learned the same principles using a system-dynamics-based approach, which provided interactivity and feedback. The results show that, quantitatively, the latter group achieved a better understanding of the principles; furthermore, qualitatively, they enjoyed the learning experience

Using learning networks to understand complex systems: a case study of biological, geophysical and social research in the Amazon

Developing high-quality scientific research will be most effective if research communities with diverse skills and interests are able to share information and knowledge, are aware of the major challenges across disciplines, and can exploit economies of scale to provide robust answers and better inform policy. We evaluate opportunities and challenges facing the development of a more interactive research environment by developing an interdisciplinary synthesis of research on a single geographic region. We focus on the Amazon as it is of enormous regional and global environmental importance and faces a highly uncertain future. To take stock of existing knowledge and provide a framework for analysis we present a set of mini-reviews from fourteen different areas of research, encompassing taxonomy, biodiversity, biogeography, vegetation dynamics, landscape ecology, earth-atmosphere interactions, ecosystem processes, fire, deforestation dynamics, hydrology, hunting, conservation planning, livelihoods, and payments for ecosystem services. Each review highlights the current state of knowledge and identifies research priorities, including major challenges and opportunities. We show that while substantial progress is being made across many areas of scientific research, our understanding of specific issues is often dependent on knowledge from other disciplines. Accelerating the acquisition of reliable and contextualized knowledge about the fate of complex pristine and modified ecosystems is partly dependent on our ability to exploit economies of scale in shared resources and technical expertise, recognise and make explicit interconnections and feedbacks among sub-disciplines, increase the temporal and spatial scale of existing studies, and improve the dissemination of scientific findings to policy makers and society at large. Enhancing interaction among research efforts is vital if we are to make the most of limited funds and overcome the challenges posed by addressing large-scale interdisciplinary questions. Bringing together a diverse scientific community with a single geographic focus can help increase awareness of research questions both within and among disciplines, and reveal the opportunities that may exist for advancing acquisition of reliable knowledge. This approach could be useful for a variety of globally important scientific questions.