Novelty, but Not Operant Aversive Learning, Enhances Fos and Egr-1 Expression in the Medial Prefrontal Cortex and Hippocampal Areas of Rats

Immediate early genes (IEG) are presumed to be activated in response to stress, novelty, and learning. Evidence supports the involvement of prefrontal and hippocampal areas in stress and learning, but also in the detection of novel events. This study examined whether a previous experience with shocks changes the pattern of Fos and Egr-1 expression in the medial prefrontal cortex (mPFC), the hippocampal cornus ammonis 1 (CA1), and dentate gyrus (DG) of adult male Wistar rats that learned to escape in an operant aversive test. Subjects previously exposed to inescapable footshocks that learned to escape from Shocks were assigned to the treated group (EXP). Subjects from Group Novelty (NOV) rested undisturbed during treatment and also learned to escape in the test. The nonshock group (NSH) rested undisturbed in both sessions. Standard immunohistochemistry procedures were used to detect the proteins in brain sections. The results show that a previous experience with shocks changed the pattern of IEG expression, then demonstrating c-fos and egr-1 induction as experience-dependent events. Compared with NSH and EXP an enhanced Fos expression was detected in the mPFC and CA1 subfield of Group NOV, which also exhibited increased Egr-1 expression in the mPFC and DG in comparison to NSH. No differences were found in the DG for Fos, or in the CA1 for Egr-1. Novelty, and not the operant aversive escape learning, seems to have generated IEG induction. The results suggest novel stimuli as a possible confounding factor in studies on Fos and/or Egr-1 expression in aversive conditions.

Subthalamic deep brain stimulation modulates small fiber-dependent sensory thresholds in Parkinson's disease

The effects of deep brain stimulation of the subthalamic nucleus on nonmotor symptoms of Parkinson's disease (PD) rarely have been investigated. Among these, sensory disturbances, including chronic pain (CP), are frequent in these patients. The aim of this study was to evaluate the changes induced by deep brain stimulation in the perception of sensory stimuli, either noxious or innocuous, mediated by small or large nerve fibers. Sensory detection and pain thresholds were assessed in 25 PD patients all in the off-medication condition with the stimulator turned on or off (on- and off-stimulation conditions, respectively). The relationship between the changes induced by surgery on quantitative sensory testing, spontaneous CP, and motor abilities were studied. Quantitative sensory test results obtained in PD patients were compared with those of age-matched healthy subjects. Chronic pain was present in 72% of patients before vs 36% after surgery (P = .019). Compared with healthy subjects, PD patients had an increased sensitivity to innocuous thermal stimuli and mechanical pain, but a reduced sensitivity to innocuous mechanical stimuli. In addition, they had an increased pain rating when painful thermal stimuli were applied, particularly in the off-stimulation condition. In the on-stimulation condition, there was an increased sensitivity to innocuous thermal stimuli but a reduced sensitivity to mechanical or thermal pain. Pain provoked by thermal stimuli was reduced when the stimulator was turned on. Motor improvement positively correlated with changes in warm detection and heat pain thresholds. Subthalamic nucleus deep brain stimulation contributes to relieve pain associated with PD and specifically modulates small fiber-mediated sensations. (C) 2012 International Association for the Study of Pain. Published by Elsevier B. V. All rights reserved.

Aprendizagem de uma habilidade motora seriada em diferentes estágios de desenvolvimento

Este estudo investigou a aprendizagem de uma tarefa motora seriada em diferentes estágios de desenvolvimento. Quinze crianças, 14 adultos e 13 idosos praticaram a tarefa de rastrear uma sequência de seis estímulos luminosos durante 10 blocos de tentativas ou até descobrir a sequência, constituindo a fase de estabilização e mais dois blocos de tentativas, referentes as fases de adaptação I e II. O desempenho foi mensurado por meio das respostas funcionais e não-funcionais e das sequências funcionais. Os resultados indicaram que os adultos foram superiores aos demais participantes, e idosos apresentaram melhor desempenho que crianças apenas no início da prática, sugerindo que o estágio de desenvolvimento interage com o processo de aprendizagem motora.

O ensino do esporte para crianças e jovens: considerações sobre uma fase do processo de desenvolvimento motor esquecida

O processo ensino-aprendizagem pode ser visto como um sistema constituído pela interação de três componentes - professor, aluno e matéria - que tem por meta promover mudanças efetivas nos comportamentos, capacidades e competências do aluno. Como numa visão sistêmica do processo ensino-aprendizagem, a função de um determinado componente implica sempre o estabelecimento de relação entre os dois componentes que restam, o papel principal do professor é estabelecer relação entre o aluno e a matéria. Neste contexto, a questão central é saber em que se basear para estabelecer essa relação. O presente ensaio parte da assunção de que o conhecimento sobre o desenvolvimento motor constitui um elemento fundamental quando a matéria de ensino é o esporte, discute uma fase desse processo que tem sido sistematicamente esquecida procurando identificar as suas possíveis causas e consequências e apresenta algumas sugestões para trabalhar com essa fase.

Learning for well being : Studies using the International Adult Literacy Survey

This thesis is a collection of five independent but closely related studies. The overall purpose is to approach the analysis of learning outcomes from a perspective that combines three major elements, namely lifelonglifewide learning, human capital, and the benefits of learning. The approach is based on an interdisciplinary perspective of the human capital paradigm. It considers the multiple learning contexts that are responsible for the development of embodied potential – including formal, nonformal and informal learning – and the multiple outcomes – including knowledge, skills, economic, social and others– that result from learning. The studies also seek to examine the extent and relative influence of learning in different contexts on the formation of embodied potential and how in turn that affects economic and social well being. The first study combines the three major elements, lifelonglifewide learning, human capital, and the benefits of learning into one common conceptual framework. This study forms a common basis for the four empirical studies that follow. All four empirical studies use data from the International Adult Literacy Survey (IALS) to investigate the relationships among the major elements of the conceptual framework presented in the first study. Study I. A conceptual framework for the analysis of learning outcomes This study brings together some key concepts and theories that are relevant for the analysis of learning outcomes. Many of the concepts and theories have emerged from varied disciplines including economics, educational psychology, cognitive science and sociology, to name only a few. Accordingly, some of the research questions inherent in the framework relate to different disciplinary perspectives. The primary purpose is to create a common basis for formulating and testing hypotheses as well as to interpret the findings in the empirical studies that follow. In particular, the framework facilitates the process of theorizing and hypothesizing on the relationships and processes concerning lifelong learning as well as their antecedents and consequences. Study II. Determinants of literacy proficiency: A lifelong-lifewide learning perspective This study investigates lifelong and lifewide processes of skill formation. In particular, it seeks to estimate the substitutability and complementarity effects of learning in multiple settings over the lifespan on literacy skill formation. This is done by investigating the predictive capacity of major determinants of literacy proficiency that are associated with a variety of learning contexts including school, home, work, community and leisure. An identical structural model based on previous research is fitted to the IALS data for 18 countries. The results show that even after accounting for all factors, education remains the most important predictor of literacy proficiency. In all countries, however, the total effect of education is significantly mediated through further learning occurring at work, at home and in the community. Therefore, the job and other literacy related factors complement education in predicting literacy proficiency. This result points to a virtual cycle of lifelong learning, particularly to how educational attainment influences other learning behaviours throughout life. In addition, results show that home background as measured by parents’ education is also a strong predictor of literacy proficiency, but in many countries this occurs only if a favourable home background is complemented with some post-secondary education. Study III. The effect of literacy proficiency on earnings: An aggregated occupational approach using the Canadian IALS data This study uses data from the Canadian Adult Literacy Survey to estimate the earnings return to literacy skills. The approach adapts a labour segmented view of the labour market by aggregating occupations into seven types, enabling the estimation of the variable impact of literacy proficiency on earnings, both within and between different types of occupations. This is done using Hierarchical Linear Modeling (HLM). The method used to construct the aggregated occupational classification is based on analysis that considers the role of cognitive and other skills in relation to the nature of occupational tasks. Substantial premiums are found to be associated with some occupational types even after adjusting for within occupational differences in individual characteristics such as schooling, literacy proficiency, labour force experience and gender. Average years of schooling and average levels of literacy proficiency at the between level account for over two-thirds of the premiums. Within occupations, there are significant returns to schooling but they vary depending on the type of occupations. In contrast, the within occupational return of literacy proficiency is not necessarily significant. The latter depends on the type of occupation. Study IV: Determinants of economic and social outcomes from a lifewide learning perspective in Canada In this study the relationship between learning in different contexts, which span the lifewide learning dimension, and individual earnings on the one hand and community participation on the other are examined in separate but comparable models. Data from the Canadian Adult Literacy Survey are used to estimate structural models, which correspond closely to the common conceptual framework outlined in Study I. The findings suggest that the relationship between formal education and economic and social outcomes is complex with confounding effects. The results indicate that learning occurring in different contexts and for different reasons leads to different kinds of benefits. The latter finding suggests a potential trade-off between realizing economic and social benefits through learning that are taken for either job-related or personal-interest related reasons. Study V: The effects of learning on economic and social well being: A comparative analysis Using the same structural model as in Study IV, hypotheses are comparatively examined using the International Adult Literacy Survey data for Canada, Denmark, the Netherlands, Norway, the United Kingdom, and the United States. The main finding from Study IV is confirmed for an additional five countries, namely that the effect of initial schooling on well being is more complex than a direct one and it is significantly mediated by subsequent learning. Additionally, findings suggest that people who devote more time to learning for job-related reasons than learning for personal-interest related reasons experience higher levels of economic well being. Moreover, devoting too much time to learning for personal-interest related reasons has a negative effect on earnings except in Denmark. But the more time people devote to learning for personal-interest related reasons tends to contribute to higher levels of social well being. These results again suggest a trade-off in learning for different reasons and in different contexts.

Práctica aleatoria y aprendizaje deportivo-motor

[ES] Los efectos de la práctica es un asunto que posee una larga tradición investigadora en el ámbito del aprendizaje motor. En las últimas décadas el interés se ha centrado en analizar si una práctica aleatoria posee unos efectos más favorables que una práctica repetitiva en el aprendizaje. Este fue el objetivo de este estudio en el que participaron voluntariamente cuarenta y ocho estudiantes universitarios.

Body ares networks for human motor assessment

Healthcare, Human Computer Interfaces (HCI), Security and Biometry are the most promising application scenario directly involved in the Body Area Networks (BANs) evolution. Both wearable devices and sensors directly integrated in garments envision a word in which each of us is supervised by an invisible assistant monitoring our health and daily-life activities. New opportunities are enabled because improvements in sensors miniaturization and transmission efficiency of the wireless protocols, that achieved the integration of high computational power aboard independent, energy-autonomous, small form factor devices. Application’s purposes are various: (I) data collection to achieve off-line knowledge discovery; (II) user notification of his/her activities or in case a danger occurs; (III) biofeedback rehabilitation; (IV) remote alarm activation in case the subject need assistance; (V) introduction of a more natural interaction with the surrounding computerized environment; (VI) users identification by physiological or behavioral characteristics. Telemedicine and mHealth [1] are two of the leading concepts directly related to healthcare. The capability to borne unobtrusiveness objects supports users’ autonomy. A new sense of freedom is shown to the user, not only supported by a psychological help but a real safety improvement. Furthermore, medical community aims the introduction of new devices to innovate patient treatments. In particular, the extension of the ambulatory analysis in the real life scenario by proving continuous acquisition. The wide diffusion of emerging wellness portable equipment extended the usability of wearable devices also for fitness and training by monitoring user performance on the working task. The learning of the right execution techniques related to work, sport, music can be supported by an electronic trainer furnishing the adequate aid. HCIs made real the concept of Ubiquitous, Pervasive Computing and Calm Technology introduced in the 1988 by Marc Weiser and John Seeley Brown. They promotes the creation of pervasive environments, enhancing the human experience. Context aware, adaptive and proactive environments serve and help people by becoming sensitive and reactive to their presence, since electronics is ubiquitous and deployed everywhere. In this thesis we pay attention to the integration of all the aspects involved in a BAN development. Starting from the choice of sensors we design the node, configure the radio network, implement real-time data analysis and provide a feedback to the user. We present algorithms to be implemented in wearable assistant for posture and gait analysis and to provide assistance on different walking conditions, preventing falls. Our aim, expressed by the idea to contribute at the development of a non proprietary solutions, driven us to integrate commercial and standard solutions in our devices. We use sensors available on the market and avoided to design specialized sensors in ASIC technologies. We employ standard radio protocol and open source projects when it was achieved. The specific contributions of the PhD research activities are presented and discussed in the following. • We have designed and build several wireless sensor node providing both sensing and actuator capability making the focus on the flexibility, small form factor and low power consumption. The key idea was to develop a simple and general purpose architecture for rapid analysis, prototyping and deployment of BAN solutions. Two different sensing units are integrated: kinematic (3D accelerometer and 3D gyroscopes) and kinetic (foot-floor contact pressure forces). Two kind of feedbacks were implemented: audio and vibrotactile. • Since the system built is a suitable platform for testing and measuring the features and the constraints of a sensor network (radio communication, network protocols, power consumption and autonomy), we made a comparison between Bluetooth and ZigBee performance in terms of throughput and energy efficiency. Test in the field evaluate the usability in the fall detection scenario. • To prove the flexibility of the architecture designed, we have implemented a wearable system for human posture rehabilitation. The application was developed in conjunction with biomedical engineers who provided the audio-algorithms to furnish a biofeedback to the user about his/her stability. • We explored off-line gait analysis of collected data, developing an algorithm to detect foot inclination in the sagittal plane, during walk. • In collaboration with the Wearable Lab – ETH, Zurich, we developed an algorithm to monitor the user during several walking condition where the user carry a load. The remainder of the thesis is organized as follows. Chapter I gives an overview about Body Area Networks (BANs), illustrating the relevant features of this technology and the key challenges still open. It concludes with a short list of the real solutions and prototypes proposed by academic research and manufacturers. The domain of the posture and gait analysis, the methodologies, and the technologies used to provide real-time feedback on detected events, are illustrated in Chapter II. The Chapter III and IV, respectively, shown BANs developed with the purpose to detect fall and monitor the gait taking advantage by two inertial measurement unit and baropodometric insoles. Chapter V reports an audio-biofeedback system to improve balance on the information provided by the use centre of mass. A walking assistant based on the KNN classifier to detect walking alteration on load carriage, is described in Chapter VI.

Genetische Analyse des Somatostatin-Systems

Somatostatin ist ein Molekül mit multifunktinonellem Charakter, dem Neurotransmitter-, Neuromodulator- und (Neuro)-Hormoneigenschaften zugeschrieben werden. Gemäß seiner ubiquitären Verteilung in Geweben beeinflusst es Stoffwechsel- und Entwicklungsprozesse, bis hin zu Lern-und Gedächtnisleistungen. Diese Wirkungen resultieren aus dem lokalen und zeitlichen Zusammenspiel eines Liganden und fünf G-Protein gekoppelter Rezeptoren (SSTR1-5). Zur Charakterisierung der biologischen Bedeutung des Somatostatin-Systems im Gesamtorganismus wurde eine Mutationsanalyse einzelner Systemkomponenten durchgeführt. Sie umfaßte die Inaktivierung der Gene für das Somatostatin-Präpropeptid und die der Rezeptoren SSTR3 und SSTR4 durch Gene Targeting. Die entsprechenden Ausfallmutationen belegen: Weder die Rezeptoren 3 und 4, noch Somatostatin sind für das Überleben des Organismus unter Standardhaltungsbedingungen notwendig. Die entsprechenden Mauslinien zeigen keine unmittelbar auffälligen Einschränkungen ihrer Biologie. Die Somatostatin-Nullmaus wurde zum Hauptgegenstand einer detaillierten Untersuchung aufgrund der übergeordneten Position des Liganden in der Signalkaskade und verfügbaren Hinweisen zu seiner Funktion. Folgende Schlußfolgerungen konnten nach eingehender Analyse gezogen werden: Der Ausfall des Somatostatin-Gens hat erhöhte Plasmakonzentrationen an Wachstumshormon (GH) zur Konsequenz. Dies steht im Einklang mit der Rolle Somatostatins als hemmender Faktor der Wachstumshormon-Freisetzung, die in der Mutante aufgehoben ist. Durch die Somatostatin-Nullmaus wurde zudem deutlich: Somatostatin interagiert als wesentliches Bindeglied zwischen der Wachstums- und Streßachse. Permanent erhöhte Corticosteron-Werte in den Mutanten implizieren einen negativen tonischen Einfluß für die Sekretion von Glukocorticoiden in vivo. Damit zeigt die Knockout-Maus, daß Somatostatin normalerweise als ein entscheidendes inhibierendes Kontrollelement der Steroidfreisetzung fungiert. Verhaltensversuche offenbarten ein Defizit im motorischen Lernen. Somatostatin-Nullmäuse bleiben im Lernparadigma “Rotierender Stabtest” hinter ihren Artgenossen zurück ohne aber generell in Motorik oder Koordination eingeschränkt zu sein. Diese motorischen Lernvorgänge sind von einem funktionierenden Kleinhirn abhängig. Da Somatostatin und seine Rezeptoren kaum im adulten, wohl aber im sich entwickelnden Kleinhirn auftreten, belegt dieses Ergebnis die Funktion transient in der Entwicklung exprimierter Neuropeptide – eine lang bestehende, aber bislang experimentell nicht nachgewiesene Hypothese. Die Überprüfung weiterer physiologischer Parameter und Verhaltenskategorien unter Standard-Laborbedingunggen ergab keine sichtbaren Abweichungen im Vergleich zu Wildtyp-Mäusen. Damit steht nun ein Tiermodell zur weiterführenden Analyse für die Somatostatin-Forschung bereit: In endokrinologischen, elektrophysiologischen und verhaltens-biologischen Experimenten ist nun eine unmittelbare Korrelation selektiv mit dem Somatostatin-Peptid bzw. mit den Rezeptoren 3 und 4 aber auch in Kombination der Ausfallmutationen nach entsprechenden Kreuzungen möglich.

Aktionenlernen mit Selbstorganisierenden Karten und Reinforcement Learning

Die vorliegende Arbeit beschäftigt sich mit der Entwicklung eines Funktionsapproximators und dessen Verwendung in Verfahren zum Lernen von diskreten und kontinuierlichen Aktionen: 1. Ein allgemeiner Funktionsapproximator – Locally Weighted Interpolating Growing Neural Gas (LWIGNG) – wird auf Basis eines Wachsenden Neuralen Gases (GNG) entwickelt. Die topologische Nachbarschaft in der Neuronenstruktur wird verwendet, um zwischen benachbarten Neuronen zu interpolieren und durch lokale Gewichtung die Approximation zu berechnen. Die Leistungsfähigkeit des Ansatzes, insbesondere in Hinsicht auf sich verändernde Zielfunktionen und sich verändernde Eingabeverteilungen, wird in verschiedenen Experimenten unter Beweis gestellt. 2. Zum Lernen diskreter Aktionen wird das LWIGNG-Verfahren mit Q-Learning zur Q-LWIGNG-Methode verbunden. Dafür muss der zugrunde liegende GNG-Algorithmus abgeändert werden, da die Eingabedaten beim Aktionenlernen eine bestimmte Reihenfolge haben. Q-LWIGNG erzielt sehr gute Ergebnisse beim Stabbalance- und beim Mountain-Car-Problem und gute Ergebnisse beim Acrobot-Problem. 3. Zum Lernen kontinuierlicher Aktionen wird ein REINFORCE-Algorithmus mit LWIGNG zur ReinforceGNG-Methode verbunden. Dabei wird eine Actor-Critic-Architektur eingesetzt, um aus zeitverzögerten Belohnungen zu lernen. LWIGNG approximiert sowohl die Zustands-Wertefunktion als auch die Politik, die in Form von situationsabhängigen Parametern einer Normalverteilung repräsentiert wird. ReinforceGNG wird erfolgreich zum Lernen von Bewegungen für einen simulierten 2-rädrigen Roboter eingesetzt, der einen rollenden Ball unter bestimmten Bedingungen abfangen soll.

Einfluss der Modulation von P-Glykoprotein (P-gp) auf die Verfügbarkeit und Wirksamkeit der pharmakologischen Behandlung psychiatrischer Erkrankungen im Tiermodell

P-Glykoprotein (P-gp) ist ein ATP-verbrauchender Transporter, der in Organschranken exprimiert wird, um Fremdstoffe auszuschleusen, darunter auch Psychopharmaka. Im Rahmen dieser Arbeit wurde im Tiermodell der Maus untersucht, welche pharmakokinetischen und pharmakodynamischen Konsequenzen sich bei Verabreichung von Risperidon als P-gp Modellsubstrat ergeben, wenn die Expression von P-gp induziert wird. Als potenzielle Induktoren wurden Dexamethason, Rifampicin, Quercetin, 5-Pregnen-3ß-ol-20-on-16α-Carbonitril (PCN) und Acitretin geprüft. Es konnte gezeigt werden, dass alle Substanzen die Verteilung von Risperidon und seinem aktiven Metaboliten 9-Hydroxyrisperidon beeinflussten. Während sich für Quercetin und Acitretin leichte P-gp inhibitorische Eigenschaften ergaben, die an Hand von erhöhten Konzentrationen von Risperidon und 9-Hydroxyrisperidon gezeigt werden konnten, führten die bekannten P-gp Induktoren Rifampicin, Dexamethason und PCN zu verringerten Konzentrationen im Vergleich zur Kontrollgruppe. Durch Western Blot Untersuchungen wurde bestätigt, dass die Induktoren die P-gp Expression im Hirngewebe tendenziell steigerten. Dies sprach dafür, dass bei Verabreichung einer Komedikation, die P-gp induziert, mit einer veränderten Verteilung von P-gp Substraten zu rechnen ist. Darüber hinaus konnte nachgewiesen werden, dass durch eine Hemmung bzw. Induktion von P-gp nicht nur die Pharmakokinetik, sondern auch die Pharmakodynamik von Risperidon und 9-Hydroxyrisperidon verändert wird. Dies wurde durch verhaltenspharmakologische Untersuchungen gezeigt. Durch Risperidon induzierte motorische Effekte auf dem RotaRod waren nach Induktion von P-gp abgeschwächt. Dies zeigte sich auch für Haloperidol, welches kein Substrat ist. Da P-gp abhängige Effekte in diesem Fall keine bedeutende Rolle spielen, ist davon auszugehen, dass neben der Induktion von P-gp an der Blut-Hirn Schranke auch andere Mechanismen wie z.B. eine Induktion von Enzymen der CYP-Familie an den beobachteten Effekten beteiligt sind. Bei Untersuchungen von kognitiven Leistungen in der Barnes Maze konnte gezeigt werden, dass Haloperidol im Gegensatz zu Risperidon das Lernverhalten negativ beeinflussen kann. Eine P-gp Induktion schien jedoch keinen deutlichen Einfluss auf das Lernverhalten unter Antipsychotika-Gabe zu haben und sprach vielmehr für substanzabhängige Effekte der einzelnen Antipsychotika bzw. P-gp Modulatoren. Zusatzuntersuchungen zur Hirngängigkeit von Acitretin, einem synthetischen Retinoid, welches derzeit als potenzielles Antidementivum geprüft wird, konnten belegen, dass es die Blut-Hirn Schranke überwindet. Bereits 1h nach Injektion war Acitretin in hoher Konzentration im Gehirn nachweisbar. Durch die Analyse zur Verteilung von Acitretin in Hirngewebe und Serum von P-gp Wildtyp und P-gp doppel knockout Mäusen konnte belegt werden, dass Acitretin nicht P-gp abhängig transportiert wird. Die Daten insgesamt betrachtet, lassen den Schluss zu, dass durch Verabreichung von Medikamenten, die P-gp Modulatoren sind, bei Antipsychotika mit pharmakokinetischen Interaktionen zu rechnen ist, welche die Wirksamkeit der Medikamente einschränken können.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Threshold-dependent effects on peripheral nerve in vivo excitability properties in the rat

Various factors, including maturity, have been shown to influence peripheral nerve excitability measures, but little is known about differences in these properties between axons with different stimulation thresholds. Multiple nerve excitability tests were performed on the caudal motor axons of immature and mature female rats, recording from tail muscles at three target compound muscle action potential (CMAP) levels: 10%, 40% ("standard" level), and 60% of the maximum CMAP amplitude. Compared to lower target levels, axons at high target levels have the following characteristics: lower strength-duration time constant, less threshold reduction during depolarizing currents and greater threshold increase to hyperpolarizing currents, most notably to long hyperpolarizing currents in mature rats. Threshold-dependent effects on peripheral nerve excitability properties depend on the maturation stage, especially inward rectification (Ih), which becomes inversely related to threshold level. Performing nerve excitability tests at different target levels is useful in understanding the variation in membrane properties between different axons within a nerve. Because of the threshold effects on nerve excitability and the possibility of increased variability between axons and altered electric recruitment order in disease conditions, excitability parameters measured only at the "standard" target level should be interpreted with caution, especially the responses to hyperpolarizing currents.

In vivo assessment of HCN channel current (I(h)) in human motor axons

The "Trond" protocol of nerve excitability tests has been used widely to assess axonal function in peripheral nerve. In this study, the routine Trond protocol was expanded to refine assessment of cAMP-dependent, hyperpolarization-activated current (I(h)) activity. I(h) activity is generated by hyperpolarization-activated, cyclic nucleotide-modulated (HCN) channels in response to hyperpolarization. It limits activity-dependent hyperpolarization, contributes to neuronal automaticity, and is implicated in chronic pain states. Published data regarding I(h) activity in motor nerve are scant. We used additional strong, prolonged hyperpolarizing conditioning stimuli in the threshold electrotonus component of the Trond protocol to demonstrate the time-course of activation of I(h) in motor axons. Fifteen healthy volunteers were tested on four occasions during 1 week. I(h) action was revealed in the threshold electrotonus by the limiting and often reversal, after about 100 ms, of the threshold increase caused by strong hyperpolarizing currents. Statistical analysis by repeated-measures analysis of variance enabled confidence limits to be established for variation between subjects and within subjects. The results demonstrate that, of all the excitability parameters, those dependent on I(h) were the most characteristic of an individual, because variance between subjects was more than four times the variance within subjects. This study demonstrates a reliable method for in vivo assessment of I(h,) and also serves to document the normal variability in nerve excitability properties within subjects.