Molecular mechanisms underlying a cellular analog of operant reward learning.

Operant conditioning is a ubiquitous but mechanistically poorly understood form of associative learning in which an animal learns the consequences of its behavior. Using a single-cell analog of operant conditioning in neuron B51 of Aplysia, we examined second-messenger pathways engaged by activity and reward and how they may provide a biochemical association underlying operant learning. Conditioning was blocked by Rp-cAMP, a peptide inhibitor of PKA, a PKC inhibitor, and by expressing a dominant-negative isoform of Ca2+-dependent PKC (apl-I). Thus, both PKA and PKC were necessary for operant conditioning. Injection of cAMP into B51 mimicked the effects of operant conditioning. Activation of PKC also mimicked conditioning but was dependent on both cAMP and PKA, suggesting that PKC acted at some point upstream of PKA activation. Our results demonstrate how these molecules can interact to mediate operant conditioning in an individual neuron important for the expression of the conditioned behavior.

Baseline hippocampal theta oscillation speeds correlate with rate of operant task acquisition

Many lines of evidence indicate that theta rhythm, a prominent neural oscillatory mode found in the mammalian hippocampus, plays a key role in the acquisition, processing, and retrieval of memories. However, a predictive neurophysiological feature of the baseline theta rhythm that correlates with the learning rate across different animals has yet to be identified. Here we show that the mean theta rhythm speed observed during baseline periods of immobility has a strong positive correlation with the rate at which rats learn an operant task. This relationship is observed across rats, during both quiet waking (r=0.82; p<0.01) and paradoxical sleep (r=0.83; p<0.01), suggesting that the basal theta frequency relates to basic neurological processes that are important in the acquisition of operant behavior. (c) 2008 Elsevier B.V. All rights reserved.

Two -way direction of influence in mother-infant interaction as denoted by concurrent reciprocal learning processes

This series of 5 single-subject studies used the operant conditioning paradigm to investigate, within the two-way influence process, how (a) contingent infant attention can reinforce maternal verbal behaviors during a period of mother-infant interaction and under subsequent experimental manipulation. Differential reinforcement was used to determine if it is possible that an infant attending to the mother (denoted by head-turns towards the image of the mother plus eye contact) increases (reinforces) the mother's verbal response (to a cue from the infant) upon which the infant behavior is contingent. There was also (b) an evaluation during the contrived parent-infant interaction for concurrent operant learning of infant vocal behavior via contingent verbal responding (reinforcement) implemented by the mother. Further, it was noted (c) whether or not the mother reported being aware that her responses were influenced by the infant's behavior. Findings showed: the operant conditioning of the maternal verbal behaviors were reinforced by contingent infant attention; and the operant conditioning of infant vocalizations was reinforced by contingent maternal verbal behaviors. No parent reported (1) being aware of the increase in their verbal response reinforced during operant conditioning of parental behavior nor a decrease in those responses during the DRA reversal phase, or (2) noticing a contingency between infant's and mother's response. By binomial 1-tail tests, the verbal-behavior patterns of the 5 mothers were conditioned by infant reinforcement (p < 0.02) and, concurrently, the vocal-response patterns of the 5 infants were conditioned by maternal reinforcement (p < 0.02). A program of systematic empirical research on the determinants of concurrent conditioning within mother-child interaction may provide a way to evaluate the differential effectiveness of interventions aimed at improving parent-child interactions. The work conducted in the present study is one step in this direction. ^

Beyond a "woman's problem" : the role of relationship processes in female genital pain

Female genital pain is a prevalent condition that can disrupt the psychosexual and relational well-being of affected women and their romantic partners. Despite the intimate context in which the pain can be elicited (i.e., during sexual intercourse), interpersonal correlates of genital pain and sexuality have not been widely studied in comparison to other psychosocial factors. This review describes several prevailing theoretical models explaining the role of the partner in female genital pain: the operant learning model, cognitive-behavioral and communal coping models, and intimacy models. The review includes a discussion of empirical research on the interpersonal and partner correlates of female genital pain and the impact of genital pain on partners’ psychosexual adjustment. Together, this research highlights a potential reciprocal interaction between both partners’ experiences of female genital pain. The direction of future theoretical, methodological, and clinical research is discussed with regard to the potential to enhance understanding of the highly interpersonal context of female genital pain

Novelty, but Not Operant Aversive Learning, Enhances Fos and Egr-1 Expression in the Medial Prefrontal Cortex and Hippocampal Areas of Rats

Immediate early genes (IEG) are presumed to be activated in response to stress, novelty, and learning. Evidence supports the involvement of prefrontal and hippocampal areas in stress and learning, but also in the detection of novel events. This study examined whether a previous experience with shocks changes the pattern of Fos and Egr-1 expression in the medial prefrontal cortex (mPFC), the hippocampal cornus ammonis 1 (CA1), and dentate gyrus (DG) of adult male Wistar rats that learned to escape in an operant aversive test. Subjects previously exposed to inescapable footshocks that learned to escape from Shocks were assigned to the treated group (EXP). Subjects from Group Novelty (NOV) rested undisturbed during treatment and also learned to escape in the test. The nonshock group (NSH) rested undisturbed in both sessions. Standard immunohistochemistry procedures were used to detect the proteins in brain sections. The results show that a previous experience with shocks changed the pattern of IEG expression, then demonstrating c-fos and egr-1 induction as experience-dependent events. Compared with NSH and EXP an enhanced Fos expression was detected in the mPFC and CA1 subfield of Group NOV, which also exhibited increased Egr-1 expression in the mPFC and DG in comparison to NSH. No differences were found in the DG for Fos, or in the CA1 for Egr-1. Novelty, and not the operant aversive escape learning, seems to have generated IEG induction. The results suggest novel stimuli as a possible confounding factor in studies on Fos and/or Egr-1 expression in aversive conditions.

Learning processes and the neural analysis of conditioning

Classical and operant conditioning principles, such as the behavioral discrepancy-derived assumption that reinforcement always selects antecedent stimulus and response relations, have been studied at the neural level, mainly by observing the strengthening of neuronal responses or synaptic connections. A review of the literature on the neural basis of behavior provided extensive scientific data that indicate a synthesis between the two conditioning processes based mainly on stimulus control in learning tasks. The resulting analysis revealed the following aspects. Dopamine acts as a behavioral discrepancy signal in the midbrain pathway of positive reinforcement, leading toward the nucleus accumbens. Dopamine modulates both types of conditioning in the Aplysia mollusk and in mammals. In vivo and in vitro mollusk preparations show convergence of both types of conditioning in the same motor neuron. Frontal cortical neurons are involved in behavioral discrimination in reversal and extinction procedures, and these neurons preferentially deliver glutamate through conditioned stimulus or discriminative stimulus pathways. Discriminative neural responses can reliably precede operant movements and can also be common to stimuli that share complex symbolic relations. The present article discusses convergent and divergent points between conditioning paradigms at the neural level of analysis to advance our knowledge on reinforcement.

Aprendizaje durante la ontogenia temprana: Estudio de la participación del sistema opioide en la modulación de aprendizajes mediados por el etanol. LEARNING AND EARLY ONTOGENY: ANALYSIS OF THE INVOLVEMENT OF OPIOID SYSTEM IN FETAL AND INFANTILE ETHANOL-MEDIATED EXPERIENCIES.

Este plan de trabajos es continuidad de una línea de investigación centrada en evaluar los mecanismos responsables de la adquisición, expresión y persistencia de experiencias con el etanol. A partir de ello, indagar acerca de los efectos de esta experiencia sobre comportamientos de búsqueda y autoadministración de etanol en neonatos e infantes de rata. Se pretende analizar la participación del sistema opiáceo en los mecanismos implicados en una memoria fetal y/o infantil, generada como consecuencia de la exposición etílica. En una primera etapa, nos proponemos establecer de qué manera experiencias prenatales con la droga modulan el patrón de auto-administración de alcohol y otros reforzadores, como sacarosa. En este primer bloque de experimentos realizaremos manipulaciones fetales para determinar con mayor grado de especificidad la posible acción del sistema opiáceo en los mecanismos de adquisición de una memoria etílica prenatal. Se realizarán administraciones de etanol y el antagonista opiáceo, directamente a nivel fetal, y se evaluará esta experiencia en un paradigma de condicionamiento neonatal positivo, mediado por la droga. De acuerdo a la evidencia previa, esperamos que la exposición prenatal con la droga facilite la expresión de conductas de consumo y búsqueda del etanol o hacia las claves que señalizan al psicotrópico, tanto durante la infancia como en el neonato. A su vez, cuando la droga es presentada bajo los efectos de un antagonista opiáceo esperamos que estas conductas muestren un perfil similar a las desplegadas por sujetos controles. El segundo bloque de experimentos ha sido ideado con el objeto de indagar acerca de la posible participación del sistema opiáceo en la modulación de los aspectos reforzantes de la droga, a través de un esquema de auto-administración etílica infantil. Se utilizará un paradigma de condicionamiento instrumental adaptado para ratas infantes que consta de dos instancias, una de adquisición de la conducta instrumental (DPs 14-17) en la cual los animales reciben un pulso de refuerzo, como consecuencia de la ejecución de la conducta operante. En una segunda fase se analiza el patrón de búsqueda del reforzador ya que se registra la respuesta instrumental, sin que ocurra el refuerzo por la misma. Para analizar la participación del sistema opiáceo, durante la fase de adquisición de la conducta operante (DPs 16 y 17) los animales serán re-expuestos a mínimas cantidades del reforzador, bajo los efectos de un antagonista opiáceo, momentos previos al ensayo instrumental correspondiente para cada uno de estos días (Exp. 3). Esperamos que el bloqueo del sistema opiáceo, durante esta re-exposición al etanol, sea suficiente para disminuir el patrón de respuesta instrumental hacia el refuerzo etílico. Un último experimento incorporará un tercer evento de re-exposición al etanol -bajo los efectos del antagonista- previo al ensayo de extinción de la conducta instrumental (DP 18). Este nuevo evento tiene por objeto analizar la participación de este sistema neurobiológico en los mecanismos de búsqueda de etanol. Si el sistema opiáceo participa en la modulación de patrones tanto de búsqueda como consumatorios del reforzamiento por etanol, se espera que la re-exposición a la droga bajo los efectos del antagonista, inhiba estas respuestas tanto durante la sesión de adquisición, como de extinción de la conducta operante. Este proyecto intenta profundizar en el conocimiento de los mecanismos que regulan reconocimiento, aceptación, búsqueda y consumo de etanol, como consecuencia de experiencias tempranas con la droga. A su vez, es importante identificar y estudiar los sistemas neurobiológicos involucrados en estos mecanismos. Es por ello que se intenta determinar el rol que ejerce el sistema opiáceo en la adquisición de estas experiencias etílicas a nivel fetal e infantil, que se conoce promueven la búsqueda y el consumo de la droga. Our work is directed to analyze the involvement of the opioid system in the generation of pre- and early postnatal ethanol-related memories. As a first step, maternal manipulations with ethanol will be done. Infants will be evaluated in a paradigm of infantile self-administration of different reinforcers (ethanol, sucrose or water), employing a model of operant conditioning adapted to infant rats. A second experiment will be conducted in order to analyze if a central administration of ethanol, directly to the fetus, modifies subsequent patterns of neonatal conditioned responses to an artificial nipple, mediated by ethanol reinforcing effects. Fetal presentation of ethanol will be accompanied with the injection of an opioid antagonist in order to analyze the involvement of this system in acquisition processes of a fetal ethanol-mediated memory. A second set of studies will be conducted to analyze appetitive and consummatory behaviors in an infant model of ethanol self-administration. Involvement of opioid system in the acquisition or expression of this experience will be also inquired. Infant rats (PDs14-17) have to display a target behavior (nose-poke) to gain access to 5% sucrose or 3.75% ethanol. On PD18 an extinction session will be included. At PDs16-17, 6-hr before training, pups will be re-exposed to ethanol under opioid antagonism effects (naloxone). In a follow up experiment, a re-exposure trial will be included at PD18. Prior extinction, pups will receive naloxone and will be re-exposed to ethanol. We aim to observe if opioid system is modulating etha¬nol reinforcing effects, in terms of both appetitive and consummatory behaviors.

Operant serial feature-positive conditional discrimination with composite features and different topography responses

In a serial feature-positive conditional discrimination procedure the properties of a target stimulus A are defined by the presence or not of a feature stimulus X preceding it. In the present experiment, composite features preceded targets associated with two different topography operant responses (right and left bar pressing); matching and non-matching-to-sample arrangements were also used. Five water-deprived Wistar rats were trained in 6 different trials: X-R®Ar and X-L®Al, in which X and A were same modality visual stimuli and the reinforcement was contingent to pressing either the right (r) or left (l) bar that had the light on during the feature (matching-to-sample); Y-R®Bl and Y-L®Br, in which Y and B were same modality auditory stimuli and the reinforcement was contingent to pressing the bar that had the light off during the feature (non-matching-to-sample); A- and B- alone. After 100 training sessions, the animals were submitted to transfer tests with the targets used plus a new one (auditory click). Average percentages of stimuli with a response were measured. Acquisition occurred completely only for Y-L®Br+; however, complex associations were established along training. Transfer was not complete during the tests since concurrent effects of extinction and response generalization also occurred. Results suggest the use of both simple conditioning and configurational strategies, favoring the most recent theories of conditional discrimination learning. The implications of the use of complex arrangements for discussing these theories are considered.

Behavioural and neural correlates of operant conditioning in Lymnaea stagnalis : role of previous experience during development /

The freshwater mollusc Lymnaea stagnalis was utilized in this study to further the understanding of how network properties change as a result of associative learning, and to determine whether or not this plasticity is dependent on previous experience during development. The respiratory and neural correlates of operant conditioning were first determined in normally reared Lymnaea. The same procedure was then applied to differentially reared Lymnaea, that is, animals that had never experienced aerial respiration during their development. The aim was to determine whether these animals would demonstrate the same responses to the training paradigm. In normally reared animals, a behavioural reduction in aerial respiration was accompanied by numerous changes within the neural network. Specifically, I provide evidence of changes at the level of the respiratory central pattern generator and the motor output. In the differentially reared animals, there was little behavioural data to suggest learning and memory. There were, however, significant differences in the network parameters, similar to those observed in normally reared animals. This demonstrated an effect of operant conditioning on differentially reared animals. In this thesis, I have identified additional correlates of operant conditioning in normally reared animals and provide evidence of associative learning in differentially reared animals. I conclude plasticity is not dependent on previous experience, but is rather ontogenetically programmed within the neural network.

Cross-situational and supervised learning in the emergence of communication

Scenarios for the emergence or bootstrap of a lexicon involve the repeated interaction between at least two agents who must reach a consensus on how to name N objects using H words. Here we consider minimal models of two types of learning algorithms: cross-situational learning, in which the individuals determine the meaning of a word by looking for something in common across all observed uses of that word, and supervised operant conditioning learning, in which there is strong feedback between individuals about the intended meaning of the words. Despite the stark differences between these learning schemes, we show that they yield the same communication accuracy in the limits of large N and H, which coincides with the result of the classical occupancy problem of randomly assigning N objects to H words.

Operant measurement of auditory threshold in prelingually deaf users of cochlear implants: II

Two experiments evaluated an operant procedure for establishing stimulus control using auditory and electrical stimuli as a baseline for measuring the electrical current threshold of electrodes implanted in the cochlea. Twenty-one prelingually deaf children, users of cochlear implants, learned a Go/No Go auditory discrimination task (i.e., pressing a button in the presence of the stimulus but not in its absence). When the simple discrimination baseline became stable, the electrical current was manipulated in descending and ascending series according to an adapted staircase method. Thresholds were determined for three electrodes, one in each location in the cochlea (basal, medial, and apical). Stimulus control was maintained within a certain range of decreasing electrical current but was eventually disrupted. Increasing the current recovered stimulus control, thus allowing the determination of a range of electrical currents that could be defined as the threshold. The present study demonstrated the feasibility of the operant procedure combined with a psychophysical method for threshold assessment, thus contributing to the routine fitting and maintenance of cochlear implants within the limitations of a hospital setting.

Spatio-temporal credit assignment in population learning

We present a model for plasticity induction in reinforcement learning which is based on a cascade of synaptic memory traces. In the cascade of these so called eligibility traces presynaptic input is first corre lated with postsynaptic events, next with the behavioral decisions and finally with the external reinforcement. A population of leaky integrate and fire neurons endowed with this plasticity scheme is studied by simulation on different tasks. For operant co nditioning with delayed reinforcement, learning succeeds even when the delay is so large that the delivered reward reflects the appropriateness, not of the immediately preceeding response, but of a decision made earlier on in the stimulus - decision sequence . So the proposed model does not rely on the temporal contiguity between decision and pertinent reward and thus provides a viable means of addressing the temporal credit assignment problem. In the same task, learning speeds up with increasing population si ze, showing that the plasticity cascade simultaneously addresses the spatial problem of assigning credit to the different population neurons. Simulations on other task such as sequential decision making serve to highlight the robustness of the proposed sch eme and, further, contrast its performance to that of temporal difference based approaches to reinforcement learning.

Spatio-Temporal Credit Assignment in Neuronal Population Learning

n learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.

Extending in vitro conditioning in Aplysia to analyze operant and classical processes in the same preparation.

Operant and classical conditioning are major processes shaping behavioral responses in all animals. Although the understanding of the mechanisms of classical conditioning has expanded significantly, the understanding of the mechanisms of operant conditioning is more limited. Recent developments in Aplysia are helping to narrow the gap in the level of understanding between operant and classical conditioning, and have raised the possibility of studying the neuronal processes underlying the interaction of operant and classical components in a relatively complex learning task. In the present study, we describe a first step toward realizing this goal, by developing a single in vitro preparation in which both operant and classical conditioning can be studied concurrently. The new paradigm reproduced previously published results, even under more conservative and homogenous selection criteria and tonic stimulation regime. Moreover, the observed learning was resistant to delay, shortening, and signaling of reinforcement.

Cellular mechanisms of operant and classical conditioning

The ability to associate a predictive stimulus with a subsequent salient event (i.e., classical conditioning) and the ability to associate an expressed behavior with the consequences (i.e., operant conditioning) allow for a predictive understanding of a changing environment. Although they are operationally distinct, there has been considerable debate whether at some fundamental level classical and operant conditioning are mechanistically distinct or similar. Feeding behavior of Aplysia (i.e., biting) was chosen as the model system and was successfully conditioned with appetitive forms of both operant and classical conditioning. The neuronal circuitry responsible for feeding is well understood and is suitable for cellular analyses, thus providing for a mechanistic comparison between these two forms of associative learning. ^ Neuron B51 is part of the feeding circuitry of Aplysia and is critical for the expression of ingestive behaviors. B51 also is a locus of plasticity following both operant and classical conditioning. Both in vivo and in vitro operant conditioning increased the input resistance and the excitability of B51. No pairing-specific changes in the input resistance were observed following both in vivo and in vitro classical conditioning. However, classical conditioning decreased the excitability of B51. Thus, both operant and classical conditioning modified the threshold level for activation of neuron B51, but in opposite directions, revealing key differences in the cellular mechanisms underlying these two forms of associative learning. ^ Next, the cellular mechanisms underlying operant conditioning were investigated in more detail using a single-cell analogue. The single-cell analogue successfully recapitulated the previous in vivo and in vitro operant conditioning results by increasing the input resistance and the excitability of B51. Both PKA and PKC were necessary for operant conditioning. Dopamine appears to be the transmitter mediating the reinforcement signal in this form of conditioning. A D1 dopamine receptor antibody revealed that the D1receptor localizes to the axon hillock, which is also the region that gives the strongest response when iontophoresing dopamine. ^ The studies presented herein, thus, provide for a greater understanding of the mechanisms underlying both of these forms of associative learning and demonstrate that they likely operate through distinct cellular mechanisms. ^