In vitro analog of classical conditioning of feeding behavior in aplysia.

The feeding behavior of Aplysia californica can be classically conditioned using tactile stimulation of the lips as a conditioned stimulus (CS) and food as an unconditioned stimulus (US). Moreover, several neural correlates of classical conditioning have been identified. The present study extended previous work by developing an in vitro analog of classical conditioning and by investigating pairing-specific changes in neuronal and synaptic properties. The preparation consisted of the isolated cerebral and buccal ganglia. Electrical stimulation of a lip nerve (AT4) and a branch of the esophageal nerve (En2) served as the CS and US, respectively. Three protocols were used: paired, unpaired, and US alone. Only the paired protocol produced a significant increase in CS-evoked fictive feeding. At the cellular level, classical conditioning enhanced the magnitude of the CS-evoked synaptic input to pattern-initiating neuron B31/32. In addition, paired training enhanced both the magnitude of the CS-evoked synaptic input and the CS-evoked spike activity in command-like neuron CBI-2. The in vitro analog of classical conditioning reproduced all of the cellular changes that previously were identified following behavioral conditioning and has led to the identification of several new learning-related neural changes. In addition, the pairing-specific enhancement of the CS response in CBI-2 indicates that some aspects of associative plasticity may occur at the level of the cerebral sensory neurons.

Behavioral and cellular analysis of classical conditioning of feeding behavior in Aplysia

In classical conditioning, an associative form of learning, animals learn to associate two stimuli. Cellular and molecular mechanisms for the induction and consolidation of associative learning and memory at the level of single cells and synaptic connections have been studied in both vertebrate and invertebrate animals. The majority of studies, however, relied on aversive stimuli to induce learning. This bias may limit the extent to which identified mechanisms generalize to other forms of associative learning and memory, such as appetitive forms. The goal of the present study was to develop a classical conditioning procedure for the marine mollusk Aplysia californica using appetitive reinforcement, and to analyze associative learning using behavioral and electrophysiological techniques. ^ Using tactile stimulation of the lips as the conditional stimulus (CS) and food as the unconditional stimulus (US) a training protocol was developed that reliably induced classical conditioning of feeding behavior. Memory persisted for at least 24 hours. The gross organization of reinforcement-mediating pathways was analyzed in additional behavioral experiments. Moreover, neurophysiological correlates of classical conditioning were identified and characterized in an in vitro preparation containing the circuitry for feeding behavior. In vitro stimulation of a nerve (AT4) that may mediate the CS during training, resulted in a greater number of buccal motor patterns (BMPs) in brains from conditioned animals, as compared to control animals. The majority of these BMPs were ingestion-like, consistent with the increased number of bites in response to the CS after classical conditioning. Moreover, classical conditioning correlated with increased excitatory synaptic input to BMP-initiating neuron B31/32, in response to stimulation of AT 4, as compared to controls. The expression of the correlates of classical conditioning identified in this study was specific to stimulation of AT 4, which is consistent the stimulus specificity that is characteristic for classical conditioning. ^ The identification of cellular correlates of classical conditioning documented here provides the basis for future, more detailed analyses of an appetitive form of associative learning and memory, that may extend the working knowledge of the cellular and molecular mechanisms for associative plasticity in general. ^

Cellular mechanisms of operant and classical conditioning

The ability to associate a predictive stimulus with a subsequent salient event (i.e., classical conditioning) and the ability to associate an expressed behavior with the consequences (i.e., operant conditioning) allow for a predictive understanding of a changing environment. Although they are operationally distinct, there has been considerable debate whether at some fundamental level classical and operant conditioning are mechanistically distinct or similar. Feeding behavior of Aplysia (i.e., biting) was chosen as the model system and was successfully conditioned with appetitive forms of both operant and classical conditioning. The neuronal circuitry responsible for feeding is well understood and is suitable for cellular analyses, thus providing for a mechanistic comparison between these two forms of associative learning. ^ Neuron B51 is part of the feeding circuitry of Aplysia and is critical for the expression of ingestive behaviors. B51 also is a locus of plasticity following both operant and classical conditioning. Both in vivo and in vitro operant conditioning increased the input resistance and the excitability of B51. No pairing-specific changes in the input resistance were observed following both in vivo and in vitro classical conditioning. However, classical conditioning decreased the excitability of B51. Thus, both operant and classical conditioning modified the threshold level for activation of neuron B51, but in opposite directions, revealing key differences in the cellular mechanisms underlying these two forms of associative learning. ^ Next, the cellular mechanisms underlying operant conditioning were investigated in more detail using a single-cell analogue. The single-cell analogue successfully recapitulated the previous in vivo and in vitro operant conditioning results by increasing the input resistance and the excitability of B51. Both PKA and PKC were necessary for operant conditioning. Dopamine appears to be the transmitter mediating the reinforcement signal in this form of conditioning. A D1 dopamine receptor antibody revealed that the D1receptor localizes to the axon hillock, which is also the region that gives the strongest response when iontophoresing dopamine. ^ The studies presented herein, thus, provide for a greater understanding of the mechanisms underlying both of these forms of associative learning and demonstrate that they likely operate through distinct cellular mechanisms. ^

Extending in vitro conditioning in Aplysia to analyze operant and classical processes in the same preparation.

Operant and classical conditioning are major processes shaping behavioral responses in all animals. Although the understanding of the mechanisms of classical conditioning has expanded significantly, the understanding of the mechanisms of operant conditioning is more limited. Recent developments in Aplysia are helping to narrow the gap in the level of understanding between operant and classical conditioning, and have raised the possibility of studying the neuronal processes underlying the interaction of operant and classical components in a relatively complex learning task. In the present study, we describe a first step toward realizing this goal, by developing a single in vitro preparation in which both operant and classical conditioning can be studied concurrently. The new paradigm reproduced previously published results, even under more conservative and homogenous selection criteria and tonic stimulation regime. Moreover, the observed learning was resistant to delay, shortening, and signaling of reinforcement.

Contingency awareness in human aversive conditioning involves the middle frontal gyrus.

In contrast to the wealth of data describing the neural mechanisms underlying classical conditioning, we know remarkably little about the mechanisms involved in acquisition of explicit contingency awareness. Subjects variably acquire contingency awareness in classical conditioning paradigms, in which they are able to describe the temporal relationship between a conditioned cue and its outcome. Previous studies have implicated the hippocampus and prefrontal cortex in the acquisition of explicit knowledge, although their specific roles remain unclear. We used functional magnetic resonance imaging to track the trial-by-trial acquisition of explicit knowledge in a concurrent trace and delay conditioning paradigm. We show that activity in bilateral middle frontal gyrus and parahippocampal gyrus correlates with the accuracy of explicit contingency awareness on each trial. In contrast, amygdala activation correlates with conditioned responses indexed by skin conductance responses (SCRs). These results demonstrate that brain regions known to be involved in other aspects of learning and memory also play a specific role, reflecting on each trial the acquisition and representation of contingency awareness.

Enhancement of sensorimotor connections by conditioning-related stimulation in Aplysia depends upon postsynaptic Ca2+.

Classical conditioning of Aplysia's siphon-withdrawal reflex is thought to be due to a presynaptic mechanism-activity-dependent presynaptic facilitation of sensorimotor connections. Recent experiments with sensorimotor synapses in dissociated cell culture, however, provide an alternative cellular mechanism for classical conditioning-Hebbian long-term potentiation (LTP) of sensorimotor connections. Induction of Hebbian LTP of these connections is mediated by activation of N-methyl-D-aspartate-related receptors and requires the postsynaptic elevation of intracellular Ca2+. To determine whether the enhancement of sensorimotor synapses during classical conditioning in Aplysia-like LTP of sensorimotor synapses in culture-also depends upon the elevation of postsynaptic Ca2+, we carried out experiments involving the cellular analog of classical conditioning of siphon withdrawal. We examined changes in the strength of monosynaptic siphon sensorimotor connections in the abdominal ganglion of Aplysia following paired presentations of sensory neuron activation and tail nerve shock. This training regimen resulted in significant enhancement of the monosynaptic sensorimotor excitatory postsynaptic potential, as compared with the sensorimotor excitatory postsynaptic potential in preparations that received only test stimulation. Infusing the motor neuron with 1,2-bis(2-aminophenoxy)ethane-N,N-N',N'-tetraacetic acid, a specific chelator of intracellular Ca2+, prior to paired stimulation training blocked this synaptic enhancement. Our results implicate a postsynaptic, possibly Hebbian, mechanism in classical conditioning in Aplysia.

Functional anatomy of human eyeblink conditioning determined with regional cerebral glucose metabolism and positron-emission tomography.

Relative cerebral glucose metabolism was examined with positron-emission tomography (PET) as a measure of neuronal activation during performance of the classically conditioned eyeblink response in 12 young adult subjects. Each subject received three sessions: (i) a control session with PET scan in which unpaired presentations of the tone conditioned stimulus and corneal airpuff unconditioned stimulus were administered, (ii) a paired training session to allow associative learning to occur, and (iii) a paired test session with PET scan. Brain regions exhibiting learning-related activation were identified as those areas that showed significant differences in glucose metabolism between the unpaired control condition and well-trained state in the 9 subjects who met the learning criterion. Areas showing significant activation included bilateral sites in the inferior cerebellar cortex/deep nuclei, anterior cerebellar vermis, contralateral cerebellar cortex and pontine tegmentum, ipsilateral inferior thalamus/red nucleus, ipsilateral hippocampal formation, ipsilateral lateral temporal cortex, and bilateral ventral striatum. Among all subjects, including those who did not meet the learning criterion, metabolic changes in ipsilateral cerebellar nuclei, bilateral cerebellar cortex, anterior vermis, contralateral pontine tegmentum, ipsilateral hippocampal formation, and bilateral striatum correlated with degree of learning. The localization to cerebellum and its associated brainstem circuitry is consistent with neurobiological studies in the rabbit model of eyeblink classical conditioning and neuropsychological studies in brain-damaged humans. In addition, these data support a role for the hippocampus in conditioning and suggest that the ventral striatum may also be involved.

Slithering snakes, angry men and out-group members : what and whom are we evolved to fear?

The preparedness theory of classical conditioning proposed by Seligman (1970, 1971) has been applied extensively over the past 40 years to explain the nature and "source" of human fear and phobias. In this review we examine the formative studies that tested the four defining characteristics of prepared learning with animal fear-relevant stimuli (typically snakes and spiders) and consider claims that fear of social stimuli, such as angry faces, or faces of racial out-group members, may also be acquired utilising the same preferential learning mechanism. Exposition of critical differences between fear learning to animal and social stimuli suggests that a single account cannot adequately explain fear learning with animal and social stimuli. We demonstrate that fear conditioned to social stimuli is less robust than fear conditioned to animal stimuli as it is susceptible to cognitive influence and propose that it may instead reflect on negative stereotypes and social norms. Thus, a theoretical model that can accommodate the influence of both biological and cultural factors is likely to have broader utility in the explanation of fear and avoidance responses than accounts based on a single mechanism.

Pavlovian fear memory circuits and phenotype models of PTSD

Pavlovian fear conditioning, also known as classical fear conditioning is an important model in the study of the neurobiology of normal and pathological fear. Progress in the neurobiology of Pavlovian fear also enhances our understanding of disorders such as posttraumatic stress disorder (PTSD) and with developing effective treatment strategies. Here we describe how Pavlovian fear conditioning is a key tool for understanding both the neurobiology of fear and the mechanisms underlying variations in fear memory strength observed across different phenotypes. First we discuss how Pavlovian fear models aspects of PTSD. Second, we describe the neural circuits of Pavlovian fear and the molecular mechanisms within these circuits that regulate fear memory. Finally, we show how fear memory strength is heritable; and describe genes which are specifically linked to both changes in Pavlovian fear behavior and to its underlying neural circuitry. These emerging data begin to define the essential genes, cells and circuits that contribute to normal and pathological fear.

Essays in behavioral and neuro-economics

This thesis examines foundational questions in behavioral economics—also called psychology and economics—and the neural foundations of varied sources of utility. We have three primary aims: First, to provide the field of behavioral economics with psychological theories of behavior that are derived from neuroscience and to use those theories to identify novel evidence for behavioral biases. Second, we provide neural and micro foundations of behavioral preferences that give rise to well-documented empirical phenomena in behavioral economics. Finally, we show how a deep understanding of the neural foundations of these behavioral preferences can feed back into our theories of social preferences and reference-dependent utility.

The first chapter focuses on classical conditioning and its application in identifying the psychological underpinnings of a pricing phenomenon. We return to classical conditioning again in the third chapter where we use fMRI to identify varied sources of utility—here, reference dependent versus direct utility—and cross-validate our interpretation with a conditioning experiment. The second chapter engages social preferences and, more broadly, causative utility (wherein the decision-maker derives utility from making or avoiding particular choices).

Conditioned responses in the reticular formation

Unit activity was recorded from the midbrain and pons of 40 freely moving rats in an appetitive classical conditioning situation. Responses to auditory stimuli were observed from 100 units before and during a conditioning procedure in which presentation of food occurred 1 sec after the onset of the auditory stimulus. Conditioned unit responses (i.e., spike rate accelerations or decelerations) were considered to be positive when 1) no similar responses appeared prior to conditioning, and 2) latencies were equal to or less than those of sensory responses derived from the inferior colliculus. Such short latency conditioned unit responses were recorded from 11 probes located in the mid-lateral pert of the ventral region of the brain stem. This region was differentiated from paramedian, far lateral and dorsal parts of the brain stem reticular formation. Conditioned unit responses of considerably longer latencies were recorded from 76 probe located in these other regions. Among the longer latency responses interesting differences appeared in experiments conducted after the first conditioning series was completed. With additional training, units in the "reticular activating system" of midbrain and pons tended to yield stabilized responses in the early portion of the CS-US interval closely related in time to the orientation responses evoked by the CS. In contrast, the responses of units in the limbic midbrain tended to stabilize in the later part of the CS-US interval closely related in time to preparatory responses tied to the US. During extinction when the auditory stimulus was no longer followed by presentation of food, many of the responses were reduced to their pre-conditioning levels. However, there was a tendency for units which had displayed short latency responses on the first conditioning day to be more resistant to extinction than units which had displayed longer latency conditioned responses. The data were interpreted as indicating a local correlate of learning in the reticular formation of midbrain end pons and a separation of the midbrain system into at least two areas: 1) the classical "reticular activating system" related to orienting reactions, and 2) the limbic midbrain areas related to drives and rewards. Because the ventral and mid-lateral area with very short latency conditioned responses was not clearly tied to either of these; it was considered as possibly representing a third division.

Predictive neural coding of reward preference involves dissociable responses in human ventral midbrain and ventral striatum.

Food preferences are acquired through experience and can exert strong influence on choice behavior. In order to choose which food to consume, it is necessary to maintain a predictive representation of the subjective value of the associated food stimulus. Here, we explore the neural mechanisms by which such predictive representations are learned through classical conditioning. Human subjects were scanned using fMRI while learning associations between arbitrary visual stimuli and subsequent delivery of one of five different food flavors. Using a temporal difference algorithm to model learning, we found predictive responses in the ventral midbrain and a part of ventral striatum (ventral putamen) that were related directly to subjects' actual behavioral preferences. These brain structures demonstrated divergent response profiles, with the ventral midbrain showing a linear response profile with preference, and the ventral striatum a bivalent response. These results provide insight into the neural mechanisms underlying human preference behavior.

Fetal memory: Does it exist? what does it do?

Whether fetal memory exists has attracted interest for many thousands of years. The following review draws on recent experimental evidence to consider two questions: does the fetus have a memory? And, if so, what function(s) does it serve? Evidence from fetal learning paradigms of classical conditioning, habituation and exposure learning reveal that the fetus does have a memory. By comparison little attention has been paid to the possible function of memory. Possible functions discussed are: practice, recognition of and attachment to the mother, promotion of breastfeeding, and language acquisition. It is concluded that the fetus does possess a memory but that more attention to the functions of fetal memory will guide future studies of fetal memory abilities.

Modèle informatique du coapprentissage des ganglions de la base et du cortex : l'apprentissage par renforcement et le développement de représentations

Tout au long de la vie, le cerveau développe des représentations de son environnement permettant à l’individu d’en tirer meilleur profit. Comment ces représentations se développent-elles pendant la quête de récompenses demeure un mystère. Il est raisonnable de penser que le cortex est le siège de ces représentations et que les ganglions de la base jouent un rôle important dans la maximisation des récompenses. En particulier, les neurones dopaminergiques semblent coder un signal d’erreur de prédiction de récompense. Cette thèse étudie le problème en construisant, à l’aide de l’apprentissage machine, un modèle informatique intégrant de nombreuses évidences neurologiques. Après une introduction au cadre mathématique et à quelques algorithmes de l’apprentissage machine, un survol de l’apprentissage en psychologie et en neuroscience et une revue des modèles de l’apprentissage dans les ganglions de la base, la thèse comporte trois articles. Le premier montre qu’il est possible d’apprendre à maximiser ses récompenses tout en développant de meilleures représentations des entrées. Le second article porte sur l'important problème toujours non résolu de la représentation du temps. Il démontre qu’une représentation du temps peut être acquise automatiquement dans un réseau de neurones artificiels faisant office de mémoire de travail. La représentation développée par le modèle ressemble beaucoup à l’activité de neurones corticaux dans des tâches similaires. De plus, le modèle montre que l’utilisation du signal d’erreur de récompense peut accélérer la construction de ces représentations temporelles. Finalement, il montre qu’une telle représentation acquise automatiquement dans le cortex peut fournir l’information nécessaire aux ganglions de la base pour expliquer le signal dopaminergique. Enfin, le troisième article évalue le pouvoir explicatif et prédictif du modèle sur différentes situations comme la présence ou l’absence d’un stimulus (conditionnement classique ou de trace) pendant l’attente de la récompense. En plus de faire des prédictions très intéressantes en lien avec la littérature sur les intervalles de temps, l’article révèle certaines lacunes du modèle qui devront être améliorées. Bref, cette thèse étend les modèles actuels de l’apprentissage des ganglions de la base et du système dopaminergique au développement concurrent de représentations temporelles dans le cortex et aux interactions de ces deux structures.

Le conditionnement classique de la réponse sexuelle humaine masculine

Thèse numérisée par la Division de la gestion de documents et des archives de l'Université de Montréal