Molecular mechanisms underlying a cellular analog of operant reward learning.

Operant conditioning is a ubiquitous but mechanistically poorly understood form of associative learning in which an animal learns the consequences of its behavior. Using a single-cell analog of operant conditioning in neuron B51 of Aplysia, we examined second-messenger pathways engaged by activity and reward and how they may provide a biochemical association underlying operant learning. Conditioning was blocked by Rp-cAMP, a peptide inhibitor of PKA, a PKC inhibitor, and by expressing a dominant-negative isoform of Ca2+-dependent PKC (apl-I). Thus, both PKA and PKC were necessary for operant conditioning. Injection of cAMP into B51 mimicked the effects of operant conditioning. Activation of PKC also mimicked conditioning but was dependent on both cAMP and PKA, suggesting that PKC acted at some point upstream of PKA activation. Our results demonstrate how these molecules can interact to mediate operant conditioning in an individual neuron important for the expression of the conditioned behavior.

Learning in the cerebellar nucleus is necessary for acquisition of Pavlovian eyelid conditioning

The cumulative work presented here supports the hypothesis that plasticity in the cerebellar cortex and cerebellar nuclei mediates a simple associative form of motor teaming-Pavlovian eyelid conditioning. It was previously demonstrated that focal ablative lesions of cerebellar anterior lobe or pharmacological block of the cerebellar cortex output disrupted the timing of the conditioned eyeblink response, unmasking a response with a relatively fixed and very short latency to onset. The results of this thesis demonstrate that the short-latency responses are due to associative learning. Unpaired training does not support the acquisition of short-latency responses while the rate of acquisition of short-latency responses during paired training is approximately the same as that of timed conditioned responses. The acquisition of short-latency responses is dependent on an intact cerebellar cortex. Both ablative lesions of the cerebellar cortex and inactivation of cerebellar cortex output with picrotoxin block the acquisition of short-latency responses. However, once the short-latency responses are acquired neither disconnection of cerebellar cortex nor inactivation of the cerebellar nucleus block reacquisition. The results are consistent with the proposal that plasticity in the cerebellar cortex is necessary for learning the timing of conditioned responses, plasticity in the interpositus nucleus mediates the short latency responses, and cerebellar cortical output and mossy fiber input are necessary for the acquisition of short latency responses. ^

Low-status monkeys “play dumb” when learning in mixed social groups

Many primates, including humans, live in complex hierarchical societies where social context and status affect daily life. Nevertheless, primate learning studies typically test single animals in limited laboratory settings where the important effects of social interactions and relationships cannot be studied. To investigate the impact of sociality on associative learning, we compared the individual performances of group-tested rhesus monkeys (Macaca mulatta) across various social contexts. We used a traditional discrimination paradigm that measures an animal’s ability to form associations between cues and the obtaining of food in choice situations; but we adapted the task for group testing. After training a 55-member colony to separate on command into two subgroups, composed of either high- or low-status families, we exposed animals to two color discrimination problems, one with all monkeys present (combined condition), the other in their “dominant” and “subordinate” cohorts (split condition). Next, we manipulated learning history by testing animals on the same problems, but with the social contexts reversed. Monkeys from dominant families excelled in all conditions, but subordinates performed well in the split condition only, regardless of learning history. Subordinate animals had learned the associations, but expressed their knowledge only when segregated from higher-ranking animals. Because aggressive behavior was rare, performance deficits probably reflected voluntary inhibition. This experimental evidence of rank-related, social modulation of performance calls for greater consideration of social factors when assessing learning and may also have relevance for the evaluation of human scholastic achievement.

Galantamine: Effect on nicotinic receptor binding, acetylcholinesterase inhibition, and learning

Classical eyeblink conditioning is a well-characterized model paradigm that engages the septohippocampal cholinergic system. This form of associative learning is impaired in normal aging and severely disrupted in Alzheimer's disease (AD). Some nicotinic cholinergic receptor subtypes are lost in AD, making the use of nicotinic allosterically potentiating ligands a promising therapeutic strategy. The allosterically potentiating ligand galantamine (Gal) modulates nicotinic cholinergic receptors to increase acetylcholine release as well as acting as an acetylcholinesterase (AChE) inhibitor. Gal was tested in two preclinical experiments. In Experiment 1 with 16 young and 16 older rabbits, Gal (3.0 mg/kg) was administered for 15 days during conditioning, and the drug significantly improved learning, reduced AChE levels, and increased nicotinic receptor binding. In Experiment 2, 53 retired breeder rabbits were tested over a 15-wk period in four conditions. Groups of rabbits received 0.0 (vehicle), 1.0, or 3.0 mg/kg Gal for the entire 15-wk period or 3.0 mg/kg Gal for 15 days and vehicle for the remainder of the experiment. Fifteen daily conditioning sessions and subsequent retention and relearning assessments were spaced at 1-month intervals. The dose of 3.0 mg/kg Gal ameliorated learning deficits significantly during acquisition and retention in the group receiving 3.0 mg/kg Gal continuously. Nicotinic receptor binding was significantly increased in rabbits treated for 15 days with 3.0 mg/kg Gal, and all Gal-treated rabbits had lower levels of brain AChE. The efficacy of Gal in a learning paradigm severely impaired in AD is consistent with outcomes in clinical studies.

Mirror-image relations in category learning

The discrimination of patterns that are mirror-symmetric counterparts of each other is difficult and requires substantial training. We explored whether mirror-image discrimination during expertise acquisition is based on associative learning strategies or involves a representational shift towards configural pattern descriptions that permit resolution of symmetry relations. Subjects were trained to discriminate between sets of unfamiliar grey-level patterns in two conditions, which either required the separation of mirror images or not. Both groups were subsequently tested in a 4-class category-learning task employing the same set of stimuli. The results show that subjects who had successfully learned to discriminate between mirror-symmetric counterparts were distinctly faster in the categorization task, indicating a transfer of conceptual knowledge between the two tasks. Additional computer simulations suggest that the development of such symmetry concepts involves the construction of configural, protoholistic descriptions, in which positions of pattern parts are encoded relative to a spatial frame of reference.

Noradrenergic modulatoins of odor learning and odor representation in the rat

How experience alters neuronal ensemble dynamics and how locus coeruleus-mediated norepinephrine release facilitates memory formation in the brain are the topics of this thesis. Here we employed a visualization technique, cellular compartment analysis of temporal activity by fluorescence in situ hybridization (catFISH), to assess activation patterns of neuronal ensembles in the olfactory bulb (OB) and anterior piriform cortex (aPC) to repeated odor inputs. Two associative learning models were used, early odor preference learning in rat pups and adult rat go-no-go odor discrimination learning. With catFISH of an immediate early gene, Arc, we showed that odor representation in the OB and aPC was sparse (~5-10%) and widely distributed. Odor associative learning enhanced the stability of the rewarded odor representation in the OB and aPC. The stable component, indexed by the overlap between the two ensembles activated by the rewarded odor at two time points, increased from ~25% to ~50% (p = 0.004-1.43E⁻4; Chapter 3 and 4). Adult odor discrimination learning promoted pattern separation between rewarded and unrewarded odor representations in the aPC. The overlap between rewarded and unrewarded odor representations reduced from ~25% to ~14% (p = 2.28E⁻⁵). However, learning an odor mixture as a rewarded odor increased the overlap of the component odor representations in the aPC from ~23% to ~44% (p = 0.010; Chapter 4). Blocking both α- and β-adrenoreceptors in the aPC prevented highly similar odor discrimination learning in adult rats, and reduced OB mitral and granule ensemble stability to the rewarded odor. Similar treatment in the OB only slowed odor discrimination learning. However, OB adrenoceptor blockade disrupted pattern separation and ensemble stability in the aPC when the rats demonstrated deficiency in discrimination (Chapter 5). In another project, the role of α₂-adrenoreceptors in the OB during early odor preference learning was studied. OB α2-adrenoceptor activation was necessary for odor learning in rat pups. α₂-adrenoceptor activation was additive with β-adrenoceptor mediated signalling to promote learning (Chapter 2). Together, these experiments suggest that odor representations are highly adaptive at the early stages of odor processing. The OB and aPC work in concert to support odor learning and top-down adrenergic input exerts a powerful modulation on both learning and odor representation.

Rodent Models of Conditioned Fear: Behavioral Measures of Fear and Memory

Pavlovian fear conditioning is a robust technique for examining behavioral and cellular components of fear learning and memory. In fear conditioning, the subject learns to associate a previously neutral stimulus with an inherently noxious co-stimulus. The learned association is reflected in the subjects' behavior upon subsequent re-exposure to the previously neutral stimulus or the training environment. Using fear conditioning, investigators can obtain a large amount of data that describe multiple aspects of learning and memory. In a single test, researchers can evaluate functional integrity in fear circuitry, which is both well characterized and highly conserved across species. Additionally, the availability of sensitive and reliable automated scoring software makes fear conditioning amenable to high-throughput experimentation in the rodent model; thus, this model of learning and memory is particularly useful for pharmacological and toxicological screening. Due to the conserved nature of fear circuitry across species, data from Pavlovian fear conditioning are highly translatable to human models. We describe equipment and techniques needed to perform and analyze conditioned fear data. We provide two examples of fear conditioning experiments, one in rats and one in mice, and the types of data that can be collected in a single experiment. © 2012 Springer Science+Business Media, LLC.

A recurrent network in the lateral amygdala : a mechanism for coincidence detection

Synaptic changes at sensory inputs to the dorsal nucleus of the lateral amygdala (LAd) play a key role in the acquisition and storage of associative fear memory. However, neither the temporal nor spatial architecture of the LAd network response to sensory signals is understood. We developed a method for the elucidation of network behavior. Using this approach, temporally patterned polysynaptic recurrent network responses were found in LAd (intra-LA), both in vitro and in vivo, in response to activation of thalamic sensory afferents. Potentiation of thalamic afferents resulted in a depression of intra-LA synaptic activity, indicating a homeostatic response to changes in synaptic strength within the LAd network. Additionally, the latencies of thalamic afferent triggered recurrent network activity within the LAd overlap with known later occurring cortical afferent latencies. Thus, this recurrent network may facilitate temporal coincidence of sensory afferents within LAd during associative learning.

Cravings and Expectancies

Drug-dependent individuals commonly report craving, which is an urge to consume a substance to experience pleasure or provide relief. However, the motivation to use drugs is also influenced by the expectancies we hold about drug ingestion. Drug expectancies are among a range of internal and external cues that can elicit craving. Scientific advances in the neurobiology of reward have illuminated individual differences in the neurotransmitter systems involved in associative learning that underpin drug use motivation and expectancies. We have begun to elaborate the interrelationships of these constructs and mechanisms to inform new treatments that incorporate recent models of reward learning.

An organization of visual and auditory fear conditioning in the lateral amygdala

Pavlovian fear conditioning is an evolutionary conserved and extensively studied form of associative learning and memory. In mammals, the lateral amygdala (LA) is an essential locus for Pavlovian fear learning and memory. Despite significant progress unraveling the cellular mechanisms responsible for fear conditioning, very little is known about the anatomical organization of neurons encoding fear conditioning in the LA. One key question is how fear conditioning to different sensory stimuli is organized in LA neuronal ensembles. Here we show that Pavlovian fear conditioning, formed through either the auditory or visual sensory modality, activates a similar density of LA neurons expressing a learning-induced phosphorylated extracellular signal-regulated kinase (p-ERK1/2). While the size of the neuron population specific to either memory was similar, the anatomical distribution differed. Several discrete sites in the LA contained a small but significant number of p-ERK1/2-expressing neurons specific to either sensory modality. The sites were anatomically localized to different levels of the longitudinal plane and were independent of both memory strength and the relative size of the activated neuronal population, suggesting some portion of the memory trace for auditory and visually cued fear conditioning is allocated differently in the LA. Presenting the visual stimulus by itself did not activate the same p-ERK1/2 neuron density or pattern, confirming the novelty of light alone cannot account for the specific pattern of activated neurons after visual fear conditioning. Together, these findings reveal an anatomical distribution of visual and auditory fear conditioning at the level of neuronal ensembles in the LA.

GABAA and NMDA receptors contribute to synaptic integration in lateral amygdala neurons

In classical fear conditioning a neutral conditioned stimulus (CS), is paired with an aversive unconditioned stimulus (US). The CS thereby acquires the capacity to elicit a fear response. This type of associative learning is thought to require co-activation of principal neurons in the lateral nucleus of the amygdala (LA) by two sets of synaptic inputs...

GABAa receptors contribute to synaptic integration in lateral amygdala neurons

In classical fear conditioning a neutral conditioned stimulus (CS) such as a tone, is paired with an aversive unconditioned stimulus (US) such as a shock. The CS thereby acquires the capacity to elicit a fear response. This type of associative learning is thought to require co-activation of principle neurons in the lateral nucleus of the amygdala (LA) by two sets of synaptic inputs, a weak CS and a strong US...

Sex and physiological state influence the rate of resource acquisition and monopolisation in urban free-ranging dogs, Canis familiaris

In animal populations, the constraints of energy and time can cause intraspecific variation in foraging behaviour. The proximate developmental mediators of such variation are often the mechanisms underlying perception and associative learning. Here, experience-dependent changes in foraging behaviour and their consequences were investigated in an urban population of free-ranging dogs, Canis familiaris by continually challenging them with the task of food extraction from specially crafted packets. Typically, males and pregnant/lactating (PL) females extracted food using the sophisticated `gap widening' technique, whereas non-pregnant/non-lactating (NPNL) females, the relatively underdeveloped `rip opening' technique. In contrast to most males and PL females (and a few NPNL females) that repeatedly used the gap widening technique and improved their performance in food extraction with experience, most NPNL females (and a few males and PL females) non-preferentially used the two extraction techniques and did not improve over successive trials. Furthermore, the ability of dogs to sophisticatedly extract food was positively related to their ability to improve their performance with experience. Collectively, these findings demonstrate that factors such as sex and physiological state can cause differences among individuals in the likelihood of learning new information and hence, in the rate of resource acquisition and monopolization.

Neural and computational representations of decision variables

These studies explore how, where, and when representations of variables critical to decision-making are represented in the brain. In order to produce a decision, humans must first determine the relevant stimuli, actions, and possible outcomes before applying an algorithm that will select an action from those available. When choosing amongst alternative stimuli, the framework of value-based decision-making proposes that values are assigned to the stimuli and that these values are then compared in an abstract “value space” in order to produce a decision. Despite much progress, in particular regarding the pinpointing of ventromedial prefrontal cortex (vmPFC) as a region that encodes the value, many basic questions remain. In Chapter 2, I show that distributed BOLD signaling in vmPFC represents the value of stimuli under consideration in a manner that is independent of the type of stimulus it is. Thus the open question of whether value is represented in abstraction, a key tenet of value-based decision-making, is confirmed. However, I also show that stimulus-dependent value representations are also present in the brain during decision-making and suggest a potential neural pathway for stimulus-to-value transformations that integrates these two results.

More broadly speaking, there is both neural and behavioral evidence that two distinct control systems are at work during action selection. These two systems compose the “goal-directed system”, which selects actions based on an internal model of the environment, and the “habitual” system, which generates responses based on antecedent stimuli only. Computational characterizations of these two systems imply that they have different informational requirements in terms of input stimuli, actions, and possible outcomes. Associative learning theory predicts that the habitual system should utilize stimulus and action information only, while goal-directed behavior requires that outcomes as well as stimuli and actions be processed. In Chapter 3, I test whether areas of the brain hypothesized to be involved in habitual versus goal-directed control represent the corresponding theorized variables.

The question of whether one or both of these neural systems drives Pavlovian conditioning is less well-studied. Chapter 4 describes an experiment in which subjects were scanned while engaged in a Pavlovian task with a simple non-trivial structure. After comparing a variety of model-based and model-free learning algorithms (thought to underpin goal-directed and habitual decision-making, respectively), it was found that subjects’ reaction times were better explained by a model-based system. In addition, neural signaling of precision, a variable based on a representation of a world model, was found in the amygdala. These data indicate that the influence of model-based representations of the environment can extend even to the most basic learning processes.

Knowledge of the state of hidden variables in an environment is required for optimal inference regarding the abstract decision structure of a given environment and therefore can be crucial to decision-making in a wide range of situations. Inferring the state of an abstract variable requires the generation and manipulation of an internal representation of beliefs over the values of the hidden variable. In Chapter 5, I describe behavioral and neural results regarding the learning strategies employed by human subjects in a hierarchical state-estimation task. In particular, a comprehensive model fit and comparison process pointed to the use of "belief thresholding". This implies that subjects tended to eliminate low-probability hypotheses regarding the state of the environment from their internal model and ceased to update the corresponding variables. Thus, in concert with incremental Bayesian learning, humans explicitly manipulate their internal model of the generative process during hierarchical inference consistent with a serial hypothesis testing strategy.

Irregular morphine administration affects the retention but not acquisition of conditioned place preference in rats

Drug addiction is increasingly viewed as the expression of abnormal associative learning following repeated exposures to the drugs of abuse Previous I studies have demonstrated that the patterns of repetition such as frequency and spacing are important to many kinds of learning and memory retention We hypothesized that drug repetition pattern might affect the reward-related learning although the total doses of the drug were the same. In the present study, we tested morphine-induced place preference following either regular or irregular pattern of morphine pairing in rats Regular morphine group received morphine administration daily at a regular time with the same dose Irregular morphine groups received morphine administration either at the same time but irregular doses, irregular time but same dose, or irregular time and irregular doses. We found that rats, who received irregular morphine pairing, exhibited similar acquisition of peace preference but different preference retentions compared with regular morphine-treated rats after the same total dose of morphine Rats, who received morphine administration at the same time but irregular doses and at irregular time and irregular doses, showed rapid disruption of place preference than the regular morphine group. Rats, who received morphine at irregular time but the same dose, showed similar retention of place preference to regular morphine group Our results suggest that the pattern of drug pairing plays an important role in the retention of reward-related memory This study may provide new evidence to broaden our understanding of the development and maintenance of drug craving (C) 2009 Elsevier B V. All rights reserved