Molecular mechanisms underlying a cellular analog of operant reward learning.

Operant conditioning is a ubiquitous but mechanistically poorly understood form of associative learning in which an animal learns the consequences of its behavior. Using a single-cell analog of operant conditioning in neuron B51 of Aplysia, we examined second-messenger pathways engaged by activity and reward and how they may provide a biochemical association underlying operant learning. Conditioning was blocked by Rp-cAMP, a peptide inhibitor of PKA, a PKC inhibitor, and by expressing a dominant-negative isoform of Ca2+-dependent PKC (apl-I). Thus, both PKA and PKC were necessary for operant conditioning. Injection of cAMP into B51 mimicked the effects of operant conditioning. Activation of PKC also mimicked conditioning but was dependent on both cAMP and PKA, suggesting that PKC acted at some point upstream of PKA activation. Our results demonstrate how these molecules can interact to mediate operant conditioning in an individual neuron important for the expression of the conditioned behavior.

Cellular mechanisms of operant and classical conditioning

The ability to associate a predictive stimulus with a subsequent salient event (i.e., classical conditioning) and the ability to associate an expressed behavior with the consequences (i.e., operant conditioning) allow for a predictive understanding of a changing environment. Although they are operationally distinct, there has been considerable debate whether at some fundamental level classical and operant conditioning are mechanistically distinct or similar. Feeding behavior of Aplysia (i.e., biting) was chosen as the model system and was successfully conditioned with appetitive forms of both operant and classical conditioning. The neuronal circuitry responsible for feeding is well understood and is suitable for cellular analyses, thus providing for a mechanistic comparison between these two forms of associative learning. ^ Neuron B51 is part of the feeding circuitry of Aplysia and is critical for the expression of ingestive behaviors. B51 also is a locus of plasticity following both operant and classical conditioning. Both in vivo and in vitro operant conditioning increased the input resistance and the excitability of B51. No pairing-specific changes in the input resistance were observed following both in vivo and in vitro classical conditioning. However, classical conditioning decreased the excitability of B51. Thus, both operant and classical conditioning modified the threshold level for activation of neuron B51, but in opposite directions, revealing key differences in the cellular mechanisms underlying these two forms of associative learning. ^ Next, the cellular mechanisms underlying operant conditioning were investigated in more detail using a single-cell analogue. The single-cell analogue successfully recapitulated the previous in vivo and in vitro operant conditioning results by increasing the input resistance and the excitability of B51. Both PKA and PKC were necessary for operant conditioning. Dopamine appears to be the transmitter mediating the reinforcement signal in this form of conditioning. A D1 dopamine receptor antibody revealed that the D1receptor localizes to the axon hillock, which is also the region that gives the strongest response when iontophoresing dopamine. ^ The studies presented herein, thus, provide for a greater understanding of the mechanisms underlying both of these forms of associative learning and demonstrate that they likely operate through distinct cellular mechanisms. ^

Operant conditioning of H-reflex changes synaptic terminals on primate motoneurons.

Operant conditioning of the primate triceps surae H-reflex, the electrical analog of the spinal stretch reflex, creates a memory trace that includes changes in the spinal cord. To define the morphological correlates of this plasticity, we analyzed the synaptic terminal coverage of triceps surae motoneurons from animals in which the triceps surae H-reflex in one leg had been increased (HRup mode) or decreased (HRdown mode) by conditioning and compared them to each other and to motoneurons from unconditioned animals. Motoneurons were labeled by intramuscular injection of cholera toxin-horseradish peroxidase. A total of 5055 terminals on the cell bodies and proximal dendrites of 114 motoneurons from 14 animals were studied by electron microscopy. Significant differences were found between HRup and HRdown animals and between HRup and naive (i.e., unconditioned) animals. F terminals (i.e., putative inhibitory terminals) were smaller and their active zone coverage on the cell body was lower on motoneurons from the conditioned side of HRup animals than on motoneurons from the conditioned side of HRdown animals. C terminals (i.e., terminals associated with postsynaptic cisterns and rough endoplasmic reticulum) were smaller and the number of C terminals in each C complex (i.e., a group of contiguous C terminals) was larger on motoneurons from the conditioned side of HRup animals than on motoneurons either from the conditioned side of HRdown animals or from naive animals. Because the treatment of HRup and HRdown animals differed only in the reward contingency, the results imply that the two contingencies had different effects on motoneuron synaptic terminals. In combination with other recent data, they show that H-reflex conditioning produces a complex pattern of spinal cord plasticity that includes changes in motoneuron physiological properties as well as in synaptic terminals. Further delineation of this pattern should reveal the contribution of the structural changes described here to the learned change in behavior.

Types and amplitudes of touch as reinforcing stimuli in operant conditioning of infant behavior

The reinforcing effects of diverse tactile stimuli were examined in this study. The study had two purposes. First, this study expanded on the Pelaez-Nogueras, Field, Gewirtz, Cigales, Gonzalez, Sanchez and Clasky (1997) finding that stroking increases infants' gaze duration, and smiling and vocalization frequencies more than tickling/poking. Instead of presenting poking and tickling as a single stimulus combination, this study separated poking and tickling in order to measure the effects of each component separately. Further, the effects of poking, tickling/tapping and stroking intensity (i.e., tactile pressure) were compared by having both mild and intense conditions. Second, this study compared the reinforcing efficacy of mother-delivered tactile stimulation to that of infant-originated tactile exploration. Twelve infants from 2- to 5-months of age participated in this study. The experiment was conducted using a repeated measures A-B-A-C-A-D reversal design. The A phases signified baselines and reversals. The B, C, and D phases consisted of alternating treatments (either mild stroking vs. mild poking vs. mild tickling/tapping, intense stroking vs. intense poking vs. intense tickling/tapping, or mother-delivered tactile stimulation vs. infant-originated tactile exploration). Three experimental hypotheses were assessed: (1) infant leg kick rate would be greater when it produced stroking or tickling/tapping (presumptive positive reinforcers), than when it produced poking (a possible punisher), regardless of tactile pressure; (2) infant leg kick rate would be greater when it produced a more intense level of stroking or tickling/tapping and lower when it produced intense poking compared to mild poking; (3) infant leg-kick rate would be greater for mother-delivered tactile stimulation than for infant-originated tactile exploration. Visual inspection and inferential statistical methods were used to analyze the results. The data supported the first two hypotheses. Mixed support emerged for the third hypothesis. This study made several important contributions to the field of psychology. First, this was the first study to quantify the pressure of tactile stimulation, via a pressure meter developed by the researcher. Additionally, the results of this study yielded valuable information about the effects of different modalities of touch. ^

Operant and respondent procedures to establish social stimuli as reinforcers in children with autism

According to the DSM-IV- TR (American Psychiatric Association, 2000), one of the core deficits in autism is in the impairment of social interaction. Some have suggested that underlying these deficits is the reality that individuals with autism do not find social stimuli to be as reinforcing as other types of stimuli (Dawson, 2008). An interesting and growing body of literature supports the notion that symptoms in autism may be caused by a general reduction in social motivation (Chevallier et al., 2012). A review of the literature suggests that social orienting and social motivation are low in individuals with autism, and including social motivation as a target for therapeutic intervention should be pursued (Helt et al., 2008). Through our understanding of learning processes, researchers in behavior analysis and related fields have been able to use conditioning procedures to change the function of neutral or ineffective stimuli, including tokens (Ayllon & Azrin, 1968), facial expressions (Gewirtz & Pelaez-Nogueras, 1992) and praise (Dozier et al., 2012). The current study aimed to use operant and respondent procedures to condition social stimuli that were empirically shown to not be reinforcing prior to conditioning. Further, this study aimed to compare the two procedures in their effectiveness to condition social stimuli to function as reinforcers, and in their maintenance of effects over time. Using a multiple-baseline, multi-element design, one social stimulus was conditioned under each procedure to compare the different response rates following conditioning. Finally, the study sought to determine if conditioning social stimuli to function as reinforcers had any effect on the social functioning of young children with autism. Six children diagnosed with autism between the ages of 18 months and 3 years participated. Results show that the respondent procedure (pairing) resulted in more robust and enduring effects than the operant procedure (Sd procedure). Results of a social communication assessment (ESCS, Mundy et al., 2003) before and after conditioning demonstrate gains in all areas of social communication, particularly in the areas of initiating and responding to joint attention.^

Evoking non-repertory verbal behavior across operant classes : the effects of motor echoic sign language training within the context of a motivating operation

The individual effects that echoic, mand, and sign language training procedures have on the acquisition of verbal behavior have been widely demonstrated, but more efficient strategies are still needed. This study combined all three treatment strategies into one treatment intervention in order to investigate the joint effects they may have on verbal behavior. Six participants took part in the study. Intervention totaled 1 hour/day for 5 days/week until mastery criterion for motor echoic behavior was achieved. Although motor echoic behavior were solely targeted for acquisition, significant increases in spontaneous motor mands were noted in all treatment participants. Additionally, 4 treatment participants also demonstrated significant gains in vocal echoics and spontaneous vocal mands. No significant increases were noted for the control participant. Results suggest that the aforementioned procedure may provide more efficient results as a first-step to teaching a functional repertoire of verbal behavior to developmentally delayed children.

Operant and Respondent Procedures to Establish Social Stimuli as Reinforcers in Children with Autism

According to the DSM-IV- TR (American Psychiatric Association, 2000), one of the core deficits in autism is in the impairment of social interaction. Some have suggested that underlying these deficits is the reality that individuals with autism do not find social stimuli to be as reinforcing as other types of stimuli (Dawson, 2008). An interesting and growing body of literature supports the notion that symptoms in autism may be caused by a general reduction in social motivation (Chevallier et al., 2012). A review of the literature suggests that social orienting and social motivation are low in individuals with autism, and including social motivation as a target for therapeutic intervention should be pursued (Helt et al., 2008). Through our understanding of learning processes, researchers in behavior analysis and related fields have been able to use conditioning procedures to change the function of neutral or ineffective stimuli, including tokens (Ayllon & Azrin, 1968), facial expressions (Gewirtz & Pelaez-Nogueras, 1992) and praise (Dozier et al., 2012). The current study aimed to use operant and respondent procedures to condition social stimuli that were empirically shown to not be reinforcing prior to conditioning. Further, this study aimed to compare the two procedures in their effectiveness to condition social stimuli to function as reinforcers, and in their maintenance of effects over time. Using a multiple-baseline, multi-element design, one social stimulus was conditioned under each procedure to compare the different response rates following conditioning. Finally, the study sought to determine if conditioning social stimuli to function as reinforcers had any effect on the social functioning of young children with autism. Six children diagnosed with autism between the ages of 18 months and 3 years participated. Results show that the respondent procedure (pairing) resulted in more robust and enduring effects than the operant procedure (Sd procedure). Results of a social communication assessment (ESCS, Mundy et al., 2003) before and after conditioning demonstrate gains in all areas of social communication, particularly in the areas of initiating and responding to joint attention.

The neurokinin 1 receptor antagonist, ezlopitant, reduces appetitive responding for sucrose and ethanol

Abstract Background: The current obesity epidemic is thought to be partly driven by over-consumption of sugar-sweetened diets and soft drinks. Loss-of-control over eating and addiction to drugs of abuse share overlapping brain mechanisms including changes in motivational drive, such that stimuli that are often no longer ‘liked’ are still intensely ‘wanted’ [7,8]. The neurokinin 1 (NK1) receptor system has been implicated in both learned appetitive behaviors and addiction to alcohol and opioids; however, its role in natural reward seeking remains unknown. Methodology/Principal Findings: We sought to determine whether the NK1-receptor system plays a role in the reinforcing properties of sucrose using a novel selective and clinically safe NK1-receptor antagonist, ezlopitant (CJ-11,974), in three animal models of sucrose consumption and seeking. Furthermore, we compared the effect of ezlopitant on ethanol consumption and seeking in rodents. The NK1-receptor antagonist, ezlopitant decreased appetitive responding for sucrose more potently than for ethanol using an operant self-administration protocol without affecting general locomotor activity. To further evaluate the selectivity of the NK1-receptor antagonist in decreasing consumption of sweetened solutions, we compared the effects of ezlopitant on water, saccharin-, and sodium chloride (NaCl) solution consumption. Ezlopitant decreased intake of saccharin but had no effect on water or salty solution consumption. Conclusions/Significance: The present study indicates that the NK1-receptor may be a part of a common pathway regulating the self-administration, motivational and reinforcing aspects of sweetened solutions, regardless of caloric value, and those of substances of abuse. Additionally, these results indicate that the NK1-receptor system may serve as a therapeutic target for obesity induced by over-consumption of natural reinforcers.

Partial agonists of the [alpha]3[beta]4[ast] neuronal nicotinic acetylcholine receptor reduce ethanol consumption and seeking in rats

Alcohol use disorders (AUDs) impact millions of individuals and there remain few effective treatment strategies. Despite evidence that neuronal nicotinic acetylcholine receptors (nAChRs) have a role in AUDs, it has not been established which subtypes of the nAChR are involved. Recent human genetic association studies have implicated the gene cluster CHRNA3-CHRNA5-CHRNB4 encoding the α3, α5, and β4 subunits of the nAChR in susceptibility to develop nicotine and alcohol dependence; however, their role in ethanol-mediated behaviors is unknown due to the lack of suitable and selective research tools. To determine the role of the α3, and β4 subunits of the nAChR in ethanol self-administration, we developed and characterized high-affinity partial agonists at α3β4 nAChRs, CP-601932, and PF-4575180. Both CP-601932 and PF-4575180 selectively decrease ethanol but not sucrose consumption and operant self-administration following long-term exposure. We show that the functional potencies of CP-601932 and PF-4575180 at α3β4 nAChRs correlate with their unbound rat brain concentrations, suggesting that the effects on ethanol self-administration are mediated via interaction with α3β4 nAChRs. Also varenicline, an approved smoking cessation aid previously shown to decrease ethanol consumption and seeking in rats and mice, reduces ethanol intake at unbound brain concentrations that allow functional interactions with α3β4 nAChRs. Furthermore, the selective α4β2(*) nAChR antagonist, DHβE, did not reduce ethanol intake. Together, these data provide further support for the human genetic association studies, implicating CHRNA3 and CHRNB4 genes in ethanol-mediated behaviors. CP-601932 has been shown to be safe in humans and may represent a potential novel treatment for AUDs.

Varenicline, an alpha-4 beta-2 nicotinic acetylcholine receptor partial agonist, selectively decreases ethanol consumption and seeking

Abstract Alcohol dependence is a disease that impacts millions of individuals worldwide. There has been some progress with pharmacotherapy for alcohol-dependent individuals; however, there remains a critical need for the development of novel and additional therapeutic approaches. Alcohol and nicotine are commonly abused together, and there is evidence that neuronal nicotinic acetylcholine receptors (nAChRs) play a role in both alcohol and nicotine dependence. Varenicline, a partial agonist at the alpha4beta2 nAChRs, reduces nicotine intake and was recently approved as a smoking cessation aid. We have investigated the role of varenicline in the modulation of ethanol consumption and seeking using three different animal models of drinking. We show that acute administration of varenicline, in doses reported to reduce nicotine reward, selectively reduced ethanol but not sucrose seeking using an operant self-administration drinking paradigm and also decreased voluntary ethanol but not water consumption in animals chronically exposed to ethanol for 2 months before varenicline treatment. Furthermore, chronic varenicline administration decreased ethanol consumption, which did not result in a rebound increase in ethanol intake when the varenicline was no longer administered. The data suggest that the alpha4beta2 nAChRs may play a role in ethanol-seeking behaviors in animals chronically exposed to ethanol. The selectivity of varenicline in decreasing ethanol consumption combined with its reported safety profile and mild side effects in humans suggest that varenicline may prove to be a treatment for alcohol dependence.

Inhibition of orexin-1/hypocretin-1 receptors inhibits yohimbine-induced reinstatement of ethanol and sucrose seeking in Long-Evans rats

Abstract RATIONALE: Previous studies have shown that orexin-1/hypocretin-1 receptors play a role in self-administration and cue-induced reinstatement of food, drug, and ethanol seeking. In the current study, we examined the role of orexin-1/hypocretin-1 receptors in operant self-administration of ethanol and sucrose and in yohimbine-induced reinstatement of ethanol and sucrose seeking. MATERIALS AND METHODS: Rats were trained to self-administer either 10% ethanol or 5% sucrose (30 min/day). The orexin-1 receptor antagonist SB334867 (0, 5, 10, 15, 20 mg/kg, i.p.) was administered 30 min before the operant self-administration sessions. After these experiments, the operant self-administration behaviors were extinguished in both the ethanol and sucrose-trained rats. Upon reaching extinction criteria, SB334867 (0, 5, 10 mg/kg, i.p.) was administered 30 min before yohimbine (0 or 2 mg/kg, i.p.). In a separate experiment, the effect of SB334867 (0, 15, or 20 mg/kg, i.p.) on general locomotor activity was determined using the open-field test. RESULTS: The orexin-1 receptor antagonist, SB334867 (10, 15 and 20 mg/kg) decreased operant self-administration of 10% ethanol but not 5% sucrose self-administration. Furthermore, SB334867 (5 and 10 mg/kg) significantly decreased yohimbine-induced reinstatement of both ethanol and sucrose seeking. SB334867 did not significantly affect locomotor activity measured using the open-field test. CONCLUSIONS: The results suggest that inhibition of OX-1/Hcrt-1 receptors modulates operant ethanol self-administration and also plays a significant role in yohimbine-induced reinstatement of both ethanol and sucrose seeking in rats.

Cabergoline Decreases Alcohol Drinking and Seeking Behaviors Via Glial Cell Line-Derived Neurotrophic Factor

Background: Cabergoline is an ergotamine derivative that increases the expression of glial cell line-derived neurotrophic factor (GDNF) in vitro. We recently showed that GDNF in the ventral tegmental area (VTA) reduces the motivation to consume alcohol. We therefore set out to determine whether cabergoline administration decreases alcohol-drinking and -seeking behaviors via GDNF. Methods: Reverse transcription polymerase chain reaction (RT-PCR) and Enzyme-Linked ImmunoSorbent Assay (ELISA) were used to measure GDNF levels. Western blot analysis was used for phosphorylation experiments. Operant self-administration in rats and a two-bottle choice procedure in mice were used to assess alcohol-drinking behaviors. Instrumental performance tested during extinction was used to measure alcohol-seeking behavior. The [35S]GTPγS binding assay was used to assess the expression and function of the dopamine D2 receptor (D2R). Results: We found that treatment of the dopaminergic-like cell line SH-SY5Y with cabergoline and systemic administration of cabergoline in rats resulted in an increase in GDNF level and in the activation of the GDNF pathway. Cabergoline treatment decreased alcohol-drinking and -seeking behaviors including relapse, and its action to reduce alcohol consumption was localized to the VTA. Finally, the increase in GDNF expression and the decrease in alcohol consumption by cabergoline were abolished in GDNF heterozygous knockout mice. Conclusions: Together, these findings suggest that cabergoline-mediated upregulation of the GDNF pathway attenuates alcohol-drinking behaviors and relapse. Alcohol abuse and addiction are devastating and costly problems worldwide. This study puts forward the possibility that cabergoline might be an effective treatment for these disorders. © 2009 Society of Biological Psychiatry.

The ghrelin signalling system is involved in the consumption of sweets

The gastric-derived orexigenic peptide ghrelin affects brain circuits involved in energy balance as well as in reward. Indeed, ghrelin activates an important reward circuit involved in natural- as well as drug-induced reward, the cholinergic-dopaminergic reward link. It has been hypothesized that there is a common reward mechanism for alcohol and sweet substances in both animals and humans. Alcohol dependent individuals have higher craving for sweets than do healthy controls and the hedonic response to sweet taste may, at least in part, depend on genetic factors. Rat selectively bred for high sucrose intake have higher alcohol consumption than non-sucrose preferring rats and vice versa. In the present study a group of alcohol-consuming individuals selected from a population cohort was investigated for genetic variants of the ghrelin signalling system in relation to both their alcohol and sucrose consumption. Moreover, the effects of GHS-R1A antagonism on voluntary sucrose- intake and operant self-administration, as well as saccharin intake were investigated in preclinical studies using rodents. The effects of peripheral grelin administration on sucrose intake were also examined. Here we found associations with the ghrelin gene haplotypes and increased sucrose consumption, and a trend for the same association was seen in the high alcohol consumers. The preclinical data show that a GHS-R1A antagonist reduces the intake and self-administration of sucrose in rats as well as saccharin intake in mice. Further, ghrelin increases the intake of sucrose in rats. Collectively, our data provide a clear indication that the GHS-R1A antagonists reduces and ghrelin increases the intake of rewarding substances and hence, the central ghrelin signalling system provides a novel target for the development of drug strategies to treat addictive behaviours. © 2011 Landgren et al.

The delta opioid receptor antagonist, SoRI-9409, decreases yohimbine stress-induced reinstatement of ethanol-seeking

A major problem in treating alcohol use disorders (AUDs) is the high rate of relapse due to stress and re-exposure to cues or an environment previously associated with alcohol use. Stressors can induce relapse to alcohol-seeking in humans or reinstatement in rodents. Delta opioid peptide receptors (DOP-Rs) play a role in cue-induced reinstatement of ethanol-seeking; however, their role in stress-induced reinstatement of ethanol-seeking is not known. The objective of this study was to determine the role of DOP-Rs in yohimbine-stress-induced reinstatement of ethanol-seeking. Male, Long-Evans rats were trained to self-administer 10% ethanol in daily 30-minute operant self-administration sessions using a FR3 schedule of reinforcement, followed by extinction training. Once extinction criteria were met, we examined the effects of the DOP-R antagonist, SoRI-9409 (0–5 mg/kg, i.p.) on yohimbine (2 mg/kg, i.p.) stress-induced reinstatement. Additionally, DOP-R-stimulated [35S]GTPS binding was measured in brain membranes and plasma levels of corticosterone (CORT) were determined. Pre-treatment with SoRI-9409 decreased yohimbine stress-induced reinstatement of ethanol-seeking but did not affect yohimbine-induced increases in plasma CORT levels. Additionally, yohimbine increased DOP-R-stimulated 35[S]GTPS binding in brain membranes of ethanol-trained rats, an effect that was inhibited by SoRI-9409. This suggests that the DOP-R plays an important role in yohimbine-stress-induced reinstatement of ethanol-seeking behavior, and DOP-R antagonists may be promising candidates for further development as a treatment for AUDs.