A key role for similarity in vicarious reward.

Humans appear to have an inherent prosocial tendency toward one another in that we often take pleasure in seeing others succeed. This fact is almost certainly exploited by game shows, yet why watching others win elicits a pleasurable vicarious rewarding feeling in the absence of personal economic gain is unclear. One explanation is that game shows use contestants who have similarities to the viewing population, thereby kindling kin-motivated responses (for example, prosocial behavior). Using a game show-inspired paradigm, we show that the interactions between the ventral striatum and anterior cingulate cortex subserve the modulation of vicarious reward by similarity, respectively. Our results support studies showing that similarity acts as a proximate neurobiological mechanism where prosocial behavior extends to unrelated strangers.

Efficacy of a cognitive and behavioural psychotherapy applied by primary care psychologists in patients with mixed anxiety-depressive disorder: a research protocol

Background: In contrast with the recommendations of clinical practice guidelines, the most common treatment for anxiety and depressive disorders in primary care is pharmacological. The aim of this study is to assess the efficacy of a cognitive-behavioural psychological intervention, delivered by primary care psychologists in patients with mixed anxiety-depressive disorder compared to usual care. Methods/Design: This is an open-label, multicentre, randomized, and controlled study with two parallel groups. A random sample of 246 patients will be recruited with mild-to-moderate mixed anxiety-depressive disorder, from the target population on the lists of 41 primary care doctors. Patients will be randomly assigned to the intervention group, who will receive standardised cognitive-behavioural therapy delivered by psychologists together with usual care, or to a control group, who will receive usual care alone. The cognitive-behavioural therapy intervention is composed of eight individual 60-minute face-to face sessions conducted in eight consecutive weeks. A follow-up session will be conducted over the telephone, for reinforcement or referral as appropriate, 6 months after the intervention, as required. The primary outcome variable will be the change in scores on the Short Form-36 General Health Survey. We will also measure the change in the frequency and intensity of anxiety symptoms (State-Trait Anxiety Inventory) and depression (Beck Depression Inventory) at baseline, and 3, 6 and 12 months later. Additionally, we will collect information on the use of drugs and health care services. Discussion: The aim of this study is to assess the efficacy of a primary care-based cognitive-behavioural psychological intervention in patients with mixed anxiety-depressive disorder. The international scientific evidence has demonstrated the need for psychologists in primary care. However, given the differences between health policies and health services, it is important to test the effect of these psychological interventions in our geographical setting.

Model-based influences on humans' choices and striatal prediction errors.

The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors, and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making.

Temporal difference models describe higher-order learning in humans.

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.

Dopaminergic and Non-Dopaminergic Value Systems in Conditioning and Outcome-Specific Revaluation

Animals are motivated to choose environmental options that can best satisfy current needs. To explain such choices, this paper introduces the MOTIVATOR (Matching Objects To Internal Values Triggers Option Revaluations) neural model. MOTIVATOR describes cognitiveemotional interactions between higher-order sensory cortices and an evaluative neuraxis composed of the hypothalamus, amygdala, and orbitofrontal cortex. Given a conditioned stimulus (CS), the model amygdala and lateral hypothalamus interact to calculate the expected current value of the subjective outcome that the CS predicts, constrained by the current state of deprivation or satiation. The amygdala relays the expected value information to orbitofrontal cells that receive inputs from anterior inferotemporal cells, and medial orbitofrontal cells that receive inputs from rhinal cortex. The activations of these orbitofrontal cells code the subjective values of objects. These values guide behavioral choices. The model basal ganglia detect errors in CS-specific predictions of the value and timing of rewards. Excitatory inputs from the pedunculopontine nucleus interact with timed inhibitory inputs from model striosomes in the ventral striatum to regulate dopamine burst and dip responses from cells in the substantia nigra pars compacta and ventral tegmental area. Learning in cortical and striatal regions is strongly modulated by dopamine. The model is used to address tasks that examine food-specific satiety, Pavlovian conditioning, reinforcer devaluation, and simultaneous visual discrimination. Model simulations successfully reproduce discharge dynamics of known cell types, including signals that predict saccadic reaction times and CS-dependent changes in systolic blood pressure.

Cumulative stress in childhood is associated with blunted reward-related brain activity in adulthood.

Early life stress (ELS) is strongly associated with negative outcomes in adulthood, including reduced motivation and increased negative mood. The mechanisms mediating these relations, however, are poorly understood. We examined the relation between exposure to ELS and reward-related brain activity, which is known to predict motivation and mood, at age 26, in a sample followed since kindergarten with annual assessments. Using functional neuroimaging, we assayed individual differences in the activity of the ventral striatum (VS) during the processing of monetary rewards associated with a simple card-guessing task, in a sample of 72 male participants. We examined associations between a cumulative measure of ELS exposure and VS activity in adulthood. We found that greater levels of cumulative stress during childhood and adolescence predicted lower reward-related VS activity in adulthood. Extending this general developmental pattern, we found that exposure to stress early in development (between kindergarten and grade 3) was significantly associated with variability in adult VS activity. Our results provide an important demonstration that cumulative life stress, especially during this childhood period, is associated with blunted reward-related VS activity in adulthood. These differences suggest neurobiological pathways through which a history of ELS may contribute to reduced motivation and increased negative mood.

LEARNING IMPULSE CONTROL IN A NOVEL ANIMAL MODEL: SYNAPTIC, CELLULAR, AND PHARMACOLOGICAL SUBSTRATES

Impulse control, an executive process that restrains inappropriate actions, is impaired in numerous psychiatric conditions. This thesis reports three experiments that utilized a novel animal model of impulse control, the response inhibition (RI) task, to examine the substrates that underlie learning this task. In the first experiment, rats were trained to withhold responding on the RI task, and then euthanized for electrophysiological testing. Training in the RI task increased the AMPA/NMDA ratio at the synapses of pyramidal neurons in the prelimbic, but not infralimbic, region of the medial prefrontal cortex. This enhancement paralleled performance as subjects underwent acquisition and extinction of the inhibitory response. AMPA/NMDA was elevated only in neurons that project to the ventral striatum. Thus, this experiment identified a synaptic correlate of impulse control. In the second experiment, a separate group of rats were trained in the RI task prior to electrophysiological testing. Training in the RI task produced a decrease in membrane excitability in prelimbic, but not infralimbic, neurons as measured by maximal spiking evoked in response to increasing current injection. Importantly, this decrease was strongly correlated with successful inhibition in the task. Fortuitously, subjects trained in an operant control condition showed elevated infralimbic, but not prelimbic, excitability, which was produced by learning an anticipatory signal that predicted imminent reward availability. These experiments revealed two cellular correlates of performance, corresponding to learning two different associations under distinct task conditions. In the final experiment, rats were trained on the RI task under three conditions: Short (4-s), long (60-s), or unpredictable (1-s to 60-s) premature phases. These conditions produced distinct errors on the RI task. Interestingly, amphetamine increased premature responding in the short and long conditions, but decreased premature responding in the unpredictable condition. This dissociation may arise from interactions between amphetamine and underlying cognitive processes, such as attention, timing, and conditioned avoidance. In summary, this thesis showed that learning to inhibit a response produces distinct synaptic, cellular, and pharmacological changes. It is hoped that these advances will provide a starting point for future therapeutic interventions of disorders of impulse control.

Network mechanisms of intentional learning

The ability to learn new tasks rapidly is a prominent characteristic of human behaviour. This ability relies on flex- ible cognitive systems that adapt in order to encode temporary programs for processing non-automated tasks. Previous functional imaging studies have revealed distinct roles for the lateral frontal cortices (LFCs) and the ven- tral striatum in intentional learning processes. However, the human LFCs are complex; they house multiple dis- tinct sub-regions, each of which co-activates with a different functional network. It remains unclear how these LFC networks differ in their functions and how they coordinate with each other, and the ventral striatum, to support intentional learning. Here, we apply a suite of fMRI connectivity methods to determine how LFC networks activate and interact at different stages of two novel tasks, in which arbitrary stimulus-response rules are learnt either from explicit instruction or by trial-and-error. We report that the networks activate en masse and in synchrony when novel rules are being learnt from instruction. However, these networks are not homogeneous in their functions; instead, the directed connectivities between them vary asymmetrically across the learning timecourse and they disengage from the task sequentially along a rostro-caudal axis. Furthermore, when negative feedback indicates the need to switch to alternative stimulus–response rules, there is additional input to the LFC networks from the ventral striatum. These results support the hypotheses that LFC networks interact as a hierarchical system during intentional learning and that signals from the ventral striatum have a driving influence on this system when the internal program for processing the task is updated.

Terapia electroconvulsiva en esquizofrenia resistente a tratamiento farmacológico

El uso de la terapia electroconvulsiva en el tratamiento de la esquizofrenia resistente a manejo farmacológico suscita en la actualidad diferentes posiciones respecto a su utilidad y eficacia en el control del espectro de síntomas presentados por este tipo de pacientes. Objetivo: Determinar la efectividad de la terapia electroconvulsiva en esquizofrenia resistente a tratamiento farmacológico. Materiales y métodos: Se realizó una revisión sistemática de la literatura. Se incluyeron dos estudios aleatorizados controlados y siete estudios abiertos cuasi experimentales. Se excluyeron 12 estudios y otros cinco estudios se clasificaron en curso, es decir, que a la fecha no se han terminado ni publicado. Resultados: La evidencia disponible, respalda el uso de la terapia electroconvulsiva como terapia coadyuvante al tratamiento farmacológico en pacientes con esquizofrenia refractaria. Un grupo de cuatro estudios cuasi experimentales mostró reducción clínicamente significativa de la escala BPRS al igual que los dos estudios aleatorizados controlados. El grupo restante de tres estudios cuasi experimentales, mostró reducción clínicamente significativa de la escala PANSS. Estos resultados, se dan en el marco de la existencia de pocos ensayos clínicos aleatorizados disponibles. Conclusiones: La TEC combinada con medicamento antipsicótico provee la mejor opción terapéutica en el tratamiento agudo y a largo plazo de la esquizofrenia refractaria. Es necesaria la realización de estudios clínicos que aporten mayor nivel de evidencia.

Maturation of limbic corticostriatal activation and connectivity associated with developmental changes in temporal discounting

Temporal discounting (TD) matures with age, alongside other markers of increased impulse control, and coherent, self-regulated behaviour. Discounting paradigms quantify the ability to refrain from preference of immediate rewards, in favour of delayed, larger rewards. As such, they measure temporal foresight and the ability to delay gratification, functions that develop slowly into adulthood. We investigated the neural maturation that accompanies the previously observed age-related behavioural changes in discounting, from early adolescence into mid-adulthood. We used functional magnetic resonance imaging of a hypothetical discounting task with monetary rewards delayed in the week to year range. We show that age-related reductions in choice impulsivity were associated with changes in activation in ventromedial prefrontal cortex (vmPFC), anterior cingulate cortex (ACC), ventral striatum (VS), insula, inferior temporal gyrus, and posterior parietal cortex. Limbic frontostriatal activation changes were specifically associated with age-dependent reductions in impulsive choice, as part of a more extensive network of brain areas showing age-related changes in activation, including dorsolateral PFC, inferior parietal cortex, and subcortical areas. The maturational pattern of functional connectivity included strengthening in activation coupling between ventromedial and dorsolateral PFC, parietal and insular cortices during selection of delayed alternatives, and between vmPFC and VS during selection of immediate alternatives. We conclude that maturational mechanisms within limbic frontostriatal circuitry underlie the observed post-pubertal reductions in impulsive choice with increasing age, and that this effect is dependent on increased activation coherence within a network of areas associated with discounting behaviour and inter-temporal decision-making.

Neural and psychological maturation of decision-making in adolescence and young adulthood

We examined the maturation of decision-making from early adolescence to mid-adulthood using fMRI of a variant of the Iowa gambling task. We have previously shown that performance in this task relies on sensitivity to accumulating negative outcomes in ventromedial PFC and dorsolateral PFC. Here, we further formalize outcome evaluation (as driven by prediction errors [PE], using a reinforcement learning model) and examine its development. Task performance improved significantly during adolescence, stabilizing in adulthood. Performance relied on greater impact of negative compared with positive PEs, the relative impact of which matured from adolescence into adulthood. Adolescents also showed increased exploratory behavior, expressed as a propensity to shift responding between options independently of outcome quality, whereas adults showed no systematic shifting patterns. The correlation between PE representation and improved performance strengthened with age for activation in ventral and dorsal PFC, ventral striatum, and temporal and parietal cortices. There was a medial-lateral distinction in the prefrontal substrates of effective PE utilization between adults and adolescents: Increased utilization of negative PEs, a hallmark of successful performance in the task, was associated with increased activation in ventromedial PFC in adults, but decreased activation in ventrolateral PFC and striatum in adolescents. These results suggest that adults and adolescents engage qualitatively distinct neural and psychological processes during decision-making, the development of which is not exclusively dependent on reward-processing maturation.

Increased neural processing of rewarding and aversive food stimuli in recovered anorexia nervosa

Background Recent evidence has shown that individuals with acute anorexia nervosa and those recovered have aberrant physiological responses to rewarding stimuli. We hypothesized that women recovered from anorexia nervosa would show aberrant neural responses to both rewarding and aversive disorder-relevant stimuli. Methods Using functional magnetic resonance imaging (fMRI), the neural response to the sight and flavor of chocolate, and their combination, in 15 women recovered from restricting-type anorexia nervosa and 16 healthy control subjects matched for age and body mass index was investigated. The neural response to a control aversive condition, consisting of the sight of moldy strawberries and a corresponding unpleasant taste, was also measured. Participants simultaneously recorded subjective ratings of “pleasantness,” “intensity,” and “wanting.” Results Despite no differences between the groups in subjective ratings, individuals recovered from anorexia nervosa showed increased neural response to the pleasant chocolate taste in the ventral striatum and pleasant chocolate sight in the occipital cortex. The recovered participants also showed increased neural response to the aversive strawberry taste in the insula and putamen and to the aversive strawberry sight in the anterior cingulate cortex and caudate. Conclusions Individuals recovered from anorexia nervosa have increased neural responses to both rewarding and aversive food stimuli. These findings suggest that even after recovery, women with anorexia nervosa have increased salience attribution to food stimuli. These results aid our neurobiological understanding and support the view that the neural response to reward may constitute a neural biomarker for anorexia nervosa.

Autistic traits modulate frontostriatal connectivity during processing of rewarding faces

Deficits in facial mimicry have been widely reported in autism. Some studies have suggested that these deficits are restricted to spontaneous mimicry and do not extend to volitional mimicry. We bridge these apparently inconsistent observations, by testing the impact of reward value on neural indices of mimicry, and how autistic traits modulate this impact. Neutral faces were conditioned with high and low reward. Subsequently, functional connectivity between the ventral striatum (VS) and inferior frontal gyrus (IFG) was measured whilst neurotypical adults (n = 30) watched happy expressions made by these conditioned faces. We found greater VS-IFG connectivity in response to high-reward vs. low-reward happy faces. This difference was negatively proportional to autistic traits, suggesting that reduced spontaneous mimicry of social stimuli seen in autism, maybe related to a failure in the modulation of the mirror system by the reward system rather than a circumscribed deficit in the mirror system.

Social equality in the number of choice options is represented in the ventromedial prefrontal cortex

A distinct aspect of the sense of fairness in humans is that we care not only about equality in material rewards but also about equality in non-material values. One such value is the opportunity to choose freely among many options, often regarded as a fundamental right to economic freedom. In modern developed societies, equal opportunities in work, living, and lifestyle are enforced by anti-discrimination laws. Despite the widespread endorsement of equal opportunity, no studies have explored how people assign value to it. We used functional magnetic resonance imaging to identify the neural substrates for subjective valuation of equality in choice opportunity. Participants performed a two-person choice task in which the number of choices available was varied across trials independently of choice outcomes. By using this procedure, we manipulated the degree of equality in choice opportunity between players and dissociated it from the value of reward outcomes and their equality. We found that activation in the ventromedial prefrontal cortex tracked the degree to which the number of options between the two players was equal. In contrast, activation in the ventral striatum tracked the number of options available to participants themselves but not the equality between players. Our results demonstrate that the vmPFC, a key brain region previously implicated in the processing of social values, is also involved in valuation of equality in choice opportunity between individuals. These findings may provide valuable insight into the human ability to value equal opportunity, a characteristic long emphasized in politics, economics, and philosophy.

The D2 antagonist sulpiride modulates the neural processing of both rewarding and aversive stimuli in healthy volunteers.

Rationale: Animal studies indicate that dopamine pathways in the ventral striatum code for the motivational salience of both rewarding and aversive stimuli, but evidence for this mechanism in humans is less established. We have developed a functional magnetic resonance imaging (fMRI) model which permits examination of the neural processing of both rewarding and aversive stimuli. Objectives: The aim of the study was to determine the effect of the dopamine receptor antagonist, sulpiride, on the neural processing of rewarding and aversive stimuli in healthy volunteers. Methods: We studied 30 healthy participants who were randomly allocated to receive a single dose of sulpiride (400 mg) or placebo, in a double-blind, parallel-group design. We used fMRI to measure the neural response to rewarding (taste or sight of chocolate) and aversive stimuli (sight of mouldy strawberries or unpleasant strawberry taste) 4 h after drug treatment. Results: Relative to placebo, sulpiride reduced blood oxygenation level-dependent responses to chocolate stimuli in the striatum (ventral striatum) and anterior cingulate cortex. Sulpiride also reduced lateral orbitofrontal cortex and insula activations to the taste and sight of the aversive condition. Conclusions: These results suggest that acute dopamine receptor blockade modulates mesolimbic and mesocortical neural activations in response to both rewarding and aversive stimuli in healthy volunteers. This effect may be relevant to the effects of dopamine receptor antagonists in the treatment of psychosis and may also have implications for the possible antidepressant properties of sulpiride.