Opponent appetitive-aversive neural processes underlie predictive learning of pain relief.

Termination of a painful or unpleasant event can be rewarding. However, whether the brain treats relief in a similar way as it treats natural reward is unclear, and the neural processes that underlie its representation as a motivational goal remain poorly understood. We used fMRI (functional magnetic resonance imaging) to investigate how humans learn to generate expectations of pain relief. Using a pavlovian conditioning procedure, we show that subjects experiencing prolonged experimentally induced pain can be conditioned to predict pain relief. This proceeds in a manner consistent with contemporary reward-learning theory (average reward/loss reinforcement learning), reflected by neural activity in the amygdala and midbrain. Furthermore, these reward-like learning signals are mirrored by opposite aversion-like signals in lateral orbitofrontal cortex and anterior cingulate cortex. This dual coding has parallels to 'opponent process' theories in psychology and promotes a formal account of prediction and expectation during pain.

Anxiety reduction through detachment: subjective, physiological, and neural effects.

The ability to volitionally regulate emotions helps to adapt behavior to changing environmental demands and can alleviate subjective distress. We show that a cognitive strategy of detachment attenuates subjective and physiological measures of anticipatory anxiety for pain and reduces reactivity to receipt of pain itself. Using functional magnetic resonance imaging, we locate the potential site and source of this modulation of anticipatory anxiety in the medial prefrontal/anterior cingulate and anterolateral prefrontal cortex, respectively.

Modulation of pain processing in hyperalgesia by cognitive demand.

The relationship between pain and cognitive function is of theoretical and clinical interest, exemplified by observations that attention-demanding activities reduce pain in chronically afflicted patients. Previous studies have concentrated on phasic pain, which bears little correspondence to clinical pain conditions. Indeed, phasic pain is often associated with differential or opposing effects to tonic pain in behavioral, lesion, and pharmacological studies. To address how cognitive engagement interacts with tonic pain, we assessed the influence of an attention-demanding cognitive task on pain-evoked neural responses in an experimental model of chronic pain, the capsaicin-induced heat hyperalgesia model. Using functional magnetic resonance imaging (fMRI), we show that activity in the orbitofrontal and medial prefrontal cortices, insula, and cerebellum correlates with the intensity of tonic pain. This pain-related activity in medial prefrontal cortex and cerebellum was modulated by the demand level of the cognitive task. Our findings highlight a role for these structures in the integration of motivational and cognitive functions associated with a physiological state of injury. Within the limitations of an experimental model of pain, we suggest that the findings are relevant to understanding both the neurobiology and pathophysiology of chronic pain and its amelioration by cognitive strategies.

Primary antibody deficiency and diagnostic delay.

AIMS: To assess the occurrence of diagnostic delay in primary antibody deficiency in the period 1989-2002, since a similar study in 1989, and to assess the impact of UK national guidelines communicated in 1995. METHODS: A retrospective case note review was performed of 89 consecutive patients with antibody deficiency referred to a regional referral centre for clinical immunology in north west England and north Wales. The delay in diagnosis and the estimated resulting morbidity in terms of infections were assessed. RESULTS: Fifty six of the 89 patients experienced delay in diagnosis. The overall median delay was 2 years (mean, 4.4), resulting in substantial morbidity (equivalent to two major infections and one minor infection). This shows a moderate improvement since the previous study in 1989 and since the introduction of UK national guidelines in 1995. Respiratory infections are the most frequent presenting infections, and respiratory physicians the most common source of referral. CONCLUSIONS: There is still considerable delay in the diagnosis of primary antibody deficiency, but the data suggest an improvement in practice since the previous study in 1989 and the distribution of national guidelines in 1995.

Temporal difference models describe higher-order learning in humans.

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.

Empathy for pain involves the affective but not sensory components of pain.

Our ability to have an experience of another's pain is characteristic of empathy. Using functional imaging, we assessed brain activity while volunteers experienced a painful stimulus and compared it to that elicited when they observed a signal indicating that their loved one--present in the same room--was receiving a similar pain stimulus. Bilateral anterior insula (AI), rostral anterior cingulate cortex (ACC), brainstem, and cerebellum were activated when subjects received pain and also by a signal that a loved one experienced pain. AI and ACC activation correlated with individual empathy scores. Activity in the posterior insula/secondary somatosensory cortex, the sensorimotor cortex (SI/MI), and the caudal ACC was specific to receiving pain. Thus, a neural response in AI and rostral ACC, activated in common for "self" and "other" conditions, suggests that the neural substrate for empathic experience does not involve the entire "pain matrix." We conclude that only that part of the pain network associated with its affective qualities, but not its sensory qualities, mediates empathy.

Dopamine and performance in a reinforcement learning task: Evidence from Parkinson's disease

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning. © 2012 The Author.

Values and actions in aversion

Model-based and model-free controllers can, in principle, learn arbitrary actions to optimize their behavior, at least those actions that can be expressed and explored. Indeed, these are often referred to as instrumental controllers because their choices are learned to be instrumental for the delivery of desired outcomes. Although this flexibility is very powerful, it comes with an attendant cost of learning. Evolution appears to have endowed everything from the simplest organisms to us with powerful, pre-specified, but inflexible alternatives. These responses are termed Pavlovian, after the famous Russian physiologist and psychologist Pavlov. The responses of the Pavlovian controller are determined by evolutionary (phylogenetic) considerations rather than (ontogenetic) aspects of the contingent development or learning of an individual. These responses directly interact with instrumental choices arising from goal-directed and habitual controllers. This interaction has been studied in a wealth of animal paradigms, and can be helpful, neutral, or harmful, according to circumstance. Although there has been less careful or analytical study of it in humans, it can be interpreted as underpinning a wealth of behavioral aberrations. © 2009 Elsevier Inc. All rights reserved.

Dread and the disvalue of future pain.

Standard theories of decision-making involving delayed outcomes predict that people should defer a punishment, whilst advancing a reward. In some cases, such as pain, people seem to prefer to expedite punishment, implying that its anticipation carries a cost, often conceptualized as 'dread'. Despite empirical support for the existence of dread, whether and how it depends on prospective delay is unknown. Furthermore, it is unclear whether dread represents a stable component of value, or is modulated by biases such as framing effects. Here, we examine choices made between different numbers of painful shocks to be delivered faithfully at different time points up to 15 minutes in the future, as well as choices between hypothetical painful dental appointments at time points of up to approximately eight months in the future, to test alternative models for how future pain is disvalued. We show that future pain initially becomes increasingly aversive with increasing delay, but does so at a decreasing rate. This is consistent with a value model in which moment-by-moment dread increases up to the time of expected pain, such that dread becomes equivalent to the discounted expectation of pain. For a minority of individuals pain has maximum negative value at intermediate delay, suggesting that the dread function may itself be prospectively discounted in time. Framing an outcome as relief reduces the overall preference to expedite pain, which can be parameterized by reducing the rate of the dread-discounting function. Our data support an account of disvaluation for primary punishments such as pain, which differs fundamentally from existing models applied to financial punishments, in which dread exerts a powerful but time-dependent influence over choice.

Decisions about decisions.

A major puzzle of decision making is how the brain decides which decision system to use at any one time. In this issue of Neuron, Lee et al. (2014) provide a theoretical, behavioral, and neurobiological account of a prefrontal reliability-based arbitration system.

Does temporal discounting explain unhealthy behavior? A systematic review and reinforcement learning perspective.

The tendency to make unhealthy choices is hypothesized to be related to an individual's temporal discount rate, the theoretical rate at which they devalue delayed rewards. Furthermore, a particular form of temporal discounting, hyperbolic discounting, has been proposed to explain why unhealthy behavior can occur despite healthy intentions. We examine these two hypotheses in turn. We first systematically review studies which investigate whether discount rates can predict unhealthy behavior. These studies reveal that high discount rates for money (and in some instances food or drug rewards) are associated with several unhealthy behaviors and markers of health status, establishing discounting as a promising predictive measure. We secondly examine whether intention-incongruent unhealthy actions are consistent with hyperbolic discounting. We conclude that intention-incongruent actions are often triggered by environmental cues or changes in motivational state, whose effects are not parameterized by hyperbolic discounting. We propose a framework for understanding these state-based effects in terms of the interplay of two distinct reinforcement learning mechanisms: a "model-based" (or goal-directed) system and a "model-free" (or habitual) system. Under this framework, while discounting of delayed health may contribute to the initiation of unhealthy behavior, with repetition, many unhealthy behaviors become habitual; if health goals then change, habitual behavior can still arise in response to environmental cues. We propose that the burgeoning development of computational models of these processes will permit further identification of health decision-making phenotypes.

The neural signature of escalating frustration in humans.

Mammalian studies show that frustration is experienced when goal-directed activity is blocked. Despite frustration's strongly negative role in health, aggression and social relationships, the neural mechanisms are not well understood. To address this we developed a task in which participants were blocked from obtaining a reward, an established method of producing frustration. Levels of experienced frustration were parametrically varied by manipulating the participants' motivation to obtain the reward prior to blocking. This was achieved by varying the participants' proximity to a reward and the amount of effort expended in attempting to acquire it. In experiment 1, we confirmed that proximity and expended effort independently enhanced participants' self-reported desire to obtain the reward, and their self-reported frustration and response vigor (key-press force) following blocking. In experiment 2, we used functional magnetic resonance imaging (fMRI) to show that both proximity and expended effort modulated brain responses to blocked reward in regions implicated in animal models of reactive aggression, including the amygdala, midbrain periaqueductal grey (PAG), insula and prefrontal cortex. Our findings suggest that frustration may serve an energizing function, translating unfulfilled motivation into aggressive-like surges via a cortical, amygdala and PAG network.