Cortical substrates for exploratory decisions in humans.

Decision making in an uncertain environment poses a conflict between the opposing demands of gathering and exploiting information. In a classic illustration of this 'exploration-exploitation' dilemma, a gambler choosing between multiple slot machines balances the desire to select what seems, on the basis of accumulated experience, the richest option, against the desire to choose a less familiar option that might turn out more advantageous (and thereby provide information for improving future decisions). Far from representing idle curiosity, such exploration is often critical for organisms to discover how best to harvest resources such as food and water. In appetitive choice, substantial experimental evidence, underpinned by computational reinforcement learning (RL) theory, indicates that a dopaminergic, striatal and medial prefrontal network mediates learning to exploit. In contrast, although exploration has been well studied from both theoretical and ethological perspectives, its neural substrates are much less clear. Here we show, in a gambling task, that human subjects' choices can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Furthermore, using this characterization to classify decisions as exploratory or exploitative, we employ functional magnetic resonance imaging to show that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions. In contrast, regions of striatum and ventromedial prefrontal cortex exhibit activity characteristic of an involvement in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes, and provide a computationally precise characterization of the contribution of key decision-related brain systems to each of these functions.

Empathic neural responses are modulated by the perceived fairness of others.

The neural processes underlying empathy are a subject of intense interest within the social neurosciences. However, very little is known about how brain empathic responses are modulated by the affective link between individuals. We show here that empathic responses are modulated by learned preferences, a result consistent with economic models of social preferences. We engaged male and female volunteers in an economic game, in which two confederates played fairly or unfairly, and then measured brain activity with functional magnetic resonance imaging while these same volunteers observed the confederates receiving pain. Both sexes exhibited empathy-related activation in pain-related brain areas (fronto-insular and anterior cingulate cortices) towards fair players. However, these empathy-related responses were significantly reduced in males when observing an unfair person receiving pain. This effect was accompanied by increased activation in reward-related areas, correlated with an expressed desire for revenge. We conclude that in men (at least) empathic responses are shaped by valuation of other people's social behaviour, such that they empathize with fair opponents while favouring the physical punishment of unfair opponents, a finding that echoes recent evidence for altruistic punishment.

Contingency awareness in human aversive conditioning involves the middle frontal gyrus.

In contrast to the wealth of data describing the neural mechanisms underlying classical conditioning, we know remarkably little about the mechanisms involved in acquisition of explicit contingency awareness. Subjects variably acquire contingency awareness in classical conditioning paradigms, in which they are able to describe the temporal relationship between a conditioned cue and its outcome. Previous studies have implicated the hippocampus and prefrontal cortex in the acquisition of explicit knowledge, although their specific roles remain unclear. We used functional magnetic resonance imaging to track the trial-by-trial acquisition of explicit knowledge in a concurrent trace and delay conditioning paradigm. We show that activity in bilateral middle frontal gyrus and parahippocampal gyrus correlates with the accuracy of explicit contingency awareness on each trial. In contrast, amygdala activation correlates with conditioned responses indexed by skin conductance responses (SCRs). These results demonstrate that brain regions known to be involved in other aspects of learning and memory also play a specific role, reflecting on each trial the acquisition and representation of contingency awareness.

Opponent appetitive-aversive neural processes underlie predictive learning of pain relief.

Termination of a painful or unpleasant event can be rewarding. However, whether the brain treats relief in a similar way as it treats natural reward is unclear, and the neural processes that underlie its representation as a motivational goal remain poorly understood. We used fMRI (functional magnetic resonance imaging) to investigate how humans learn to generate expectations of pain relief. Using a pavlovian conditioning procedure, we show that subjects experiencing prolonged experimentally induced pain can be conditioned to predict pain relief. This proceeds in a manner consistent with contemporary reward-learning theory (average reward/loss reinforcement learning), reflected by neural activity in the amygdala and midbrain. Furthermore, these reward-like learning signals are mirrored by opposite aversion-like signals in lateral orbitofrontal cortex and anterior cingulate cortex. This dual coding has parallels to 'opponent process' theories in psychology and promotes a formal account of prediction and expectation during pain.

Anxiety reduction through detachment: subjective, physiological, and neural effects.

The ability to volitionally regulate emotions helps to adapt behavior to changing environmental demands and can alleviate subjective distress. We show that a cognitive strategy of detachment attenuates subjective and physiological measures of anticipatory anxiety for pain and reduces reactivity to receipt of pain itself. Using functional magnetic resonance imaging, we locate the potential site and source of this modulation of anticipatory anxiety in the medial prefrontal/anterior cingulate and anterolateral prefrontal cortex, respectively.

Modulation of pain processing in hyperalgesia by cognitive demand.

The relationship between pain and cognitive function is of theoretical and clinical interest, exemplified by observations that attention-demanding activities reduce pain in chronically afflicted patients. Previous studies have concentrated on phasic pain, which bears little correspondence to clinical pain conditions. Indeed, phasic pain is often associated with differential or opposing effects to tonic pain in behavioral, lesion, and pharmacological studies. To address how cognitive engagement interacts with tonic pain, we assessed the influence of an attention-demanding cognitive task on pain-evoked neural responses in an experimental model of chronic pain, the capsaicin-induced heat hyperalgesia model. Using functional magnetic resonance imaging (fMRI), we show that activity in the orbitofrontal and medial prefrontal cortices, insula, and cerebellum correlates with the intensity of tonic pain. This pain-related activity in medial prefrontal cortex and cerebellum was modulated by the demand level of the cognitive task. Our findings highlight a role for these structures in the integration of motivational and cognitive functions associated with a physiological state of injury. Within the limitations of an experimental model of pain, we suggest that the findings are relevant to understanding both the neurobiology and pathophysiology of chronic pain and its amelioration by cognitive strategies.

Temporal difference models describe higher-order learning in humans.

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.

Dopamine and performance in a reinforcement learning task: Evidence from Parkinson's disease

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning. © 2012 The Author.

Novel fluid grid and voidage calculation techniques for a discrete element model of a 3D cylindrical fluidized bed

A discrete element model (DEM) combined with computational fluid dynamics (CFD) was developed to model particle and fluid behaviour in 3D cylindrical fluidized beds. Novel techniques were developed to (1) keep fluid cells, defined in cylindrical coordinates, at a constant volume in order to ensure the conditions for validity of the volume-averaged fluid equations were satisfied and (2) smoothly and accurately measure voidage in arbitrarily shaped fluid cells. The new technique for calculating voidage was more stable than traditional techniques, also examined in the paper, whilst remaining computationally-effective. The model was validated by quantitative comparison with experimental results from the magnetic resonance imaging of a fluidised bed analysed to give time-averaged particle velocities. Comparisons were also made between theoretical determinations of slug rise velocity in a tall bed. It was concluded that the DEM-CFD model is able to investigate aspects of the underlying physics of fluidisation not readily investigated by experiment. © 2014 The Authors.

The neural signature of escalating frustration in humans.

Mammalian studies show that frustration is experienced when goal-directed activity is blocked. Despite frustration's strongly negative role in health, aggression and social relationships, the neural mechanisms are not well understood. To address this we developed a task in which participants were blocked from obtaining a reward, an established method of producing frustration. Levels of experienced frustration were parametrically varied by manipulating the participants' motivation to obtain the reward prior to blocking. This was achieved by varying the participants' proximity to a reward and the amount of effort expended in attempting to acquire it. In experiment 1, we confirmed that proximity and expended effort independently enhanced participants' self-reported desire to obtain the reward, and their self-reported frustration and response vigor (key-press force) following blocking. In experiment 2, we used functional magnetic resonance imaging (fMRI) to show that both proximity and expended effort modulated brain responses to blocked reward in regions implicated in animal models of reactive aggression, including the amygdala, midbrain periaqueductal grey (PAG), insula and prefrontal cortex. Our findings suggest that frustration may serve an energizing function, translating unfulfilled motivation into aggressive-like surges via a cortical, amygdala and PAG network.

Strong three-level resonant magnetopolaron effect due to the intersubband coupling in heavily modulation-doped GaAs/AlxGa1-xAs single quantum wells at high magnetic-fields

Electron cyclotron resonance CR) measurements have been carried out in magnetic fields up to 32 T to study electron-phonon interaction in two heavily modulation-delta -doped GaAs/Al0.3Ga0.7As single-quantum-well samples. No measurable resonant magnetopolaron effects were observed in either sample in the region of the GaAs longitudinal optical (LO) phonons. However, when the CR frequency is above LO phonon frequency, omega (LO)=E-LO/(h) over bar, at high magnetic fields (B>27 T), electron CR exhibits a strong avoided-level-crossing splitting for both samples at frequencies close to (omega (LO)+ (E-2-E-1)1 (h) over bar, where E-2, and E-1 are the energies of the bottoms of the second and the first subbands, respectively. The energy separation between the two branches is large with the minimum separation of 40 cm(-1) occurring at around 30.5 T. A detailed theoretical analysis, which includes a self-consistent calculation of the band structure and the effects of electron-phonon interaction on the CR, shows that this type of splitting is due to a three-level resonance between the second Landau level of the first electron subband and the lowest Landau level of the second subband plus one GaAs LO phonon. The absence of occupation effects in the final states and weak screening or this three-level process yields large energy separation even in the presence of high electron densities. Excellent agreement between the theory and the experimental results is obtained.

新型靶向性磁共振成像造影剂的合成及性质研究

磁共振成像(magnetic resonance imaging, MRI)以其分辨率高、对人体无电离辐射损伤、多参数成像等优点而得到迅速发展和广泛应用。目前，MRI已从单一形态学向分子影像学的深度发展，对医学临床和医学研究产生了巨大影响。为了提高病变部位与正常组织间信号的对比度，约30%~40%的诊断需要使用磁共振成像造影剂。它是一类能缩短成像时间、提高成像对比度和清晰度、显示组织器官功能状态的诊断用药。下一代磁共振成像造影剂的设计目标将集中在对特定组织或器官具有选择性或靶向性、高弛豫性能和减少用药剂量等方面。本论文在此领域的研究内容可归纳如下： (1) 以多糖为载体的MRI造影剂 设计合成了阿拉伯半乳聚糖修饰的Gd-DTPA配合物(Gd-DTPA-CMAG-An)和葡聚糖修饰的Gd-DTPA配合物(Gd-DTPA-CMDn-Cyst)。通过体外弛豫时间测试和体内磁共振成像实验研究了Gd-DTPA-CMAG-An弛豫性能、器官选择性、体内滞留时间和代谢情况，结合体外稳定性综合评价了其应用于临床的可能性。研究结果表明，Gd-DTPA-CMAG-An配合物在水溶液中弛豫性能为Gd-DTPA的1.4倍左右，Gd-DTPA-CMAG-A2对肝脏信号的增强效果是Gd-DTPA的2.0倍左右，并且能在较长时间内产生良好稳定的增强效果。这与肝脏表面的去唾液酸糖蛋白受体的专一性识别有关。Gd-DTPA-CMAG-A2良好的肝脏选择性和肾脏代谢能力，有望成为有前景的肝脏选择性造影剂。通过小鼠MRI实验初步评价了Gd-DTPA-CMDn-Cyst配合物造影剂对血管信号的增强作用。Gd-DTPA-CMD4-Cyst对血管产生了良好的增强效果，并且能在较长时间内对血管产生良好稳定的增强，从而有充分的时间优化成像窗口获得理想的成像效果。但造影剂在体内的分布和代谢是一个非常复杂的过程，Gd-DTPA-CMD4-Cyst在血液中的滞留情况及能否用于血管造影仍需进一步的实验证实。 (2) MnNaY 型分子筛作为胃肠道MRI造影剂 离子交换法制备了Mn2+交换的NaY分子筛MnNaY，从对造影剂的一般要求出发，对其酸性水溶液中的稳定性和离子交换选择性、体外弛豫性能和体内成像等方面进行研究，并对器官的选择性及体内滞留时间和代谢情况进行了分析，从而对其应用于临床的可能性进行了探讨。研究结果表明，MnNaY悬浮液能长时间在较低的酸性条件下保持良好的稳定性，其弛豫效率高于目前临床所用造影剂Gd-DTPA，随Mn2+的含量在NaY分子筛中的增加(3.2%~5.2%)，弛豫效率反而降低。MnNaY (3.2% Mn)对胃部具有良好的增强效果，并且能在较长时间内产生良好稳定的增强效果，有利于获得理想的成像效果。它是一种比较好的潜在口服胃肠道造影剂。 (3) 甘草酸为载体的MRI造影剂 合成了甘草酸为载体的配合物GL-(A-Gd-DTPA)3，对其体外弛豫性能和体内成像等方面进行了研究，结果表明，其在水中的弛豫效率约为目前临床所用造影剂Gd-DTPA的1.4倍，体内成像表明它能在较长时间内对大鼠肝脏产生良好稳定的增强效果，这是由于肝(实质)细胞膜表面存在GL和GA受体，Gd-DTPA 以GL为载体后具有良好的趋肝性与肝细胞靶向性。 (4) 中性的Gd-DTPA双酰胺衍生物 合成了两种中性的Gd-DTPA双酰胺衍生物Gd-DTPA-BBA和Gd-DTPA-BtBA，其弛豫效率与Gd-DTPA相近，对肝脏和肾脏具有较好的增强效果，由于这两种配合物均为电中性化合物，这样配合物溶液的渗透压值与血液的渗透压值较接近，可能更易为生物体所接受。

吗啡给药及戒断期间大鼠眶额叶脑区功能活动特征户外环境下自由活动猕猴顶叶及海马神经元单位放电的胞外记录方法

已有的研究表明，眶额叶在解剖上与现在已知的药物滥用相关的脑区是紧密联系在一起的。例如，眶额叶在药物滥用和强迫性重复行为中起作用，且随着脑成像技术的应用，越来越多的证据表明眶额叶参与了药物滥用。但是我们并不了解在阿片给药和戒断期间眶额叶脑区活动是如何变化的。因此，我们在实验中采用了Mn2+增强的核磁共振成像（Manganese-enhanced magnetic resonance　imaging，MEMRI，4.7T）技术和脑电（EEG）记录的方法，以研究大鼠眶额叶在给与阿片类药物（盐酸吗啡）以及戒断过程中的动态变化。 MEMRI是一种近年才发展起来的新型技术。研究表明，Mn2+是Ca2+的类似物，可以通过Ca2+通道进入兴奋性的神经元里面并结合到胞内的蛋白质和核酸上的Ca2+和Mg2+结合位点上 (MILDVAN and COHN, 1963; EISINGER et al., 1965)。另外，Mn2+的顺磁性也为它成为核磁共振成像的造影剂提供了前提条件。可是成功应用MEMRI的前提就是要在适当的时间把合适剂量的Mn2+传递到靶点上。因此，Mn2+在注射到靶点后，是否能够在有效的时间内反映大脑活动的变化就成为一个非常重要并且在技术上较为棘手的问题。在给实验大鼠脑区微量注射Mn2+（80mMol/L，200nl）的同时，通过微量注射兴奋性神经递质谷氨酸（Glu 0.5mM/L）或抑制性递质γ-aminobutyric acid（GABA 0.5M/L）以改变靶点神经元兴奋性的方法，检测Mn2+能否反映脑区的活动变化。另外，我们随机选取实验动物，分别在注射Mn2+ 3小时、5小时和8小时后对三组大鼠（n=5）进行10％福尔马林灌流，并且通过观察大鼠眶额叶脑区Mn2+强度的变化来研究最佳的灌流时间。我们的实验结果表明，Mn2++Glu组的右侧脑区/左侧脑区的Mn2+亮度比Mn2+空白对照组增加了20％（p=0.016, student t-test, *p <0.05），也远大于Mn2++GABA组（p=0.047, *p<0.05）。结果表明，当神经元被兴奋的时候，较多的Mn2+可以通过Ca2+通道进入兴奋的神经元内，使得Mn2+的成像亮度增加。由于Mn2+成像亮度的增加可以反映神经元的兴奋活动，因此可以显示出靶点区的脑活动。另外，在研究灌流时间对Mn2+亮度影响的实验中发现，注射Mn2+ 5小时后灌流得到的信噪比分别比注射Mn2+3小时（p＝0.055）和8小时（p=0.004，*p<0.05）高出24％和32％。总之，我们采用微量注射Mn2+（80mM/L,200nl）后5小时用10%福尔马林心脏灌流的方法获得了较好的结果。另外在试验中我们首先观测了大鼠吗啡戒断后的行为学指标和检测大鼠戒断后条件化位置偏好的程度。实验结果表明大鼠可以建立非常明显的条件化位置偏好，但在湿狗抖等行为学指标上无明显症状。这说明大鼠对于吗啡(10mg/kg, 一天两次，持续12天)形成了明显的心理依赖而无明显的生理依赖。此外，MEMRI的结果表明，在吗啡给药的第1天和第6天，大鼠眶额叶的Mn2+强度与空白对照组相比有显著的降低( one-way ANOVA, Post Hoc Dunnett’s C Tests), F (6,28)=7.242, P<0.001)；而在戒断第3天又恢复到正常水平，在戒断第5天和第7天Mn2+强度跟空白对照组相比没有显著性差别(one-way ANOVA, *p<0.05)。脑电（EEG）的结果表明，急性吗啡诱导的gamma波段的EEG显著降低(Two-way ANOVA, F(1,10)=13.626,p=0.006)。然而在戒断第1天gamma波段的EEG与空白对照组相比是增加的。在戒断第3天和戒断第5天，gamma波段的EEG与空白对照组相比也有显著性增强。以上研究结果表明：大鼠眶额叶脑区的动态变化与整个吗啡给药和戒断过程是密切相关的；此外，MEMRI在探讨药物滥用以及成瘾等机制上有很大的应用前景。

Metabolic profiling of serum from gadolinium chloride-treated rats by H-1 NMR spectroscopy

Metabolic profiling of serum from gadolinium chloride (GdCl3, 10 and 50 mg/kg body weight, intraperitoneal [i.p.])-treated rats was investigated by the NMR spectroscopic-based metabonomic strategy. Serum samples were collected at 48, 96, and 168 h postdose (p.d.) after exposure to GdCl3. H-1 NMR spectra of serum were analyzed by pattern recognition using principal components analysis. The studies showed that there was a dose-related biochemical effect of GdCl3 treatment on the levels of a range of low-molecular weight compounds in serum. The liver damage induced by GdCl3 was characterized by the elevation of lactate, pyruvate, and creatine as well as the decrease of branched-chain amino acids (valine and isoleucine), alanine, glucose, and trimethylamine-N-oxide concentration in serum samples. The biochemical effects of GdCl3 in rats could be consulted when evaluating the biochemical profile of gadolinium-containing compounds that are being developed for nuclear magnetic resonance imaging.

The gadolinium complexes with polyoxometalates as potential MRI contrast agents

The two gadolinium (Gd) polyoxometalates, K-15[Gd(BW11O39)(2)] [Gd(BW11)(2)] and K-17[Gd(CuW11O39)(2)] [Gd(CuW11)(2)] have been evaluated by in vivo and in vitro experiments as the candidates of potential tissue-specific magnetic resonance imaging (MRI) contrast agents. T-1 relaxivities of 17.12 mM(-1) . s(-1) for Gd(BW11)(2) and 19.95 mM(-1) . s(-1) for Gd(CuW11)(2) (400MHz, 25 degrees C) were much higher than that of the commercial MRI contrast agent (GdDTPA). Their relaxivities in bovine serum albumin and human serum transferrin solutions were also reported. After administration of Gd(BW11)(2) and Gd(CuW11)(2) to Wistar rats, MRI showed longer and remarkable enhancement in rat liver and favorable renal excretion capability. The signal intensity increased by 37.63 +/- 3.45% for the liver during the whole imaging period (100 min) and by 61.47 +/- 10.03% for kidney within 5-40 min after injection at 40 +/- 1-mu mol . kg(-1) dose for Gd(CuW11)(2), and Gd(BW11)(2) induced 50.44 +/- 3.51% enhancement in the liver in 5-50-min range and 61.47 +/- 10.03% enhancement for kidney within 5-40 min after injection at 39 +/- 4 mu mol . kg(-1) dose. In vitro and in vivo study showed that Gd(BW11)(2) and Gd(CuW11)(2) are favorable candidates as tissue-specific contrast agents for MRI.