Differential encoding of losses and gains in the human striatum.

Studies on human monetary prediction and decision making emphasize the role of the striatum in encoding prediction errors for financial reward. However, less is known about how the brain encodes financial loss. Using Pavlovian conditioning of visual cues to outcomes that simultaneously incorporate the chance of financial reward and loss, we show that striatal activation reflects positively signed prediction errors for both. Furthermore, we show functional segregation within the striatum, with more anterior regions showing relative selectivity for rewards and more posterior regions for losses. These findings mirror the anteroposterior valence-specific gradient reported in rodents and endorse the role of the striatum in aversive motivational learning about financial losses, illustrating functional and anatomical consistencies with primary aversive outcomes such as pain.

The misbehavior of value and the discipline of the will.

Most reinforcement learning models of animal conditioning operate under the convenient, though fictive, assumption that Pavlovian conditioning concerns prediction learning whereas instrumental conditioning concerns action learning. However, it is only through Pavlovian responses that Pavlovian prediction learning is evident, and these responses can act against the instrumental interests of the subjects. This can be seen in both experimental and natural circumstances. In this paper we study the consequences of importing this competition into a reinforcement learning context, and demonstrate the resulting effects in an omission schedule and a maze navigation task. The misbehavior created by Pavlovian values can be quite debilitating; we discuss how it may be disciplined.

Opponent appetitive-aversive neural processes underlie predictive learning of pain relief.

Termination of a painful or unpleasant event can be rewarding. However, whether the brain treats relief in a similar way as it treats natural reward is unclear, and the neural processes that underlie its representation as a motivational goal remain poorly understood. We used fMRI (functional magnetic resonance imaging) to investigate how humans learn to generate expectations of pain relief. Using a pavlovian conditioning procedure, we show that subjects experiencing prolonged experimentally induced pain can be conditioned to predict pain relief. This proceeds in a manner consistent with contemporary reward-learning theory (average reward/loss reinforcement learning), reflected by neural activity in the amygdala and midbrain. Furthermore, these reward-like learning signals are mirrored by opposite aversion-like signals in lateral orbitofrontal cortex and anterior cingulate cortex. This dual coding has parallels to 'opponent process' theories in psychology and promotes a formal account of prediction and expectation during pain.

Temporal difference models describe higher-order learning in humans.

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.

Soil conditioning under London

The CTRL Contract 220 covered 7.5km twin-bore tunnels excavated between late 2002 and early 2004 from Stratford Box to St Pancras station in Central London. To ensure efficient machine operation as well as the transport and disposal of soil, soil conditioning treatments were applied. Specifically, the foam injection ratio (FIR) and the polymer injection ratio (PIR) (injected volume of foam and polymer solution expressed as a percentage of the excavated soil volume) were employed. It was found that carefully selected soil conditioning allowed chamber pressures of 200kPa or more to be accurately controlled in the stiff London Clay and to an extent, in the very stiff clays of the Lambeth Group. Average FIRs of 50% and PIRs of 7 and 9% were used in the Thanet Sand and in the Lambeth Group Clays. In contrast, much lower quantities of foam were used in the London Clay.

Human pavlovian-instrumental transfer.

The vigor with which a participant performs actions that produce valuable outcomes is subject to a complex set of motivational influences. Many of these are believed to involve the amygdala and the nucleus accumbens, which act as an interface between limbic and motor systems. One prominent class of influences is called pavlovian-instrumental transfer (PIT), in which the motivational characteristics of a predictor influence the vigor of an action with respect to which it is formally completely independent. We provide a demonstration of behavioral PIT in humans, with an audiovisual predictor of the noncontingent delivery of money inducing participants to perform more avidly an action involving squeezing a handgrip to earn money. Furthermore, using functional magnetic resonance imaging, we show that this enhanced motivation was associated with a trial-by-trial correlation with the blood oxygenation level-dependent (BOLD) signal in the nucleus accumbens and a subject-by-subject correlation with the BOLD signal in the amygdala. Our data dovetails well with the animal literature and sheds light on the neural control of vigor.

Contingency awareness in human aversive conditioning involves the middle frontal gyrus.

In contrast to the wealth of data describing the neural mechanisms underlying classical conditioning, we know remarkably little about the mechanisms involved in acquisition of explicit contingency awareness. Subjects variably acquire contingency awareness in classical conditioning paradigms, in which they are able to describe the temporal relationship between a conditioned cue and its outcome. Previous studies have implicated the hippocampus and prefrontal cortex in the acquisition of explicit knowledge, although their specific roles remain unclear. We used functional magnetic resonance imaging to track the trial-by-trial acquisition of explicit knowledge in a concurrent trace and delay conditioning paradigm. We show that activity in bilateral middle frontal gyrus and parahippocampal gyrus correlates with the accuracy of explicit contingency awareness on each trial. In contrast, amygdala activation correlates with conditioned responses indexed by skin conductance responses (SCRs). These results demonstrate that brain regions known to be involved in other aspects of learning and memory also play a specific role, reflecting on each trial the acquisition and representation of contingency awareness.

Laser microstructuring of Si surfaces for low-threshold field-electron emission

Dense arrays of high aspect ratio Si micro-pyramids have been formed by cumulative high intensity laser irradiation of doped Si wafers in an SF6 environment. A comparative study using nanosecond (XeCl, 308 nm) and femtosecond (Ti: Sapphire, 800 nm and KrF, 248 nm) laser pulses has been performed in this work. The influence of pulse duration and ambient gas pressure (SF6) is also presented. Scanning electron microscopy has shown that upon laser irradiation conical features appear on the Si surface in a rather homogenous distribution and with a spontaneous self alignment into arrays. Their lowest tip diameter is 800 nm; while their height reaches up to 90 mum. Secondary tip decoration appears on the surface of the formed spikes. Areas of 2 X 2 mm(2) covered with Si cones have been tested as cold cathode field emitters. After several conditioning cycles, the field emission threshold for the studied Si tips is as low as 2 V/mum, with an emission current of 10(-3) A/cm(2) at 4 V/mum. Even though these structures have smaller aspect ratios than good quality carbon nanotubes, their field emission properties are similar. The simple and direct formation of field emission Si arrays over small pre-selected areas by laser irradiation could lead to a novel approach for the development of electron sources. (C) 2003 Elsevier B.V. All rights reserved.

Influence of cluster-assembly parameters on the field emission properties of nanostructured carbon films

Supersonic cluster beam deposition has been used to produce films with different nanostructures by controlling the deposition parameters such as the film thickness, substrate temperature and cluster mass distribution. The field emission properties of cluster-assembled carbon films have been characterized and correlated to the evolution of the film nanostructure. Threshold fields ranging between 4 and 10 V/mum and saturation current densities as high as 0.7 mA have been measured for samples heated during deposition. A series of voltage ramps, i.e., a conditioning process, was found to initiate more stable and reproducible emission. It was found that the presence of graphitic particles (onions, nanotube embryos) in the films substantially enhances the field emission performance. Films patterned on a micrometer scale have been conditioned spot by spot by a ball-tip anode, showing that a relatively high emission site density can be achieved from the cluster-assembled material. (C) 2002 American Institute of Physics.

Wave energy emulator and AC/DC rectifiers for direct drive wave energy converters

This paper describes a solid state electrical emulator devised for laboratory testing of power conditioning electronics for direct drive linear wave energy converters (DDLWEC). Two rectification strategies are considered; a uni-directional boost topology, and an H-bridge which may be controlled in either uni- or bidirectional modes.

Application of the fixed point method for solution in time stepping finite element analysis using the inverse vector Jiles-Atherton model

An implementation of the inverse vector Jiles-Atherton model for the solution of non-linear hysteretic finite element problems is presented. The implementation applies the fixed point method with differential reluctivity values obtained from the Jiles-Atherton model. Differential reluctivities are usually computed using numerical differentiation, which is ill-posed and amplifies small perturbations causing large sudden increases or decreases of differential reluctivity values, which may cause numerical problems. A rule based algorithm for conditioning differential reluctivity values is presented. Unwanted perturbations on the computed differential reluctivity values are eliminated or reduced with the aim to guarantee convergence. Details of the algorithm are presented together with an evaluation of the algorithm by a numerical example. The algorithm is shown to guarantee convergence, although the rate of convergence depends on the choice of algorithm parameters. © 2011 IEEE.