Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans.

Theories of instrumental learning are centred on understanding how success and failure are used to improve future decisions. These theories highlight a central role for reward prediction errors in updating the values associated with available actions. In animals, substantial evidence indicates that the neurotransmitter dopamine might have a key function in this type of learning, through its ability to modulate cortico-striatal synaptic efficacy. However, no direct evidence links dopamine, striatal activity and behavioural choice in humans. Here we show that, during instrumental learning, the magnitude of reward prediction error expressed in the striatum is modulated by the administration of drugs enhancing (3,4-dihydroxy-L-phenylalanine; L-DOPA) or reducing (haloperidol) dopaminergic function. Accordingly, subjects treated with L-DOPA have a greater propensity to choose the most rewarding action relative to subjects treated with haloperidol. Furthermore, incorporating the magnitude of the prediction errors into a standard action-value learning algorithm accurately reproduced subjects' behavioural choices under the different drug conditions. We conclude that dopamine-dependent modulation of striatal activity can account for how the human brain uses reward prediction errors to improve future decisions.

Functional requirements for reward-modulated spike-timing-dependent plasticity.

Recent experiments have shown that spike-timing-dependent plasticity is influenced by neuromodulation. We derive theoretical conditions for successful learning of reward-related behavior for a large class of learning rules where Hebbian synaptic plasticity is conditioned on a global modulatory factor signaling reward. We show that all learning rules in this class can be separated into a term that captures the covariance of neuronal firing and reward and a second term that presents the influence of unsupervised learning. The unsupervised term, which is, in general, detrimental for reward-based learning, can be suppressed if the neuromodulatory signal encodes the difference between the reward and the expected reward-but only if the expected reward is calculated for each task and stimulus separately. If several tasks are to be learned simultaneously, the nervous system needs an internal critic that is able to predict the expected reward for arbitrary stimuli. We show that, with a critic, reward-modulated spike-timing-dependent plasticity is capable of learning motor trajectories with a temporal resolution of tens of milliseconds. The relation to temporal difference learning, the relevance of block-based learning paradigms, and the limitations of learning with a critic are discussed.

Slowness: an objective for spike-timing-dependent plasticity?

Our nervous system can efficiently recognize objects in spite of changes in contextual variables such as perspective or lighting conditions. Several lines of research have proposed that this ability for invariant recognition is learned by exploiting the fact that object identities typically vary more slowly in time than contextual variables or noise. Here, we study the question of how this "temporal stability" or "slowness" approach can be implemented within the limits of biologically realistic spike-based learning rules. We first show that slow feature analysis, an algorithm that is based on slowness, can be implemented in linear continuous model neurons by means of a modified Hebbian learning rule. This approach provides a link to the trace rule, which is another implementation of slowness learning. Then, we show analytically that for linear Poisson neurons, slowness learning can be implemented by spike-timing-dependent plasticity (STDP) with a specific learning window. By studying the learning dynamics of STDP, we show that for functional interpretations of STDP, it is not the learning window alone that is relevant but rather the convolution of the learning window with the postsynaptic potential. We then derive STDP learning windows that implement slow feature analysis and the "trace rule." The resulting learning windows are compatible with physiological data both in shape and timescale. Moreover, our analysis shows that the learning window can be split into two functionally different components that are sensitive to reversible and irreversible aspects of the input statistics, respectively. The theory indicates that irreversible input statistics are not in favor of stable weight distributions but may generate oscillatory weight dynamics. Our analysis offers a novel interpretation for the functional role of STDP in physiological neurons.

Context-dependent decay of motor memories during skill acquisition.

Current models of motor learning posit that skill acquisition involves both the formation and decay of multiple motor memories that can be engaged in different contexts. Memory formation is assumed to be context dependent, so that errors most strongly update motor memories associated with the current context. In contrast, memory decay is assumed to be context independent, so that movement in any context leads to uniform decay across all contexts. We demonstrate that for both object manipulation and force-field adaptation, contrary to previous models, memory decay is highly context dependent. We show that the decay of memory associated with a given context is greatest for movements made in that context, with more distant contexts showing markedly reduced decay. Thus, both memory formation and decay are strongest for the current context. We propose that this apparently paradoxical organization provides a mechanism for optimizing performance. While memory decay tends to reduce force output, memory formation can correct for any errors that arise, allowing the motor system to regulate force output so as to both minimize errors and avoid unnecessary energy expenditure. The motor commands for any given context thus result from a balance between memory formation and decay, while memories for other contexts are preserved.

Measurement of filling-factor-dependent magnetophonon resonances in graphene using Raman spectroscopy

We perform polarization-resolved Raman spectroscopy on graphene in magnetic fields up to 45 T. This reveals a filling-factor-dependent, multicomponent anticrossing structure of the Raman G peak, resulting from magnetophonon resonances between magnetoexcitons and E2g phonons. This is explained with a model of Raman scattering taking into account the effects of spatially inhomogeneous carrier densities and strain. Random fluctuations of strain-induced pseudomagnetic fields lead to increased scattering intensity inside the anticrossing gap, consistent with the experiments. © 2013 American Physical Society.

Stable multivariate rational interpolation for parameter-dependent aerospace models

A multivariate, robust, rational interpolation method for propagating uncertainties in several dimensions is presented. The algorithm for selecting numerator and denominator polynomial orders is based on recent work that uses a singular value decomposition approach. In this paper we extend this algorithm to higher dimensions and demonstrate its efficacy in terms of convergence and accuracy, both as a method for response suface generation and interpolation. To obtain stable approximants for continuous functions, we use an L2 error norm indicator to rank optimal numerator and denominator solutions. For discontinous functions, a second criterion setting an upper limit on the approximant value is employed. Analytical examples demonstrate that, for the same stencil, rational methods can yield more rapid convergence compared to pseudospectral or collocation approaches for certain problems. © 2012 AIAA.

Catalyst composition and impurity-dependent kinetics of nanowire heteroepitaxy.

The mechanisms and kinetics of axial Ge-Si nanowire heteroepitaxial growth based on the tailoring of the Au catalyst composition via Ga alloying are studied by environmental transmission electron microscopy combined with systematic ex situ CVD calibrations. The morphology of the Ge-Si heterojunction, in particular, the extent of a local, asymmetric increase in nanowire diameter, is found to depend on the Ga composition of the catalyst, on the TMGa precursor exposure temperature, and on the presence of dopants. To rationalize the findings, a general nucleation-based model for nanowire heteroepitaxy is established which is anticipated to be relevant to a wide range of material systems and device-enabling heterostructures.

Time-dependent fracture toughness of cornea.

The fracture and time-dependent properties of cornea are very important for the development of corneal scaffolds and prostheses. However, there has been no systematic study of cornea fracture; time-dependent behavior of cornea has never been investigated in a fracture context. In this work, fracture toughness of cornea was characterized by trouser tear tests, and time-dependent properties of cornea were examined by stress-relaxation and uniaxial tensile tests. Control experiments were performed on a photoelastic rubber sheet. Corneal fracture resistance was found to be strain-rate dependent, with values ranging from 3.39±0.57 to 5.40±0.48kJm(-2) over strain rates from 3 to 300mmmin(-1). Results from stress-relaxation tests confirmed that cornea is a nonlinear viscoelastic material. The cornea behaved closer to a viscous fluid at small strain but became relatively more elastic at larger strain. Although cornea properties are greatly dependent on time, the stress-strain responses of cornea were found to be insensitive to the strain rate when subjected to tensile loading.

Excitation energy dependent raman signatures of ABA- and ABC-stacked few-layer graphene

The dependence of the Raman spectrum on the excitation energy has been investigated for ABA-and ABC- stacked few-layer graphene in order to establish the fingerprint of the stacking order and the number of layers, which affect the transport and optical properties of few-layer graphene. Five different excitation sources with energies of 1.96, 2.33, 2.41, 2.54 and 2.81â €...eV were used. The position and the line shape of the Raman 2D, G*, N, M, and other combination modes show dependence on the excitation energy as well as the stacking order and the thickness. One can unambiguously determine the stacking order and the thickness by comparing the 2D band spectra measured with 2 different excitation energies or by carefully comparing weaker combination Raman modes such as N, M, or LOLA modes. The criteria for unambiguous determination of the stacking order and the number of layers up to 5 layers are established.