Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans.

Theories of instrumental learning are centred on understanding how success and failure are used to improve future decisions. These theories highlight a central role for reward prediction errors in updating the values associated with available actions. In animals, substantial evidence indicates that the neurotransmitter dopamine might have a key function in this type of learning, through its ability to modulate cortico-striatal synaptic efficacy. However, no direct evidence links dopamine, striatal activity and behavioural choice in humans. Here we show that, during instrumental learning, the magnitude of reward prediction error expressed in the striatum is modulated by the administration of drugs enhancing (3,4-dihydroxy-L-phenylalanine; L-DOPA) or reducing (haloperidol) dopaminergic function. Accordingly, subjects treated with L-DOPA have a greater propensity to choose the most rewarding action relative to subjects treated with haloperidol. Furthermore, incorporating the magnitude of the prediction errors into a standard action-value learning algorithm accurately reproduced subjects' behavioural choices under the different drug conditions. We conclude that dopamine-dependent modulation of striatal activity can account for how the human brain uses reward prediction errors to improve future decisions.

Frames, biases, and rational decision-making in the human brain.

Human choices are remarkably susceptible to the manner in which options are presented. This so-called "framing effect" represents a striking violation of standard economic accounts of human rationality, although its underlying neurobiology is not understood. We found that the framing effect was specifically associated with amygdala activity, suggesting a key role for an emotional system in mediating decision biases. Moreover, across individuals, orbital and medial prefrontal cortex activity predicted a reduced susceptibility to the framing effect. This finding highlights the importance of incorporating emotional processes within models of human choice and suggests how the brain may modulate the effect of these biasing influences to approximate rationality.

Sensing through body dynamics

It has been shown that sensory morphology and sensory-motor coordination enhance the capabilities of sensing in robotic systems. The tasks of categorization and category learning, for example, can be significantly simplified by exploiting the morphological constraints, sensory-motor couplings and the interaction with the environment. This paper argues that, in the context of sensory-motor control, it is essential to consider body dynamics derived from morphological properties and the interaction with the environment in order to gain additional insight into the underlying mechanisms of sensory-motor coordination, and more generally the nature of perception. A locomotion model of a four-legged robot is used for the case studies in both simulation and real world. The locomotion model demonstrates how attractor states derived from body dynamics influence the sensory information, which can then be used for the recognition of stable behavioral patterns and of physical properties in the environment. A comprehensive analysis of behavior and sensory information leads to a deeper understanding of the underlying mechanisms by which body dynamics can be exploited for category learning of autonomous robotic systems. © 2006 Elsevier Ltd. All rights reserved.