Temporal difference models describe higher-order learning in humans.

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.

Adaptive combustion instability control with saturation

The control of a class of combustion systems, suceptible to damage from self-excited combustion oscillations, is considered. An adaptive stable controller, called Self-Tuning Regulator (STR), has recently been developed, which meets the apparently contradictory challenge of relying as little as possible on a particular combustion model while providing some guarantee that the controller will cause no harm. The controller injects some fuel unsteadily into the burning region, thereby altering the heat release, in response to an input signal detecting the oscillation. This paper focuses on an extension of the STR design, when, due to stringent emission requirements and to the danger of flame extension, the amount of fuel used for control is limited in amplitude. A Lyapunov stability analysis is used to prove the stability of the modified STR when the saturation constraint is imposed. The practical implementation of the modified STR remains straightforward, and simulation results, based on the nonlinear premixed flame model developed by Dowling, show that in the presence of a saturation constraint, the self-excited oscillations are damped more rapidly with the modified STR than with the original STR. © 2001 by S. Evesque. Published by the American Institute of Aeronautics and Astronautics, Inc.