Model-based influences on humans' choices and striatal prediction errors.

The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors, and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making.

Neural activity associated with the passive prediction of ambiguity and risk for aversive events.

In economic decision making, outcomes are described in terms of risk (uncertain outcomes with certain probabilities) and ambiguity (uncertain outcomes with uncertain probabilities). Humans are more averse to ambiguity than to risk, with a distinct neural system suggested as mediating this effect. However, there has been no clear disambiguation of activity related to decisions themselves from perceptual processing of ambiguity. In a functional magnetic resonance imaging (fMRI) experiment, we contrasted ambiguity, defined as a lack of information about outcome probabilities, to risk, where outcome probabilities are known, or ignorance, where outcomes are completely unknown and unknowable. We modified previously learned pavlovian CS+ stimuli such that they became an ambiguous cue and contrasted evoked brain activity both with an unmodified predictive CS+ (risky cue), and a cue that conveyed no information about outcome probabilities (ignorance cue). Compared with risk, ambiguous cues elicited activity in posterior inferior frontal gyrus and posterior parietal cortex during outcome anticipation. Furthermore, a similar set of regions was activated when ambiguous cues were compared with ignorance cues. Thus, regions previously shown to be engaged by decisions about ambiguous rewarding outcomes are also engaged by ambiguous outcome prediction in the context of aversive outcomes. Moreover, activation in these regions was seen even when no actual decision is made. Our findings suggest that these regions subserve a general function of contextual analysis when search for hidden information during outcome anticipation is both necessary and meaningful.

Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans.

Theories of instrumental learning are centred on understanding how success and failure are used to improve future decisions. These theories highlight a central role for reward prediction errors in updating the values associated with available actions. In animals, substantial evidence indicates that the neurotransmitter dopamine might have a key function in this type of learning, through its ability to modulate cortico-striatal synaptic efficacy. However, no direct evidence links dopamine, striatal activity and behavioural choice in humans. Here we show that, during instrumental learning, the magnitude of reward prediction error expressed in the striatum is modulated by the administration of drugs enhancing (3,4-dihydroxy-L-phenylalanine; L-DOPA) or reducing (haloperidol) dopaminergic function. Accordingly, subjects treated with L-DOPA have a greater propensity to choose the most rewarding action relative to subjects treated with haloperidol. Furthermore, incorporating the magnitude of the prediction errors into a standard action-value learning algorithm accurately reproduced subjects' behavioural choices under the different drug conditions. We conclude that dopamine-dependent modulation of striatal activity can account for how the human brain uses reward prediction errors to improve future decisions.

Using sub-word-level information for confidence estimation with conditional random field models

The task of word-level confidence estimation (CE) for automatic speech recognition (ASR) systems stands to benefit from the combination of suitably defined input features from multiple information sources. However, the information sources of interest may not necessarily operate at the same level of granularity as the underlying ASR system. The research described here builds on previous work on confidence estimation for ASR systems using features extracted from word-level recognition lattices, by incorporating information at the sub-word level. Furthermore, the use of Conditional Random Fields (CRFs) with hidden states is investigated as a technique to combine information for word-level CE. Performance improvements are shown using the sub-word-level information in linear-chain CRFs with appropriately engineered feature functions, as well as when applying the hidden-state CRF model at the word level.

Change prediction using interface data

Managing change can be challenging due to the high levels of interdependency in concurrent engineering processes. A key activity in engineering change management is propagation analysis, which can be supported using the change prediction method. In common with most other change prediction approaches, the change prediction method has three important limitations: L1: it depends on highly subjective input data; L2: it is capable of modelling 'generalised cases' only and cannot be; customised to assess specific changes; and L3: the input data are static, and thus, guidance does not reflect changes in the design. This article contributes to resolving these limitations by incorporating interface information into the change prediction method. The enhanced method is illustrated using an example based on a flight simulator. © The Author(s) 2013.

Bayesian Structured Prediction Using Gaussian Processes

We introduce a conceptually novel structured prediction model, GPstruct, which is kernelized, non-parametric and Bayesian, by design. We motivate the model with respect to existing approaches, among others, conditional random fields (CRFs), maximum margin Markov networks (M3N), and structured support vector machines (SVMstruct), which embody only a subset of its properties. We present an inference procedure based on Markov Chain Monte Carlo. The framework can be instantiated for a wide range of structured objects such as linear chains, trees, grids, and other general graphs. As a proof of concept, the model is benchmarked on several natural language processing tasks and a video gesture segmentation task involving a linear chain structure. We show prediction accuracies for GPstruct which are comparable to or exceeding those of CRFs and SVMstruct.

Vibration from railways: Can we achieve better than +/-10dB prediction accuracy?

This paper examines the sources of uncertainly in models used to predict vibration from underground railways. It will become clear from this presentation that by varying parameters by a small amount, consistent with uncertainties in measured data, the predicted vibration levels vary significantly, often by more than 10dB. This error cannot be forecast. Small changes made to soil parameters (Compressive and Shear Wave velocities and density), to slab bending stiffness and mass and to the measurement position give rise to changes in vibration levels of more than lOdB. So if 10dB prediction error results from small uncertainties in soil parameters and measurement position it cannot be sensible to rely on prediction models for accuracy better than 10dB. The presentation will demonstrate in real time the use of the new - and freely-available - PiP software for calculating vibration from railway tunnels in real time.

A single-Stream jet noise prediction model for a family of chevron nozzles

This study develops a single-stream jet noise prediction model for a family of chevron nozzles. An original equation is proposed for the fourth-order space-time cross-correlations. They are expressed in flow parameters such as streamwise circulation and turbulent kinetic energy. The cross-correlations based on a Reynolds Averaged Navier-Stokes (RANS) flowfield showed a good agreement with those based on a Large Eddy Simulation (LES) flowfield. This proves that the proposed equation describes the cross-correlations accurately. With this novel source description, there is an excellent agreement between our model's far-field noise predictions and measurements1 for a wide range of frequencies and radiation angles. Our model captures the spectral shape, amplitude and peak frequency very well. This establishes that our model holds good for a family of chevron nozzles. As our model provides quick and accurate predictions, a parametric study was performed to understand the effects of a chevron nozzle geometry on jet noise and thrust loss. Chevron penetration is the underpinning factor for jet noise reduction. The reduction of jet noise per unit thrust loss decreases linearly with chevron penetration. The number of chevrons also has a considerable effect on jet noise.

A coupled hydro-mechanical analysis for prediction of hydraulic fracture propagation in saturated porous media using EFG mesh-less method

The details of the Element Free Galerkin (EFG) method are presented with the method being applied to a study on hydraulic fracturing initiation and propagation process in a saturated porous medium using coupled hydro-mechanical numerical modelling. In this EFG method, interpolation (approximation) is based on nodes without using elements and hence an arbitrary discrete fracture path can be modelled.The numerical approach is based upon solving two governing partial differential equations of equilibrium and continuity of pore water simultaneously. Displacement increment and pore water pressure increment are discretized using the same EFG shape functions. An incremental constrained Galerkin weak form is used to create the discrete system of equations and a fully implicit scheme is used for discretization in the time domain. Implementation of essential boundary conditions is based on the penalty method. In order to model discrete fractures, the so-called diffraction method is used.Examples are presented and the results are compared to some closed-form solutions and FEM approximations in order to demonstrate the validity of the developed model and its capabilities. The model is able to take the anisotropy and inhomogeneity of the material into account. The applicability of the model is examined by simulating hydraulic fracture initiation and propagation process from a borehole by injection of fluid. The maximum tensile strength criterion and Mohr-Coulomb shear criterion are used for modelling tensile and shear fracture, respectively. The model successfully simulates the leak-off of fluid from the fracture into the surrounding material. The results indicate the importance of pore fluid pressure in the initiation and propagation pattern of fracture in saturated soils. © 2013 Elsevier Ltd.

Functional magnetic resonance imaging of impaired sensory prediction in schizophrenia.

IMPORTANCE: Forward models predict the sensory consequences of planned actions and permit discrimination of self- and non-self-elicited sensation; their impairment in schizophrenia is implied by an abnormality in behavioral force-matching and the flawed agency judgments characteristic of positive symptoms, including auditory hallucinations and delusions of control. OBJECTIVE: To assess attenuation of sensory processing by self-action in individuals with schizophrenia and its relation to current symptom severity. DESIGN, SETTING, AND PARTICIPANTS: Functional magnetic resonance imaging data were acquired while medicated individuals with schizophrenia (n = 19) and matched controls (n = 19) performed a factorially designed sensorimotor task in which the occurrence and relative timing of action and sensation were manipulated. The study took place at the neuroimaging research unit at the Institute of Cognitive Neuroscience, University College London, and the Maudsley Hospital. RESULTS: In controls, a region of secondary somatosensory cortex exhibited attenuated activation when sensation and action were synchronous compared with when the former occurred after an unexpected delay or alone. By contrast, reduced attenuation was observed in the schizophrenia group, suggesting that these individuals were unable to predict the sensory consequences of their own actions. Furthermore, failure to attenuate secondary somatosensory cortex processing was predicted by current hallucinatory severity. CONCLUSIONS AND RELEVANCE: Although comparably reduced attenuation has been reported in the verbal domain, this work implies that a more general physiologic deficit underlies positive symptoms of schizophrenia.