Risk-sensitivity and the mean-variance trade-off: decision making in sensorimotor control.

Numerous psychophysical studies suggest that the sensorimotor system chooses actions that optimize the average cost associated with a movement. Recently, however, violations of this hypothesis have been reported in line with economic theories of decision-making that not only consider the mean payoff, but are also sensitive to risk, that is the variability of the payoff. Here, we examine the hypothesis that risk-sensitivity in sensorimotor control arises as a mean-variance trade-off in movement costs. We designed a motor task in which participants could choose between a sure motor action that resulted in a fixed amount of effort and a risky motor action that resulted in a variable amount of effort that could be either lower or higher than the fixed effort. By changing the mean effort of the risky action while experimentally fixing its variance, we determined indifference points at which participants chose equiprobably between the sure, fixed amount of effort option and the risky, variable effort option. Depending on whether participants accepted a variable effort with a mean that was higher, lower or equal to the fixed effort, they could be classified as risk-seeking, risk-averse or risk-neutral. Most subjects were risk-sensitive in our task consistent with a mean-variance trade-off in effort, thereby, underlining the importance of risk-sensitivity in computational models of sensorimotor control.

Variance prediction in sea

A method is presented for predicting the variance of the energy levels in a built-up system to accompany the mean values predicted by SEA. Closed form expressions for the variance are obtained in terms of the standard SEA parameters and an additional set of parameters αk that describe the nature of the power input to each subsystem k, and αks that describe the nature of the coupling between subsystems k and s.

Effects of Lewis number on scalar variance transport in premixed flames

The influences of differential diffusion rates of heat and mass on the transport of the variances of Favre fluctuations of reaction progress variable and non-dimensional temperature have been studied using three-dimensional simplified chemistry based Direct Numerical Simulation (DNS) data of statistically planar turbulent premixed flames with global Lewis number ranging from Le = 0.34 to 1.2. The Lewis number effects on the statistical behaviours of the various terms of the transport equations of variances of Favre fluctuations of reaction progress variable and non-dimensional temperature have been analysed in the context of Reynolds Averaged Navier Stokes (RANS) simulations. It has been found that the turbulent fluxes of the progress variable and temperature variances exhibit counter-gradient transport for the flames with Lewis number significantly smaller than unity whereas the extent of this counter-gradient transport is found to decrease with increasing Lewis number. The Lewis number is also shown to have significant influences on the magnitudes of the chemical reaction and scalar dissipation rate contributions to the scalar variance transport. The modelling of the unclosed terms in the scalar variance equations for the non-unity Lewis number flames have been discussed in detail. The performances of the existing models for the unclosed terms are assessed based on a-priori analysis of DNS data. Based on the present analysis, new models for the unclosed terms of the active scalar variance transport equations are proposed, whenever necessary, which are shown to satisfactorily capture the behaviours of unclosed terms for all the flames considered in this study. © 2010 Springer Science+Business Media B.V.

Modal scattering at an impedance transition in a lined flow duct

An explicit Wiener-Hopf solution is derived to describe the scattering of duct modes at a hard-soft wall impedance transition in a circular duct with uniform mean flow. Specifically, we have a circular duct r = 1, - ∞ < x < ∞ with mean flow Mach number M > 0 and a hard wall along x < 0 and a wall of impedance Z along x > 0. A minimum edge condition at x = 0 requires a continuous wall streamline r = 1 + h(x, t), no more singular than h = Ο(x1/2) for x ↓ 0. A mode, incident from x < 0, scatters at x = 0 into a series of reflected modes and a series of transmitted modes. Of particular interest is the role of a possible instability along the lined wall in combination with the edge singularity. If one of the "upstream" running modes is to be interpreted as a downstream-running instability, we have an extra degree of freedom in the Wiener-Hopf analysis that can be resolved by application of some form of Kutta condition at x = 0, for example a more stringent edge condition where h = Ο(x3/2) at the downstream side. The question of the instability requires an investigation of the modes in the complex frequency plane and therefore depends on the chosen impedance model, since Z = Z (ω) is essentially frequency dependent. The usual causality condition by Briggs and Bers appears to be not applicable here because it requires a temporal growth rate bounded for all real axial wave numbers. The alternative Crighton-Leppington criterion, however, is applicable and confirms that the suspected mode is usually unstable. In general, the effect of this Kutta condition is significant, but it is particularly large for the plane wave at low frequencies and should therefore be easily measurable. For ω → 0, the modulus fends to |R001| → (1 + M)/(1 -M) without and to 1 with Kutta condition, while the end correction tends to ∞ without and to a finite value with Kutta condition. This is exactly the same behaviour as found for reflection at a pipe exit with flow, irrespective if this is uniform or jet flow.

Sequential decision making in general state space models

Many problems in control and signal processing can be formulated as sequential decision problems for general state space models. However, except for some simple models one cannot obtain analytical solutions and has to resort to approximation. In this thesis, we have investigated problems where Sequential Monte Carlo (SMC) methods can be combined with a gradient based search to provide solutions to online optimisation problems. We summarise the main contributions of the thesis as follows. Chapter 4 focuses on solving the sensor scheduling problem when cast as a controlled Hidden Markov Model. We consider the case in which the state, observation and action spaces are continuous. This general case is important as it is the natural framework for many applications. In sensor scheduling, our aim is to minimise the variance of the estimation error of the hidden state with respect to the action sequence. We present a novel SMC method that uses a stochastic gradient algorithm to find optimal actions. This is in contrast to existing works in the literature that only solve approximations to the original problem. In Chapter 5 we presented how an SMC can be used to solve a risk sensitive control problem. We adopt the use of the Feynman-Kac representation of a controlled Markov chain flow and exploit the properties of the logarithmic Lyapunov exponent, which lead to a policy gradient solution for the parameterised problem. The resulting SMC algorithm follows a similar structure with the Recursive Maximum Likelihood(RML) algorithm for online parameter estimation. In Chapters 6, 7 and 8, dynamic Graphical models were combined with with state space models for the purpose of online decentralised inference. We have concentrated more on the distributed parameter estimation problem using two Maximum Likelihood techniques, namely Recursive Maximum Likelihood (RML) and Expectation Maximization (EM). The resulting algorithms can be interpreted as an extension of the Belief Propagation (BP) algorithm to compute likelihood gradients. In order to design an SMC algorithm, in Chapter 8 uses a nonparametric approximations for Belief Propagation. The algorithms were successfully applied to solve the sensor localisation problem for sensor networks of small and medium size.

Sequential Monte Carlo computation of the score and observed information matrix in state-space models with application to parameter estimation

Sequential Monte Carlo (SMC) methods are popular computational tools for Bayesian inference in non-linear non-Gaussian state-space models. For this class of models, we propose SMC algorithms to compute the score vector and observed information matrix recursively in time. We propose two different SMC implementations, one with computational complexity $\mathcal{O}(N)$ and the other with complexity $\mathcal{O}(N^{2})$ where $N$ is the number of importance sampling draws. Although cheaper, the performance of the $\mathcal{O}(N)$ method degrades quickly in time as it inherently relies on the SMC approximation of a sequence of probability distributions whose dimension is increasing linearly with time. In particular, even under strong \textit{mixing} assumptions, the variance of the estimates computed with the $\mathcal{O}(N)$ method increases at least quadratically in time. The $\mathcal{O}(N^{2})$ is a non-standard SMC implementation that does not suffer from this rapid degrade. We then show how both methods can be used to perform batch and recursive parameter estimation.