Stochastic dynamic programming for election timing: A game theory approach

In this paper, we consider dynamic programming for the election timing in the majoritarian parliamentary system such as in Australia, where the government has a constitutional right to call an early election. This right can give the government an advantage to remain in power for as long as possible by calling an election, when its popularity is high. On the other hand, the opposition's natural objective is to gain power, and it will apply controls termed as "boosts" to reduce the chance of the government being re-elected by introducing policy and economic responses. In this paper, we explore equilibrium solutions to the government, and the opposition strategies in a political game using stochastic dynamic programming. Results are given in terms of the expected remaining life in power, call and boost probabilities at each time at any level of popularity.

Accelerated leap methods for simulating discrete stochastic chemical kinetics

Biologists are increasingly conscious of the critical role that noise plays in cellular functions such as genetic regulation, often in connection with fluctuations in small numbers of key regulatory molecules. This has inspired the development of models that capture this fundamentally discrete and stochastic nature of cellular biology - most notably the Gillespie stochastic simulation algorithm (SSA). The SSA simulates a temporally homogeneous, discrete-state, continuous-time Markov process, and of course the corresponding probabilities and numbers of each molecular species must all remain positive. While accurately serving this purpose, the SSA can be computationally inefficient due to very small time stepping so faster approximations such as the Poisson and Binomial τ-leap methods have been suggested. This work places these leap methods in the context of numerical methods for the solution of stochastic differential equations (SDEs) driven by Poisson noise. This allows analogues of Euler-Maruyuma, Milstein and even higher order methods to be developed through the Itô-Taylor expansions as well as similar derivative-free Runge-Kutta approaches. Numerical results demonstrate that these novel methods compare favourably with existing techniques for simulating biochemical reactions by more accurately capturing crucial properties such as the mean and variance than existing methods.

Approximate Bayesian techniques for inference in stochastic dynamical systems

This thesis is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variant of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here two new extended frameworks are derived and presented that are based on basis function expansions and local polynomial approximations of a recently proposed variational Bayesian algorithm. It is shown that the new extensions converge to the original variational algorithm and can be used for state estimation (smoothing). However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new methods are numerically validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein-Uhlenbeck process, for which the exact likelihood can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz '63 (3-dimensional model). The algorithms are also applied to the 40 dimensional stochastic Lorenz '96 system. In this investigation these new approaches are compared with a variety of other well known methods such as the ensemble Kalman filter / smoother, a hybrid Monte Carlo sampler, the dual unscented Kalman filter (for jointly estimating the systems states and model parameters) and full weak-constraint 4D-Var. Empirical analysis of their asymptotic behaviour as a function of observation density or length of time window increases is provided.

A comparison of variational and Markov chain Monte Carlo methods for inference in partially observed stochastic dynamic systems

In recent work we have developed a novel variational inference method for partially observed systems governed by stochastic differential equations. In this paper we provide a comparison of the Variational Gaussian Process Smoother with an exact solution computed using a Hybrid Monte Carlo approach to path sampling, applied to a stochastic double well potential model. It is demonstrated that the variational smoother provides us a very accurate estimate of mean path while conditional variance is slightly underestimated. We conclude with some remarks as to the advantages and disadvantages of the variational smoother. © 2008 Springer Science + Business Media LLC.

Estimating parameters in stochastic systems:a variational Bayesian approach

This work is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variation of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here a new extended framework is derived that is based on a local polynomial approximation of a recently proposed variational Bayesian algorithm. The paper begins by showing that the new extension of this variational algorithm can be used for state estimation (smoothing) and converges to the original algorithm. However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new approach is validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein–Uhlenbeck process, the exact likelihood of which can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz ’63 (3D model). As a special case the algorithm is also applied to the 40 dimensional stochastic Lorenz ’96 system. In our investigation we compare this new approach with a variety of other well known methods, such as the hybrid Monte Carlo, dual unscented Kalman filter, full weak-constraint 4D-Var algorithm and analyse empirically their asymptotic behaviour as a function of observation density or length of time window increases. In particular we show that we are able to estimate parameters in both the drift (deterministic) and the diffusion (stochastic) part of the model evolution equations using our new methods.

Stochastic anti-resonance in a fibre Raman amplifier

Stochastic anti-resonance, that is resonant enhancement of randomness caused by polarization mode beatings, is analyzed both numerically and analytically on an example of fibre Raman amplifier with randomly varying birefringence. As a result of such anti-resonance, the polarization mode dispersion growth causes an escape of the signal state of polarization from a metastable state corresponding to the pulling of the signal to the pump state of polarization.This phenomenon reveals itself in abrupt growth of gain fluctuations as well as in dropping of Hurst parameter and Kramers length characterizing long memory in a system and noise induced escape from the polarization pulling state. The results based on analytical multiscale averaging technique agree perfectly with the numerical data obtained by direct numerical simulations of underlying stochastic differential equations. This challenging outcome would allow replacing the cumbersome numerical simulations for real-world extra-long high-speed communication systems.

Quantum criticality

We investigate the theory of quantum fluctuations in non-equilibrium systems having large critical fluctuations. This allows us to treat the limits imposed by nonlinearities to quantum squeezing and noise reduction, and also to envisage future tests of quantum theory in regions of macroscopic quantum fluctuations. A long-term objective of this research is to identify suitable physical systems in which macroscopic 'Schrodinger cat'-like behaviour may be observed. We investigate two systems in particular of much current experimental interest, namely the degenerate parametric oscillator near threshold, and the evaporatively cooled (BEC). We compare the results obtained in the positive-P representation, as a fully quantum mechanical calculation, with the truncated Wigner phase space equation, also known as semi-classical theory. We show when these results agree and differ in calculations taken beyond the linearized approximation. In the region where the largest quantum fluctuations and Schrodinger cat-like behaviour might be expected, we find that the quantum predictions correspond very closely to the semi-classical theory. Nature abhors observing a Schrodinger car.

Atom lasers, coherent states, and coherence. I. Physically realizable ensembles of pure states

A laser, be it an optical laser or an atom laser, is an open quantum system that produces a coherent beam of bosons (photons or atoms, respectively). Far above threshold, the stationary state rho(ss) of the laser mode is a mixture of coherent-field states with random phase, or, equivalently, a Poissonian mixture of number states. This paper answers the question: can descriptions such as these, of rho(ss) as a stationary ensemble of pure states, be physically realized? Here physical realization is as defined previously by us [H. M. Wiseman and J. A. Vaccaro, Phys. Lett. A 250, 241 (1998)]: an ensemble of pure states for a particular system can be physically realized if, without changing the dynamics of the system, an experimenter can (in principle) know at any time that the system is in one of the pure-state members of the ensemble. Such knowledge can be obtained by monitoring the baths to which the system is coupled, provided that coupling is describable by a Markovian master equation. Using a family of master equations for the (atom) laser, we solve for the physically realizable (PR) ensembles. We find that for any finite self-energy chi of the bosons in the laser mode, the coherent-state ensemble is not PR; the closest one can come to it is an ensemble of squeezed states. This is particularly relevant for atom lasers, where the self-energy arising from elastic collisions is expected to be large. By contrast, the number-state ensemble is always PR. As the self-energy chi increases, the states in the PR ensemble closest to the coherent-state ensemble become increasingly squeezed. Nevertheless, there are values of chi for which states with well-defined coherent amplitudes are PR, even though the atom laser is not coherent (in the sense of having a Bose-degenerate output). We discuss the physical significance of this anomaly in terms of conditional coherence (and hence conditional Bose degeneracy).

Critical quantum fluctuations in the degenerate parametric oscillator

We develop a systematic theory of critical quantum fluctuations in the driven parametric oscillator. Our analytic results agree well with stochastic numerical simulations. We also compare the results obtained in the positive-P representation, as a fully quantum-mechanical calculation, with the truncated Wigner phase-space equation, also known as the semiclassical theory. We show when these results agree and differ in calculations taken beyond the linearized approximation. We find that the optimal broadband noise reduction occurs just above threshold. In this region where there are large quantum fluctuations in the conjugate variance and macroscopic quantum superposition states might be expected, we find that the quantum predictions correspond very closely to the semiclassical theory.

Limits to squeezing in the degenerate optical parametric oscillator

We develop a systematic theory of quantum fluctuations in the driven optical parametric oscillator, including the region near threshold. This allows us to treat the limits imposed by nonlinearities to quantum squeezing and noise reduction in this nonequilibrium quantum phase transition. In particular, we compute the squeezing spectrum near threshold and calculate the optimum value. We find that the optimal noise reduction occurs at different driving fields, depending on the ratio of damping rates. The largest spectral noise reductions are predicted to occur with a very high-Q second-harmonic cavity. Our analytic results agree well with stochastic numerical simulations. We also compare the results obtained in the positive-P representation, as a fully quantum-mechanical calculation, with the truncated Wigner phase-space equation, also known as the semiclassical theory.

Bayesian feedback versus Markovian feedback in a two-level atom

We compare two different approaches to the control of the dynamics of a continuously monitored open quantum system. The first is Markovian feedback, as introduced in quantum optics by Wiseman and Milburn [Phys. Rev. Lett. 70, 548 (1993)]. The second is feedback based on an estimate of the system state, developed recently by Doherty and Jacobs [Phys. Rev. A 60, 2700 (1999)]. Here we choose to call it, for brevity, Bayesian feedback. For systems with nonlinear dynamics, we expect these two methods of feedback control to give markedly different results. The simplest possible nonlinear system is a driven and damped two-level atom, so we choose this as our model system. The monitoring is taken to be homodyne detection of the atomic fluorescence, and the control is by modulating the driving. The aim of the feedback in both cases is to stabilize the internal state of the atom as close as possible to an arbitrarily chosen pure state, in the presence of inefficient detection and other forms of decoherence. Our results (obtained without recourse to stochastic simulations) prove that Bayesian feedback is never inferior, and is usually superior, to Markovian feedback. However, it would be far more difficult to implement than Markovian feedback and it loses its superiority when obvious simplifying approximations are made. It is thus not clear which form of feedback would be better in the face of inevitable experimental imperfections.

Dynamic programming for a Markov-switching jump–diffusion

We consider an optimal control problem with a deterministic finite horizon and state variable dynamics given by a Markov-switching jump–diffusion stochastic differential equation. Our main results extend the dynamic programming technique to this larger family of stochastic optimal control problems. More specifically, we provide a detailed proof of Bellman’s optimality principle (or dynamic programming principle) and obtain the corresponding Hamilton–Jacobi–Belman equation, which turns out to be a partial integro-differential equation due to the extra terms arising from the Lévy process and the Markov process. As an application of our results, we study a finite horizon consumption– investment problem for a jump–diffusion financial market consisting of one risk-free asset and one risky asset whose coefficients are assumed to depend on the state of a continuous time finite state Markov process. We provide a detailed study of the optimal strategies for this problem, for the economically relevant families of power utilities and logarithmic utilities.

Lp-solutions to BSDEs with super-linear growth coefficient. Application to degenerate semilinear PDEs.

We consider multidimensional backward stochastic differential equations (BSDEs). We prove the existence and uniqueness of solutions when the coefficient grow super-linearly, and moreover, can be neither locally Lipschitz in the variable y nor in the variable z. This is done with super-linear growth coefficient and a p-integrable terminal condition (p & 1). As application, we establish the existence and uniqueness of solutions to degenerate semilinear PDEs with superlinear growth generator and an Lp-terminal data, p & 1. Our result cover, for instance, the case of PDEs with logarithmic nonlinearities.

Large deviation for BSDE with subdifferential operator

In this paper we prove that the solution of a backward stochastic differential equation, which involves a subdifferential operator and associated to a family of reflecting diffusion processes, converges to the solution of a deterministic backward equation and satisfes a large deviation principle.

A decision-making Fokker-Planck model in computational neuroscience

Minimal models for the explanation of decision-making in computational neuroscience are based on the analysis of the evolution for the average firing rates of two interacting neuron populations. While these models typically lead to multi-stable scenario for the basic derived dynamical systems, noise is an important feature of the model taking into account finite-size effects and robustness of the decisions. These stochastic dynamical systems can be analyzed by studying carefully their associated Fokker-Planck partial differential equation. In particular, we discuss the existence, positivity and uniqueness for the solution of the stationary equation, as well as for the time evolving problem. Moreover, we prove convergence of the solution to the the stationary state representing the probability distribution of finding the neuron families in each of the decision states characterized by their average firing rates. Finally, we propose a numerical scheme allowing for simulations performed on the Fokker-Planck equation which are in agreement with those obtained recently by a moment method applied to the stochastic differential system. Our approach leads to a more detailed analytical and numerical study of this decision-making model in computational neuroscience.