Variational mean-field algorithm for efficient inference in large systems of stochastic differential equations

This work introduces a Gaussian variational mean-field approximation for inference in dynamical systems which can be modeled by ordinary stochastic differential equations. This new approach allows one to express the variational free energy as a functional of the marginal moments of the approximating Gaussian process. A restriction of the moment equations to piecewise polynomial functions, over time, dramatically reduces the complexity of approximate inference for stochastic differential equation models and makes it comparable to that of discrete time hidden Markov models. The algorithm is demonstrated on state and parameter estimation for nonlinear problems with up to 1000 dimensional state vectors and compares the results empirically with various well-known inference methodologies.

Gaussian process approximations of stochastic differential equations

Stochastic differential equations arise naturally in a range of contexts, from financial to environmental modeling. Current solution methods are limited in their representation of the posterior process in the presence of data. In this work, we present a novel Gaussian process approximation to the posterior measure over paths for a general class of stochastic differential equations in the presence of observations. The method is applied to two simple problems: the Ornstein-Uhlenbeck process, of which the exact solution is known and can be compared to, and the double-well system, for which standard approaches such as the ensemble Kalman smoother fail to provide a satisfactory result. Experiments show that our variational approximation is viable and that the results are very promising as the variational approximate solution outperforms standard Gaussian process regression for non-Gaussian Markov processes.

Quasi-separation of the biharmonic partial differential equation

In this paper, we consider analytical and numerical solutions to the Dirichlet boundary-value problem for the biharmonic partial differential equation on a disc of finite radius in the plane. The physical interpretation of these solutions is that of the harmonic oscillations of a thin, clamped plate. For the linear, fourth-order, biharmonic partial differential equation in the plane, it is well known that the solution method of separation in polar coordinates is not possible, in general. However, in this paper, for circular domains in the plane, it is shown that a method, here called quasi-separation of variables, does lead to solutions of the partial differential equation. These solutions are products of solutions of two ordinary linear differential equations: a fourth-order radial equation and a second-order angular differential equation. To be expected, without complete separation of the polar variables, there is some restriction on the range of these solutions in comparison with the corresponding separated solutions of the second-order harmonic differential equation in the plane. Notwithstanding these restrictions, the quasi-separation method leads to solutions of the Dirichlet boundary-value problem on a disc with centre at the origin, with boundary conditions determined by the solution and its inward drawn normal taking the value 0 on the edge of the disc. One significant feature for these biharmonic boundary-value problems, in general, follows from the form of the biharmonic differential expression when represented in polar coordinates. In this form, the differential expression has a singularity at the origin, in the radial variable. This singularity translates to a singularity at the origin of the fourth-order radial separated equation; this singularity necessitates the application of a third boundary condition in order to determine a self-adjoint solution to the Dirichlet boundary-value problem. The penultimate section of the paper reports on numerical solutions to the Dirichlet boundary-value problem; these results are also presented graphically. Two specific cases are studied in detail and numerical values of the eigenvalues are compared with the results obtained in earlier studies.

Approximate Bayesian techniques for inference in stochastic dynamical systems

This thesis is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variant of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here two new extended frameworks are derived and presented that are based on basis function expansions and local polynomial approximations of a recently proposed variational Bayesian algorithm. It is shown that the new extensions converge to the original variational algorithm and can be used for state estimation (smoothing). However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new methods are numerically validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein-Uhlenbeck process, for which the exact likelihood can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz '63 (3-dimensional model). The algorithms are also applied to the 40 dimensional stochastic Lorenz '96 system. In this investigation these new approaches are compared with a variety of other well known methods such as the ensemble Kalman filter / smoother, a hybrid Monte Carlo sampler, the dual unscented Kalman filter (for jointly estimating the systems states and model parameters) and full weak-constraint 4D-Var. Empirical analysis of their asymptotic behaviour as a function of observation density or length of time window increases is provided.

A comparison of variational and Markov chain Monte Carlo methods for inference in partially observed stochastic dynamic systems

In recent work we have developed a novel variational inference method for partially observed systems governed by stochastic differential equations. In this paper we provide a comparison of the Variational Gaussian Process Smoother with an exact solution computed using a Hybrid Monte Carlo approach to path sampling, applied to a stochastic double well potential model. It is demonstrated that the variational smoother provides us a very accurate estimate of mean path while conditional variance is slightly underestimated. We conclude with some remarks as to the advantages and disadvantages of the variational smoother. © 2008 Springer Science + Business Media LLC.

Estimating parameters in stochastic systems:a variational Bayesian approach

This work is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variation of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here a new extended framework is derived that is based on a local polynomial approximation of a recently proposed variational Bayesian algorithm. The paper begins by showing that the new extension of this variational algorithm can be used for state estimation (smoothing) and converges to the original algorithm. However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new approach is validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein–Uhlenbeck process, the exact likelihood of which can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz ’63 (3D model). As a special case the algorithm is also applied to the 40 dimensional stochastic Lorenz ’96 system. In our investigation we compare this new approach with a variety of other well known methods, such as the hybrid Monte Carlo, dual unscented Kalman filter, full weak-constraint 4D-Var algorithm and analyse empirically their asymptotic behaviour as a function of observation density or length of time window increases. In particular we show that we are able to estimate parameters in both the drift (deterministic) and the diffusion (stochastic) part of the model evolution equations using our new methods.

Stochastic anti-resonance in a fibre Raman amplifier

Stochastic anti-resonance, that is resonant enhancement of randomness caused by polarization mode beatings, is analyzed both numerically and analytically on an example of fibre Raman amplifier with randomly varying birefringence. As a result of such anti-resonance, the polarization mode dispersion growth causes an escape of the signal state of polarization from a metastable state corresponding to the pulling of the signal to the pump state of polarization.This phenomenon reveals itself in abrupt growth of gain fluctuations as well as in dropping of Hurst parameter and Kramers length characterizing long memory in a system and noise induced escape from the polarization pulling state. The results based on analytical multiscale averaging technique agree perfectly with the numerical data obtained by direct numerical simulations of underlying stochastic differential equations. This challenging outcome would allow replacing the cumbersome numerical simulations for real-world extra-long high-speed communication systems.

Localized waves in optical systems with periodic dispersion and nonlinearity management

We overview our recent developments in the theory of dispersion-managed (DM) solitons within the context of optical applications. First, we present a class of localized solutions with a period multiple to that of the standard DM soliton in the nonlinear Schrödinger equation with periodic variations of the dispersion. In the framework of a reduced ordinary differential equation-based model, we discuss the key features of these structures, such as a smaller energy compared to traditional DM solitons with the same temporal width. Next, we present new results on dissipative DM solitons, which occur in the context of mode-locked lasers. By means of numerical simulations and a reduced variational model of the complex Ginzburg-Landau equation, we analyze the influence of the different dissipative processes that take place in a laser.

On the theory of self-similar parabolic optical pulses

Self-similar optical pulses (or “similaritons”) of parabolic intensity profile can be found as asymptotic solutions of the nonlinear Schr¨odinger equation in a gain medium such as a fiber amplifier or laser resonator. These solutions represent a wide-ranging significance example of dissipative nonlinear structures in optics. Here, we address some issues related to the formation and evolution of parabolic pulses in a fiber gain medium by means of semi-analytic approaches. In particular, the effect of the third-order dispersion on the structure of the asymptotic solution is examined. Our analysis is based on the resolution of ordinary differential equations, which enable us to describe the main properties of the pulse propagation and structural characteristics observable through direct numerical simulations of the basic partial differential equation model with sufficient accuracy.

On the theory of self-similar parabolic optical pulses

Self-similar optical pulses (or “similaritons”) of parabolic intensity profile can be found as asymptotic solutions of the nonlinear Schr¨odinger equation in a gain medium such as a fiber amplifier or laser resonator. These solutions represent a wide-ranging significance example of dissipative nonlinear structures in optics. Here, we address some issues related to the formation and evolution of parabolic pulses in a fiber gain medium by means of semi-analytic approaches. In particular, the effect of the third-order dispersion on the structure of the asymptotic solution is examined. Our analysis is based on the resolution of ordinary differential equations, which enable us to describe the main properties of the pulse propagation and structural characteristics observable through direct numerical simulations of the basic partial differential equation model with sufficient accuracy.

Function interval arithmetic

We propose an arithmetic of function intervals as a basis for convenient rigorous numerical computation. Function intervals can be used as mathematical objects in their own right or as enclosures of functions over the reals. We present two areas of application of function interval arithmetic and associated software that implements the arithmetic: (1) Validated ordinary differential equation solving using the AERN library and within the Acumen hybrid system modeling tool. (2) Numerical theorem proving using the PolyPaver prover. © 2014 Springer-Verlag.

Localized waves in optical systems with periodic dispersion and nonlinearity management

We overview our recent developments in the theory of dispersion-managed (DM) solitons within the context of optical applications. First, we present a class of localized solutions with a period multiple to that of the standard DM soliton in the nonlinear Schrödinger equation with periodic variations of the dispersion. In the framework of a reduced ordinary differential equation-based model, we discuss the key features of these structures, such as a smaller energy compared to traditional DM solitons with the same temporal width. Next, we present new results on dissipative DM solitons, which occur in the context of mode-locked lasers. By means of numerical simulations and a reduced variational model of the complex Ginzburg-Landau equation, we analyze the influence of the different dissipative processes that take place in a laser.

Coherence and incoherence in an optical comb

We demonstrate a coexistence of coherent and incoherent modes in the optical comb generated by a passively mode-locked quantum dot laser. This is experimentally achieved by means of optical linewidth, radio frequency spectrum, and optical spectrum measurements and confirmed numerically by a delay-differential equation model showing excellent agreement with the experiment. We interpret the state as a chimera state. © 2014 American Physical Society.

Weak form equation–based finite-element modeling of viscoelastic asphalt mixtures

The objective of this study is to demonstrate using weak form partial differential equation (PDE) method for a finite-element (FE) modeling of a new constitutive relation without the need of user subroutine programming. The viscoelastic asphalt mixtures were modeled by the weak form PDE-based FE method as the examples in the paper. A solid-like generalized Maxwell model was used to represent the deforming mechanism of a viscoelastic material, the constitutive relations of which were derived and implemented in the weak form PDE module of Comsol Multiphysics, a commercial FE program. The weak form PDE modeling of viscoelasticity was verified by comparing Comsol and Abaqus simulations, which employed the same loading configurations and material property inputs in virtual laboratory test simulations. Both produced identical results in terms of axial and radial strain responses. The weak form PDE modeling of viscoelasticity was further validated by comparing the weak form PDE predictions with real laboratory test results of six types of asphalt mixtures with two air void contents and three aging periods. The viscoelastic material properties such as the coefficients of a Prony series model for the relaxation modulus were obtained by converting from the master curves of dynamic modulus and phase angle. Strain responses of compressive creep tests at three temperatures and cyclic load tests were predicted using the weak form PDE modeling and found to be comparable with the measurements of the real laboratory tests. It was demonstrated that the weak form PDE-based FE modeling can serve as an efficient method to implement new constitutive models and can free engineers from user subroutine programming.

Interaction of Kelvin waves and nonlocality of energy transfer in superfluids

We argue that the physics of interacting Kelvin Waves (KWs) is highly nontrivial and cannot be understood on the basis of pure dimensional reasoning. A consistent theory of KW turbulence in superfluids should be based upon explicit knowledge of their interactions. To achieve this, we present a detailed calculation and comprehensive analysis of the interaction coefficients for KW turbuelence, thereby, resolving previous mistakes stemming from unaccounted contributions. As a first application of this analysis, we derive a local nonlinear (partial differential) equation. This equation is much simpler for analysis and numerical simulations of KWs than the Biot-Savart equation, and in contrast to the completely integrable local induction approximation (in which the energy exchange between KWs is absent), describes the nonlinear dynamics of KWs. Second, we show that the previously suggested Kozik-Svistunov energy spectrum for KWs, which has often been used in the analysis of experimental and numerical data in superfluid turbulence, is irrelevant, because it is based upon an erroneous assumption of the locality of the energy transfer through scales. Moreover, we demonstrate the weak nonlocality of the inverse cascade spectrum with a constant particle-number flux and find resulting logarithmic corrections to this spectrum.