Variational mean-field algorithm for efficient inference in large systems of stochastic differential equations

This work introduces a Gaussian variational mean-field approximation for inference in dynamical systems which can be modeled by ordinary stochastic differential equations. This new approach allows one to express the variational free energy as a functional of the marginal moments of the approximating Gaussian process. A restriction of the moment equations to piecewise polynomial functions, over time, dramatically reduces the complexity of approximate inference for stochastic differential equation models and makes it comparable to that of discrete time hidden Markov models. The algorithm is demonstrated on state and parameter estimation for nonlinear problems with up to 1000 dimensional state vectors and compares the results empirically with various well-known inference methodologies.

Integration of the Manakov-PMD Equation with Precomputed M(w) Matrices for PMD Simulation

A novel direct integration technique of the Manakov-PMD equation for the simulation of polarisation mode dispersion (PMD) in optical communication systems is demonstrated and shown to be numerically as efficient as the commonly used coarse-step method. The main advantage of using a direct integration of the Manakov-PMD equation over the coarse-step method is a higher accuracy of the PMD model. The new algorithm uses precomputed M(w) matrices to increase the computational speed compared to a full integration without loss of accuracy. The simulation results for the probability distribution function (PDF) of the differential group delay (DGD) and the autocorrelation function (ACF) of the polarisation dispersion vector for varying numbers of precomputed M(w) matrices are compared to analytical models and results from the coarse-step method. It is shown that the coarse-step method achieves a significantly inferior reproduction of the statistical properties of PMD in optical fibres compared to a direct integration of the Manakov-PMD equation.

Quasi-separation of the biharmonic partial differential equation

In this paper, we consider analytical and numerical solutions to the Dirichlet boundary-value problem for the biharmonic partial differential equation on a disc of finite radius in the plane. The physical interpretation of these solutions is that of the harmonic oscillations of a thin, clamped plate. For the linear, fourth-order, biharmonic partial differential equation in the plane, it is well known that the solution method of separation in polar coordinates is not possible, in general. However, in this paper, for circular domains in the plane, it is shown that a method, here called quasi-separation of variables, does lead to solutions of the partial differential equation. These solutions are products of solutions of two ordinary linear differential equations: a fourth-order radial equation and a second-order angular differential equation. To be expected, without complete separation of the polar variables, there is some restriction on the range of these solutions in comparison with the corresponding separated solutions of the second-order harmonic differential equation in the plane. Notwithstanding these restrictions, the quasi-separation method leads to solutions of the Dirichlet boundary-value problem on a disc with centre at the origin, with boundary conditions determined by the solution and its inward drawn normal taking the value 0 on the edge of the disc. One significant feature for these biharmonic boundary-value problems, in general, follows from the form of the biharmonic differential expression when represented in polar coordinates. In this form, the differential expression has a singularity at the origin, in the radial variable. This singularity translates to a singularity at the origin of the fourth-order radial separated equation; this singularity necessitates the application of a third boundary condition in order to determine a self-adjoint solution to the Dirichlet boundary-value problem. The penultimate section of the paper reports on numerical solutions to the Dirichlet boundary-value problem; these results are also presented graphically. Two specific cases are studied in detail and numerical values of the eigenvalues are compared with the results obtained in earlier studies.

Gaussian process approximations of stochastic differential equations

Stochastic differential equations arise naturally in a range of contexts, from financial to environmental modeling. Current solution methods are limited in their representation of the posterior process in the presence of data. In this work, we present a novel Gaussian process approximation to the posterior measure over paths for a general class of stochastic differential equations in the presence of observations. The method is applied to two simple problems: the Ornstein-Uhlenbeck process, of which the exact solution is known and can be compared to, and the double-well system, for which standard approaches such as the ensemble Kalman smoother fail to provide a satisfactory result. Experiments show that our variational approximation is viable and that the results are very promising as the variational approximate solution outperforms standard Gaussian process regression for non-Gaussian Markov processes.

Numerical implementation of the Manakov-PMD equation with precomputed M(Ω) matrices

The Manakov-PMD equation can be integrated with the same numerical efficiency as the coarse-step method by using precomputed M(Ω) matrices, which entirely avoids the somewhat ad-hoc rescaling of coefficients necessary in the coarse-step method.

Solvability and asymptotics of the heat equation with mixed variable lateral conditions and applications in the opening of the exocytotic fusion pore in cells

We investigate a mixed problem with variable lateral conditions for the heat equation that arises in modelling exocytosis, i.e. the opening of a cell boundary in specific biological species for the release of certain molecules to the exterior of the cell. The Dirichlet condition is imposed on a surface patch of the boundary and this patch is occupying a larger part of the boundary as time increases modelling where the cell is opening (the fusion pore), and on the remaining part, a zero Neumann condition is imposed (no molecules can cross this boundary). Uniform concentration is assumed at the initial time. We introduce a weak formulation of this problem and show that there is a unique weak solution. Moreover, we give an asymptotic expansion for the behaviour of the solution near the opening point and for small values in time. We also give an integral equation for the numerical construction of the leading term in this expansion.

Travelling waves in a complex Ginzburg-Landau equation with time-delay feedback

One of the simplest ways to create nonlinear oscillations is the Hopf bifurcation. The spatiotemporal dynamics observed in an extended medium with diffusion (e.g., a chemical reaction) undergoing this bifurcation is governed by the complex Ginzburg-Landau equation, one of the best-studied generic models for pattern formation, where besides uniform oscillations, spiral waves, coherent structures and turbulence are found. The presence of time delay terms in this equation changes the pattern formation scenario, and different kind of travelling waves have been reported. In particular, we study the complex Ginzburg-Landau equation that contains local and global time-delay feedback terms. We focus our attention on plane wave solutions in this model. The first novel result is the derivation of the plane wave solution in the presence of time-delay feedback with global and local contributions. The second and more important result of this study consists of a linear stability analysis of plane waves in that model. Evaluation of the eigenvalue equation does not show stabilisation of plane waves for the parameters studied. We discuss these results and compare to results of other models.

Conditional probability calculations for the nonlinear Schrödinger equation with additive noise

The method for the computation of the conditional probability density function for the nonlinear Schrödinger equation with additive noise is developed. We present in a constructive form the conditional probability density function in the limit of small noise and analytically derive it in a weakly nonlinear case. The general theory results are illustrated using fiber-optic communications as a particular, albeit practically very important, example.

Approximate Bayesian techniques for inference in stochastic dynamical systems

This thesis is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variant of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here two new extended frameworks are derived and presented that are based on basis function expansions and local polynomial approximations of a recently proposed variational Bayesian algorithm. It is shown that the new extensions converge to the original variational algorithm and can be used for state estimation (smoothing). However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new methods are numerically validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein-Uhlenbeck process, for which the exact likelihood can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz '63 (3-dimensional model). The algorithms are also applied to the 40 dimensional stochastic Lorenz '96 system. In this investigation these new approaches are compared with a variety of other well known methods such as the ensemble Kalman filter / smoother, a hybrid Monte Carlo sampler, the dual unscented Kalman filter (for jointly estimating the systems states and model parameters) and full weak-constraint 4D-Var. Empirical analysis of their asymptotic behaviour as a function of observation density or length of time window increases is provided.

A comparison of variational and Markov chain Monte Carlo methods for inference in partially observed stochastic dynamic systems

In recent work we have developed a novel variational inference method for partially observed systems governed by stochastic differential equations. In this paper we provide a comparison of the Variational Gaussian Process Smoother with an exact solution computed using a Hybrid Monte Carlo approach to path sampling, applied to a stochastic double well potential model. It is demonstrated that the variational smoother provides us a very accurate estimate of mean path while conditional variance is slightly underestimated. We conclude with some remarks as to the advantages and disadvantages of the variational smoother. © 2008 Springer Science + Business Media LLC.

Estimating parameters in stochastic systems:a variational Bayesian approach

This work is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variation of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here a new extended framework is derived that is based on a local polynomial approximation of a recently proposed variational Bayesian algorithm. The paper begins by showing that the new extension of this variational algorithm can be used for state estimation (smoothing) and converges to the original algorithm. However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new approach is validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein–Uhlenbeck process, the exact likelihood of which can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz ’63 (3D model). As a special case the algorithm is also applied to the 40 dimensional stochastic Lorenz ’96 system. In our investigation we compare this new approach with a variety of other well known methods, such as the hybrid Monte Carlo, dual unscented Kalman filter, full weak-constraint 4D-Var algorithm and analyse empirically their asymptotic behaviour as a function of observation density or length of time window increases. In particular we show that we are able to estimate parameters in both the drift (deterministic) and the diffusion (stochastic) part of the model evolution equations using our new methods.

Developments and extensions of Roth's method of double Fourier series with applications to the solution of electromagnetic field problems

The first part of the thesis compares Roth's method with other methods, in particular the method of separation of variables and the finite cosine transform method, for solving certain elliptic partial differential equations arising in practice. In particular we consider the solution of steady state problems associated with insulated conductors in rectangular slots. Roth's method has two main disadvantages namely the slow rate of con­vergence of the double Fourier series and the restrictive form of the allowable boundary conditions. A combined Roth-separation of variables method is derived to remove the restrictions on the form of the boundary conditions and various Chebyshev approximations are used to try to improve the rate of convergence of the series. All the techniques are then applied to the Neumann problem arising from balanced rectangular windings in a transformer window. Roth's method is then extended to deal with problems other than those resulting from static fields. First we consider a rectangular insulated conductor in a rectangular slot when the current is varying sinusoidally with time. An approximate method is also developed and compared with the exact method.The approximation is then used to consider the problem of an insulated conductor in a slot facing an air gap. We also consider the exact method applied to the determination of the eddy-current loss produced in an isolated rectangular conductor by a transverse magnetic field varying sinusoidally with time. The results obtained using Roth's method are critically compared with those obtained by other authors using different methods. The final part of the thesis investigates further the application of Chebyshdev methods to the solution of elliptic partial differential equations; an area where Chebyshev approximations have rarely been used. A poisson equation with a polynomial term is treated first followed by a slot problem in cylindrical geometry.

Localized waves in optical systems with periodic dispersion and nonlinearity management

We overview our recent developments in the theory of dispersion-managed (DM) solitons within the context of optical applications. First, we present a class of localized solutions with a period multiple to that of the standard DM soliton in the nonlinear Schrödinger equation with periodic variations of the dispersion. In the framework of a reduced ordinary differential equation-based model, we discuss the key features of these structures, such as a smaller energy compared to traditional DM solitons with the same temporal width. Next, we present new results on dissipative DM solitons, which occur in the context of mode-locked lasers. By means of numerical simulations and a reduced variational model of the complex Ginzburg-Landau equation, we analyze the influence of the different dissipative processes that take place in a laser.

A procedure for the reconstruction of a stochastic stationary temperature field

An iterative procedure is proposed for the reconstruction of a temperature field from a linear stationary heat equation with stochastic coefficients, and stochastic Cauchy data given on a part of the boundary of a bounded domain. In each step, a series of mixed well-posed boundary-value problems are solved for the stochastic heat operator and its adjoint. Well-posedness of these problems is shown to hold and convergence in the mean of the procedure is proved. A discretized version of this procedure, based on a Monte Carlo Galerkin finite-element method, suitable for numerical implementation is discussed. It is demonstrated that the solution to the discretized problem converges to the continuous as the mesh size tends to zero.

On an indirect integral equation approach for stationary heat transfer in semi-infinite layered domains in R3 with cavities

Regions containing internal boundaries such as composite materials arise in many applications.We consider a situation of a layered domain in IR3 containing a nite number of bounded cavities. The model is stationary heat transfer given by the Laplace equation with piecewise constant conductivity. The heat ux (a Neumann condition) is imposed on the bottom of the layered region and various boundary conditions are imposed on the cavities. The usual transmission (interface) conditions are satised at the interface layer, that is continuity of the solution and its normal derivative. To eciently calculate the stationary temperature eld in the semi-innite region, we employ a Green's matrix technique and reduce the problem to boundary integral equations (weakly singular) over the bounded surfaces of the cavities. For the numerical solution of these integral equations, we use Wienert's approach [20]. Assuming that each cavity is homeomorphic with the unit sphere, a fully discrete projection method with super-algebraic convergence order is proposed. A proof of an error estimate for the approximation is given as well. Numerical examples are presented that further highlights the eciency and accuracy of the proposed method.