Variational mean-field algorithm for efficient inference in large systems of stochastic differential equations

This work introduces a Gaussian variational mean-field approximation for inference in dynamical systems which can be modeled by ordinary stochastic differential equations. This new approach allows one to express the variational free energy as a functional of the marginal moments of the approximating Gaussian process. A restriction of the moment equations to piecewise polynomial functions, over time, dramatically reduces the complexity of approximate inference for stochastic differential equation models and makes it comparable to that of discrete time hidden Markov models. The algorithm is demonstrated on state and parameter estimation for nonlinear problems with up to 1000 dimensional state vectors and compares the results empirically with various well-known inference methodologies.

Gaussian process approximations of stochastic differential equations

Stochastic differential equations arise naturally in a range of contexts, from financial to environmental modeling. Current solution methods are limited in their representation of the posterior process in the presence of data. In this work, we present a novel Gaussian process approximation to the posterior measure over paths for a general class of stochastic differential equations in the presence of observations. The method is applied to two simple problems: the Ornstein-Uhlenbeck process, of which the exact solution is known and can be compared to, and the double-well system, for which standard approaches such as the ensemble Kalman smoother fail to provide a satisfactory result. Experiments show that our variational approximation is viable and that the results are very promising as the variational approximate solution outperforms standard Gaussian process regression for non-Gaussian Markov processes.

Some aspects of the solution of non-linear differential equations

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

Approximate Bayesian techniques for inference in stochastic dynamical systems

This thesis is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variant of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here two new extended frameworks are derived and presented that are based on basis function expansions and local polynomial approximations of a recently proposed variational Bayesian algorithm. It is shown that the new extensions converge to the original variational algorithm and can be used for state estimation (smoothing). However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new methods are numerically validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein-Uhlenbeck process, for which the exact likelihood can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz '63 (3-dimensional model). The algorithms are also applied to the 40 dimensional stochastic Lorenz '96 system. In this investigation these new approaches are compared with a variety of other well known methods such as the ensemble Kalman filter / smoother, a hybrid Monte Carlo sampler, the dual unscented Kalman filter (for jointly estimating the systems states and model parameters) and full weak-constraint 4D-Var. Empirical analysis of their asymptotic behaviour as a function of observation density or length of time window increases is provided.

Simultaneous biochemical reaction and separation in a continuous rotating annular chromatograph

The aim of this work has been to investigate the behaviour of a continuous rotating annular chromatograph (CRAC) under a combined biochemical reaction and separation duty. Two biochemical reactions have been employed, namely the inversion of sucrose to glucose and fructose in the presence of the enzyme invertase and the saccharification of liquefied starch to maltose and dextrin using the enzyme maltogenase. Simultaneous biochemical reaction and separation has been successfully carried out for the first time in a CRAC by inverting sucrose to fructose and glucose using the enzyme invertase and collecting continuously pure fractions of glucose and fructose from the base of the column. The CRAC was made of two concentric cylinders which form an annulus 140 cm long by 1.2 cm wide, giving an annular space of 14.5 dm3. The ion exchange resin used was an industrial grade calcium form Dowex 50W-X4 with a mean diameter of 150 microns. The mobile phase used was deionised and dearated water and contained the appropriate enzyme. The annular column was slowly rotated at speeds of up to 240°h-1 while the sucrose substrate was fed continuously through a stationary feed pipe to the top of the resin bed. A systematic investigation of the factors affecting the performance of the CRAC under simultaneous biochemical reaction and separation conditions was carried out by employing a factorial experimental procedure. The main factors affecting the performance of the system were found to be the feed rate, feed concentrations and eluent rate. Results from the experiments indicated that complete conversion could be achieved for feed concentrations of up to 50% w/v sucrose and at feed throughputs of up to 17.2 kg sucrose per m3 resin/h. The second enzymic reaction, namely the saccharification of liquefied starch to maltose employing the enzyme maltogenase has also been successfully carried out on a CRAC. Results from the experiments using soluble potato starch showed that conversions of up to 79% were obtained for a feed concentration of 15.5% w/v at a feed flowrate of 400 cm3/h. The product maltose obtained was over 95% pure. Mathematical modelling and computer simulation of the sucrose inversion system has been carried out. A finite difference method was used to solve the partial differential equations and the simulation results showed good agreement with the experimental results obtained.

Bioreaction and separation in batch chromatographic columns

The objective of this work has been to study the behaviour and performance of a batch chromatographic column under simultaneous bioreaction and separation conditions for several carbohydrate feedstocks. Four bioreactions were chosen, namely the hydrolysis of sucrose to glucose and fructose using the enzyme invertase, the hydrolysis of inulin to fructose and glucose using inulinase, the hydrolysis of lactose to glucose and galactose using lactase and the isomerization of glucose to fructose using glucose isomerase. The chromatographic columns employed were jacketed glass columns ranging from 1 m to 2 m long and the internal diameter ranging from 0.97 cm to 1.97 cm. The stationary phase used was a cation exchange resin (PUROLITE PCR-833) in the Ca2+ form for the hydrolysis and the Mg2+ form for the isomerization reactions. The mobile phase used was a diluted enzyme solution which was continuously pumped through the chromatographic bed. The substrate was injected at the top of the bed as a pulse. The effect of the parameters pulse size, the amount of substrate solution introduced into the system corresponding to a percentage of the total empty column volume (% TECV), pulse concentration, eluent flowrate and the enzyme activity of the eluent were investigated. For the system sucrose-invertase complete conversions of substrate were achieved for pulse sizes and pulse concentrations of up to 20% TECV and 60% w/v, respectively. Products with purity above 90% were obtained. The enzyme consumption was 45% of the amount theoretically required to produce the same amount of product as in a conventional batch reactor. A value of 27 kg sucrose/m3 resin/h for the throughput of the system was achieved. The systematic investigation of the factors affecting the performance of the batch chromatographic bioreactor-separator was carried out by employing a factorial experimental procedure. The main factors affecting the performance of the system were the flowrate and enzyme activity. For the system inulin-inulinase total conversions were also obtained for pulses sizes of up to 20 % TECV and a pulse concentration of 10 % w/v. Fructose rich fractions with 100 % purity and representing up to 99.4 % of the total fructose generated were obtained with an enzyme consumption of 32 % of the amount theoretically required to produce the same amount of product in a conventional batch reactor. The hydrolysis of lactose by lactase was studied in the glass columns and also in an SCCR-S unit adapted for batch operation, in co-operation with Dr. Shieh, a fellow researcher in the Chemical Engineering and Applied Chemistry Department at Aston University. By operating at up to 30 % w/v lactose feed concentrations complete conversions were obtained and the purities of the products generated were above 90%. An enzyme consumption of 48 % of the amount theoretically required to produce the same amount of product in a conventional batch reactor was achieved. On working with the system glucose-glucose isomerase, which is a reversible reaction, the separation obtained with the stationary phase conditioned in the magnesium form was very poor although the conversion obtained was compatible with those for conventional batch reactors. By working with a mixed pulse of enzyme and substrate, up to 82.5 % of the fructose generated with a purity of 100 % was obtained. The mathematical modelling and computer simulation of the batch chromatographic bioreaction-separation has been performed on a personal computer. A finite difference method was used to solve the partial differential equations and the simulation results showed good agreement with the experimental results.

Physics and mathematics of dispersion-managed optical solitons

We review the main physical and mathematical properties of dispersion-managed (DM) optical solitons. Theory of DM solitons can be presented at two levels of accuracy: first, simple, but nevertheless, quantitative models based on ordinary differential equations governing evolution of the soliton width and phase parameter (the so-called chirp); and second, a comprehensive path-average theory that is capable of describing in detail both the fine structure of DM soliton form and its evolution along the fiber line. An analogy between DM soliton and a macroscopic nonlinear quantum oscillator model is also discussed. © 2003 Académie des sciences/Éditions scientifiques et médicales Elsevier SAS. All rights reserved.