A variational method and approximations of a Cauchy problem for elliptic equations

A Cauchy problem for general elliptic second-order linear partial differential equations in which the Dirichlet data in H½(?1 ? ?3) is assumed available on a larger part of the boundary ? of the bounded domain O than the boundary portion ?1 on which the Neumann data is prescribed, is investigated using a conjugate gradient method. We obtain an approximation to the solution of the Cauchy problem by minimizing a certain discrete functional and interpolating using the finite diference or boundary element method. The minimization involves solving equations obtained by discretising mixed boundary value problems for the same operator and its adjoint. It is proved that the solution of the discretised optimization problem converges to the continuous one, as the mesh size tends to zero. Numerical results are presented and discussed.

A variational method for identifying a spacewise-dependent heat source

The inverse problem of determining a spacewise-dependent heat source for the parabolic heat equation using the usual conditions of the direct problem and information from one supplementary temperature measurement at a given instant of time is studied. This spacewise-dependent temperature measurement ensures that this inverse problem has a unique solution, but the solution is unstable and hence the problem is ill-posed. We propose a variational conjugate gradient-type iterative algorithm for the stable reconstruction of the heat source based on a sequence of well-posed direct problems for the parabolic heat equation which are solved at each iteration step using the boundary element method. The instability is overcome by stopping the iterative procedure at the first iteration for which the discrepancy principle is satisfied. Numerical results are presented which have the input measured data perturbed by increasing amounts of random noise. The numerical results show that the proposed procedure yields stable and accurate numerical approximations after only a few iterations.

Bayesian invariant measurements of generalisation for continuous distributions

A family of measurements of generalisation is proposed for estimators of continuous distributions. In particular, they apply to neural network learning rules associated with continuous neural networks. The optimal estimators (learning rules) in this sense are Bayesian decision methods with information divergence as loss function. The Bayesian framework guarantees internal coherence of such measurements, while the information geometric loss function guarantees invariance. The theoretical solution for the optimal estimator is derived by a variational method. It is applied to the family of Gaussian distributions and the implications are discussed. This is one in a series of technical reports on this topic; it generalises the results of ¸iteZhu95:prob.discrete to continuous distributions and serve as a concrete example of a larger picture ¸iteZhu95:generalisation.

Analysis of power jitter induced by interchannel interactions in dispersion-managed optical soliton transmission systems

Collision-induced power jitter is theoretically and numerically examined in dispersion-managed wavelength-division-multiplexed optical soliton transmission systems. The variational method is mainly used to develop a time efficient jitter calculation approach. The power jitter causes a serious problem for a singly periodic dispersion managed line having almost zero average dispersion, which can be reduced by applying doubly periodic dispersion management.

Auto-solitons guided by in-line nonlinear optical loop mirrors in dispersion-managed communication systems

Summary form only given. Both dispersion management and the use of a nonlinear optical loop mirror (NOLM) as a saturable absorber can improve the performance of a soliton-based communication system. Dispersion management gives the benefits of low average dispersion while allowing pulses with higher powers to propagate, which helps to suppress Gordon-Haus timing jitter without sacrificing the signal-to-noise ratio. The NOLM suppresses the buildup of amplifier spontaneous emission noise and background dispersive radiation which, if allowed to interact with the soliton, can lead to its breakup. We examine optical pulse propagation in dispersion-managed (DM) transmission system with periodically inserted in-line NOLMs. To describe basic features of the signal transmission in such lines, we develop a simple theory based on a variational approach involving Gaussian trial functions. It, has already been proved that the variational method is an extremely effective tool for description of DM solitons. In the work we manage to include in the variational description the point action of the NOLM on pulse parameters, assuming that the Gaussian pulse shape is inherently preserved by propagation through the NOLM. The obtained results are verified by direct numerical simulations

Reconstruction of an unsteady flow from incomplete boundary data:Part I. Theory

Purpose – To propose and investigate a stable numerical procedure for the reconstruction of the velocity of a viscous incompressible fluid flow in linear hydrodynamics from knowledge of the velocity and fluid stress force given on a part of the boundary of a bounded domain. Design/methodology/approach – Earlier works have involved the similar problem but for stationary case (time-independent fluid flow). Extending these ideas a procedure is proposed and investigated also for the time-dependent case. Findings – The paper finds a novel variation method for the Cauchy problem. It proves convergence and also proposes a new boundary element method. Research limitations/implications – The fluid flow domain is limited to annular domains; this restriction can be removed undertaking analyses in appropriate weighted spaces to incorporate singularities that can occur on general bounded domains. Future work involves numerical investigations and also to consider Oseen type flow. A challenging problem is to consider non-linear Navier-Stokes equation. Practical implications – Fluid flow problems where data are known only on a part of the boundary occur in a range of engineering situations such as colloidal suspension and swimming of microorganisms. For example, the solution domain can be the region between to spheres where only the outer sphere is accessible for measurements. Originality/value – A novel variational method for the Cauchy problem is proposed which preserves the unsteady Stokes operator, convergence is proved and using recent for the fundamental solution for unsteady Stokes system, a new boundary element method for this system is also proposed.

Auto-solitons guided by in-line nonlinear optical loop mirrors in dispersion-managed communication systems

Summary form only given. Both dispersion management and the use of a nonlinear optical loop mirror (NOLM) as a saturable absorber can improve the performance of a soliton-based communication system. Dispersion management gives the benefits of low average dispersion while allowing pulses with higher powers to propagate, which helps to suppress Gordon-Haus timing jitter without sacrificing the signal-to-noise ratio. The NOLM suppresses the buildup of amplifier spontaneous emission noise and background dispersive radiation which, if allowed to interact with the soliton, can lead to its breakup. We examine optical pulse propagation in dispersion-managed (DM) transmission system with periodically inserted in-line NOLMs. To describe basic features of the signal transmission in such lines, we develop a simple theory based on a variational approach involving Gaussian trial functions. It, has already been proved that the variational method is an extremely effective tool for description of DM solitons. In the work we manage to include in the variational description the point action of the NOLM on pulse parameters, assuming that the Gaussian pulse shape is inherently preserved by propagation through the NOLM. The obtained results are verified by direct numerical simulations

A variational conjugate gradient method for determining the fluid velocity of a slow viscous flow

The problem considered is that of determining the fluid velocity for linear hydrostatics Stokes flow of slow viscous fluids from measured velocity and fluid stress force on a part of the boundary of a bounded domain. A variational conjugate gradient iterative procedure is proposed based on solving a series of mixed well-posed boundary value problems for the Stokes operator and its adjoint. In order to stabilize the Cauchy problem, the iterations are ceased according to an optimal order discrepancy principle stopping criterion. Numerical results obtained using the boundary element method confirm that the procedure produces a convergent and stable numerical solution.

A variational approach to learning curves

We combine the replica approach from statistical physics with a variational approach to analyze learning curves analytically. We apply the method to Gaussian process regression. As a main result we derive approximative relations between empirical error measures, the generalization error and the posterior variance.

A comparison of variational and Markov chain Monte Carlo methods for inference in partially observed stochastic dynamic systems

In recent work we have developed a novel variational inference method for partially observed systems governed by stochastic differential equations. In this paper we provide a comparison of the Variational Gaussian Process Smoother with an exact solution computed using a Hybrid Monte Carlo approach to path sampling, applied to a stochastic double well potential model. It is demonstrated that the variational smoother provides us a very accurate estimate of mean path while conditional variance is slightly underestimated. We conclude with some remarks as to the advantages and disadvantages of the variational smoother. © 2008 Springer Science + Business Media LLC.

Variational inference for Student-t MLP models

This paper presents a novel methodology to infer parameters of probabilistic models whose output noise is a Student-t distribution. The method is an extension of earlier work for models that are linear in parameters to nonlinear multi-layer perceptrons (MLPs). We used an EM algorithm combined with variational approximation, the evidence procedure, and an optimisation algorithm. The technique was tested on two regression applications. The first one is a synthetic dataset and the second is gas forward contract prices data from the UK energy market. The results showed that forecasting accuracy is significantly improved by using Student-t noise models.

Variational Markov Chain Monte Carlo for Bayesian smoothing of non-linear niffusions

In this paper we develop set of novel Markov chain Monte Carlo algorithms for Bayesian smoothing of partially observed non-linear diffusion processes. The sampling algorithms developed herein use a deterministic approximation to the posterior distribution over paths as the proposal distribution for a mixture of an independence and a random walk sampler. The approximating distribution is sampled by simulating an optimized time-dependent linear diffusion process derived from the recently developed variational Gaussian process approximation method. Flexible blocking strategies are introduced to further improve mixing, and thus the efficiency, of the sampling algorithms. The algorithms are tested on two diffusion processes: one with double-well potential drift and another with SINE drift. The new algorithm's accuracy and efficiency is compared with state-of-the-art hybrid Monte Carlo based path sampling. It is shown that in practical, finite sample, applications the algorithm is accurate except in the presence of large observation errors and low observation densities, which lead to a multi-modal structure in the posterior distribution over paths. More importantly, the variational approximation assisted sampling algorithm outperforms hybrid Monte Carlo in terms of computational efficiency, except when the diffusion process is densely observed with small errors in which case both algorithms are equally efficient.

Variational Markov chain Monte Carlo for Bayesian smoothing of non-linear diffusions

In this paper we develop set of novel Markov Chain Monte Carlo algorithms for Bayesian smoothing of partially observed non-linear diffusion processes. The sampling algorithms developed herein use a deterministic approximation to the posterior distribution over paths as the proposal distribution for a mixture of an independence and a random walk sampler. The approximating distribution is sampled by simulating an optimized time-dependent linear diffusion process derived from the recently developed variational Gaussian process approximation method. The novel diffusion bridge proposal derived from the variational approximation allows the use of a flexible blocking strategy that further improves mixing, and thus the efficiency, of the sampling algorithms. The algorithms are tested on two diffusion processes: one with double-well potential drift and another with SINE drift. The new algorithm's accuracy and efficiency is compared with state-of-the-art hybrid Monte Carlo based path sampling. It is shown that in practical, finite sample applications the algorithm is accurate except in the presence of large observation errors and low to a multi-modal structure in the posterior distribution over paths. More importantly, the variational approximation assisted sampling algorithm outperforms hybrid Monte Carlo in terms of computational efficiency, except when the diffusion process is densely observed with small errors in which case both algorithms are equally efficient. © 2011 Springer-Verlag.