Transients and asymptotics of natural gradient learning

We analyse natural gradient learning in a two-layer feed-forward neural network using a statistical mechanics framework which is appropriate for large input dimension. We find significant improvement over standard gradient descent in both the transient and asymptotic phases of learning.

Intermediate asymptotics in nonlinear optical systems

Using a fiber laser system as a specific illustrative example, we introduce the concept of intermediate asymptotic states in finite nonlinear optical systems. We show that intermediate asymptotics of nonlinear equations (e.g., coherent structures with a finite lifetime or distance) can be used in applications similar to those of truly stable asymptotic solutions, such as, e.g., solitons and dissipative nonlinear waves. Applying this general idea to a particular, albeit practically important, physical system, we demonstrate a novel type of nonlinear pulse-shaping regime in a mode-locked fiber laser leading to the generation of linearly chirped pulses with a triangular distribution of the intensity.

Solvability and asymptotics of the heat equation with mixed variable lateral conditions and applications in the opening of the exocytotic fusion pore in cells

We investigate a mixed problem with variable lateral conditions for the heat equation that arises in modelling exocytosis, i.e. the opening of a cell boundary in specific biological species for the release of certain molecules to the exterior of the cell. The Dirichlet condition is imposed on a surface patch of the boundary and this patch is occupying a larger part of the boundary as time increases modelling where the cell is opening (the fusion pore), and on the remaining part, a zero Neumann condition is imposed (no molecules can cross this boundary). Uniform concentration is assumed at the initial time. We introduce a weak formulation of this problem and show that there is a unique weak solution. Moreover, we give an asymptotic expansion for the behaviour of the solution near the opening point and for small values in time. We also give an integral equation for the numerical construction of the leading term in this expansion.

Intermediate asymptotics in nonlinear optical systems

Using a fiber laser system as a specific illustrative example, we introduce the concept of intermediate asymptotic states in finite nonlinear optical systems. We show that intermediate asymptotics of nonlinear equations (e.g., coherent structures with a finite lifetime or distance) can be used in applications similar to those of truly stable asymptotic solutions, such as, e.g., solitons and dissipative nonlinear waves. Applying this general idea to a particular, albeit practically important, physical system, we demonstrate a novel type of nonlinear pulse-shaping regime in a mode-locked fiber laser leading to the generation of linearly chirped pulses with a triangular distribution of the intensity.

Simple approximate MAP inference for Dirichlet processes mixtures

The Dirichlet process mixture model (DPMM) is a ubiquitous, flexible Bayesian nonparametric statistical model. However, full probabilistic inference in this model is analytically intractable, so that computationally intensive techniques such as Gibbs sampling are required. As a result, DPMM-based methods, which have considerable potential, are restricted to applications in which computational resources and time for inference is plentiful. For example, they would not be practical for digital signal processing on embedded hardware, where computational resources are at a serious premium. Here, we develop a simplified yet statistically rigorous approximate maximum a-posteriori (MAP) inference algorithm for DPMMs. This algorithm is as simple as DP-means clustering, solves the MAP problem as well as Gibbs sampling, while requiring only a fraction of the computational effort. (For freely available code that implements the MAP-DP algorithm for Gaussian mixtures see http://www.maxlittle.net/.) Unlike related small variance asymptotics (SVA), our method is non-degenerate and so inherits the “rich get richer” property of the Dirichlet process. It also retains a non-degenerate closed-form likelihood which enables out-of-sample calculations and the use of standard tools such as cross-validation. We illustrate the benefits of our algorithm on a range of examples and contrast it to variational, SVA and sampling approaches from both a computational complexity perspective as well as in terms of clustering performance. We demonstrate the wide applicabiity of our approach by presenting an approximate MAP inference method for the infinite hidden Markov model whose performance contrasts favorably with a recently proposed hybrid SVA approach. Similarly, we show how our algorithm can applied to a semiparametric mixed-effects regression model where the random effects distribution is modelled using an infinite mixture model, as used in longitudinal progression modelling in population health science. Finally, we propose directions for future research on approximate MAP inference in Bayesian nonparametrics.