A variational perspective on noise-robust speech recognition

Model compensation methods for noise-robust speech recognition have shown good performance. Predictive linear transformations can approximate these methods to balance computational complexity and compensation accuracy. This paper examines both of these approaches from a variational perspective. Using a matched-pair approximation at the component level yields a number of standard forms of model compensation and predictive linear transformations. However, a tighter bound can be obtained by using variational approximations at the state level. Both model-based and predictive linear transform schemes can be implemented in this framework. Preliminary results show that the tighter bound obtained from the state-level variational approach can yield improved performance over standard schemes. © 2011 IEEE.

Two problems with variational expectation maximisation for time-series models

Variational methods are a key component of the approximate inference and learning toolbox. These methods fill an important middle ground, retaining distributional information about uncertainty in latent variables, unlike maximum a posteriori methods (MAP), and yet generally requiring less computational time than Monte Carlo Markov Chain methods. In particular the variational Expectation Maximisation (vEM) and variational Bayes algorithms, both involving variational optimisation of a free-energy, are widely used in time-series modelling. Here, we investigate the success of vEM in simple probabilistic time-series models. First we consider the inference step of vEM, and show that a consequence of the well-known compactness property of variational inference is a failure to propagate uncertainty in time, thus limiting the usefulness of the retained distributional information. In particular, the uncertainty may appear to be smallest precisely when the approximation is poorest. Second, we consider parameter learning and analytically reveal systematic biases in the parameters found by vEM. Surprisingly, simpler variational approximations (such a mean-field) can lead to less bias than more complicated structured approximations.

Gaussian processes for machine learning (GPML) toolbox

The GPML toolbox provides a wide range of functionality for Gaussian process (GP) inference and prediction. GPs are specified by mean and covariance functions; we offer a library of simple mean and covariance functions and mechanisms to compose more complex ones. Several likelihood functions are supported including Gaussian and heavy-tailed for regression as well as others suitable for classification. Finally, a range of inference methods is provided, including exact and variational inference, Expectation Propagation, and Laplace’s method dealing with non-Gaussian likelihoods and FITC for dealing with large regression tasks.

Bayesian adaptation and adaptively trained systems

As the use of found data increases, more systems are being built using adaptive training. Here transforms are used to represent unwanted acoustic variability, e.g. speaker and acoustic environment changes, allowing a canonical model that models only the "pure" variability of speech to be trained. Adaptive training may be described within a Bayesian framework. By using complexity control approaches to ensure robust parameter estimates, the standard point estimate adaptive training can be justified within this Bayesian framework. However during recognition there is usually no control over the amount of data available. It is therefore preferable to be able to use a full Bayesian approach to applying transforms during recognition rather than the standard point estimates. This paper discusses various approximations to Bayesian approaches including a new variational Bayes approximation. The application of these approaches to state-of-the-art adaptively trained systems using both CAT and MLLR transforms is then described and evaluated on a large vocabulary speech recognition task. © 2005 IEEE.

Gaussian Process Regression Networks

We introduce a new regression framework, Gaussian process regression networks (GPRN), which combines the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian processes. This model accommodates input dependent signal and noise correlations between multiple response variables, input dependent length-scales and amplitudes, and heavy-tailed predictive distributions. We derive both efficient Markov chain Monte Carlo and variational Bayes inference procedures for this model. We apply GPRN as a multiple output regression and multivariate volatility model, demonstrating substantially improved performance over eight popular multiple output (multi-task) Gaussian process models and three multivariate volatility models on benchmark datasets, including a 1000 dimensional gene expression dataset.

Phase noise analysis of a mechanical autonomous impact oscillator with a MEMS resonator

In this paper, phase noise analysis of a mechanical autonomous impact oscillator with a MEMS resonator is performed. Since the circuit considered belongs to the class of hybrid systems, methods based on the variational model for the evaluation of either phase noise or steady state solutions cannot be directly applied. As a matter of fact, the monodromy matrix is not defined at impact events in these systems. By introducing saltation matrices, this limit is overcome and the aforementioned methods are extended. In particular, the unified theory developed by Demir is used to analyze the phase noise after evaluating the asymptotically stable periodic solution of the system by resorting to the shooting method. Numerical results are presented to show how noise sources affect the phase noise performances. © 2011 IEEE.

On the stability of a rod adhering to a rigid surface: Shear-induced stable adhesion and the instability of peeling

Using variational methods, we establish conditions for the nonlinear stability of adhesive states between an elastica and a rigid halfspace. The treatment produces coupled criteria for adhesion and buckling instabilities by exploiting classical techniques from Legendre and Jacobi. Three examples that arise in a broad range of engineered systems, from microelectronics to biologically inspired fiber array adhesion, are used to illuminate the stability criteria. The first example illustrates buckling instabilities in adhered rods, while the second shows the instability of a peeling process and the third illustrates the stability of a shear-induced adhesion. The latter examples can also be used to explain how microfiber array adhesives can be activated by shearing and deactivated by peeling. The nonlinear stability criteria developed in this paper are also compared to other treatments. © 2012 Elsevier Ltd. All rights reserved.

A unifying resolution-independent formulation for early vision

We present a model for early vision tasks such as denoising, super-resolution, deblurring, and demosaicing. The model provides a resolution-independent representation of discrete images which admits a truly rotationally invariant prior. The model generalizes several existing approaches: variational methods, finite element methods, and discrete random fields. The primary contribution is a novel energy functional which has not previously been written down, which combines the discrete measurements from pixels with a continuous-domain world viewed through continous-domain point-spread functions. The value of the functional is that simple priors (such as total variation and generalizations) on the continous-domain world become realistic priors on the sampled images. We show that despite its apparent complexity, optimization of this model depends on just a few computational primitives, which although tedious to derive, can now be reused in many domains. We define a set of optimization algorithms which greatly overcome the apparent complexity of this model, and make possible its practical application. New experimental results include infinite-resolution upsampling, and a method for obtaining subpixel superpixels. © 2012 IEEE.

Power absorption invariance for brownian spring forcing

In [5] it was shown that, for a standard quarter-car vehicle model and a road disturbance whose velocity profile is white noise of intensity A, the mean power dissipated in the suspension is equal to kA/2 where k is the tyre vertical stiffness. It is remarkable that the power dissipation turns out to be independent of all masses and suspension parameters. The proof in [5] makes use of a spectral formulation of white noise and is specific to linear systems. This paper casts the result in a more general form and shows that it follows from a simple application of Ito calculus. © 2012 IEEE.

A Hierarchical Model for Ordinal Matrix Factorization

This paper proposes a hierarchical probabilistic model for ordinal matrix factorization. Unlike previous approaches, we model the ordinal nature of the data and take a principled approach to incorporating priors for the hidden variables. Two algorithms are presented for inference, one based on Gibbs sampling and one based on variational Bayes. Importantly, these algorithms may be implemented in the factorization of very large matrices with missing entries. The model is evaluated on a collaborative filtering task, where users have rated a collection of movies and the system is asked to predict their ratings for other movies. The Netflix data set is used for evaluation, which consists of around 100 million ratings. Using root mean-squared error (RMSE) as an evaluation metric, results show that the suggested model outperforms alternative factorization techniques. Results also show how Gibbs sampling outperforms variational Bayes on this task, despite the large number of ratings and model parameters. Matlab implementations of the proposed algorithms are available from cogsys.imm.dtu.dk/ordinalmatrixfactorization.

Modelling the vibration of tyre sidewalls

The dynamical behaviour of the sidewall has an important influence on tyre vibration characteristics. Nonetheless, it remains crudely represented in many existing models. The current work considers a geometrically accurate, two-dimensional, sidewall description, with a view to identifying potential shortcomings in the approximate formulations and identifying the physical characteristics that must be accounted for. First, the mean stress state under pressurisation and centrifugal loading is investigated. Finite-Element calculations show that, while the loaded sidewall shape remains close to a toroid, its in-plane tensions differ appreciably from the associated analytical solution. This is largely due to the inability of the anisotropic sidewall material to sustain significant azimuthal stress. An approximate analysis, based on the meridional tension alone, is therefore developed, and shown to yield accurate predictions. In conjunction with a set of formulae for the 'engineering constants' of the sidewall material, the approximate solutions provide a straightforward and efficient means of determining the base state for the vibration analysis. The latter is implemented via a 'waveguide' discretisation of a variational formulation. Its results show that, while the full geometrical description is necessary for a complete and reliable characterisation of the sidewall's vibrational properties, a one-dimensional approximation will often be satisfactory in practice. Meridional thickness variations only become important at higher frequencies (above 500 Hz for the example considered here), and rotational inertia effects appear to be minor at practical vehicle speeds. © 2013 Elsevier Ltd. All rights reserved.

Computational analysis of liquid crystalline elastomer membranes: Changing Gaussian curvature without stretch energy

Liquid crystalline elastomers (LCEs) can undergo extremely large reversible shape changes when exposed to external stimuli, such as mechanical deformations, heating or illumination. The deformation of LCEs result from a combination of directional reorientation of the nematic director and entropic elasticity. In this paper, we study the energetics of initially flat, thin LCE membranes by stress driven reorientation of the nematic director. The energy functional used in the variational formulation includes contributions depending on the deformation gradient and the second gradient of the deformation. The deformation gradient models the in-plane stretching of the membrane. The second gradient regularises the non-convex membrane energy functional so that infinitely fine in-plane microstructures and infinitely fine out-of-plane membrane wrinkling are penalised. For a specific example, our computational results show that a non-developable surface can be generated from an initially flat sheet at cost of only energy terms resulting from the second gradients. That is, Gaussian curvature can be generated in LCE membranes without the cost of stretch energy in contrast to conventional materials. © 2013 Elsevier Ltd. All rights reserved.

Unified form language: A domain-specific language for weak formulations of partial differential equations

We present the Unified Form Language (UFL), which is a domain-specific language for representing weak formulations of partial differential equations with a view to numerical approximation. Features of UFL include support for variational forms and functionals, automatic differentiation of forms and expressions, arbitrary function space hierarchies formultifield problems, general differential operators and flexible tensor algebra. With these features, UFL has been used to effortlessly express finite element methods for complex systems of partial differential equations in near-mathematical notation, resulting in compact, intuitive and readable programs. We present in this work the language and its construction. An implementation of UFL is freely available as an open-source software library. The library generates abstract syntax tree representations of variational problems, which are used by other software libraries to generate concrete low-level implementations. Some application examples are presented and libraries that support UFL are highlighted. © 2014 ACM.

MoT - Mixture of trees probabilistic graphical model for video segmentation

We present a novel mixture of trees (MoT) graphical model for video segmentation. Each component in this mixture represents a tree structured temporal linkage between super-pixels from the first to the last frame of a video sequence. Our time-series model explicitly captures the uncertainty in temporal linkage between adjacent frames which improves segmentation accuracy. We provide a variational inference scheme for this model to estimate super-pixel labels and their confidences in nearly realtime. The efficacy of our approach is demonstrated via quantitative comparisons on the challenging SegTrack joint segmentation and tracking dataset [23].