Two problems with variational expectation maximisation for time-series models

Variational methods are a key component of the approximate inference and learning toolbox. These methods fill an important middle ground, retaining distributional information about uncertainty in latent variables, unlike maximum a posteriori methods (MAP), and yet generally requiring less computational time than Monte Carlo Markov Chain methods. In particular the variational Expectation Maximisation (vEM) and variational Bayes algorithms, both involving variational optimisation of a free-energy, are widely used in time-series modelling. Here, we investigate the success of vEM in simple probabilistic time-series models. First we consider the inference step of vEM, and show that a consequence of the well-known compactness property of variational inference is a failure to propagate uncertainty in time, thus limiting the usefulness of the retained distributional information. In particular, the uncertainty may appear to be smallest precisely when the approximation is poorest. Second, we consider parameter learning and analytically reveal systematic biases in the parameters found by vEM. Surprisingly, simpler variational approximations (such a mean-field) can lead to less bias than more complicated structured approximations.

Accelerating Bayesian hierarchical clustering of time series data with a randomised algorithm.

We live in an era of abundant data. This has necessitated the development of new and innovative statistical algorithms to get the most from experimental data. For example, faster algorithms make practical the analysis of larger genomic data sets, allowing us to extend the utility of cutting-edge statistical methods. We present a randomised algorithm that accelerates the clustering of time series data using the Bayesian Hierarchical Clustering (BHC) statistical method. BHC is a general method for clustering any discretely sampled time series data. In this paper we focus on a particular application to microarray gene expression data. We define and analyse the randomised algorithm, before presenting results on both synthetic and real biological data sets. We show that the randomised algorithm leads to substantial gains in speed with minimal loss in clustering quality. The randomised time series BHC algorithm is available as part of the R package BHC, which is available for download from Bioconductor (version 2.10 and above) via http://bioconductor.org/packages/2.10/bioc/html/BHC.html. We have also made available a set of R scripts which can be used to reproduce the analyses carried out in this paper. These are available from the following URL. https://sites.google.com/site/randomisedbhc/.

Dynamic Covariance Models for Multivariate Financial Time Series

The accurate prediction of time-changing covariances is an important problem in the modeling of multivariate financial data. However, some of the most popular models suffer from a) overfitting problems and multiple local optima, b) failure to capture shifts in market conditions and c) large computational costs. To address these problems we introduce a novel dynamic model for time-changing covariances. Over-fitting and local optima are avoided by following a Bayesian approach instead of computing point estimates. Changes in market conditions are captured by assuming a diffusion process in parameter values, and finally computationally efficient and scalable inference is performed using particle filters. Experiments with financial data show excellent performance of the proposed method with respect to current standard models.

Flow field measurements of a series of turbulent premixed and stratified methane/air flames

This paper presents flow field measurements for the turbulent stratified burner introduced in two previous publications in which high resolution scalar measurements were made by Sweeney et al. [1,2] for model validation. The flow fields of the series of premixed and stratified methane/air flames are investigated under turbulent, globally lean conditions (φg=0.75). Velocity data acquired with laser Doppler anemometry (LDA) and particle image velocimetry (PIV) are presented and discussed. Pairwise 2-component LDA measurements provide profiles of axial velocity, radial velocity, tangential velocity and corresponding fluctuating velocities. The LDA measurements of axial and tangential velocities enable the swirl number to be evaluated and the degree of swirl characterized. Power spectral density and autocorrelation functions derived from the LDA data acquired at 10kHz are optimized to calculate the integral time scales. Flow patterns are obtained using a 2-component PIV system operated at 7Hz. Velocity profiles and spatial correlations derived from the PIV and LDA measurements are shown to be in very good agreement, thus offering 3D mapping of the velocities. A strong correlation was observed between the shape of the recirculation zones above the central bluff body and the effects of heat release, stoichiometry and swirl. Detailed analyses of the LDA data further demonstrate that the flow behavior changes significantly with the levels of swirl and stratification, which combines the contributions of dilatation, recirculation and swirl. Key turbulence parameters are derived from the total velocity components, combining axial, radial and tangential velocities. © 2013 The Combustion Institute.

2.1 GHz MOS mixer linearisation approach for IIP2 enhancement using multiple factor compensation

An integrated downconversion CMOS mixer incorporating a comprehensive compensation scheme is presented which aims to minimise second-order intermodulation distortion (IMD2). Unlike previously reported IMD2 calibration schemes which tune only one nonlinear factor at a time, the presented solution allows simultaneous adjustment of several different factors thus achieving a better compensation. The mixer has been implemented in UMC 0.18 μm CMOS to verify the proposed scheme and for comparison with alternative compensation methods. Measurements show that the solution described can improve the input intercept point (IIP2) by over 20 dB while maintaining good amplification and noise performance. IMD2 calibration results are presented and show useful advantages over other approaches. To the best of the authors' knowledge, this scheme for IMD2 calibration has not been previously reported. © The Institution of Engineering and Technology 2013.

Degenerate mode-group division multiplexing using MIMO digital signal processing

MIMO DSP is employed to improve the performance of degenerate mode-group division multiplexing in 8 km of conventional GI-MMF. Compensation of the mode coupling, induced by the launch and propagation, between and inside each degenerate mode-group is investigated in order to reduce the DSP complexity. © 2013 IEEE.

Multilinear function factorisation for time series feature extraction

This work applies a variety of multilinear function factorisation techniques to extract appropriate features or attributes from high dimensional multivariate time series for classification. Recently, a great deal of work has centred around designing time series classifiers using more and more complex feature extraction and machine learning schemes. This paper argues that complex learners and domain specific feature extraction schemes of this type are not necessarily needed for time series classification, as excellent classification results can be obtained by simply applying a number of existing matrix factorisation or linear projection techniques, which are simple and computationally inexpensive. We highlight this using a geometric separability measure and classification accuracies obtained though experiments on four different high dimensional multivariate time series datasets. © 2013 IEEE.

Dopant compensation in HfO2 and other high K oxides

The theory of doping limits in semiconductors and insulators is applied to the case of wide gap oxides, crystalline, or amorphous, and used to explain that impurities do not in general give rise to gap states or a doping response. Instead, the system tends to form defect complexes or undergo symmetry-lowering reconstructions to expel gap states out of the band gap. The model is applied to impurities, such as trivalent metals, carbon, N, P, and B, in HfO2, the main gate dielectric used in field effect transistors. © 2014 AIP Publishing LLC.

Cell types, network homeostasis, and pathological compensation from a biologically plausible ion channel expression model.

How do neurons develop, control, and maintain their electrical signaling properties in spite of ongoing protein turnover and perturbations to activity? From generic assumptions about the molecular biology underlying channel expression, we derive a simple model and show how it encodes an "activity set point" in single neurons. The model generates diverse self-regulating cell types and relates correlations in conductance expression observed in vivo to underlying channel expression rates. Synaptic as well as intrinsic conductances can be regulated to make a self-assembling central pattern generator network; thus, network-level homeostasis can emerge from cell-autonomous regulation rules. Finally, we demonstrate that the outcome of homeostatic regulation depends on the complement of ion channels expressed in cells: in some cases, loss of specific ion channels can be compensated; in others, the homeostatic mechanism itself causes pathological loss of function.