Bayesian and L1 Approaches to Sparse Unsupervised Learning

The use of L1 regularisation for sparse learning has generated immense research interest, with successful application in such diverse areas as signal acquisition, image coding, genomics and collaborative filtering. While existing work highlights the many advantages of L1 methods, in this paper we find that L1 regularisation often dramatically underperforms in terms of predictive performance when compared with other methods for inferring sparsity. We focus on unsupervised latent variable models, and develop L1 minimising factor models, Bayesian variants of "L1", and Bayesian models with a stronger L0-like sparsity induced through spike-and-slab distributions. These spike-and-slab Bayesian factor models encourage sparsity while accounting for uncertainty in a principled manner and avoiding unnecessary shrinkage of non-zero values. We demonstrate on a number of data sets that in practice spike-and-slab Bayesian methods outperform L1 minimisation, even on a computational budget. We thus highlight the need to re-assess the wide use of L1 methods in sparsity-reliant applications, particularly when we care about generalising to previously unseen data, and provide an alternative that, over many varying conditions, provides improved generalisation performance.

Immersed Boundary Method for generalised finite volume and finite difference Navier-Stokes solvers

In Immersed Boundary Methods (IBM) the effect of complex geometries is introduced through the forces added in the Navier-Stokes solver at the grid points in the vicinity of the immersed boundaries. Most of the methods in the literature have been used with Cartesian grids. Moreover many of the methods developed in the literature do not satisfy some basic conservation properties (the conservation of torque, for instance) on non-uniform meshes. In this paper we will follow the RKPM method originated by Liu et al. [1] to build locally regularized functions that verify a number of integral conditions. These local approximants will be used both for interpolating the velocity field and for spreading the singular force field in the framework of a pressure correction scheme for the incompressible Navier-Stokes equations. We will also demonstrate the robustness and effectiveness of the scheme through various examples. Copyright © 2010 by ASME.

Fast L0-based sparse signal recovery

This paper develops an algorithm for finding sparse signals from limited observations of a linear system. We assume an adaptive Gaussian model for sparse signals. This model results in a least square problem with an iteratively reweighted L2 penalty that approximates the L0-norm. We propose a fast algorithm to solve the problem within a continuation framework. In our examples, we show that the correct sparsity map and sparsity level are gradually learnt during the iterations even when the number of observations is reduced, or when observation noise is present. In addition, with the help of sophisticated interscale signal models, the algorithm is able to recover signals to a better accuracy and with reduced number of observations than typical L1-norm and reweighted L1 norm methods. ©2010 IEEE.

Imaging the femoral cortex: thickness, density and mass from clinical CT.

There is growing evidence that focal thinning of cortical bone in the proximal femur may predispose a hip to fracture. Detecting such defects in clinical CT is challenging, since cortices may be significantly thinner than the imaging system's point spread function. We recently proposed a model-fitting technique to measure sub-millimetre cortices, an ill-posed problem which was regularized by assuming a specific, fixed value for the cortical density. In this paper, we develop the work further by proposing and evaluating a more rigorous method for estimating the constant cortical density, and extend the paradigm to encompass the mapping of cortical mass (mineral mg/cm(2)) in addition to thickness. Density, thickness and mass estimates are evaluated on sixteen cadaveric femurs, with high resolution measurements from a micro-CT scanner providing the gold standard. The results demonstrate robust, accurate measurement of peak cortical density and cortical mass. Cortical thickness errors are confined to regions of thin cortex and are bounded by the extent to which the local density deviates from the peak, averaging 20% for 0.5mm cortex.

Imaging the femoral cortex: Thickness, density and mass from clinical CT

There is growing evidence that focal thinning of cortical bone in the proximal femur may predispose a hip to fracture. Detecting such defects in clinical CT is challenging, since cortices may be significantly thinner than the imaging system's point spread function. We recently proposed a model-fitting technique to measure sub-millimetre cortices, an ill-posed problem which was regularized by assuming a specific, fixed value for the cortical density. In this paper, we develop the work further by proposing and evaluating a more rigorous method for estimating the constant cortical density, and extend the paradigm to encompass the mapping of cortical mass (mineral mg/cm 2) in addition to thickness. Density, thickness and mass estimates are evaluated on sixteen cadaveric femurs, with high resolution measurements from a micro-CT scanner providing the gold standard. The results demonstrate robust, accurate measurement of peak cortical density and cortical mass. Cortical thickness errors are confined to regions of thin cortex and are bounded by the extent to which the local density deviates from the peak, averaging 20% for 0.5mm cortex. © 2012 Elsevier B.V.

Improved bounds for subband-adaptive iterative shrinkage/thresholding algorithms.

This paper presents new methods for computing the step sizes of the subband-adaptive iterative shrinkage-thresholding algorithms proposed by Bayram & Selesnick and Vonesch & Unser. The method yields tighter wavelet-domain bounds of the system matrix, thus leading to improved convergence speeds. It is directly applicable to non-redundant wavelet bases, and we also adapt it for cases of redundant frames. It turns out that the simplest and most intuitive setting for the step sizes that ignores subband aliasing is often satisfactory in practice. We show that our methods can be used to advantage with reweighted least squares penalty functions as well as L1 penalties. We emphasize that the algorithms presented here are suitable for performing inverse filtering on very large datasets, including 3D data, since inversions are applied only to diagonal matrices and fast transforms are used to achieve all matrix-vector products.

Change prediction using interface data

Managing change can be challenging due to the high levels of interdependency in concurrent engineering processes. A key activity in engineering change management is propagation analysis, which can be supported using the change prediction method. In common with most other change prediction approaches, the change prediction method has three important limitations: L1: it depends on highly subjective input data; L2: it is capable of modelling 'generalised cases' only and cannot be; customised to assess specific changes; and L3: the input data are static, and thus, guidance does not reflect changes in the design. This article contributes to resolving these limitations by incorporating interface information into the change prediction method. The enhanced method is illustrated using an example based on a flight simulator. © The Author(s) 2013.