Bayesian and L1 Approaches to Sparse Unsupervised Learning

The use of L1 regularisation for sparse learning has generated immense research interest, with successful application in such diverse areas as signal acquisition, image coding, genomics and collaborative filtering. While existing work highlights the many advantages of L1 methods, in this paper we find that L1 regularisation often dramatically underperforms in terms of predictive performance when compared with other methods for inferring sparsity. We focus on unsupervised latent variable models, and develop L1 minimising factor models, Bayesian variants of "L1", and Bayesian models with a stronger L0-like sparsity induced through spike-and-slab distributions. These spike-and-slab Bayesian factor models encourage sparsity while accounting for uncertainty in a principled manner and avoiding unnecessary shrinkage of non-zero values. We demonstrate on a number of data sets that in practice spike-and-slab Bayesian methods outperform L1 minimisation, even on a computational budget. We thus highlight the need to re-assess the wide use of L1 methods in sparsity-reliant applications, particularly when we care about generalising to previously unseen data, and provide an alternative that, over many varying conditions, provides improved generalisation performance.

Movement of deposited water on turbomachinery rotor blade surfaces

A theoretical approach for calculating the movement of liquid water following deposition onto a turbomachine rotor blade is described. Such a situation can occur during operation of an aero-engine in rain. The equation of motion of the deposited water is developed on an arbitrarily oriented plane triangular surface facet. By dividing the blade surface into a large number of facets and calculating the water trajectory over each one crossed in turn, the overall trajectory can be constructed. Apart from the centrifugal and Coriolis inertia effects, the forces acting on the water arise from the blade surface friction, and the aerodynamic shear and pressure gradient. Non- dimensionalisation of the equations of motion provides considerable insight and a detailed study of water flow on a flat rotating plate set at different stagger angles demonstrates the paramount importance of blade surface friction. The extreme cases of low and high blade friction are examined and it is concluded that the latter (which allows considerable mathematical generalisation) is the most likely in practice. It is also shown that the aerodynamic shear force, but not the pressure force, may influence the water motion. Calculations of water movement on a low-speed compressor blade and the fan blade of a high bypass ratio aero-engine suggest that in low rotational speed situations most of the deposited water is centrifuged rapidly to the blade tip region. Copyright © 2006 by ASME.

Gaussian mixture modeling with Gaussian process latent variable models

Density modeling is notoriously difficult for high dimensional data. One approach to the problem is to search for a lower dimensional manifold which captures the main characteristics of the data. Recently, the Gaussian Process Latent Variable Model (GPLVM) has successfully been used to find low dimensional manifolds in a variety of complex data. The GPLVM consists of a set of points in a low dimensional latent space, and a stochastic map to the observed space. We show how it can be interpreted as a density model in the observed space. However, the GPLVM is not trained as a density model and therefore yields bad density estimates. We propose a new training strategy and obtain improved generalisation performance and better density estimates in comparative evaluations on several benchmark data sets. © 2010 Springer-Verlag.

Making a shallow network deep: Conversion of a boosting classifier into a decision tree by boolean optimisation

This paper presents a novel way to speed up the evaluation time of a boosting classifier. We make a shallow (flat) network deep (hierarchical) by growing a tree from decision regions of a given boosting classifier. The tree provides many short paths for speeding up while preserving the reasonably smooth decision regions of the boosting classifier for good generalisation. For converting a boosting classifier into a decision tree, we formulate a Boolean optimization problem, which has been previously studied for circuit design but limited to a small number of binary variables. In this work, a novel optimisation method is proposed for, firstly, several tens of variables i.e. weak-learners of a boosting classifier, and then any larger number of weak-learners by using a two-stage cascade. Experiments on the synthetic and face image data sets show that the obtained tree achieves a significant speed up both over a standard boosting classifier and the Fast-exit-a previously described method for speeding-up boosting classification, at the same accuracy. The proposed method as a general meta-algorithm is also useful for a boosting cascade, where it speeds up individual stage classifiers by different gains. The proposed method is further demonstrated for fast-moving object tracking and segmentation problems. © 2011 Springer Science+Business Media, LLC.

Ventilation effectiveness measures based on heat removal: Part 1. Definitions

The effectiveness of ventilation flows is considered from the perspective of buoyancy (or heat) removal from a space. This perspective is distinct from the standard in which the effectiveness is based on the concentrations of a neutrally buoyant contaminant/passive tracer. Three new measures of effectiveness are proposed based on the ability of a flow to flush buoyancy from a ventilated space. These measures provide estimates of instantaneous and time-averaged effectiveness for the entire space, and local effectiveness at any height of interest. From a generalisation of the latter, a vertical profile of effectiveness is defined. These measures enable quantitative comparisons to be made between different flows and they are applicable when there is a difference in density (as is typical due to temperature differences) between the interior environment and the replacement air. Applications, therefore, include natural ventilation, hybrid ventilation and a range of forced ventilation flows. Finally, we demonstrate how the ventilation effectiveness of a room may be assessed from simple traces of temperature versus time. © 2006 Elsevier Ltd. All rights reserved.