Nonparametric Bayesian sparse factor models with application to gene expression modeling

A nonparametric Bayesian extension of Factor Analysis (FA) is proposed where observed data $\mathbf{Y}$ is modeled as a linear superposition, $\mathbf{G}$, of a potentially infinite number of hidden factors, $\mathbf{X}$. The Indian Buffet Process (IBP) is used as a prior on $\mathbf{G}$ to incorporate sparsity and to allow the number of latent features to be inferred. The model's utility for modeling gene expression data is investigated using randomly generated data sets based on a known sparse connectivity matrix for E. Coli, and on three biological data sets of increasing complexity.

Construction of nonparametric Bayesian models from parametric Bayes equations

We consider the general problem of constructing nonparametric Bayesian models on infinite-dimensional random objects, such as functions, infinite graphs or infinite permutations. The problem has generated much interest in machine learning, where it is treated heuristically, but has not been studied in full generality in non-parametric Bayesian statistics, which tends to focus on models over probability distributions. Our approach applies a standard tool of stochastic process theory, the construction of stochastic processes from their finite-dimensional marginal distributions. The main contribution of the paper is a generalization of the classic Kolmogorov extension theorem to conditional probabilities. This extension allows a rigorous construction of nonparametric Bayesian models from systems of finite-dimensional, parametric Bayes equations. Using this approach, we show (i) how existence of a conjugate posterior for the nonparametric model can be guaranteed by choosing conjugate finite-dimensional models in the construction, (ii) how the mapping to the posterior parameters of the nonparametric model can be explicitly determined, and (iii) that the construction of conjugate models in essence requires the finite-dimensional models to be in the exponential family. As an application of our constructive framework, we derive a model on infinite permutations, the nonparametric Bayesian analogue of a model recently proposed for the analysis of rank data.

Physical consequences of a nonparametric uncertainty model in structural dynamics

One of the main claims of the nonparametric model of random uncertainty introduced by Soize (2000) [3] is its ability to account for model uncertainty. The present paper investigates this claim by examining the statistics of natural frequencies, total energy and underlying dispersion equation yielded by the nonparametric approach for two simple systems: a thin plate in bending and a one-dimensional finite periodic massspring chain. Results for the plate show that the average modal density and the underlying dispersion equation of the structure are gradually and systematically altered with increasing uncertainty. The findings for the massspring chain corroborate the findings for the plate and show that the remote coupling of nonadjacent degrees of freedom induced by the approach suppresses the phenomenon of mode localization. This remote coupling also leads to an instantaneous response of all points in the chain when one mass is excited. In the light of these results, it is argued that the nonparametric approach can deal with a certain type of model uncertainty, in this case the presence of unknown terms of higher or lower order in the governing differential equation, but that certain expectations about the system such as the average modal density may conflict with these results. © 2012 Elsevier Ltd.

Internal representations of temporal statistics and feedback calibrate motor-sensory interval timing.

Humans have been shown to adapt to the temporal statistics of timing tasks so as to optimize the accuracy of their responses, in agreement with the predictions of Bayesian integration. This suggests that they build an internal representation of both the experimentally imposed distribution of time intervals (the prior) and of the error (the loss function). The responses of a Bayesian ideal observer depend crucially on these internal representations, which have only been previously studied for simple distributions. To study the nature of these representations we asked subjects to reproduce time intervals drawn from underlying temporal distributions of varying complexity, from uniform to highly skewed or bimodal while also varying the error mapping that determined the performance feedback. Interval reproduction times were affected by both the distribution and feedback, in good agreement with a performance-optimizing Bayesian observer and actor model. Bayesian model comparison highlighted that subjects were integrating the provided feedback and represented the experimental distribution with a smoothed approximation. A nonparametric reconstruction of the subjective priors from the data shows that they are generally in agreement with the true distributions up to third-order moments, but with systematically heavier tails. In particular, higher-order statistical features (kurtosis, multimodality) seem much harder to acquire. Our findings suggest that humans have only minor constraints on learning lower-order statistical properties of unimodal (including peaked and skewed) distributions of time intervals under the guidance of corrective feedback, and that their behavior is well explained by Bayesian decision theory.

Bayesian parameter identification for vibro-acoustic models with nonparametric uncertainty

Vibration and acoustic analysis at higher frequencies faces two challenges: computing the response without using an excessive number of degrees of freedom, and quantifying its uncertainty due to small spatial variations in geometry, material properties and boundary conditions. Efficient models make use of the observation that when the response of a decoupled vibro-acoustic subsystem is sufficiently sensitive to uncertainty in such spatial variations, the local statistics of its natural frequencies and mode shapes saturate to universal probability distributions. This holds irrespective of the causes that underly these spatial variations and thus leads to a nonparametric description of uncertainty. This work deals with the identification of uncertain parameters in such models by using experimental data. One of the difficulties is that both experimental errors and modeling errors, due to the nonparametric uncertainty that is inherent to the model type, are present. This is tackled by employing a Bayesian inference strategy. The prior probability distribution of the uncertain parameters is constructed using the maximum entropy principle. The likelihood function that is subsequently computed takes the experimental information, the experimental errors and the modeling errors into account. The posterior probability distribution, which is computed with the Markov Chain Monte Carlo method, provides a full uncertainty quantification of the identified parameters, and indicates how well their uncertainty is reduced, with respect to the prior information, by the experimental data. © 2013 Taylor & Francis Group, London.

Nonparametric Bayesian Density Modeling with Gaussian Processes

We present the Gaussian process density sampler (GPDS), an exchangeable generative model for use in nonparametric Bayesian density estimation. Samples drawn from the GPDS are consistent with exact, independent samples from a distribution defined by a density that is a transformation of a function drawn from a Gaussian process prior. Our formulation allows us to infer an unknown density from data using Markov chain Monte Carlo, which gives samples from the posterior distribution over density functions and from the predictive distribution on data space. We describe two such MCMC methods. Both methods also allow inference of the hyperparameters of the Gaussian process.

The statistics of the second order response of an FPSO in spreading seas

An expression for the probability density function of the second order response of a general FPSO in spreading seas is derived by using the Kac-Siegert approach. Various approximations of the second order force transfer functions are investigated for a ship-shaped FPSO. It is found that, when expressed in non-dimensional form, the probability density function of the response is not particularly sensitive to wave spreading, although the mean squared response and the resulting dimensional extreme values can be sensitive. The analysis is then applied to a Sevan FPSO, which is a large cylindrical buoy-like structure. The second order force transfer functions are derived by using an efficient semi-analytical hydrodynamic approach, and these are then employed to yield the extreme response. However, a significant effect of wave spreading on the statistics for a Sevan FPSO is found even in non-dimensional form. It implies that the exact statistics of a general ship-shaped FPSO may be sensitive to the wave direction, which needs to be verified in future work. It is also pointed out that the Newman's approximation regarding the frequency dependency of force transfer function is acceptable even for the spreading seas. An improvement on the results may be attained when considering the angular dependency exactly. Copyright © 2009 by ASME.

The ensemble statistics of the band-averaged energy of a random system

This paper is concerned with the response statistics of a dynamic system that has random properties. The frequency-band-averaged energy of the system is considered, and a closed form expression is derived for the relative variance of this quantity. The expression depends upon three parameters: the modal overlap factor m, a bandwidth parameter B, and a parameter α that defines the nature of the loading (for example single point forcing or rain-on-the-roof loading). The result is applicable to any single structural component or acoustic volume, and a comparison is made here with simulation results for a mass loaded plate. Good agreement is found between the simulations and the theory. © 2003 Published by Elsevier Ltd.

The ensemble statistics of the energy of a random system subjected to harmonic excitation

This paper is concerned with the ensemble statistics of the response to harmonic excitation of a single dynamic system such as a plate or an acoustic volume. Random point process theory is employed, and various statistical assumptions regarding the system natural frequencies are compared, namely: (i) Poisson natural frequency spacings, (ii) statistically independent Rayleigh natural frequency spacings, and (iii) natural frequency spacings conforming to the Gaussian orthogonal ensemble (GOE). The GOE is found to be the most realistic assumption, and simple formulae are derived for the variance of the energy of the system under either point loading or rain-on-the-roof excitation. The theoretical results are compared favourably with numerical simulations and experimental data for the case of a mass loaded plate. © 2003 Elsevier Ltd. All rights reserved.

Tractable nonparametric bayesian inference in poisson processes with Gaussian process intensities

The inhomogeneous Poisson process is a point process that has varying intensity across its domain (usually time or space). For nonparametric Bayesian modeling, the Gaussian process is a useful way to place a prior distribution on this intensity. The combination of a Poisson process and GP is known as a Gaussian Cox process, or doubly-stochastic Poisson process. Likelihood-based inference in these models requires an intractable integral over an infinite-dimensional random function. In this paper we present the first approach to Gaussian Cox processes in which it is possible to perform inference without introducing approximations or finitedimensional proxy distributions. We call our method the Sigmoidal Gaussian Cox Process, which uses a generative model for Poisson data to enable tractable inference via Markov chain Monte Carlo. We compare our methods to competing methods on synthetic data and apply it to several real-world data sets. Copyright 2009.

Tractable nonparametric Bayesian inference in Poisson processes with Gaussian process intensities

The inhomogeneous Poisson process is a point process that has varying intensity across its domain (usually time or space). For nonparametric Bayesian modeling, the Gaussian process is a useful way to place a prior distribution on this intensity. The combination of a Poisson process and GP is known as a Gaussian Cox process, or doubly-stochastic Poisson process. Likelihood-based inference in these models requires an intractable integral over an infinite-dimensional random function. In this paper we present the first approach to Gaussian Cox processes in which it is possible to perform inference without introducing approximations or finite-dimensional proxy distributions. We call our method the Sigmoidal Gaussian Cox Process, which uses a generative model for Poisson data to enable tractable inference via Markov chain Monte Carlo. We compare our methods to competing methods on synthetic data and apply it to several real-world data sets.