Accelerating Pseudo-Marginal MCMC using Gaussian Processes

Pseudo-marginal methods such as the grouped independence Metropolis-Hastings (GIMH) and Markov chain within Metropolis (MCWM) algorithms have been introduced in the literature as an approach to perform Bayesian inference in latent variable models. These methods replace intractable likelihood calculations with unbiased estimates within Markov chain Monte Carlo algorithms. The GIMH method has the posterior of interest as its limiting distribution, but suffers from poor mixing if it is too computationally intensive to obtain high-precision likelihood estimates. The MCWM algorithm has better mixing properties, but less theoretical support. In this paper we propose to use Gaussian processes (GP) to accelerate the GIMH method, whilst using a short pilot run of MCWM to train the GP. Our new method, GP-GIMH, is illustrated on simulated data from a stochastic volatility and a gene network model.

Pseudo-marginal algorithms with multiple CPUs

The emergence of pseudo-marginal algorithms has led to improved computational efficiency for dealing with complex Bayesian models with latent variables. Here an unbiased estimator of the likelihood replaces the true likelihood in order to produce a Bayesian algorithm that remains on the marginal space of the model parameter (with latent variables integrated out), with a target distribution that is still the correct posterior distribution. Very efficient proposal distributions can be developed on the marginal space relative to the joint space of model parameter and latent variables. Thus psuedo-marginal algorithms tend to have substantially better mixing properties. However, for pseudo-marginal approaches to perform well, the likelihood has to be estimated rather precisely. This can be difficult to achieve in complex applications. In this paper we propose to take advantage of multiple central processing units (CPUs), that are readily available on most standard desktop computers. Here the likelihood is estimated independently on the multiple CPUs, with the ultimate estimate of the likelihood being the average of the estimates obtained from the multiple CPUs. The estimate remains unbiased, but the variability is reduced. We compare and contrast two different technologies that allow the implementation of this idea, both of which require a negligible amount of extra programming effort. The superior performance of this idea over the standard approach is demonstrated on simulated data from a stochastic volatility model.

A pseudo-marginal sequential Monte Carlo algorithm for random effects models in Bayesian sequential design

A computationally efficient sequential Monte Carlo algorithm is proposed for the sequential design of experiments for the collection of block data described by mixed effects models. The difficulty in applying a sequential Monte Carlo algorithm in such settings is the need to evaluate the observed data likelihood, which is typically intractable for all but linear Gaussian models. To overcome this difficulty, we propose to unbiasedly estimate the likelihood, and perform inference and make decisions based on an exact-approximate algorithm. Two estimates are proposed: using Quasi Monte Carlo methods and using the Laplace approximation with importance sampling. Both of these approaches can be computationally expensive, so we propose exploiting parallel computational architectures to ensure designs can be derived in a timely manner. We also extend our approach to allow for model uncertainty. This research is motivated by important pharmacological studies related to the treatment of critically ill patients.

Seismic reliability assessment of external stability of reinforced soil walls using pseudo-dynamic method

The paper presents a method for the evaluation of external stability of reinforced soil walls subjected to earthquakes in the framework of the pseudo-dynamic method. The seismic reliability of the wall is evaluated by considering the different possible failure modes such as sliding along the base, overturning about the toe point of the wall, bearing capacity and the eccentricity of the resultant force. The analysis is performed considering properties of the reinforced backfill, foundation soil below the base of the wall, length of the geosynthetic reinforcement and characteristics of earthquake ground motions such as shear wave and primary wave velocity as random variables. The optimum length of reinforcement needed to maintain stability against four modes of failure by targeting various component reliability indices is obtained. Differences between pseudo-static and pseudo-dynamic methods are clearly highlighted in the paper. A complete analysis of pseudo-static and pseudo-dynamic methodologies shows that the pseudodynamic method results in realistic design values for the length of geosynthetic reinforcement under earthquake conditions.

Reliability Based Design Optimization for Internal Stability of Reinforced Soil Structures Using Pseudo‐Static Method

Seismic design of reinforced soil structures involves many uncertainties that arise from the backfill soil properties and tensile strength of the reinforcement which is not addressed in current design guidelines. This paper highlights the significance of variability in the internal stability assessment of reinforced soil structures. Reliability analysis is applied to estimate probability of failure and pseudo‐static approach has been used for the calculation of the tensile strength and length of the reinforcement needed to maintain the internal stability against tension and pullout failures. Logarithmic spiral failure surface has been considered in conjunction with the limit equilibrium method. Two modes of failure namely, tension failure and pullout failure have been considered. The influence of variations of the backfill soil friction angle, the tensile strength of reinforcement, horizontal seismic acceleration on the reliability index against tension failure and pullout failure of reinforced earth structure have been discussed.

Seismic reliability assessment of internal stability of reinforced soil walls using the pseudo-dynamic method

The study focuses on probabilistic assessment of the internal seismic stability of reinforced soil structures (RSS) subjected to earthquake loading in the framework of the pseudo-dynamic method. In the literature, the pseudo-static approach has been used to compute reliability indices against the tension and pullout failure modes, and the real dynamic nature of earthquake accelerations cannot be considered. The work presented in this paper makes use of the horizontal and vertical sinusoidal accelerations, amplification of vibrations, shear wave and primary wave velocities and time period. This approach is applied to quantify the influence of the backfill properties, geosynthetic reinforcement and characteristics of earthquake ground motions on reliability indices in relation to the tension and pullout failure modes. Seismic reliability indices at different levels of geosynthetic layers are determined for different magnitudes of seismic acceleration, soil amplification, shear wave and primary wave velocities. The results are compared with the pseudo-static method, and the significance of the present methodology for designing reinforced soil structures is discussed.

Influence of strain localization on reliability based design of bridge abutments using pseudo-dynamic method

The paper focuses on reliability based design of bridge abutments when subjected to earthquake loading. Planar failure surface has been used in conjunction with pseudo-dynamic approach to compute the seismic active earth pressures on the bridge abutment. The proposed pseudo dynamic method, considers the effects of strain localization in the backfill soil and associated post-peak reduction in the shear resistance from peak to residual values along a previously formed failure plane, phase difference in shear waves and soil amplification along with the horizontal seismic accelerations. Four modes of stability viz. sliding, overturning, eccentricity and bearing capacity of the foundation soil are considered for the reliability analysis. The influence of various design parameters on the seismic reliability indices against four modes of failure is presented, following the suggestions of Japan Road Association, Caltrans Bridge Design Specifications and U.S Department of the Army.

Electron-impact ionization of argon using the R-matrix with a pseudo-states method

Electron-impact ionization cross sections for argon are calculated using both non-perturbative R-matrix with pseudo-states (RMPS) and perturbative distorted-wave methods. At twice the ionization potential, the 3p(61)S ground-term cross section from a distorted-wave calculation is found to be a factor of 4 above crossed-beams experimental measurements, while with the inclusion of term-dependent continuum effects in the distorted-wave method, the perturbative cross section still remains almost a factor of 2 above experiment. In the case of ionization from the metastable 3p(5)4s(3)P term, the distorted-wave ionization cross section is also higher than the experimental cross section. On the other hand, the ground-term cross section determined from a nonperturbative RMPS calculation that includes 27 LS spectroscopic terms and another 282 LS pseudo-state terms to represent the high Rydberg states, and the target continuum is found to be in excellent agreement with experimental measurements, while the RMPS result is below the experimental cross section for ionization from the metastable term. We conclude that both continuum term dependence and interchannel coupling effects, which are included in the RMPS method, are important for ionization from the ground term, and interchannel coupling is also significant for ionization from the metastable term

Scalable Bayesian Computation for intractable likelihoods in image analysis

The inverse temperature hyperparameter of the hidden Potts model governs the strength of spatial cohesion and therefore has a substantial influence over the resulting model fit. The difficulty arises from the dependence of an intractable normalising constant on the value of the inverse temperature, thus there is no closed form solution for sampling from the distribution directly. We review three computational approaches for addressing this issue, namely pseudolikelihood, path sampling, and the approximate exchange algorithm. We compare the accuracy and scalability of these methods using a simulation study.

Alive SMC^2: Bayesian model selection for low-count time series models with intractable likelihoods

In this paper we present a new method for performing Bayesian parameter inference and model choice for low count time series models with intractable likelihoods. The method involves incorporating an alive particle filter within a sequential Monte Carlo (SMC) algorithm to create a novel pseudo-marginal algorithm, which we refer to as alive SMC^2. The advantages of this approach over competing approaches is that it is naturally adaptive, it does not involve between-model proposals required in reversible jump Markov chain Monte Carlo and does not rely on potentially rough approximations. The algorithm is demonstrated on Markov process and integer autoregressive moving average models applied to real biological datasets of hospital-acquired pathogen incidence, animal health time series and the cumulative number of poison disease cases in mule deer.

An economical method for direct numerical simulation studies of transitional round jets

A pseudo-spectral method based on Fourier expansions in a Cartesian coordinate system is shown to be an economical method for direct numerical simulation studies of transitional round jets, Several characteristics of the solutions are presented to establish the validity of the solutions in spite of the unnatural choices. We show that neither periodicity, nor the use of a Cartesian system have adversely affected the simulations, Instead, there are benefits in terms of ease of computing and lack of the usual restrictions due to grid structure near the jet axis. By computing the simultaneous evolution of passive scalers, the process of reaction in round jet burners, between a fuel-laden jet and an ambient oxidizer, was also simulated. Some typical solutions are shown and then the results of analysis of these data are summarized. (C) 2001 Elsevier Science Ltd, All rights reserved.

Pseudo-Static and Pseudo-Dynamic Stability Analysis of Tailings Dam Under Seismic Conditions

In this paper the seismic slope stability analyses are performed for a typical section of 44 m high water retention type tailings earthen dam located in the eastern part of India, using both the conventional pseudo-static and recent pseudo-dynamic methods. The tailings earthen dam is analyzed for different upstream conditions of reservoir like filled up with compacted and non-compacted dumped waste materials with different water levels of the pond tailings portion. Phreatic surface is generated using seepage analysis in geotechnical software SEEP/W and that same is used in the pseudo-static and pseudo-dynamic analyses to make the approach more realistic. The minimum values of factor of safety using pseudo-static and pseudo-dynamic method are obtained as 1.18 and 1.09 respectively for the chosen seismic zone in India. These values of factor of safety show clearly the demerits of conventional pseudo-static analysis compared to recent pseudo-dynamic analysis, where in addition to the seismic accelerations, duration, frequency of earthquake, body waves traveling during earthquake and amplification effects are considered.

Leveraging Web 2.0 data for scalable semi-supervised learning of domain-specific sentiment lexicons

Since manually constructing domain-specific sentiment lexicons is extremely time consuming and it may not even be feasible for domains where linguistic expertise is not available. Research on the automatic construction of domain-specific sentiment lexicons has become a hot topic in recent years. The main contribution of this paper is the illustration of a novel semi-supervised learning method which exploits both term-to-term and document-to-term relations hidden in a corpus for the construction of domain specific sentiment lexicons. More specifically, the proposed two-pass pseudo labeling method combines shallow linguistic parsing and corpusbase statistical learning to make domain-specific sentiment extraction scalable with respect to the sheer volume of opinionated documents archived on the Internet these days. Another novelty of the proposed method is that it can utilize the readily available user-contributed labels of opinionated documents (e.g., the user ratings of product reviews) to bootstrap the performance of sentiment lexicon construction. Our experiments show that the proposed method can generate high quality domain-specific sentiment lexicons as directly assessed by human experts. Moreover, the system generated domain-specific sentiment lexicons can improve polarity prediction tasks at the document level by 2:18% when compared to other well-known baseline methods. Our research opens the door to the development of practical and scalable methods for domain-specific sentiment analysis.

Bayesian synthetic likelihood

Having the ability to work with complex models can be highly beneficial, but the computational cost of doing so is often large. Complex models often have intractable likelihoods, so methods that directly use the likelihood function are infeasible. In these situations, the benefits of working with likelihood-free methods become apparent. Likelihood-free methods, such as parametric Bayesian indirect likelihood that uses the likelihood of an alternative parametric auxiliary model, have been explored throughout the literature as a good alternative when the model of interest is complex. One of these methods is called the synthetic likelihood (SL), which assumes a multivariate normal approximation to the likelihood of a summary statistic of interest. This paper explores the accuracy and computational efficiency of the Bayesian version of the synthetic likelihood (BSL) approach in comparison to a competitor known as approximate Bayesian computation (ABC) and its sensitivity to its tuning parameters and assumptions. We relate BSL to pseudo-marginal methods and propose to use an alternative SL that uses an unbiased estimator of the exact working normal likelihood when the summary statistic has a multivariate normal distribution. Several applications of varying complexity are considered to illustrate the findings of this paper.