Squared Extrapolation Methods (SQUAREM): A New Class of Simple and Efficient Numerical Schemes for Accelerating the Convergence of the EM Algorithm

We derive a new class of iterative schemes for accelerating the convergence of the EM algorithm, by exploiting the connection between fixed point iterations and extrapolation methods. First, we present a general formulation of one-step iterative schemes, which are obtained by cycling with the extrapolation methods. We, then square the one-step schemes to obtain the new class of methods, which we call SQUAREM. Squaring a one-step iterative scheme is simply applying it twice within each cycle of the extrapolation method. Here we focus on the first order or rank-one extrapolation methods for two reasons, (1) simplicity, and (2) computational efficiency. In particular, we study two first order extrapolation methods, the reduced rank extrapolation (RRE1) and minimal polynomial extrapolation (MPE1). The convergence of the new schemes, both one-step and squared, is non-monotonic with respect to the residual norm. The first order one-step and SQUAREM schemes are linearly convergent, like the EM algorithm but they have a faster rate of convergence. We demonstrate, through five different examples, the effectiveness of the first order SQUAREM schemes, SqRRE1 and SqMPE1, in accelerating the EM algorithm. The SQUAREM schemes are also shown to be vastly superior to their one-step counterparts, RRE1 and MPE1, in terms of computational efficiency. The proposed extrapolation schemes can fail due to the numerical problems of stagnation and near breakdown. We have developed a new hybrid iterative scheme that combines the RRE1 and MPE1 schemes in such a manner that it overcomes both stagnation and near breakdown. The squared first order hybrid scheme, SqHyb1, emerges as the iterative scheme of choice based on our numerical experiments. It combines the fast convergence of the SqMPE1, while avoiding near breakdowns, with the stability of SqRRE1, while avoiding stagnations. The SQUAREM methods can be incorporated very easily into an existing EM algorithm. They only require the basic EM step for their implementation and do not require any other auxiliary quantities such as the complete data log likelihood, and its gradient or hessian. They are an attractive option in problems with a very large number of parameters, and in problems where the statistical model is complex, the EM algorithm is slow and each EM step is computationally demanding.

Whither PQL?

Generalized linear mixed models (GLMM) are generalized linear models with normally distributed random effects in the linear predictor. Penalized quasi-likelihood (PQL), an approximate method of inference in GLMMs, involves repeated fitting of linear mixed models with “working” dependent variables and iterative weights that depend on parameter estimates from the previous cycle of iteration. The generality of PQL, and its implementation in commercially available software, has encouraged the application of GLMMs in many scientific fields. Caution is needed, however, since PQL may sometimes yield badly biased estimates of variance components, especially with binary outcomes. Recent developments in numerical integration, including adaptive Gaussian quadrature, higher order Laplace expansions, stochastic integration and Markov chain Monte Carlo (MCMC) algorithms, provide attractive alternatives to PQL for approximate likelihood inference in GLMMs. Analyses of some well known datasets, and simulations based on these analyses, suggest that PQL still performs remarkably well in comparison with more elaborate procedures in many practical situations. Adaptive Gaussian quadrature is a viable alternative for nested designs where the numerical integration is limited to a small number of dimensions. Higher order Laplace approximations hold the promise of accurate inference more generally. MCMC is likely the method of choice for the most complex problems that involve high dimensional integrals.

Percutaneous coronary interventions in Europe: prevalence, numerical estimates, and projections based on data up to 2004

AIMS: A registry mandated by the European Society of Cardiology collects data on trends in interventional cardiology within Europe. Special interest focuses on relative increases and ratios in new techniques and their distributions across Europe. We report the data through 2004 and give an overview of the development of coronary interventions since the first data collection in 1992. METHODS AND RESULTS: Questionnaires were distributed yearly to delegates of all national societies of cardiology represented in the European Society of Cardiology. The goal was to collect the case numbers of all local institutions and operators. The overall numbers of coronary angiographies increased from 1992 to 2004 from 684 000 to 2 238 000 (from 1250 to 3930 per million inhabitants). The respective numbers for percutaneous coronary interventions (PCIs) and coronary stenting procedures increased from 184 000 to 885 000 (from 335 to 1550) and from 3000 to 770 000 (from 5 to 1350), respectively. Germany was the most active country with 712 000 angiographies (8600), 249 000 angioplasties (3000), and 200 000 stenting procedures (2400) in 2004. The indication has shifted towards acute coronary syndromes, as demonstrated by rising rates of interventions for acute myocardial infarction over the last decade. The procedures are more readily performed and perceived safer, as shown by increasing rate of "ad hoc" PCIs and decreasing need for emergency coronary artery bypass grafting (CABG). In 2004, the use of drug-eluting stents continued to rise. However, an enormous variability is reported with the highest rate in Switzerland (70%). If the rate of progression remains constant until 2010 the projected number of coronary angiographies will be over three million, and the number of PCIs about 1.5 million with a stenting rate of almost 100%. CONCLUSION: Interventional cardiology in Europe is ever expanding. New coronary revascularization procedures, alternative or complementary to balloon angioplasty, have come and gone. Only stenting has stood the test of time and matured to the default technique. Facilitated access to PCI, more complete and earlier detection of coronary artery disease promise continued growth of the procedure despite the uncontested success of prevention.

Posterior Simulation in the Generalized Linear Model with Semiparmetric Random Effects

Generalized linear mixed models with semiparametric random effects are useful in a wide variety of Bayesian applications. When the random effects arise from a mixture of Dirichlet process (MDP) model, normal base measures and Gibbs sampling procedures based on the Pólya urn scheme are often used to simulate posterior draws. These algorithms are applicable in the conjugate case when (for a normal base measure) the likelihood is normal. In the non-conjugate case, the algorithms proposed by MacEachern and Müller (1998) and Neal (2000) are often applied to generate posterior samples. Some common problems associated with simulation algorithms for non-conjugate MDP models include convergence and mixing difficulties. This paper proposes an algorithm based on the Pólya urn scheme that extends the Gibbs sampling algorithms to non-conjugate models with normal base measures and exponential family likelihoods. The algorithm proceeds by making Laplace approximations to the likelihood function, thereby reducing the procedure to that of conjugate normal MDP models. To ensure the validity of the stationary distribution in the non-conjugate case, the proposals are accepted or rejected by a Metropolis-Hastings step. In the special case where the data are normally distributed, the algorithm is identical to the Gibbs sampler.