Squared Extrapolation Methods (SQUAREM): A New Class of Simple and Efficient Numerical Schemes for Accelerating the Convergence of the EM Algorithm

We derive a new class of iterative schemes for accelerating the convergence of the EM algorithm, by exploiting the connection between fixed point iterations and extrapolation methods. First, we present a general formulation of one-step iterative schemes, which are obtained by cycling with the extrapolation methods. We, then square the one-step schemes to obtain the new class of methods, which we call SQUAREM. Squaring a one-step iterative scheme is simply applying it twice within each cycle of the extrapolation method. Here we focus on the first order or rank-one extrapolation methods for two reasons, (1) simplicity, and (2) computational efficiency. In particular, we study two first order extrapolation methods, the reduced rank extrapolation (RRE1) and minimal polynomial extrapolation (MPE1). The convergence of the new schemes, both one-step and squared, is non-monotonic with respect to the residual norm. The first order one-step and SQUAREM schemes are linearly convergent, like the EM algorithm but they have a faster rate of convergence. We demonstrate, through five different examples, the effectiveness of the first order SQUAREM schemes, SqRRE1 and SqMPE1, in accelerating the EM algorithm. The SQUAREM schemes are also shown to be vastly superior to their one-step counterparts, RRE1 and MPE1, in terms of computational efficiency. The proposed extrapolation schemes can fail due to the numerical problems of stagnation and near breakdown. We have developed a new hybrid iterative scheme that combines the RRE1 and MPE1 schemes in such a manner that it overcomes both stagnation and near breakdown. The squared first order hybrid scheme, SqHyb1, emerges as the iterative scheme of choice based on our numerical experiments. It combines the fast convergence of the SqMPE1, while avoiding near breakdowns, with the stability of SqRRE1, while avoiding stagnations. The SQUAREM methods can be incorporated very easily into an existing EM algorithm. They only require the basic EM step for their implementation and do not require any other auxiliary quantities such as the complete data log likelihood, and its gradient or hessian. They are an attractive option in problems with a very large number of parameters, and in problems where the statistical model is complex, the EM algorithm is slow and each EM step is computationally demanding.

Quick divergence but slow convergence during ecotype formation in lake and stream stickleback pairs of variable age

When genetic constraints restrict phenotypic evolution, diversification can be predicted to evolve along so-called lines of least resistance. To address the importance of such constraints and their resolution, studies of parallel phenotypic divergence that differ in their age are valuable. Here, we investigate the parapatric evolution of six lake and stream threespine stickleback systems from Iceland and Switzerland, ranging in age from a few decades to several millennia. Using phenotypic data, we test for parallelism in ecotypic divergence between parapatric lake and stream populations and compare the observed patterns to an ancestral-like marine population. We find strong and consistent phenotypic divergence, both among lake and stream populations and between our freshwater populations and the marine population. Interestingly, ecotypic divergence in low-dimensional phenotype space (i.e. single traits) is rapid and seems to be often completed within 100 years. Yet, the dimensionality of ecotypic divergence was highest in our oldest systems and only there parallel evolution of unrelated ecotypes was strong enough to overwrite phylogenetic contingency. Moreover, the dimensionality of divergence in different systems varies between trait complexes, suggesting different constraints and evolutionary pathways to their resolution among freshwater systems.

Accumulation rates and grain sizes of eolian dust in sediments of the Intertropical Convergence Zone in the northeast Pacific

A 328 cm-long piston core (KODOS 02-01-02) collected from the northeast equatorial Pacific at 16°12'N, 125°59'W was investigated for eolian mass fluxes and grain sizes to test these proxies as a tool for the paleo-position of the Intertropical Convergence Zone (ITCZ). The eolian mass fluxes of the lower interval below 250 cm (15.5-7.6 Ma) are very uniform at 5 +/- 1 mg/cm**2/kyr, while those of the upper interval above 250 cm (from 7.6 Ma) are over 2 times higher than the lower interval at 12 +/- 1 mg/cm**2/kyr. The median grain size of the eolian dusts in the lower interval increases from 8.4 Phi to 8.0 Phi downward, while that of the upper interval varies in a narrow range from 8.8 Phi to 8.6 Phi. The determined values compare well in magnitude to those of central Pacific sediments for the upper interval and equatorial and southeast Pacific sediments for the lower interval. This result suggests a possibility that the study site had been under the influence of southeast trade winds at its earlier depositional period due to the northerly position of the ITCZ, and subsequently of the northeast trade winds for a later period when the upper sediments were deposited. This interpretation is consistent with a mineralogical and geochemical study published elsewhere that assigned the provenance of the study core dust to Central/South America for the lower interval and to Asia for the upper interval. This study suggests that the distinct differences in eolian mass flux and grain size observed across the ITCZ can be used to trace the paleo-latitude of the ITCZ.

Social cleavages and electoral support in Turkey -- toward convergence?

Studies on Western democracies have shown that deep-seated social cleavages stabilize the electoral behavior and thus reduce electoral volatility. But how do social cleavages affect a party system that is undergoing democratic consolidation, such as in Turkey? In this study, investigations were carried out on long- and short-term relationships between social cleavages (religiosity, ethnicity, and sectarism) and electoral volatility in Turkey during the 1961-2002 period. Cross-sectional multiple regressions were applied to electoral and demographic data at the provincial level. The results showed that in the long-term, social cleavages on the whole have increased volatility rather than reduced it. The cleavage-volatility relationship, however, has changed over time. Repeated elections have mitigated the volatile effect of social cleavages on the voting behavior, as political parties have become more representative of the existent social cleavages.

Network convergence and QoS for future multimedia services in the VISION project

The emerging use of real-time 3D-based multimedia applications imposes strict quality of service (QoS) requirements on both access and core networks. These requirements and their impact to provide end-to-end 3D videoconferencing services have been studied within the Spanish-funded VISION project, where different scenarios were implemented showing an agile stereoscopic video call that might be offered to the general public in the near future. In view of the requirements, we designed an integrated access and core converged network architecture which provides the requested QoS to end-to-end IP sessions. Novel functional blocks are proposed to control core optical networks, the functionality of the standard ones is redefined, and the signaling improved to better meet the requirements of future multimedia services. An experimental test-bed to assess the feasibility of the solution was also deployed. In such test-bed, set-up and release of end-to-end sessions meeting specific QoS requirements are shown and the impact of QoS degradation in terms of the user perceived quality degradation is quantified. In addition, scalability results show that the proposed signaling architecture is able to cope with large number of requests introducing almost negligible delay.