On Convergence Properties of the EM Algorithm for Gaussian Mixtures

"Expectation-Maximization'' (EM) algorithm and gradient-based approaches for maximum likelihood learning of finite Gaussian mixtures. We show that the EM step in parameter space is obtained from the gradient via a projection matrix $P$, and we provide an explicit expression for the matrix. We then analyze the convergence of EM in terms of special properties of $P$ and provide new results analyzing the effect that $P$ has on the likelihood surface. Based on these mathematical results, we present a comparative discussion of the advantages and disadvantages of EM and other algorithms for the learning of Gaussian mixture models.

On The Critical Points of some Iteration Methods for Solving Algebraic Equations. Global Convergence Properties

In this work we give su±cient conditions for k-th approximations of the polynomial roots of f(x) when the Maehly{Aberth{Ehrlich, Werner-Borsch-Supan, Tanabe, Improved Borsch-Supan iteration methods fail on the next step. For these methods all non-attractive sets are found. This is a subsequent improvement of previously developed techniques and known facts. The users of these methods can use the results presented here for software implementation in Distributed Applications and Simulation Environ- ments. Numerical examples with graphics are shown.

On the convergence of EM-like algorithms for image segmentation using Markov random fields.

Inference of Markov random field images segmentation models is usually performed using iterative methods which adapt the well-known expectation-maximization (EM) algorithm for independent mixture models. However, some of these adaptations are ad hoc and may turn out numerically unstable. In this paper, we review three EM-like variants for Markov random field segmentation and compare their convergence properties both at the theoretical and practical levels. We specifically advocate a numerical scheme involving asynchronous voxel updating, for which general convergence results can be established. Our experiments on brain tissue classification in magnetic resonance images provide evidence that this algorithm may achieve significantly faster convergence than its competitors while yielding at least as good segmentation results.

Boundary condition effects in the simulation study of equilibrium properties of magnetic dipolar fluids

In this paper we investigate the equilibrium properties of magnetic dipolar (ferro-) fluids and discuss finite-size effects originating from the use of different boundary conditions in computer simulations. Both periodic boundary conditions and a finite spherical box are studied. We demonstrate that periodic boundary conditions and subsequent use of Ewald sum to account for the long-range dipolar interactions lead to a much faster convergence (in terms of the number of investigated dipolar particles) of the magnetization curve and the initial susceptibility to their thermodynamic limits. Another unwanted effect of the simulations in a finite spherical box geometry is a considerable sensitivity to the container size. We further investigate the influence of the surface term in the Ewald sum-that is, due to the surrounding continuum with magnetic permeability mu(BC)-on the convergence properties of our observables and on the final results. The two different ways of evaluating the initial susceptibility, i.e., (1) by the magnetization response of the system to an applied field and (2) by the zero-field fluctuation of the mean-square dipole moment of the system, are compared in terms of speed and accuracy.

1/N expansion in noncommutative quantum mechanics

We study the 1/N expansion in noncommutative quantum mechanics for the anharmonic and Coulombian potentials. The expansion for the anharmonic oscillator presented good convergence properties, but for the Coulombian potential, we found a divergent large N expansion when using the usual noncommutative generalization of the potential. We proposed a modified version of the noncommutative Coulombian potential which provides a well-behaved 1/N expansion.

Automatic tuning method for the design of supplementary damping controllers for flexible alternating current transmission system devices

The design of supplementary damping controllers to mitigate the effects of electromechanical oscillations in power systems is a highly complex and time-consuming process, which requires a significant amount of knowledge from the part of the designer. In this study, the authors propose an automatic technique that takes the burden of tuning the controller parameters away from the power engineer and places it on the computer. Unlike other approaches that do the same based on robust control theories or evolutionary computing techniques, our proposed procedure uses an optimisation algorithm that works over a formulation of the classical tuning problem in terms of bilinear matrix inequalities. Using this formulation, it is possible to apply linear matrix inequality solvers to find a solution to the tuning problem via an iterative process, with the advantage that these solvers are widely available and have well-known convergence properties. The proposed algorithm is applied to tune the parameters of supplementary controllers for thyristor controlled series capacitors placed in the New England/New York benchmark test system, aiming at the improvement of the damping factor of inter-area modes, under several different operating conditions. The results of the linear analysis are validated by non-linear simulation and demonstrate the effectiveness of the proposed procedure.

Distributed Estimation Over an Adaptive Incremental Network Based on the Affine Projection Algorithm

We study the problem of distributed estimation based on the affine projection algorithm (APA), which is developed from Newton`s method for minimizing a cost function. The proposed solution is formulated to ameliorate the limited convergence properties of least-mean-square (LMS) type distributed adaptive filters with colored inputs. The analysis of transient and steady-state performances at each individual node within the network is developed by using a weighted spatial-temporal energy conservation relation and confirmed by computer simulations. The simulation results also verify that the proposed algorithm provides not only a faster convergence rate but also an improved steady-state performance as compared to an LMS-based scheme. In addition, the new approach attains an acceptable misadjustment performance with lower computational and memory cost, provided the number of regressor vectors and filter length parameters are appropriately chosen, as compared to a distributed recursive-least-squares (RLS) based method.

The composite Euler method for stiff stochastic differential equations

In this paper we present the composite Euler method for the strong solution of stochastic differential equations driven by d-dimensional Wiener processes. This method is a combination of the semi-implicit Euler method and the implicit Euler method. At each step either the semi-implicit Euler method or the implicit Euler method is used in order to obtain better stability properties. We give criteria for selecting the semi-implicit Euler method or the implicit Euler method. For the linear test equation, the convergence properties of the composite Euler method depend on the criteria for selecting the methods. Numerical results suggest that the convergence properties of the composite Euler method applied to nonlinear SDEs is the same as those applied to linear equations. The stability properties of the composite Euler method are shown to be far superior to those of the Euler methods, and numerical results show that the composite Euler method is a very promising method. (C) 2001 Elsevier Science B.V. All rights reserved.

Implicit Taylor methods for stiff stochastic differential equations

In this paper we discuss implicit Taylor methods for stiff Ito stochastic differential equations. Based on the relationship between Ito stochastic integrals and backward stochastic integrals, we introduce three implicit Taylor methods: the implicit Euler-Taylor method with strong order 0.5, the implicit Milstein-Taylor method with strong order 1.0 and the implicit Taylor method with strong order 1.5. The mean-square stability properties of the implicit Euler-Taylor and Milstein-Taylor methods are much better than those of the corresponding semi-implicit Euler and Milstein methods and these two implicit methods can be used to solve stochastic differential equations which are stiff in both the deterministic and the stochastic components. Numerical results are reported to show the convergence properties and the stability properties of these three implicit Taylor methods. The stability analysis and numerical results show that the implicit Euler-Taylor and Milstein-Taylor methods are very promising methods for stiff stochastic differential equations.

R&D spillovers in an endogenous growth model with physical capital, human capital and varieties

There is a family of models with Physical, Human capital and R&D for which convergence properties have been discussed (Arnold, 2000a; Gómez, 2005). However, spillovers in R&D have been ignored in this context. We introduce spillovers in this model and derive its steady-state and stability properties. This new feature implies that the model is characterized by a system of four differential equations. A unique Balanced Growth Path along with a two dimensional stable manifold are obtained under simple and reasonable conditions. Transition is oscillatory toward the steady-state for plausible values of parameters.

Transitional dynamics of an endogenous growth model with an erosion effect

The convergence features of an Endogenous Growth model with Physical capital, Human Capital and R&D have been studied. We add an erosion effect (supported by empirical evidence) to this model, and fully characterize its convergence properties. The dynamics is described by a fourth-order system of differential equations. We show that the model converges along a one-dimensional stable manifold and that its equilibrium is saddle-path stable. We also argue that one of the implications of considering this “erosion effect” is the increase in the adherence of the model to data.

Distance Transforms on Gray-Level Surfaces

This thesis studies gray-level distance transforms, particularly the Distance Transform on Curved Space (DTOCS). The transform is produced by calculating distances on a gray-level surface. The DTOCS is improved by definingmore accurate local distances, and developing a faster transformation algorithm. The Optimal DTOCS enhances the locally Euclidean Weighted DTOCS (WDTOCS) with local distance coefficients, which minimize the maximum error from the Euclideandistance in the image plane, and produce more accurate global distance values.Convergence properties of the traditional mask operation, or sequential localtransformation, and the ordered propagation approach are analyzed, and compared to the new efficient priority pixel queue algorithm. The Route DTOCS algorithmdeveloped in this work can be used to find and visualize shortest routes between two points, or two point sets, along a varying height surface. In a digital image, there can be several paths sharing the same minimal length, and the Route DTOCS visualizes them all. A single optimal path can be extracted from the route set using a simple backtracking algorithm. A new extension of the priority pixel queue algorithm produces the nearest neighbor transform, or Voronoi or Dirichlet tessellation, simultaneously with the distance map. The transformation divides the image into regions so that each pixel belongs to the region surrounding the reference point, which is nearest according to the distance definition used. Applications and application ideas for the DTOCS and its extensions are presented, including obstacle avoidance, image compression and surface roughness evaluation.

Fuzzy Topological Games and Related Topics

The main purpose of study is to extend the concept of the topological game G(K, X) and some other kinds of games into fuzzy topological games and to obtain some results regarding them. Owing to the fact that topological games have plenty of applications in covering properties, it made an attempt to explore some inter relations of games and covering properties in fuzzy topological spaces. Even though the main focus is on fuzzy para-meta compact spaces and closure preserving shading families, some brief sketches regarding fuzzy P-spaces and Shading Dimension is also provided. In a topological game players choose some objects related to the topological structure of a space such as points, closed subsets, open covers etc. More over the condition on a play to be winning for a player may also include topological notions such as closure, convergence, etc. It turns out that topological games are related to the Baire property, Baire spaces, Completeness properties, Convergence properties, Separation properties, Covering and Base properties, Continuous images, Suslin sets, Singular spaces etc.

Numerical Methods for Non-Stationary Stokes Flow

We consider a first order implicit time stepping procedure (Euler scheme) for the non-stationary Stokes equations in smoothly bounded domains of R3. Using energy estimates we can prove optimal convergence properties in the Sobolev spaces Hm(G) (m = 0;1;2) uniformly in time, provided that the solution of the Stokes equations has a certain degree of regularity. For the solution of the resulting Stokes resolvent boundary value problems we use a representation in form of hydrodynamical volume and boundary layer potentials, where the unknown source densities of the latter can be determined from uniquely solvable boundary integral equations’ systems. For the numerical computation of the potentials and the solution of the boundary integral equations a boundary element method of collocation type is used. Some simulations of a model problem are carried out and illustrate the efficiency of the method.

On Nonlinear Preconditioners in Newton-Krylov-Methods for Unsteady Flows

The application of nonlinear schemes like dual time stepping as preconditioners in matrix-free Newton-Krylov-solvers is considered and analyzed. We provide a novel formulation of the left preconditioned operator that says it is in fact linear in the matrix-free sense, but changes the Newton scheme. This allows to get some insight in the convergence properties of these schemes which are demonstrated through numerical results.