Distributed maximum likelihood for simultaneous self-localization and tracking in sensor networks

We show that the sensor self-localization problem can be cast as a static parameter estimation problem for Hidden Markov Models and we implement fully decentralized versions of the Recursive Maximum Likelihood and on-line Expectation-Maximization algorithms to localize the sensor network simultaneously with target tracking. For linear Gaussian models, our algorithms can be implemented exactly using a distributed version of the Kalman filter and a novel message passing algorithm. The latter allows each node to compute the local derivatives of the likelihood or the sufficient statistics needed for Expectation-Maximization. In the non-linear case, a solution based on local linearization in the spirit of the Extended Kalman Filter is proposed. In numerical examples we demonstrate that the developed algorithms are able to learn the localization parameters. © 2012 IEEE.

Occlusive Components Analysis

We study unsupervised learning in a probabilistic generative model for occlusion. The model uses two types of latent variables: one indicates which objects are present in the image, and the other how they are ordered in depth. This depth order then determines how the positions and appearances of the objects present, specified in the model parameters, combine to form the image. We show that the object parameters can be learnt from an unlabelled set of images in which objects occlude one another. Exact maximum-likelihood learning is intractable. However, we show that tractable approximations to Expectation Maximization (EM) can be found if the training images each contain only a small number of objects on average. In numerical experiments it is shown that these approximations recover the correct set of object parameters. Experiments on a novel version of the bars test using colored bars, and experiments on more realistic data, show that the algorithm performs well in extracting the generating causes. Experiments based on the standard bars benchmark test for object learning show that the algorithm performs well in comparison to other recent component extraction approaches. The model and the learning algorithm thus connect research on occlusion with the research field of multiple-causes component extraction methods.

Convex relaxation of mixture regression with efficient algorithms

We develop a convex relaxation of maximum a posteriori estimation of a mixture of regression models. Although our relaxation involves a semidefinite matrix variable, we reformulate the problem to eliminate the need for general semidefinite programming. In particular, we provide two reformulations that admit fast algorithms. The first is a max-min spectral reformulation exploiting quasi-Newton descent. The second is a min-min reformulation consisting of fast alternating steps of closed-form updates. We evaluate the methods against Expectation-Maximization in a real problem of motion segmentation from video data.

Generalized power method for sparse principal component analysis

In this paper we develop a new approach to sparse principal component analysis (sparse PCA). We propose two single-unit and two block optimization formulations of the sparse PCA problem, aimed at extracting a single sparse dominant principal component of a data matrix, or more components at once, respectively. While the initial formulations involve nonconvex functions, and are therefore computationally intractable, we rewrite them into the form of an optimization program involving maximization of a convex function on a compact set. The dimension of the search space is decreased enormously if the data matrix has many more columns (variables) than rows. We then propose and analyze a simple gradient method suited for the task. It appears that our algorithm has best convergence properties in the case when either the objective function or the feasible set are strongly convex, which is the case with our single-unit formulations and can be enforced in the block case. Finally, we demonstrate numerically on a set of random and gene expression test problems that our approach outperforms existing algorithms both in quality of the obtained solution and in computational speed. © 2010 Michel Journée, Yurii Nesterov, Peter Richtárik and Rodolphe Sepulchre.

Application of the multi-objective Alliance algorithm to a benchmark aerodynamic optimization problem

This paper introduces a new version of the multiobjective Alliance Algorithm (MOAA) applied to the optimization of the NACA 0012 airfoil section, for minimization of drag and maximization of lift coefficients, based on eight section shape parameters. Two software packages are used: XFoil which evaluates each new candidate airfoil section in terms of its aerodynamic efficiency, and a Free-Form Deformation tool to manage the section geometry modifications. Two versions of the problem are formulated with different design variable bounds. The performance of this approach is compared, using two indicators and a statistical test, with that obtained using NSGA-II and multi-objective Tabu Search (MOTS) to guide the optimization. The results show that the MOAA outperforms MOTS and obtains comparable results with NSGA-II on the first problem, while in the other case NSGA-II is not able to find feasible solutions and the MOAA is able to outperform MOTS. © 2013 IEEE.

A self-adaptive genetic algorithm applied to multi-objective optimization of an airfoil

Genetic algorithms (GAs) have been used to tackle non-linear multi-objective optimization (MOO) problems successfully, but their success is governed by key parameters which have been shown to be sensitive to the nature of the particular problem, incorporating concerns such as the numbers of objectives and variables, and the size and topology of the search space, making it hard to determine the best settings in advance. This work describes a real-encoded multi-objective optimizing GA (MOGA) that uses self-adaptive mutation and crossover, and which is applied to optimization of an airfoil, for minimization of drag and maximization of lift coefficients. The MOGA is integrated with a Free-Form Deformation tool to manage the section geometry, and XFoil which evaluates each airfoil in terms of its aerodynamic efficiency. The performance is compared with those of the heuristic MOO algorithms, the Multi-Objective Tabu Search (MOTS) and NSGA-II, showing that this GA achieves better convergence.