A species conserving genetic algorithm for multimodal function optimization

This paper introduces a new technique called species conservation for evolving parallel subpopulations. The technique is based on the concept of dividing the population into several species according to their similarity. Each of these species is built around a dominating individual called the species seed. Species seeds found in the current generation are saved (conserved) by moving them into the next generation. Our technique has proved to be very effective in finding multiple solutions of multimodal optimization problems. We demonstrate this by applying it to a set of test problems, including some problems known to be deceptive to genetic algorithms.

Low-rank optimization on the cone of positive semidefinite matrices

We propose an algorithm for solving optimization problems defined on a subset of the cone of symmetric positive semidefinite matrices. This algorithm relies on the factorization X = Y Y T , where the number of columns of Y fixes an upper bound on the rank of the positive semidefinite matrix X. It is thus very effective for solving problems that have a low-rank solution. The factorization X = Y Y T leads to a reformulation of the original problem as an optimization on a particular quotient manifold. The present paper discusses the geometry of that manifold and derives a second-order optimization method with guaranteed quadratic convergence. It furthermore provides some conditions on the rank of the factorization to ensure equivalence with the original problem. In contrast to existing methods, the proposed algorithm converges monotonically to the sought solution. Its numerical efficiency is evaluated on two applications: the maximal cut of a graph and the problem of sparse principal component analysis. © 2010 Society for Industrial and Applied Mathematics.

Collective optimization over average quantities

This paper addresses the design of algorithms for the collective optimization of a cost function defined over average quantities in the presence of limited communication. We argue that several meaningful collective optimization problems can be formulated in this way. As an application of the proposed approach, we propose a novel algorithm that achieves synchronization or balancing in phase models of coupled oscillators under mild connectedness assumptions on the (possibly time-varying and unidirectional) communication graphs. © 2006 IEEE.

Predictive entropy search for efficient global optimization of black-box functions

We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the global maximum. PES codifies this intractable acquisition function in terms of the expected reduction in the differential entropy of the predictive distribution. This reformulation allows PES to obtain approximations that are both more accurate and efficient than other alternatives such as Entropy Search (ES). Furthermore, PES can easily perform a fully Bayesian treatment of the model hyperparameters while ES cannot. We evaluate PES in both synthetic and real-world applications, including optimization problems in machine learning, finance, biotechnology, and robotics. We show that the increased accuracy of PES leads to significant gains in optimization performance.

Bayesian separation and recovery of convolutively mixed autoregressive sources

In this paper we address the problem of the separation and recovery of convolutively mixed autoregressive processes in a Bayesian framework. Solving this problem requires the ability to solve integration and/or optimization problems of complicated posterior distributions. We thus propose efficient stochastic algorithms based on Markov chain Monte Carlo (MCMC) methods. We present three algorithms. The first one is a classical Gibbs sampler that generates samples from the posterior distribution. The two other algorithms are stochastic optimization algorithms that allow to optimize either the marginal distribution of the sources, or the marginal distribution of the parameters of the sources and mixing filters, conditional upon the observation. Simulations are presented.

The most persistent soft-clique in a set of sampled graphs

When searching for characteristic subpatterns in potentially noisy graph data, it appears self-evident that having multiple observations would be better than having just one. However, it turns out that the inconsistencies introduced when different graph instances have different edge sets pose a serious challenge. In this work we address this challenge for the problem of finding maximum weighted cliques. We introduce the concept of most persistent soft-clique. This is subset of vertices, that 1) is almost fully or at least densely connected, 2) occurs in all or almost all graph instances, and 3) has the maximum weight. We present a measure of clique-ness, that essentially counts the number of edge missing to make a subset of vertices into a clique. With this measure, we show that the problem of finding the most persistent soft-clique problem can be cast either as: a) a max-min two person game optimization problem, or b) a min-min soft margin optimization problem. Both formulations lead to the same solution when using a partial Lagrangian method to solve the optimization problems. By experiments on synthetic data and on real social network data we show that the proposed method is able to reliably find soft cliques in graph data, even if that is distorted by random noise or unreliable observations. Copyright 2012 by the author(s)/owner(s).

Differential geometry and design coupling in MDO

The notion of coupling within a design, particularly within the context of Multidisciplinary Design Optimization (MDO), is much used but ill-defined. There are many different ways of measuring design coupling, but these measures vary in both their conceptions of what design coupling is and how such coupling may be calculated. Within the differential geometry framework which we have previously developed for MDO systems, we put forth our own design coupling metric for consideration. Our metric is not commensurate with similar types of coupling metrics, but we show that it both provides a helpful geo- metric interpretation of coupling (and uncoupledness in particular) and exhibits greater generality and potential for analysis than those similar metrics. Furthermore, we discuss how the metric might be profitably extended to time-varying problems and show how the metric's measure of coupling can be applied to multi-objective optimization problems (in unconstrained optimization and in MDO). © 2013 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.

Output disturbance rejection using parallel model predictive control

The solution time of the online optimization problems inherent to Model Predictive Control (MPC) can become a critical limitation when working in embedded systems. One proposed approach to reduce the solution time is to split the optimization problem into a number of reduced order problems, solve such reduced order problems in parallel and selecting the solution which minimises a global cost function. This approach is known as Parallel MPC. The potential capabilities of disturbance rejection are introduced using a simulation example. The algorithm is implemented in a linearised model of a Boeing 747-200 under nominal flight conditions and with an induced wind disturbance. Under significant output disturbances Parallel MPC provides a significant improvement in performance when compared to Multiplexed MPC (MMPC) and Linear Quadratic Synchronous MPC (SMPC). © 2013 IEEE.