Predictive entropy search for efficient global optimization of black-box functions

We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the global maximum. PES codifies this intractable acquisition function in terms of the expected reduction in the differential entropy of the predictive distribution. This reformulation allows PES to obtain approximations that are both more accurate and efficient than other alternatives such as Entropy Search (ES). Furthermore, PES can easily perform a fully Bayesian treatment of the model hyperparameters while ES cannot. We evaluate PES in both synthetic and real-world applications, including optimization problems in machine learning, finance, biotechnology, and robotics. We show that the increased accuracy of PES leads to significant gains in optimization performance.

Rigorous finite-time guarantees for optimization on continuous domains by simulated annealing

Simulated annealing is a popular method for approaching the solution of a global optimization problem. Existing results on its performance apply to discrete combinatorial optimization where the optimization variables can assume only a finite set of possible values. We introduce a new general formulation of simulated annealing which allows one to guarantee finite-time performance in the optimization of functions of continuous variables. The results hold universally for any optimization problem on a bounded domain and establish a connection between simulated annealing and up-to-date theory of convergence of Markov chain Monte Carlo methods on continuous domains. This work is inspired by the concept of finite-time learning with known accuracy and confidence developed in statistical learning theory.

Simulated Annealing: Rigorous finite-time guarantees for optimization on continuous domains

Simulated annealing is a popular method for approaching the solution of a global optimization problem. Existing results on its performance apply to discrete combinatorial optimization where the optimization variables can assume only a finite set of possible values. We introduce a new general formulation of simulated annealing which allows one to guarantee finite-time performance in the optimization of functions of continuous variables. The results hold universally for any optimization problem on a bounded domain and establish a connection between simulated annealing and up-to-date theory of convergence of Markov chain Monte Carlo methods on continuous domains. This work is inspired by the concept of finite-time learning with known accuracy and confidence developed in statistical learning theory.

Design and simulation of SMES system using YBCO tapes for direct drive wave energy converters

The ocean represents a huge energy reservoir since waves can be exploited to generate clean and renewable electricity; however, a hybrid energy storage system is needed to smooth the fluctuation. In this paper a hybrid energy storage system using a superconducting magnetic energy system (SMES) and Li-ion battery is proposed. The SMES is designed using Yttrium Barium Copper Oxide (YBCO) tapes, which store 60 kJ electrical energy. The magnet component of the SMES is designed using global optimization algorithm. Mechanical stress, coupled with electromagnetic field, is calculated using COMSOL and Matlab. A cooling system is presented and a suitable refrigerator is chosen to maintain a cold working temperature taking into account four heat sources. Then a microgrid system of direct drive linear wave energy converters is designed. The interface circuit connecting the generator and storage system is given. The result reveals that the fluctuated power from direct drive linear wave energy converters is smoothed by the hybrid energy storage system. The maximum power of the wave energy converter is 10 kW. © 2012 IEEE.

Multi-fidelity simulation modelling in optimization of a hybrid submarine propulsion system

Multi-objective Genetic Algorithms have become a popular choice to aid in optimising the size of the whole hybrid power train. Within these optimisation processes, other optimisation techniques for the control strategy are implemented. This optimisation within an optimisation requires many simulations to be run, so reducing the computational cost is highly desired. This paper presents an optimisation framework consisting of a series hybrid optimisation algorithm, in which a global search optimizes a submarine propulsion system using low-fidelity models and, in order to refine the results, a local search is used with high-fidelity models. The effectiveness of the Hybrid optimisation algorithm is demonstrated with the optimisation of a submarine propulsion system. © 2011 EPE Association - European Power Electr.

Low-rank optimization for distance matrix completion

This paper addresses the problem of low-rank distance matrix completion. This problem amounts to recover the missing entries of a distance matrix when the dimension of the data embedding space is possibly unknown but small compared to the number of considered data points. The focus is on high-dimensional problems. We recast the considered problem into an optimization problem over the set of low-rank positive semidefinite matrices and propose two efficient algorithms for low-rank distance matrix completion. In addition, we propose a strategy to determine the dimension of the embedding space. The resulting algorithms scale to high-dimensional problems and monotonically converge to a global solution of the problem. Finally, numerical experiments illustrate the good performance of the proposed algorithms on benchmarks. © 2011 IEEE.

Optimization on the orthogonal group for independent component analysis

This paper derives a new algorithm that performs independent component analysis (ICA) by optimizing the contrast function of the RADICAL algorithm. The core idea of the proposed optimization method is to combine the global search of a good initial condition with a gradient-descent algorithm. This new ICA algorithm performs faster than the RADICAL algorithm (based on Jacobi rotations) while still preserving, and even enhancing, the strong robustness properties that result from its contrast. © Springer-Verlag Berlin Heidelberg 2007.

An approach to multi-fidelity optimization of aeroengine compression systems

The aerodynamic design of turbomachinery presents the design optimisation community with a number of exquisite challenges. Chief among these are the size of the design space and the extent of discontinuity therein. This discontinuity can serve to limit the full exploitation of high-fidelity computational fluid dynamics (CFD): such codes require detailed geometric information often available only sometime after the basic configuration of the machine has been set by other means. The premise of this paper is that it should be possible to produce higher performing designs in less time by exploiting multi-fidelity techniques to effectively harness CFD earlier in the design process, specifically by facilitating its participation in configuration selection. The adopted strategy of local multi-fidelity correction, generated on demand, combined with a global search algorithm via an adaptive trust region is first tested on a modest, smooth external aerodynamic problem. Speed-up of an order of magnitude is demonstrated, comparable to established techniques applied to smooth problems. A number of enhancements aimed principally at effectively evaluating a wide range of configurations quickly is then applied to the basic strategy, and the emerging technique is tested on a generic aeroengine core compression system. A similar order of magnitude speed-up is achieved on this relatively large and highly discontinuous problem. A five-fold increase in the number of configurations assessed with CFD is observed. As the technique places constraints neither on the underlying physical modelling of the constituent analysis codes nor on first-order agreement between those codes, it has potential applicability to a range of multidisciplinary design challenges. © 2012 by Jerome Jarrett and Tiziano Ghisu.

Low-rank optimization with trace norm penalty

The paper addresses the problem of low-rank trace norm minimization. We propose an algorithm that alternates between fixed-rank optimization and rank-one updates. The fixed-rank optimization is characterized by an efficient factorization that makes the trace norm differentiable in the search space and the computation of duality gap numerically tractable. The search space is nonlinear but is equipped with a Riemannian structure that leads to efficient computations. We present a second-order trust-region algorithm with a guaranteed quadratic rate of convergence. Overall, the proposed optimization scheme converges superlinearly to the global solution while maintaining complexity that is linear in the number of rows and columns of the matrix. To compute a set of solutions efficiently for a grid of regularization parameters we propose a predictor-corrector approach that outperforms the naive warm-restart approach on the fixed-rank quotient manifold. The performance of the proposed algorithm is illustrated on problems of low-rank matrix completion and multivariate linear regression. © 2013 Society for Industrial and Applied Mathematics.