Global non-smooth optimization in robust multivariate regression

Robust regression in statistics leads to challenging optimization problems. Here, we study one such problem, in which the objective is non-smooth, non-convex and expensive to calculate. We study the numerical performance of several derivative-free optimization algorithms with the aim of computing robust multivariate estimators. Our experiences demonstrate that the existing algorithms often fail to deliver optimal solutions. We introduce three new methods that use Powell's derivative-free algorithm. The proposed methods are reliable and can be used when processing very large data sets containing outliers.

A new approach based on support vector machine for solving stochastic optimization

Making decision usually occurs in the state of being uncertain. These kinds of problems often expresses in a formula as optimization problems. It is desire for decision makers to find a solution for optimization problems. Typically, solving optimization problems in uncertain environment is difficult. This paper proposes a new hybrid intelligent algorithm to solve a kind of stochastic optimization i.e. dependent chance programming (DCP) model. In order to speed up the solution process, we used support vector machine regression (SVM regression) to approximate chance functions which is the probability of a sequence of uncertain event occurs based on the training data generated by the stochastic simulation. The proposed algorithm consists of three steps: (1) generate data to estimate the objective function, (2) utilize SVM regression to reveal a trend hidden in the data (3) apply genetic algorithm (GA) based on SVM regression to obtain an estimation for the chance function. Numerical example is presented to show the ability of algorithm in terms of time-consuming and precision.

A hybrid cuckoo search and variable neighborhood descent for single and multiobjective scheduling problems

Cuckoo search (CS) is a relatively new meta-heuristic that has proven its strength in solving continuous optimization problems. This papers applies cuckoo search to the class of sequencing problems by hybridizing it with a variable neighborhood descent local search for enhancing the quality of the obtained solutions. The Lévy flight operator proposed in the original CS is modified to address the discrete nature of scheduling problems. Two well-known problems are used to demonstrate the effectiveness of the proposed hybrid CS approach. The first is the NP-hard single objective problem of minimizing the weighted total tardiness time (Formula presented.) and the second is the multiobjective problem of minimizing the flowtime ¯ and the maximum tardiness Tmaxfor single machine (Formula presented.). For the first problem, computational results show that the hybrid CS is able to find the optimal solutions for all benchmark test instances with 40, 50, and 100 jobs and for most instances with 150, 200, 250, and 300 jobs. For the second problem, the hybrid CS generated solutions on and very close to the exact Pareto fronts of test instances with 10, 20, 30, and 40 jobs. In general, the results reveal that the hybrid CS is an adequate and robust method for tackling single and multiobjective scheduling problems.

Neural network training by hybrid accelerated cuckoo particle swarm optimization algorithm

Metaheuristic algorithm is one of the most popular methods in solving many optimization problems. This paper presents a new hybrid approach comprising of two natures inspired metaheuristic algorithms i.e. Cuckoo Search (CS) and Accelerated Particle Swarm Optimization (APSO) for training Artificial Neural Networks (ANN). In order to increase the probability of the egg’s survival, the cuckoo bird migrates by traversing more search space. It can successfully search better solutions by performing levy flight with APSO. In the proposed Hybrid Accelerated Cuckoo Particle Swarm Optimization (HACPSO) algorithm, the communication ability for the cuckoo birds have been provided by APSO, thus making cuckoo bird capable of searching for the best nest with better solution. Experimental results are carried-out on benchmarked datasets, and the performance of the proposed hybrid algorithm is compared with Artificial Bee Colony (ABC) and similar hybrid variants. The results show that the proposed HACPSO algorithm performs better than other algorithms in terms of convergence and accuracy.

Evolutionary optimization: a big data perspective

Stochastic search techniques such as evolutionary algorithms (EA) are known to be better explorer of search space as compared to conventional techniques including deterministic methods. However, in the era of big data like most other search methods and learning algorithms, suitability of evolutionary algorithms is naturally questioned. Big data pose new computational challenges including very high dimensionality and sparseness of data. Evolutionary algorithms' superior exploration skills should make them promising candidates for handling optimization problems involving big data. High dimensional problems introduce added complexity to the search space. However, EAs need to be enhanced to ensure that majority of the potential winner solutions gets the chance to survive and mature. In this paper we present an evolutionary algorithm with enhanced ability to deal with the problems of high dimensionality and sparseness of data. In addition to an informed exploration of the solution space, this technique balances exploration and exploitation using a hierarchical multi-population approach. The proposed model uses informed genetic operators to introduce diversity by expanding the scope of search process at the expense of redundant less promising members of the population. Next phase of the algorithm attempts to deal with the problem of high dimensionality by ensuring broader and more exhaustive search and preventing premature death of potential solutions. To achieve this, in addition to the above exploration controlling mechanism, a multi-tier hierarchical architecture is employed, where, in separate layers, the less fit isolated individuals evolve in dynamic sub-populations that coexist alongside the original or main population. Evaluation of the proposed technique on well known benchmark problems ascertains its superior performance. The algorithm has also been successfully applied to a real world problem of financial portfolio management. Although the proposed method cannot be considered big data-ready, it is certainly a move in the right direction.

A novel approach based on recurrent neural networks applied to nonlinear systems optimization

This paper presents an efficient approach based on recurrent neural network for solving nonlinear optimization. More specifically, a modified Hopfield network is developed and its internal parameters are computed using the valid subspace technique. These parameters guarantee the convergence of the network to the equilibrium points that represent an optimal feasible solution. The main advantage of the developed network is that it treats optimization and constraint terms in different stages with no interference with each other. Moreover, the proposed approach does not require specification of penalty and weighting parameters for its initialization. A study of the modified Hopfield model is also developed to analyze its stability and convergence. Simulation results are provided to demonstrate the performance of the proposed neural network. (c) 2005 Elsevier B.V. All rights reserved.

Stability and convergence analysis of a neural model applied in nonlinear systems optimization

A neural model for solving nonlinear optimization problems is presented in this paper. More specifically, a modified Hopfield network is developed and its internal parameters are computed using the valid-subspace technique. These parameters guarantee the convergence of the network to the equilibrium points that represent an optimal feasible solution. The network is shown to be completely stable and globally convergent to the solutions of nonlinear optimization problems. A study of the modified Hopfield model is also developed to analyze its stability and convergence. Simulation results are presented to validate the developed methodology.

Nonlinear optimization using a modified Hopfield model

Systems based on artificial neural networks have high computational rates due to the use of a massive number of simple processing elements. Neural networks with feedback connections provide a computing model capable of solving a rich class of optimization problems. In this paper, a modified Hopfield network is developed for solving constrained nonlinear optimization problems. The internal parameters of the network are obtained using the valid-subspace technique. Simulated examples are presented as an illustration of the proposed approach.

Studying properties of Lagrangian bounds for many-to-many assignment problems

Classical and modified Lagrangian bounds for the optimal value of optimization problems with a double decomposable structure are studied. For the class of many-to-many assignment problems, this property of constraints is used to design a subgradient algorithm for solving the modified dual problem. Numerical results are presented to compare the quality of classical and modified bounds, as well as the properties of the corresponding Lagrangian solutions.

Comparison of Lagrangian Bounds for One Class of Generalized Assignment Problems

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

A note on KKT-invexity in nonsmooth continuous-time optimization

We introduce the notion of KKT-inverity for nonsmooth continuous-time nonlinear optimization problems and prove that this notion is a necessary and sufficient condition for every KKT solution to be a global optimal solution.

Application of an iterative method and an evolutionary algorithm in fuzzy optimization

This work develops two approaches based on the fuzzy set theory to solve a class of fuzzy mathematical optimization problems with uncertainties in the objective function and in the set of constraints. The first approach is an adaptation of an iterative method that obtains cut levels and later maximizes the membership function of fuzzy decision making using the bound search method. The second one is a metaheuristic approach that adapts a standard genetic algorithm to use fuzzy numbers. Both approaches use a decision criterion called satisfaction level that reaches the best solution in the uncertain environment. Selected examples from the literature are presented to compare and to validate the efficiency of the methods addressed, emphasizing the fuzzy optimization problem in some import-export companies in the south of Spain. © 2012 Brazilian Operations Research Society.

Global optimization using a genetic algorithm with hierarchically structured population

This paper applies a genetic algorithm with hierarchically structured population to solve unconstrained optimization problems. The population has individuals distributed in several overlapping clusters, each one with a leader and a variable number of support individuals. The hierarchy establishes that leaders must be fitter than its supporters with the topological organization of the clusters following a tree. Computational tests evaluate different population structures, population sizes and crossover operators for better algorithm performance. A set of known benchmark test problems is solved and the results found are compared with those obtained from other methods described in the literature, namely, two genetic algorithms, a simulated annealing, a differential evolution and a particle swarm optimization. The results indicate that the method employed is capable of achieving better performance than the previous approaches in regard as the two criteria usually employed for comparisons: the number of function evaluations and rate of success. The method also has a superior performance if the number of problems solved is taken into account. (C) 2013 Elsevier B.V. All rights reserved.

Discovering promising regions to help global numerical optimization algorithms

We have developed an algorithm using a Design of Experiments technique for reduction of search-space in global optimization problems. Our approach is called Domain Optimization Algorithm. This approach can efficiently eliminate search-space regions with low probability of containing a global optimum. The Domain Optimization Algorithm approach is based on eliminating non-promising search-space regions, which are identifyed using simple models (linear) fitted to the data. Then, we run a global optimization algorithm starting its population inside the promising region. The proposed approach with this heuristic criterion of population initialization has shown relevant results for tests using hard benchmark functions.

A smooth evolutionary structural optimization procedure applied to plane stress problem

Topological optimization problems based on stress criteria are solved using two techniques in this paper. The first technique is the conventional Evolutionary Structural Optimization (ESO), which is known as hard kill, because the material is discretely removed; that is, the elements under low stress that are being inefficiently utilized have their constitutive matrix has suddenly reduced. The second technique, proposed in a previous paper, is a variant of the ESO procedure and is called Smooth ESO (SESO), which is based on the philosophy that if an element is not really necessary for the structure, its contribution to the structural stiffness will gradually diminish until it no longer influences the structure; its removal is thus performed smoothly. This procedure is known as "soft-kill"; that is, not all of the elements removed from the structure using the ESO criterion are discarded. Thus, the elements returned to the structure must provide a good conditioning system that will be resolved in the next iteration, and they are considered important to the optimization process. To evaluate elasticity problems numerically, finite element analysis is applied, but instead of using conventional quadrilateral finite elements, a plane-stress triangular finite element was implemented with high-order modes for solving complex geometric problems. A number of typical examples demonstrate that the proposed approach is effective for solving problems of bi-dimensional elasticity. (C) 2014 Elsevier Ltd. All rights reserved.