Methods for integrating machine learning and constrained optimization

In the framework of industrial problems, the application of Constrained Optimization is known to have overall very good modeling capability and performance and stands as one of the most powerful, explored, and exploited tool to address prescriptive tasks. The number of applications is huge, ranging from logistics to transportation, packing, production, telecommunication, scheduling, and much more. The main reason behind this success is to be found in the remarkable effort put in the last decades by the OR community to develop realistic models and devise exact or approximate methods to solve the largest variety of constrained or combinatorial optimization problems, together with the spread of computational power and easily accessible OR software and resources. On the other hand, the technological advancements lead to a data wealth never seen before and increasingly push towards methods able to extract useful knowledge from them; among the data-driven methods, Machine Learning techniques appear to be one of the most promising, thanks to its successes in domains like Image Recognition, Natural Language Processes and playing games, but also the amount of research involved. The purpose of the present research is to study how Machine Learning and Constrained Optimization can be used together to achieve systems able to leverage the strengths of both methods: this would open the way to exploiting decades of research on resolution techniques for COPs and constructing models able to adapt and learn from available data. In the first part of this work, we survey the existing techniques and classify them according to the type, method, or scope of the integration; subsequently, we introduce a novel and general algorithm devised to inject knowledge into learning models through constraints, Moving Target. In the last part of the thesis, two applications stemming from real-world projects and done in collaboration with Optit will be presented.

Spectral Region Optimization for Raman-Based Optical Biopsy of Inflammatory Lesions

Objective: The biochemical alterations between inflammatory fibrous hyperplasia (IFH) and normal tissues of buccal mucosa were probed by using the FT-Raman spectroscopy technique. The aim was to find the minimal set of Raman bands that would furnish the best discrimination. Background: Raman-based optical biopsy is a widely recognized potential technique for noninvasive real-time diagnosis. However, few studies had been devoted to the discrimination of very common subtle or early pathologic states as inflammatory processes that are always present on, for example, cancer lesion borders. Methods: Seventy spectra of IFH from 14 patients were compared with 30 spectra of normal tissues from six patients. The statistical analysis was performed with principal components analysis and soft independent modeling class analogy cross-validated, leave-one-out methods. Results: Bands close to 574, 1,100, 1,250 to 1,350, and 1,500 cm(-1) (mainly amino acids and collagen bands) showed the main intragroup variations that are due to the acanthosis process in the IFH epithelium. The 1,200 (C-C aromatic/DNA), 1,350 (CH(2) bending/collagen 1), and 1,730 cm(-1) (collagen III) regions presented the main intergroup variations. This finding was interpreted as originating in an extracellular matrix-degeneration process occurring in the inflammatory tissues. The statistical analysis results indicated that the best discrimination capability (sensitivity of 95% and specificity of 100%) was found by using the 530-580 cm(-1) spectral region. Conclusions: The existence of this narrow spectral window enabling normal and inflammatory diagnosis also had useful implications for an in vivo dispersive Raman setup for clinical applications.

Optimization and validation of a LIBS method for the determination of macro and micronutrients in sugar cane leaves

Laser induced breakdown spectrometry (LIBS) was applied for the determination of macro (P, K, Ca, Mg) and micronutrients (B, Cu, Fe, Mn and Zn) in sugar cane leaves, which is one of the most economically important crops in Brazil. Operational conditions were previously optimized by a neuro-genetic approach, by using a laser Nd:YAG at 1064 nm with 110 mJ per pulse focused on a pellet surface prepared with ground plant samples. Emission intensities were measured after 2.0 mu s delay time, with 4.5 mu s integration time gate and 25 accumulated laser pulses. Measurements of LIBS spectra were based on triplicate and each replicate consisted of an average of ten spectra collected in different sites (craters) of the pellet. Quantitative determinations were carried out by using univariate calibration and chemometric methods, such as PLSR and iPLS. The calibration models were obtained by using 26 laboratory samples and the validation was carried out by using 15 test samples. For comparative purpose, these samples were also microwave-assisted digested and further analyzed by ICP OES. In general, most results obtained by LIBS did not differ significantly from ICP OES data by applying a t-test at 95% confidence level. Both LIBS multivariate and univariate calibration methods produced similar results, except for Fe where better results were achieved by the multivariate approach. Repeatability precision varied from 0.7 to 15% and 1.3 to 20% from measurements obtained by multivariate and univariate calibration, respectively. It is demonstrated that LIBS is a powerful tool for analysis of pellets of plant materials for determination of macro and micronutrients by choosing calibration and validation samples with similar matrix composition.

Parallel Dynamic Optimization of Steel Risers

This paper addresses the use of optimization techniques in the design of a steel riser. Two methods are used: the genetic algorithm, which imitates the process of natural selection, and the simulated annealing, which is based on the process of annealing of a metal. Both of them are capable of searching a given solution space for the best feasible riser configuration according to predefined criteria. Optimization issues are discussed, such as problem codification, parameter selection, definition of objective function, and restrictions. A comparison between the results obtained for economic and structural objective functions is made for a case study. Optimization method parallelization is also addressed. [DOI: 10.1115/1.4001955]

Topology optimization for designing strain-gauge load cells

Load cells are used extensively in engineering fields. This paper describes a novel structural optimization method for single- and multi-axis load cell structures. First, we briefly explain the topology optimization method that uses the solid isotropic material with penalization (SIMP) method. Next, we clarify the mechanical requirements and design specifications of the single- and multi-axis load cell structures, which are formulated as an objective function. In the case of multi-axis load cell structures, a methodology based on singular value decomposition is used. The sensitivities of the objective function with respect to the design variables are then formulated. On the basis of these formulations, an optimization algorithm is constructed using finite element methods and the method of moving asymptotes (MMA). Finally, we examine the characteristics of the optimization formulations and the resultant optimal configurations. We confirm the usefulness of our proposed methodology for the optimization of single- and multi-axis load cell structures.

Multi-actuated functionally graded piezoelectric micro-tools design: A multiphysics topology optimization approach

Micro-tools offer significant promise in a wide range of applications Such as cell Manipulation, microsurgery, and micro/nanotechnology processes. Such special micro-tools consist of multi-flexible structures actuated by two or more piezoceramic devices that must generate output displacements and forces lit different specified points of the domain and at different directions. The micro-tool Structure acts as a mechanical transformer by amplifying and changing the direction of the piezoceramics Output displacements. The design of these micro-tools involves minimization of the coupling among movements generated by various piezoceramics. To obtain enhanced micro-tool performance, the concept of multifunctional and functionally graded materials is extended by, tailoring elastic and piezoelectric properties Of the piezoceramics while simultaneously optimizing the multi-flexible structural configuration using multiphysics topology optimization. The design process considers the influence of piezoceramic property gradation and also its polarization sign. The method is implemented considering continuum material distribution with special interpolation of fictitious densities in the design domain. As examples, designs of a single piezoactuator, an XY nano-positioner actuated by two graded piezoceramics, and a micro-gripper actuated by three graded piezoceramics are considered. The results show that material gradation plays an important role to improve actuator performance, which may also lead to optimal displacements and coupling ratios with reduced amount of piezoelectric material. The present examples are limited to two-dimensional models because many of the applications for Such micro-tools are planar devices. Copyright (c) 2008 John Wiley & Sons, Ltd.

Robust integration of real time optimization with linear model predictive control

Here, we study the stable integration of real time optimization (RTO) with model predictive control (MPC) in a three layer structure. The intermediate layer is a quadratic programming whose objective is to compute reachable targets to the MPC layer that lie at the minimum distance to the optimum set points that are produced by the RTO layer. The lower layer is an infinite horizon MPC with guaranteed stability with additional constraints that force the feasibility and convergence of the target calculation layer. It is also considered the case in which there is polytopic uncertainty in the steady state model considered in the target calculation. The dynamic part of the MPC model is also considered unknown but it is assumed to be represented by one of the models of a discrete set of models. The efficiency of the methods presented here is illustrated with the simulation of a low order system. (C) 2010 Elsevier Ltd. All rights reserved.

S/MIMO MC-CDMA Heuristic Multiuser Detectors Based on Single-Objective Optimization

This paper analyzes the complexity-performance trade-off of several heuristic near-optimum multiuser detection (MuD) approaches applied to the uplink of synchronous single/multiple-input multiple-output multicarrier code division multiple access (S/MIMO MC-CDMA) systems. Genetic algorithm (GA), short term tabu search (STTS) and reactive tabu search (RTS), simulated annealing (SA), particle swarm optimization (PSO), and 1-opt local search (1-LS) heuristic multiuser detection algorithms (Heur-MuDs) are analyzed in details, using a single-objective antenna-diversity-aided optimization approach. Monte- Carlo simulations show that, after convergence, the performances reached by all near-optimum Heur-MuDs are similar. However, the computational complexities may differ substantially, depending on the system operation conditions. Their complexities are carefully analyzed in order to obtain a general complexity-performance framework comparison and to show that unitary Hamming distance search MuD (uH-ds) approaches (1-LS, SA, RTS and STTS) reach the best convergence rates, and among them, the 1-LS-MuD provides the best trade-off between implementation complexity and bit error rate (BER) performance.

Box-Behnken design for the optimization of an enantioselective method for the simultaneous analysis of propranolol and 4-hydroxypropranolol by CE

An experimental design optimization (Box-Behnken design, BBD) was used to develop a CE method for the simultaneous resolution of propranolol (Prop) and 4-hydroxypropranolol enantiomers and acetaminophen (internal standard). The method was optimized using an uncoated fused silica capillary, carboxymethyl-beta-cyclodextrin (CM-beta-CD) as chiral selector and triethylamine/phosphoric acid buffer in alkaline conditions. A BBD for four factors was selected to observe the effects of buffer electrolyte concentration, pH, CM-beta-CD concentration and voltage on separation responses. Each factor was studied at three levels: high, central and low, and three center points were added. The buffer electrolyte concentration ranged from 25 to 75 mM, the pH ranged from 8 to 9, the CM-beta-CD concentration ranged from 3.5 to 4.5%w/v, and the applied run voltage ranged from 14 to 20 W. The responses evaluated were resolution and migration time for the last peak. The obtained responses were processed by Minitab (R) to evaluate the significance of the effects and to find the optimum analysis conditions. The best results were obtained using 4%w/v CM-beta-CD in 25 mM triethylamine/H(3)PO(4) buffer at pH 9 as running electrolyte and 17 kV of voltage. Resolution values of 1.98 and 1.95 were obtained for Prop and 4-hydroxypropranolol enantiomers, respectively. The total analysis time was around of 15 min. The BBD showed to be an adequate design for the development of a CE method, resulting in a rapid and efficient optimization of the pH and concentration of the buffer, cyclodextrin concentration and applied voltage.

Analysis of small signal stability margins using genetic optimization

Power system small signal stability analysis aims to explore different small signal stability conditions and controls, namely: (1) exploring the power system security domains and boundaries in the space of power system parameters of interest, including load flow feasibility, saddle node and Hopf bifurcation ones; (2) finding the maximum and minimum damping conditions; and (3) determining control actions to provide and increase small signal stability. These problems are presented in this paper as different modifications of a general optimization to a minimum/maximum, depending on the initial guesses of variables and numerical methods used. In the considered problems, all the extreme points are of interest. Additionally, there are difficulties with finding the derivatives of the objective functions with respect to parameters. Numerical computations of derivatives in traditional optimization procedures are time consuming. In this paper, we propose a new black-box genetic optimization technique for comprehensive small signal stability analysis, which can effectively cope with highly nonlinear objective functions with multiple minima and maxima, and derivatives that can not be expressed analytically. The optimization result can then be used to provide such important information such as system optimal control decision making, assessment of the maximum network's transmission capacity, etc. (C) 1998 Elsevier Science S.A. All rights reserved.

Structure optimization combining soft-core interaction functions, the diffusion equation method, and molecular dynamics

Smoothing the potential energy surface for structure optimization is a general and commonly applied strategy. We propose a combination of soft-core potential energy functions and a variation of the diffusion equation method to smooth potential energy surfaces, which is applicable to complex systems such as protein structures; The performance of the method was demonstrated by comparison with simulated annealing using the refinement of the undecapeptide Cyclosporin A as a test case. Simulations were repeated many times using different initial conditions and structures since the methods are heuristic and results are only meaningful in a statistical sense.

Mathematical methods for spatially cohesive reserve design

The problem of designing spatially cohesive nature reserve systems that meet biodiversity objectives is formulated as a nonlinear integer programming problem. The multiobjective function minimises a combination of boundary length, area and failed representation of the biological attributes we are trying to conserve. The task is to reserve a subset of sites that best meet this objective. We use data on the distribution of habitats in the Northern Territory, Australia, to show how simulated annealing and a greedy heuristic algorithm can be used to generate good solutions to such large reserve design problems, and to compare the effectiveness of these methods.

Interval training program optimization in highly trained endurance cyclists

Purpose: The purpose of this study was to examine the influence of three different high-intensity interval training (HIT) regimens on endurance performance in highly trained endurance athletes. Methods: Before, and after 2 and 4 wk of training, 38 cyclists and triathletes (mean +/- SD; age = 25 +/- 6 yr; mass = 75 +/- 7 kg; (V)over dot O-2peak = 64.5 +/- 5.2 mL.kg(-1).min(-1)) performed: 1) a progressive cycle test to measure peak oxygen consumption ((V)over dotO(2peak)) and peak aerobic power output (PPO), 2) a time to exhaustion test (T-max) at their (V)over dotO(2peak) power output (P-max), as well as 3) a 40-kin time-trial (TT40). Subjects were matched and assigned to one of four training groups (G(1), N = 8, 8 X 60% T-max P-max, 1:2 work:recovery ratio; G(2), N = 9, 8 X 60% T-max at P-max, recovery at 65% HRmax; G(3), N = 10, 12 X 30 s at 175% PPO, 4.5-min recovery; G(CON), N = 11). In addition to G(1) G(2), and G(3) performing HIT twice per week, all athletes maintained their regular low-intensity training throughout the experimental period. Results: All HIT groups improved TT40 performance (+4.4 to +5.8%) and PPO (+3.0 to +6.2%) significantly more than G(CON) (-0.9 to + 1.1 %; P < 0.05). Furthermore, G(1) (+5.4%) and G(2) (+8.1%) improved their (V)over dot O-2peak significantly more than G(CON) (+ 1.0%; P < 0.05). Conclusion: The present study has shown that when HIT incorporates P-max as the interval intensity and 60% of T-max as the interval duration, already highly trained cyclists can significantly improve their 40-km time trial performance. Moreover, the present data confirm prior research, in that repeated supramaximal HIT can significantly improve 40-km time trial performance.

A genetic algorithm/method of moments approach to the optimization of an RF coil for MRI applications - Theoretical considerations - Abstract

A Combined Genetic Algorithm and Method of Moments design methods is presented for the design of unusual near-field antennas for use in Magnetic Resonance Imaging systems. The method is successfully applied to the design of an asymmetric coil structure for use at 190MHz and demonstrates excellent radiofrequency field homogeneity.