Algoritmo Q-learning como estratégia de exploração e/ou explotação para metaheurísticas GRASP e algoritmo genético

Techniques of optimization known as metaheuristics have achieved success in the resolution of many problems classified as NP-Hard. These methods use non deterministic approaches that reach very good solutions which, however, don t guarantee the determination of the global optimum. Beyond the inherent difficulties related to the complexity that characterizes the optimization problems, the metaheuristics still face the dilemma of xploration/exploitation, which consists of choosing between a greedy search and a wider exploration of the solution space. A way to guide such algorithms during the searching of better solutions is supplying them with more knowledge of the problem through the use of a intelligent agent, able to recognize promising regions and also identify when they should diversify the direction of the search. This way, this work proposes the use of Reinforcement Learning technique - Q-learning Algorithm - as exploration/exploitation strategy for the metaheuristics GRASP (Greedy Randomized Adaptive Search Procedure) and Genetic Algorithm. The GRASP metaheuristic uses Q-learning instead of the traditional greedy-random algorithm in the construction phase. This replacement has the purpose of improving the quality of the initial solutions that are used in the local search phase of the GRASP, and also provides for the metaheuristic an adaptive memory mechanism that allows the reuse of good previous decisions and also avoids the repetition of bad decisions. In the Genetic Algorithm, the Q-learning algorithm was used to generate an initial population of high fitness, and after a determined number of generations, where the rate of diversity of the population is less than a certain limit L, it also was applied to supply one of the parents to be used in the genetic crossover operator. Another significant change in the hybrid genetic algorithm is the proposal of a mutually interactive cooperation process between the genetic operators and the Q-learning algorithm. In this interactive/cooperative process, the Q-learning algorithm receives an additional update in the matrix of Q-values based on the current best solution of the Genetic Algorithm. The computational experiments presented in this thesis compares the results obtained with the implementation of traditional versions of GRASP metaheuristic and Genetic Algorithm, with those obtained using the proposed hybrid methods. Both algorithms had been applied successfully to the symmetrical Traveling Salesman Problem, which was modeled as a Markov decision process

Metodologia estatística na solução do problema do caixeiro viajante e na avaliação de algoritmos : um estudo aplicado à transgenética computacional

The problems of combinatory optimization have involved a large number of researchers in search of approximative solutions for them, since it is generally accepted that they are unsolvable in polynomial time. Initially, these solutions were focused on heuristics. Currently, metaheuristics are used more for this task, especially those based on evolutionary algorithms. The two main contributions of this work are: the creation of what is called an -Operon- heuristic, for the construction of the information chains necessary for the implementation of transgenetic (evolutionary) algorithms, mainly using statistical methodology - the Cluster Analysis and the Principal Component Analysis; and the utilization of statistical analyses that are adequate for the evaluation of the performance of the algorithms that are developed to solve these problems. The aim of the Operon is to construct good quality dynamic information chains to promote an -intelligent- search in the space of solutions. The Traveling Salesman Problem (TSP) is intended for applications based on a transgenetic algorithmic known as ProtoG. A strategy is also proposed for the renovation of part of the chromosome population indicated by adopting a minimum limit in the coefficient of variation of the adequation function of the individuals, with calculations based on the population. Statistical methodology is used for the evaluation of the performance of four algorithms, as follows: the proposed ProtoG, two memetic algorithms and a Simulated Annealing algorithm. Three performance analyses of these algorithms are proposed. The first is accomplished through the Logistic Regression, based on the probability of finding an optimal solution for a TSP instance by the algorithm being tested. The second is accomplished through Survival Analysis, based on a probability of the time observed for its execution until an optimal solution is achieved. The third is accomplished by means of a non-parametric Analysis of Variance, considering the Percent Error of the Solution (PES) obtained by the percentage in which the solution found exceeds the best solution available in the literature. Six experiments have been conducted applied to sixty-one instances of Euclidean TSP with sizes of up to 1,655 cities. The first two experiments deal with the adjustments of four parameters used in the ProtoG algorithm in an attempt to improve its performance. The last four have been undertaken to evaluate the performance of the ProtoG in comparison to the three algorithms adopted. For these sixty-one instances, it has been concluded on the grounds of statistical tests that there is evidence that the ProtoG performs better than these three algorithms in fifty instances. In addition, for the thirty-six instances considered in the last three trials in which the performance of the algorithms was evaluated through PES, it was observed that the PES average obtained with the ProtoG was less than 1% in almost half of these instances, having reached the greatest average for one instance of 1,173 cities, with an PES average equal to 3.52%. Therefore, the ProtoG can be considered a competitive algorithm for solving the TSP, since it is not rare in the literature find PESs averages greater than 10% to be reported for instances of this size.

Técnicas neuronais de agrupamentos aplicadas à detecção de anomalias em mamogramas digitais

This work proposes the development of an intelligent system for analysis of digital mammograms, capable to detect and to classify masses and microcalcifications. The digital mammograms will be pre-processed through techniques of digital processing of images with the purpose of adapting the image to the detection system and automatic classification of the existent calcifications in the suckles. The model adopted for the detection and classification of the mammograms uses the neural network of Kohonen by the algorithm Self Organization Map - SOM. The algorithm of Vector quantization, Kmeans it is also used with the same purpose of the SOM. An analysis of the performance of the two algorithms in the automatic classification of digital mammograms is developed. The developed system will aid the radiologist in the diagnosis and accompaniment of the development of abnormalities

Algoritmos genéricos para otimização de uma arquitetura de controle inteligente híerárquico

A hierarchical fuzzy control scheme is applied to improve vibration suppression by using an electro-mechanical system based on the lever principle. The hierarchical intelligent controller consists of a hierarchical fuzzy supervisor, one fuzzy controller and one robust controller. The supervisor combines controllers output signal to generate the control signal that will be applied on the plant. The objective is to improve the performance of the electromechanical system, considering that the supervisor could take advantage of the different techniques based controllers. The robust controller design is based on a linear mathematical model. Genetic algorithms are used on the fuzzy controller and the supervisor tuning, which are based on non-linear mathematical model. In order to attest the efficiency of the hierarchical fuzzy control scheme, digital simulations were employed. Some comparisons involving the optimized hierarchical controller and the non-optimized hierarchical controller will be made to prove the efficiency of the genetic algorithms and the advantages of its use

Classificação de padrões através de um comitê de máquinas aprimorado por aprendizagem por reforço

Reinforcement learning is a machine learning technique that, although finding a large number of applications, maybe is yet to reach its full potential. One of the inadequately tested possibilities is the use of reinforcement learning in combination with other methods for the solution of pattern classification problems. It is well documented in the literature the problems that support vector machine ensembles face in terms of generalization capacity. Algorithms such as Adaboost do not deal appropriately with the imbalances that arise in those situations. Several alternatives have been proposed, with varying degrees of success. This dissertation presents a new approach to building committees of support vector machines. The presented algorithm combines Adaboost algorithm with a layer of reinforcement learning to adjust committee parameters in order to avoid that imbalances on the committee components affect the generalization performance of the final hypothesis. Comparisons were made with ensembles using and not using the reinforcement learning layer, testing benchmark data sets widely known in area of pattern classification

Sistema inteligente para detecção de manchas de óleo na superfície marinha através de imagens de SAR

Oil spill on the sea, accidental or not, generates enormous negative consequences for the affected area. The damages are ambient and economic, mainly with the proximity of these spots of preservation areas and/or coastal zones. The development of automatic techniques for identification of oil spots on the sea surface, captured through Radar images, assist in a complete monitoring of the oceans and seas. However spots of different origins can be visualized in this type of imaging, which is a very difficult task. The system proposed in this work, based on techniques of digital image processing and artificial neural network, has the objective to identify the analyzed spot and to discern between oil and other generating phenomena of spot. Tests in functional blocks that compose the proposed system allow the implementation of different algorithms, as well as its detailed and prompt analysis. The algorithms of digital image processing (speckle filtering and gradient), as well as classifier algorithms (Multilayer Perceptron, Radial Basis Function, Support Vector Machine and Committe Machine) are presented and commented.The final performance of the system, with different kind of classifiers, is presented by ROC curve. The true positive rates are considered agreed with the literature about oil slick detection through SAR images presents

Search schemes for random optimization algorithms that preserve the asymptotic distribution

Markovian algorithms for estimating the global maximum or minimum of real valued functions defined on some domain Omega subset of R-d are presented. Conditions on the search schemes that preserve the asymptotic distribution are derived. Global and local search schemes satisfying these conditions are analysed and shown to yield sharper confidence intervals when compared to the i.i.d. case.

An Intelligent System for Petroleum Well Drilling Cutting Analysis

Cutting analysis is a important and crucial task task to detect and prevent problems during the petroleum well drilling process. Several studies have been developed for drilling inspection, but none of them takes care about analysing the generated cutting at the vibrating shale shakers. Here we proposed a system to analyse the cutting's concentration at the vibrating shale shakers, which can indicate problems during the petroleum well drilling process, such that the collapse of the well borehole walls. Cutting's images are acquired and sent to the data analysis module, which has as the main goal to extract features and to classify frames according to one of three previously classes of cutting's volume. A collection of supervised classifiers were applied in order to allow comparisons about their accuracy and efficiency. We used the Optimum-Path Forest (OPF), Artificial Neural Network using Multi layer Perceptrons (ANN-MLP), Support Vector Machines (SVM) and a Bayesian Classifier (BC) for this task. The first one outperformed all the remaining classifiers. Recall that we are also the first to introduce the OPF classifier in this field of knowledge. Very good results show the robustness of the proposed system, which can be also integrated with other commonly system (Mud-Logging) in order to improve the last one's efficiency.

Application of intelligent systems for specification of automotive equipments using LEDs

The advantages offered by the electronic component light emitting diode ( LED) have caused a quick and wide application of this device in replacement of incandescent lights. However, in its combined application, the relationship between the design variables and the desired effect or result is very complex and it becomes difficult to model by conventional techniques. This work consists of the development of a technique, through artificial neural networks, to make possible to obtain the luminous intensity values of brake lights using LEDs from design data. (C) 2005 Elsevier B.V. All rights reserved.

A novel intelligent system to nitrogen content prediction in plants using indirect chlorophyll measurements

The accurate identification of the nitrogen content in plants is extremely important since it involves economic aspects and environmental impacts, Several experimental tests have been carried out to obtain characteristics and parameters associated with the health of plants and its growing. The nitrogen content identification in plants involves a lot of non-linear parameters and complexes mathematical models. This paper describes a novel approach for identification of nitrogen content thought SPAD index using artificial neural networks (ANN). The network acts as identifier of relationships among, crop varieties, fertilizer treatments, type of leaf and nitrogen content in the plants (target). So, nitrogen content can be generalized and estimated and from an input parameter set. This approach can form the basis for development of an accurate real time system to predict nitrogen content in plants.

Beam Search Algorithms for Minimizing Tool Switches on a Flexible Manufacturing System

In the minimization of tool switches problem we seek a sequence to process a set of jobs so that the number of tool switches required is minimized. In this work different variations of a heuristic based on partial ordered job sequences are implemented and evaluated. All variations adopt a depth first strategy of the enumeration tree. The computational test results indicate that good results can be obtained by a variation which keeps the best three branches at each node of the enumeration tree, and randomly choose, among all active nodes, the next node to branch when backtracking.

Allocation of protective devices in distribution circuits using nonlinear programming models and genetic algorithms

The optimized allocation of protective devices in strategic points of the circuit improves the quality of the energy supply and the system reliability index. This paper presents a nonlinear integer programming (NLIP) model with binary variables, to deal with the problem of protective device allocation in the main feeder and all branches of an overhead distribution circuit, to improve the reliability index and to provide customers with service of high quality and reliability. The constraints considered in the problem take into account technical and economical limitations, such as coordination problems of serial protective devices, available equipment, the importance of the feeder and the circuit topology. The use of genetic algorithms (GAs) is proposed to solve this problem, using a binary representation that does (1) or does not (0) show allocation of protective devices (reclosers, sectionalizers and fuses) in predefined points of the circuit. Results are presented for a real circuit (134 busses), with the possibility of protective device allocation in 29 points. Also the ability of the algorithm in finding good solutions while improving significantly the indicators of reliability is shown. (C) 2003 Elsevier B.V. All rights reserved.

Design of a control system using linear matrix inequalities for the active vibration control of a plate

The study of algorithms for active vibration control in smart structures is an area of interest, mainly due to the demand for better performance of mechanical systems, such as aircraft and aerospace structures. Smart structures, formed using actuators and sensors, can improve the dynamic performance with the application of several kinds of controllers. This article describes the application of a technique based on linear matrix inequalities (LMI) to design an active control system. The positioning of the actuators, the design of a robust state feedback controller and the design of an observer are all achieved using LMI. The following are considered in the controller design: limited actuator input, bounded output (energy) and robustness to parametric uncertainties. Active vibration control of a flat plate is chosen as an application example. The model is identified using experimental data by an eigensystem realization algorithm (ERA) and the placement of the two piezoelectric actuators and single sensor is determined using a finite element model (FEM) and an optimization procedure. A robust controller for active damping is designed using an LMI framework, and a reduced model with observation and control spillover effects is implemented using a computer. The simulation results demonstrate the efficacy of the approach, and show that the control system increases the damping in some of the modes.