Algoritmo Q-learning como estratégia de exploração e/ou explotação para metaheurísticas GRASP e algoritmo genético

Techniques of optimization known as metaheuristics have achieved success in the resolution of many problems classified as NP-Hard. These methods use non deterministic approaches that reach very good solutions which, however, don t guarantee the determination of the global optimum. Beyond the inherent difficulties related to the complexity that characterizes the optimization problems, the metaheuristics still face the dilemma of xploration/exploitation, which consists of choosing between a greedy search and a wider exploration of the solution space. A way to guide such algorithms during the searching of better solutions is supplying them with more knowledge of the problem through the use of a intelligent agent, able to recognize promising regions and also identify when they should diversify the direction of the search. This way, this work proposes the use of Reinforcement Learning technique - Q-learning Algorithm - as exploration/exploitation strategy for the metaheuristics GRASP (Greedy Randomized Adaptive Search Procedure) and Genetic Algorithm. The GRASP metaheuristic uses Q-learning instead of the traditional greedy-random algorithm in the construction phase. This replacement has the purpose of improving the quality of the initial solutions that are used in the local search phase of the GRASP, and also provides for the metaheuristic an adaptive memory mechanism that allows the reuse of good previous decisions and also avoids the repetition of bad decisions. In the Genetic Algorithm, the Q-learning algorithm was used to generate an initial population of high fitness, and after a determined number of generations, where the rate of diversity of the population is less than a certain limit L, it also was applied to supply one of the parents to be used in the genetic crossover operator. Another significant change in the hybrid genetic algorithm is the proposal of a mutually interactive cooperation process between the genetic operators and the Q-learning algorithm. In this interactive/cooperative process, the Q-learning algorithm receives an additional update in the matrix of Q-values based on the current best solution of the Genetic Algorithm. The computational experiments presented in this thesis compares the results obtained with the implementation of traditional versions of GRASP metaheuristic and Genetic Algorithm, with those obtained using the proposed hybrid methods. Both algorithms had been applied successfully to the symmetrical Traveling Salesman Problem, which was modeled as a Markov decision process

Arquitetura multiagente baseada em nuvem de partículas para hibridização de metaheurísticas

This thesis proposes an architecture of a new multiagent system framework for hybridization of metaheuristics inspired on the general Particle Swarm Optimization framework (PSO). The main contribution is to propose an effective approach to solve hard combinatory optimization problems. The choice of PSO as inspiration was given because it is inherently multiagent, allowing explore the features of multiagent systems, such as learning and cooperation techniques. In the proposed architecture, particles are autonomous agents with memory and methods for learning and making decisions, using search strategies to move in the solution space. The concepts of position and velocity originally defined in PSO are redefined for this approach. The proposed architecture was applied to the Traveling Salesman Problem and to the Quadratic Assignment Problem, and computational experiments were performed for testing its effectiveness. The experimental results were promising, with satisfactory performance, whereas the potential of the proposed architecture has not been fully explored. For further researches, the proposed approach will be also applied to multiobjective combinatorial optimization problems, which are closer to real-world problems. In the context of applied research, we intend to work with both students at the undergraduate level and a technical level in the implementation of the proposed architecture in real-world problems

Algoritmos científicos

The Scientific Algorithms are a new metaheuristics inspired in the scientific research process. The new method introduces the idea of theme to search the solution space of hard problems. The inspiration for this class of algorithms comes from the act of researching that comprises thinking, knowledge sharing and disclosing new ideas. The ideas of the new method are illustrated in the Traveling Salesman Problem. A computational experiment applies the proposed approach to a new variant of the Traveling Salesman Problem named Car Renter Salesman Problem. The results are compared to state-of-the-art algorithms for the latter problem

Avaliação sistemática de abordagens de derivação de produto

Product derivation tools are responsible for automating the development process of software product lines. The configuration knowledge, which is responsible for mapping the problem space to the solution space, plays a fundamental role on product derivation approaches. Each product derivation approach adopts different strategies and techniques to manage the existing variabilities in code assets. There is a lack of empirical studies to analyze these different approaches. This dissertation has the aim of comparing systematically automatic product derivation approaches through of the development of two different empirical studies. The studies are analyzed under two perspectives: (i) qualitative that analyzes the characteristics of approaches using specific criteria; and (ii) quantitative that quantifies specific properties of product derivation artifacts produced for the different approaches. A set of criteria and metrics are also being proposed with the aim of providing support to the qualitative and quantitative analysis. Two software product lines from the web and mobile application domains are targets of our study

Avaliação de um método de cromatografia em fase gasosa head space e estudo da estabilidade do etanol em amostras de sangue

The ethanol is the most overused psychoactive drug over the world; this fact makes it one of the main substances required in toxicological exams nowadays. The development of an analytical method, adaptation or implementation of a method known, involves a process of validation that estimates its efficiency in the laboratory routine and credibility of the method. The stability is defined as the ability of the sample of material to keep the initial value of a quantitative measure for a defined period within specific limits when stored under defined conditions. This study aimed to evaluate the method of Gas chromatography and study the stability of ethanol in blood samples, considering the variables time and temperature of storage, and the presence of preservative and, with that check if the conditions of conservation and storage used in this study maintain the quality of the sample and preserve the originally amount of analyte present. Blood samples were collected from 10 volunteers to evaluate the method and to study the stability of ethanol. For the evaluation of the method, part of the samples was added to known concentrations of ethanol. In the study of stability, the other side of the pool of blood was placed in two containers: one containing the preservative sodium fluoride 1% and the anticoagulant heparin and the other only heparin, was added ethanol at a concentration of 0.6 g/L, fractionated in two bottles, one being stored at 4ºC (refrigerator) and another at -20ºC (freezer), the tests were performed on the same day (time zero) and after 1, 3, 7, 14, 30 and 60 days of storage. The assessment found the difference in results during storage in relation to time zero. It used the technique of headspace associated with gas chromatography with the FID and capillary column with stationary phase of polyethylene. The best analysis of chromatographic conditions were: temperature of 50ºC (column), 150ºC (jet) and 250ºC (detector), with retention time for ethanol from 9.107 ± 0.026 and the tercbutanol (internal standard) of 8.170 ± 0.081 minutes, the ethanol being separated properly from acetaldehyde, acetone, methanol and 2-propanol, which are potential interfering in the determination of ethanol. The technique showed linearity in the concentration range of 0.01 and 3.2 g/L (0.8051 x + y = 0.6196; r2 = 0.999). The calibration curve showed the following equation of the line: y = x 0.7542 + 0.6545, with a linear correlation coefficient equal to 0.996. The average recovery was 100.2%, the coefficients of variation of accuracy and inter intra test showed values of up to 7.3%, the limit of detection and quantification was 0.01 g/L and showed coefficient of variation within the allowed. The analytical method evaluated in this study proved to be fast, efficient and practical, given the objective of this work satisfactorily. The study of stability has less than 20% difference in the response obtained under the conditions of storage and stipulated period, compared with the response obtained at time zero and at the significance level of 5%, no statistical difference in the concentration of ethanol was observed between analysis. The results reinforce the reliability of the method of gas chromatography and blood samples in search of ethanol, either in the toxicological, forensic, social or clinic

Metodologia estatística na solução do problema do caixeiro viajante e na avaliação de algoritmos : um estudo aplicado à transgenética computacional

The problems of combinatory optimization have involved a large number of researchers in search of approximative solutions for them, since it is generally accepted that they are unsolvable in polynomial time. Initially, these solutions were focused on heuristics. Currently, metaheuristics are used more for this task, especially those based on evolutionary algorithms. The two main contributions of this work are: the creation of what is called an -Operon- heuristic, for the construction of the information chains necessary for the implementation of transgenetic (evolutionary) algorithms, mainly using statistical methodology - the Cluster Analysis and the Principal Component Analysis; and the utilization of statistical analyses that are adequate for the evaluation of the performance of the algorithms that are developed to solve these problems. The aim of the Operon is to construct good quality dynamic information chains to promote an -intelligent- search in the space of solutions. The Traveling Salesman Problem (TSP) is intended for applications based on a transgenetic algorithmic known as ProtoG. A strategy is also proposed for the renovation of part of the chromosome population indicated by adopting a minimum limit in the coefficient of variation of the adequation function of the individuals, with calculations based on the population. Statistical methodology is used for the evaluation of the performance of four algorithms, as follows: the proposed ProtoG, two memetic algorithms and a Simulated Annealing algorithm. Three performance analyses of these algorithms are proposed. The first is accomplished through the Logistic Regression, based on the probability of finding an optimal solution for a TSP instance by the algorithm being tested. The second is accomplished through Survival Analysis, based on a probability of the time observed for its execution until an optimal solution is achieved. The third is accomplished by means of a non-parametric Analysis of Variance, considering the Percent Error of the Solution (PES) obtained by the percentage in which the solution found exceeds the best solution available in the literature. Six experiments have been conducted applied to sixty-one instances of Euclidean TSP with sizes of up to 1,655 cities. The first two experiments deal with the adjustments of four parameters used in the ProtoG algorithm in an attempt to improve its performance. The last four have been undertaken to evaluate the performance of the ProtoG in comparison to the three algorithms adopted. For these sixty-one instances, it has been concluded on the grounds of statistical tests that there is evidence that the ProtoG performs better than these three algorithms in fifty instances. In addition, for the thirty-six instances considered in the last three trials in which the performance of the algorithms was evaluated through PES, it was observed that the PES average obtained with the ProtoG was less than 1% in almost half of these instances, having reached the greatest average for one instance of 1,173 cities, with an PES average equal to 3.52%. Therefore, the ProtoG can be considered a competitive algorithm for solving the TSP, since it is not rare in the literature find PESs averages greater than 10% to be reported for instances of this size.

Incorporação do vínculo de suavidade no ajuste de histórico de reservatórios de petróleo

The history match procedure in an oil reservoir is of paramount importance in order to obtain a characterization of the reservoir parameters (statics and dynamics) that implicates in a predict production more perfected. Throughout this process one can find reservoir model parameters which are able to reproduce the behaviour of a real reservoir.Thus, this reservoir model may be used to predict production and can aid the oil file management. During the history match procedure the reservoir model parameters are modified and for every new set of reservoir model parameters found, a fluid flow simulation is performed so that it is possible to evaluate weather or not this new set of parameters reproduces the observations in the actual reservoir. The reservoir is said to be matched when the discrepancies between the model predictions and the observations of the real reservoir are below a certain tolerance. The determination of the model parameters via history matching requires the minimisation of an objective function (difference between the observed and simulated productions according to a chosen norm) in a parameter space populated by many local minima. In other words, more than one set of reservoir model parameters fits the observation. With respect to the non-uniqueness of the solution, the inverse problem associated to history match is ill-posed. In order to reduce this ambiguity, it is necessary to incorporate a priori information and constraints in the model reservoir parameters to be determined. In this dissertation, the regularization of the inverse problem associated to the history match was performed via the introduction of a smoothness constraint in the following parameter: permeability and porosity. This constraint has geological bias of asserting that these two properties smoothly vary in space. In this sense, it is necessary to find the right relative weight of this constrain in the objective function that stabilizes the inversion and yet, introduces minimum bias. A sequential search method called COMPLEX was used to find the reservoir model parameters that best reproduce the observations of a semi-synthetic model. This method does not require the usage of derivatives when searching for the minimum of the objective function. Here, it is shown that the judicious introduction of the smoothness constraint in the objective function formulation reduces the associated ambiguity and introduces minimum bias in the estimates of permeability and porosity of the semi-synthetic reservoir model