Algoritmo Q-learning como estratégia de exploração e/ou explotação para metaheurísticas GRASP e algoritmo genético

Techniques of optimization known as metaheuristics have achieved success in the resolution of many problems classified as NP-Hard. These methods use non deterministic approaches that reach very good solutions which, however, don t guarantee the determination of the global optimum. Beyond the inherent difficulties related to the complexity that characterizes the optimization problems, the metaheuristics still face the dilemma of xploration/exploitation, which consists of choosing between a greedy search and a wider exploration of the solution space. A way to guide such algorithms during the searching of better solutions is supplying them with more knowledge of the problem through the use of a intelligent agent, able to recognize promising regions and also identify when they should diversify the direction of the search. This way, this work proposes the use of Reinforcement Learning technique - Q-learning Algorithm - as exploration/exploitation strategy for the metaheuristics GRASP (Greedy Randomized Adaptive Search Procedure) and Genetic Algorithm. The GRASP metaheuristic uses Q-learning instead of the traditional greedy-random algorithm in the construction phase. This replacement has the purpose of improving the quality of the initial solutions that are used in the local search phase of the GRASP, and also provides for the metaheuristic an adaptive memory mechanism that allows the reuse of good previous decisions and also avoids the repetition of bad decisions. In the Genetic Algorithm, the Q-learning algorithm was used to generate an initial population of high fitness, and after a determined number of generations, where the rate of diversity of the population is less than a certain limit L, it also was applied to supply one of the parents to be used in the genetic crossover operator. Another significant change in the hybrid genetic algorithm is the proposal of a mutually interactive cooperation process between the genetic operators and the Q-learning algorithm. In this interactive/cooperative process, the Q-learning algorithm receives an additional update in the matrix of Q-values based on the current best solution of the Genetic Algorithm. The computational experiments presented in this thesis compares the results obtained with the implementation of traditional versions of GRASP metaheuristic and Genetic Algorithm, with those obtained using the proposed hybrid methods. Both algorithms had been applied successfully to the symmetrical Traveling Salesman Problem, which was modeled as a Markov decision process

Metaheurística GRASP para o problema de planejamento da expansão de sistemas de transmissão de energia elétrica

Pós-graduação em Engenharia Elétrica - FEIS

Aplicação do algoritmo de otimização por colônia de formigas sobre o problema do passeio do robô seletivo

This work seeks to propose and evaluate a change to the Ant Colony Optimization based on the results of experiments performed on the problem of Selective Ride Robot (PRS, a new problem, also proposed in this paper. Four metaheuristics are implemented, GRASP, VNS and two versions of Ant Colony Optimization, and their results are analyzed by running the algorithms over 32 instances created during this work. The metaheuristics also have their results compared to an exact approach. The results show that the algorithm implemented using the GRASP metaheuristic show good results. The version of the multicolony ant colony algorithm, proposed and evaluated in this work, shows the best results

Modelo estocástico de programação matemática de alocação de medidores de tensão em alimentadores radiais de distribuição de energia elétrica para localização de faltas e monitoramento do perfil de tensão

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Fluxo de carga não-iterativo para a análise de sistemas de distribuição de energia elétrica radiais e malhados

Pós-graduação em Engenharia Elétrica - FEIS

Reconfiguração de alimentadores em sistemas de distribuição usando a metaheurística GRASP

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Algoritmo GRASP especializado aplicado ao problema de reconfiguração de alimentadores em sistemas de distribuição radial

Pós-graduação em Engenharia Elétrica - FEIS

GRASP and statistical bounds for heuristic solutions to combinatorial problems

The quality of a heuristic solution to a NP-hard combinatorial problem is hard to assess. A few studies have advocated and tested statistical bounds as a method for assessment. These studies indicate that statistical bounds are superior to the more widely known and used deterministic bounds. However, the previous studies have been limited to a few metaheuristics and combinatorial problems and, hence, the general performance of statistical bounds in combinatorial optimization remains an open question. This work complements the existing literature on statistical bounds by testing them on the metaheuristic Greedy Randomized Adaptive Search Procedures (GRASP) and four combinatorial problems. Our findings confirm previous results that statistical bounds are reliable for the p-median problem, while we note that they also seem reliable for the set covering problem. For the quadratic assignment problem, the statistical bounds has previously been found reliable when obtained from the Genetic algorithm whereas in this work they found less reliable. Finally, we provide statistical bounds to four 2-path network design problem instances for which the optimum is currently unknown.

Metaheuristic approaches to the hybrid flow shop scheduling problem with a cost-related criterion

In the paper, the flow-shop scheduling problem with parallel machines at each stage (machine center) is studied. For each job its release and due date as well as a processing time for its each operation are given. The scheduling criterion consists of three parts: the total weighted earliness, the total weighted tardiness and the total weighted waiting time. The criterion takes into account the costs of storing semi-manufactured products in the course of production and ready-made products as well as penalties for not meeting the deadlines stated in the conditions of the contract with customer. To solve the problem, three constructive algorithms and three metaheuristics (based one Tabu Search and Simulated Annealing techniques) are developed and experimentally analyzed. All the proposed algorithms operate on the notion of so-called operation processing order, i.e. the order of operations on each machine. We show that the problem of schedule construction on the base of a given operation processing order can be reduced to the linear programming task. We also propose some approximation algorithm for schedule construction and show the conditions of its optimality.

Touch, grasp, deliver and control : functional cross-talk between microtubules and cell adhesions

Cross-talk between microtubule networks and sites of cell-matrix and cell-cell adhesion has profound impact on these structures and is essential for proper cell organization, polarization and motility. Components of adhesion sites can interact directly with microtubules or with proteins that specifically associate with microtubule plus ends and minus ends and in this way capture, stabilize or destabilize microtubules. In their turn, microtubules can serve as routes for delivery of structural and regulatory factors that control adhesion site turnover. In addition, the microtubule lattice or growing microtubule plus ends can serve as diffusional sinks that accumulate and scaffold regulatory molecules, thereby affecting their activity in the vicinity of adhesions. Combination of these mechanisms underlies the functional co-operation between microtubules and adhesion sites and defines their dynamic behavior.

Hybrid metaheuristic techniques for optimising sugarcane rail operations

Mixed integer programming and parallel-machine job shop scheduling are used to solve the sugarcane rail transport scheduling problem. Constructive heuristics and metaheuristics were developed to produce a more efficient scheduling system and so reduce operating costs. The solutions were tested on small and large size problems. High-quality solutions and improved CPU time are the result of developing new hybrid techniques which consist of different ways of integrating simulated annealing and Tabu search techniques.

Inversão de funções do plano no plano aplicada ao cálculo de azeótropos

Azeotropia é um fenômeno termodinâmico onde um líquido em ebulição produz um vapor com composição idêntica. Esta situação é um desafio para a Engenharia de Separação, já que os processos de destilação exploram as diferenças entre as volatilidades relativas e, portanto, um azeótropo pode ser uma barreira para a separação. Em misturas binárias, o cálculo da azeotropia é caracterizado por um sistema não-linear do tipo 2 × 2. Um interessante e raro caso é o denominado azeotropia dupla, que pode ser verificado quando este sistema não-linear tem duas soluções, correspondendo a dois azeótropos distintos. Diferentes métodos tem sido utilizados na resolução de problemas desta natureza, como métodos estocásticos de otimização e as técnicas intervalares (do tipo Newton intervalar/bisseção generalizada). Nesta tese apresentamos a formulação do problema de azeotropia dupla e uma nova e robusta abordagem para a resolução dos sistemas não-lineares do tipo 2 × 2, que é a inversão de funções do plano no plano (MALTA; SALDANHA; TOMEI, 1996). No método proposto, as soluções são obtidas através de um conjunto de ações: obtenção de curvas críticas e de pré-imagens de pontos arbritários, inversão da função e por fim, as soluções esperadas para o problema de azeotropia. Esta metodologia foi desenvolvida para resolver sistemas não-lineares do tipo 2 × 2, tendo como objetivo dar uma visão global da função que modela o fenômeno em questão, além, é claro, de gerar as soluções esperadas. Serão apresentados resultados numéricos para o cálculo dos azeótropos no sistema benzeno + hexafluorobenzeno a baixas pressões por este método de inversão. Como ferramentas auxiliares, serão também apresentados aspectos numéricos usando aproximações clássicas, tais como métodos de Newton com técnicas de globalização e o algorítmo de otimização não-linear C-GRASP, para efeito de comparação.

Multiple grasp-specific representations of tool dynamics mediate skillful manipulation.

Skillful tool use requires knowledge of the dynamic properties of tools in order to specify the mapping between applied force and tool motion. Importantly, this mapping depends on the orientation of the tool in the hand. Here we investigate the representation of dynamics during skillful manipulation of a tool that can be grasped at different orientations. We ask whether the motor system uses a single general representation of dynamics for all grasp contexts or whether it uses multiple grasp-specific representations. Using a novel robotic interface, subjects rotated a virtual tool whose orientation relative to the hand could be varied. Subjects could immediately anticipate the force direction for each orientation of the tool based on its visual geometry, and, with experience, they learned to parameterize the force magnitude. Surprisingly, this parameterization of force magnitude showed limited generalization when the orientation of the tool changed. Had subjects parameterized a single general representation, full generalization would be expected. Thus, our results suggest that object dynamics are captured by multiple representations, each of which encodes the mapping associated with a specific grasp context. We suggest that the concept of grasp-specific representations may provide a unifying framework for interpreting previous results related to dynamics learning.