Algoritmo Q-learning como estratégia de exploração e/ou explotação para metaheurísticas GRASP e algoritmo genético

Techniques of optimization known as metaheuristics have achieved success in the resolution of many problems classified as NP-Hard. These methods use non deterministic approaches that reach very good solutions which, however, don t guarantee the determination of the global optimum. Beyond the inherent difficulties related to the complexity that characterizes the optimization problems, the metaheuristics still face the dilemma of xploration/exploitation, which consists of choosing between a greedy search and a wider exploration of the solution space. A way to guide such algorithms during the searching of better solutions is supplying them with more knowledge of the problem through the use of a intelligent agent, able to recognize promising regions and also identify when they should diversify the direction of the search. This way, this work proposes the use of Reinforcement Learning technique - Q-learning Algorithm - as exploration/exploitation strategy for the metaheuristics GRASP (Greedy Randomized Adaptive Search Procedure) and Genetic Algorithm. The GRASP metaheuristic uses Q-learning instead of the traditional greedy-random algorithm in the construction phase. This replacement has the purpose of improving the quality of the initial solutions that are used in the local search phase of the GRASP, and also provides for the metaheuristic an adaptive memory mechanism that allows the reuse of good previous decisions and also avoids the repetition of bad decisions. In the Genetic Algorithm, the Q-learning algorithm was used to generate an initial population of high fitness, and after a determined number of generations, where the rate of diversity of the population is less than a certain limit L, it also was applied to supply one of the parents to be used in the genetic crossover operator. Another significant change in the hybrid genetic algorithm is the proposal of a mutually interactive cooperation process between the genetic operators and the Q-learning algorithm. In this interactive/cooperative process, the Q-learning algorithm receives an additional update in the matrix of Q-values based on the current best solution of the Genetic Algorithm. The computational experiments presented in this thesis compares the results obtained with the implementation of traditional versions of GRASP metaheuristic and Genetic Algorithm, with those obtained using the proposed hybrid methods. Both algorithms had been applied successfully to the symmetrical Traveling Salesman Problem, which was modeled as a Markov decision process

A expressão do condicionado contrafactual em construções se p, então q no português brasileiro

Conselho Nacional de Desenvolvimento Científico e Tecnológico

Fases e criticalidade no modelo ashkin - teller de três cores

The usual Ashkin-Teller (AT) model is obtained as a superposition of two Ising models coupled through a four-spin interaction term. In two dimension the AT model displays a line of fixed points along which the exponents vary continuously. On this line the model becomes soluble via a mapping onto the Baxter model. Such richness of multicritical behavior led Grest and Widom to introduce the N-color Ashkin-Teller model (N-AT). Those authors made an extensive analysis of the model thus introduced both in the isotropic as well as in the anisotropic cases by several analytical and computational methods. In the present work we define a more general version of the 3-color Ashkin-Teller model by introducing a 6-spin interaction term. We investigate the corresponding symmetry structure presented by our model in conjunction with an analysis of possible phase diagrams obtained by real space renormalization group techniques. The phase diagram are obtained at finite temperature in the region where the ferromagnetic behavior is predominant. Through the use of the transmissivities concepts we obtain the recursion relations in some periodical as well as aperiodic hierarchical lattices. In a first analysis we initially consider the two-color Ashkin-Teller model in order to obtain some results with could be used as a guide to our main purpose. In the anisotropic case the model was previously studied on the Wheatstone bridge by Claudionor Bezerra in his Master Degree dissertation. By using more appropriated computational resources we obtained isomorphic critical surfaces described in Bezerra's work but not properly identified. Besides, we also analyzed the isotropic version in an aperiodic hierarchical lattice, and we showed how the geometric fluctuations are affected by such aperiodicity and its consequences in the corresponding critical behavior. Those analysis were carried out by the use of appropriated definitions of transmissivities. Finally, we considered the modified 3-AT model with a 6-spin couplings. With the inclusion of such term the model becomes more attractive from the symmetry point of view. For some hierarchical lattices we derived general recursion relations in the anisotropic version of the model (3-AAT), from which case we can obtain the corresponding equations for the isotropic version (3-IAT). The 3-IAT was studied extensively in the whole region where the ferromagnetic couplings are dominant. The fixed points and the respective critical exponents were determined. By analyzing the attraction basins of such fixed points we were able to find the three-parameter phase diagram (temperature £ 4-spin coupling £ 6-spin coupling). We could identify fixed points corresponding to the universality class of Ising and 4- and 8-state Potts model. We also obtained a fixed point which seems to be a sort of reminiscence of a 6-state Potts fixed point as well as a possible indication of the existence of a Baxter line. Some unstable fixed points which do not belong to any aforementioned q-state Potts universality class was also found

Aplicações da q-Álgebra em física da matéria condensada

We address the generalization of thermodynamic quantity q-deformed by q-algebra that describes a general algebra for bosons and fermions . The motivation for our study stems from an interest to strengthen our initial ideas, and a possible experimental application. On our journey, we met a generalization of the recently proposed formalism of the q-calculus, which is the application of a generalized sequence described by two parameters deformation positive real independent and q1 and q2, known for Fibonacci oscillators . We apply the wellknown problem of Landau diamagnetism immersed in a space D-dimensional, which still generates good discussions by its nature, and dependence with the number of dimensions D, enables us future extend its application to systems extra-dimensional, such as Modern Cosmology, Particle Physics and String Theory. We compare our results with some experimentally obtained performing major equity. We also use the formalism of the oscillators to Einstein and Debye solid, strengthening the interpretation of the q-deformation acting as a factor of disturbance or impurity in a given system, modifying the properties of the same. Our results show that the insertion of two parameters of disorder, allowed a wider range of adjustment , i.e., enabling change only the desired property, e.g., the thermal conductivity of a same element without the waste essence

Estudo comparativo de abordagens de inversão do fator de qualidade Q

Seismic wave dispersion and attenuation studies have become an important tool for lithology and fluid discrimination in hydrocarbon reservoirs. The processes associated to attenuation are complex and are encapsulated in a single quantitative description called quality factor (Q). The present dissertation has the objective of comparing different approaches of Q determination and is divided in two parts. Firstly, we made performance and robustness tests of three different approaches for Q determination in the frequency domain. They are: peak shift, centroid shift and spectral ratio. All these tests were performed in a three-layered model. In the suite of tests performed here, we varied the thickness, Q and inclination of the layers for propagation pulses with central frequency of 30, 40 and 60 Hz. We found that the centroid shift method is produces robust results for the entire suíte of tests. Secondly, we inverted for Q values using the peak and centroid shift methods using an sequential grid search algorithm. In this case, centroid shift method also produced more robust results than the peak shift method, despite being of slower convergence