Algoritmo Q-learning como estratégia de exploração e/ou explotação para metaheurísticas GRASP e algoritmo genético

Techniques of optimization known as metaheuristics have achieved success in the resolution of many problems classified as NP-Hard. These methods use non deterministic approaches that reach very good solutions which, however, don t guarantee the determination of the global optimum. Beyond the inherent difficulties related to the complexity that characterizes the optimization problems, the metaheuristics still face the dilemma of xploration/exploitation, which consists of choosing between a greedy search and a wider exploration of the solution space. A way to guide such algorithms during the searching of better solutions is supplying them with more knowledge of the problem through the use of a intelligent agent, able to recognize promising regions and also identify when they should diversify the direction of the search. This way, this work proposes the use of Reinforcement Learning technique - Q-learning Algorithm - as exploration/exploitation strategy for the metaheuristics GRASP (Greedy Randomized Adaptive Search Procedure) and Genetic Algorithm. The GRASP metaheuristic uses Q-learning instead of the traditional greedy-random algorithm in the construction phase. This replacement has the purpose of improving the quality of the initial solutions that are used in the local search phase of the GRASP, and also provides for the metaheuristic an adaptive memory mechanism that allows the reuse of good previous decisions and also avoids the repetition of bad decisions. In the Genetic Algorithm, the Q-learning algorithm was used to generate an initial population of high fitness, and after a determined number of generations, where the rate of diversity of the population is less than a certain limit L, it also was applied to supply one of the parents to be used in the genetic crossover operator. Another significant change in the hybrid genetic algorithm is the proposal of a mutually interactive cooperation process between the genetic operators and the Q-learning algorithm. In this interactive/cooperative process, the Q-learning algorithm receives an additional update in the matrix of Q-values based on the current best solution of the Genetic Algorithm. The computational experiments presented in this thesis compares the results obtained with the implementation of traditional versions of GRASP metaheuristic and Genetic Algorithm, with those obtained using the proposed hybrid methods. Both algorithms had been applied successfully to the symmetrical Traveling Salesman Problem, which was modeled as a Markov decision process

Estudo do acoplamento entre superfícies seletivas de frequência assimétricas em estruturas de multicamadas

This work presents the development of new microwaves structures, filters and high gain antenna, through the cascading of frequency selective surfaces, which uses fractals Dürer and Minkowski patches as elements, addition of an element obtained from the combination of the other two simple the cross dipole and the square spiral. Frequency selective surfaces (FSS) includes a large area of Telecommunications and have been widely used due to its low cost, low weight and ability to integrate with others microwaves circuits. They re especially important in several applications, such as airplane, antennas systems, radomes, rockets, missiles, etc. FSS applications in high frequency ranges have been investigated, as well as applications of cascading structures or multi-layer, and active FSS. In this work, we present results for simulated and measured transmission characteristics of cascaded structures (multilayer), aiming to investigate the behavior of the operation in terms of bandwidth, one of the major problems presented by frequency selective surfaces. Comparisons are made with simulated results, obtained using commercial software such as Ansoft DesignerTM v3 and measured results in the laboratory. Finally, some suggestions are presented for future works on this subject

Modelo bioeconômico para cálculo de custos e receitas em sistemas de produção de gado de corte visando à obtenção de valores econômicos de características produtivas e reprodutivas

Foi desenvolvido um modelo bioeconômico para o cálculo do desempenho, dos custos e das receitas para obtenção de valores econômicos de características de interesse econômico em sistemas de produção de gado de corte no Brasil. As informações de desempenho e de parâmetros biológicos e econômicos foram obtidas em uma propriedade de gado Nelore que realiza ciclo completo com venda de reprodutores. O modelo é determinístico e estático e foram usadas planilhas Excel para a realização dos cálculos. Com base nas informações originais, foram simulados dois sistemas de produção, um fazendo o ciclo completo com venda de reprodutores (CcoR) e o outro, de cria (Cc). Foram calculados os custos e as receitas para esses dois sistemas e, a partir disto, foram obtidos seus lucros anuais. Para o cálculo dos valores econômicos foram escolhidas as características peso à desmama (PD), peso adulto da vaca (PAV), taxa de prenhez (TP) e taxa de desmama (TD), que são de interesse nos dois sistemas de produção. Para avaliar o impacto das mudanças no desempenho das características sobre o lucro anual do sistema de produção, os valores iniciais das características foram aumentados em 1%. Esse aumento resultou em mudanças positivas no lucro, observando-se que a TD foi a característica que apresentou maior impacto nos dois sistemas. Para o CcoR, os valores econômicos para PD, PAV, TP e TD foram, respectivamente, R$ 0,40/kg, R$ 0,09/kg, R$ 3,20/1% e R$ 10,15/1%. Para Cc, estes valores foram, respectivamente, de R$ 1,31/kg, R$ 0,09/kg, R$ 2,41/1% e R$ 3,36/1%. O modelo foi capaz de reproduzir satisfatoriamente o sistema de produção de gado de corte estudado e pode ser adaptado para outras circunstâncias de produção.

Antenas de microfita com patch quase-fractal para aplicações em Redes WPAN/WLAN

The microstrip antennas are in constant evidence in current researches due to several advantages that it presents. Fractal geometry coupled with good performance and convenience of the planar structures are an excellent combination for design and analysis of structures with ever smaller features and multi-resonant and broadband. This geometry has been applied in such patch microstrip antennas to reduce its size and highlight its multi-band behavior. Compared with the conventional microstrip antennas, the quasifractal patch antennas have lower frequencies of resonance, enabling the manufacture of more compact antennas. The aim of this work is the design of quasi-fractal patch antennas through the use of Koch and Minkowski fractal curves applied to radiating and nonradiating antenna s edges of conventional rectangular patch fed by microstrip inset-fed line, initially designed for the frequency of 2.45 GHz. The inset-fed technique is investigated for the impedance matching of fractal antennas, which are fed through lines of microstrip. The efficiency of this technique is investigated experimentally and compared with simulations carried out by commercial software Ansoft Designer used for precise analysis of the electromagnetic behavior of antennas by the method of moments and the neural model proposed. In this dissertation a study of literature on theory of microstrip antennas is done, the same study is performed on the fractal geometry, giving more emphasis to its various forms, techniques for generation of fractals and its applicability. This work also presents a study on artificial neural networks, showing the types/architecture of networks used and their characteristics as well as the training algorithms that were used for their implementation. The equations of settings of the parameters for networks used in this study were derived from the gradient method. It will also be carried out research with emphasis on miniaturization of the proposed new structures, showing how an antenna designed with contours fractals is capable of a miniaturized antenna conventional rectangular patch. The study also consists of a modeling through artificial neural networks of the various parameters of the electromagnetic near-fractal antennas. The presented results demonstrate the excellent capacity of modeling techniques for neural microstrip antennas and all algorithms used in this work in achieving the proposed models were implemented in commercial software simulation of Matlab 7. In order to validate the results, several prototypes of antennas were built, measured on a vector network analyzer and simulated in software for comparison

Análise e projeto de superfícies seletivas de frequência com elementos pré-fractais para aplicações em comunicações indoor

In this thesis, a frequency selective surface (FSS) consists of a two-dimensional periodic structure mounted on a dielectric substrate, which is capable of selecting signals in one or more frequency bands of interest. In search of better performance, more compact dimensions, low cost manufacturing, among other characteristics, these periodic structures have been continually optimized over time. Due to its spectral characteristics, which are similar to band-stop or band-pass filters, the FSSs have been studied and used in several applications for more than four decades. The design of an FSS with a periodic structure composed by pre-fractal elements facilitates the tuning of these spatial filters and the adjustment of its electromagnetic parameters, enabling a compact design which generally has a stable frequency response and superior performance relative to its euclidean counterpart. The unique properties of geometric fractals have shown to be useful, mainly in the production of antennas and frequency selective surfaces, enabling innovative solutions and commercial applications in microwave range. In recent applications, the FSSs modify the indoor propagation environments (emerging concept called wireless building ). In this context, the use of pre-fractal elements has also shown promising results, allowing a more effective filtering of more than one frequency band with a single-layer structure. This thesis approaches the design of FSSs using pre-fractal elements based on Vicsek, Peano and teragons geometries, which act as band-stop spatial filters. The transmission properties of the periodic surfaces are analyzed to design compact and efficient devices with stable frequency responses, applicable to microwave frequency range and suitable for use in indoor communications. The results are discussed in terms of the electromagnetic effect resulting from the variation of parameters such as: fractal iteration number (or fractal level), scale factor, fractal dimension and periodicity of FSS, according the pre-fractal element applied on the surface. The analysis of the fractal dimension s influence on the resonant properties of a FSS is a new contribution in relation to researches about microwave devices that use fractal geometry. Due to its own characteristics and the geometric shape of the Peano pre-fractal elements, the reconfiguration possibility of these structures is also investigated and discussed. This thesis also approaches, the construction of efficient selective filters with new configurations of teragons pre-fractal patches, proposed to control the WLAN coverage in indoor environments by rejecting the signals in the bands of 2.4~2.5 GHz (IEEE 802.11 b) and 5.0~6.0 GHz (IEEE 802.11a). The FSSs are initially analyzed through simulations performed by commercial software s: Ansoft DesignerTM and HFSSTM. The fractal design methodology is validated by experimental characterization of the built prototypes, using alternatively, different measurement setups, with commercial horn antennas and microstrip monopoles fabricated for low cost measurements

Uma implementação paralela híbrida para o problema do caixeiro viajante usando algoritmos genéticos, GRASP e aprendizagem por reforço

The metaheuristics techiniques are known to solve optimization problems classified as NP-complete and are successful in obtaining good quality solutions. They use non-deterministic approaches to generate solutions that are close to the optimal, without the guarantee of finding the global optimum. Motivated by the difficulties in the resolution of these problems, this work proposes the development of parallel hybrid methods using the reinforcement learning, the metaheuristics GRASP and Genetic Algorithms. With the use of these techniques, we aim to contribute to improved efficiency in obtaining efficient solutions. In this case, instead of using the Q-learning algorithm by reinforcement learning, just as a technique for generating the initial solutions of metaheuristics, we use it in a cooperative and competitive approach with the Genetic Algorithm and GRASP, in an parallel implementation. In this context, was possible to verify that the implementations in this study showed satisfactory results, in both strategies, that is, in cooperation and competition between them and the cooperation and competition between groups. In some instances were found the global optimum, in others theses implementations reach close to it. In this sense was an analyze of the performance for this proposed approach was done and it shows a good performance on the requeriments that prove the efficiency and speedup (gain in speed with the parallel processing) of the implementations performed

Otimização de superfícies seletivas de frequência com elementos pré-fractais utilizando rede neural MLP e algoritmos de busca populacional

This thesis describes design methodologies for frequency selective surfaces (FSSs) composed of periodic arrays of pre-fractals metallic patches on single-layer dielectrics (FR4, RT/duroid). Shapes presented by Sierpinski island and T fractal geometries are exploited to the simple design of efficient band-stop spatial filters with applications in the range of microwaves. Initial results are discussed in terms of the electromagnetic effect resulting from the variation of parameters such as, fractal iteration number (or fractal level), fractal iteration factor, and periodicity of FSS, depending on the used pre-fractal element (Sierpinski island or T fractal). The transmission properties of these proposed periodic arrays are investigated through simulations performed by Ansoft DesignerTM and Ansoft HFSSTM commercial softwares that run full-wave methods. To validate the employed methodology, FSS prototypes are selected for fabrication and measurement. The obtained results point to interesting features for FSS spatial filters: compactness, with high values of frequency compression factor; as well as stable frequency responses at oblique incidence of plane waves. This thesis also approaches, as it main focus, the application of an alternative electromagnetic (EM) optimization technique for analysis and synthesis of FSSs with fractal motifs. In application examples of this technique, Vicsek and Sierpinski pre-fractal elements are used in the optimal design of FSS structures. Based on computational intelligence tools, the proposed technique overcomes the high computational cost associated to the full-wave parametric analyzes. To this end, fast and accurate multilayer perceptron (MLP) neural network models are developed using different parameters as design input variables. These neural network models aim to calculate the cost function in the iterations of population-based search algorithms. Continuous genetic algorithm (GA), particle swarm optimization (PSO), and bees algorithm (BA) are used for FSSs optimization with specific resonant frequency and bandwidth. The performance of these algorithms is compared in terms of computational cost and numerical convergence. Consistent results can be verified by the excellent agreement obtained between simulations and measurements related to FSS prototypes built with a given fractal iteration

Estudo comparativo de técnicas de cascateamento de superfícies seletivas em frequência

This work presents a theoretical and numerical analysis for the cascading of frequency selective surfaces, which uses rectangular patches and triangular Koch fractals as elements. Two cascading techniques are used to determine the transmission and reflection characteristics. Frequency selective surfaces includes a large area of Telecommunications and have been widely used due to its low cost, low weight and ability to integrate with others microwaves circuits. They re especially important in several applications, such as airplane, antennas systems, radomes, rockets, missiles, etc.. FSS applications in high frequency ranges have been investigated, as well as applications of cascading structures or multi-layer, and active FSS. Furthermore, the analyses uses the microwave circuit theory, with the Floquet harmonics, it allows to obtain the expressions of the scattering parameters of each structure and also of the composed structure of two or more FSS. In this work, numeric results are presented for the transmission characteristics. Comparisons are made with experimental results and simulated results using the commercial software Ansoft Designer® v3. Finally, some suggestions are presented for future works on this subject

Análise e projeto de superfícies seletivas em frequência multibanda e/ou banda larga

Frequency selective surfaces (Frequency Selective Surface - FSS) are often used in various applications in telecommunications. Some of these applications may require that these structures have response with multiple resonance bands. Other applications require that the FSS response have large frequency range, to meet the necessary requirements. FSS to design with these features there are numerous techniques cited in the scientific literature. Thus, the purpose of this paper is to examine some common techniques such as: Overlap of FSS; Elements combined; Elements Elements convolucionados and fractals. And designing multiband FSS and / or broadband selecting simple ways in terms of construction and occupy the smallest possible space, aiming at practical applications. Given these requirements, three projects FSS were performed: a technology applied to IEEE 802.11 a/b/g/n and two projects for application in UWB. In project development, commercial software Ansoft DesignerTM and experimental results were satisfactory was used

Digital monitoring of mycelium growth kinetics and vigor of shiitake (Lentinula edodes (Berk.) Pegler) on agar medium

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Modelagem e simulação de um separador bifásico com inversão de fases induzida

The present work has the main goal to study the modeling and simulation of a biphasic separator with induced phase inversion, the MDIF, with the utilization of the finite differences method for the resolution of the partial differencial equations which describe the transport of contaminant s mass fraction inside the equipment s settling chamber. With this aim, was developed the deterministic differential model AMADDA, wich was admensionalizated and then semidiscretizated with the method of lines. The integration of the resultant system of ordinary differential equations was realized by means of a modified algorithm of the Adam-Bashfort- Moulton method, and the sthocastic optimization routine of Basin-Hopping was used in the model s parameter estimation procedure . With the aim to establish a comparative referential for the results obtained with the model AMADDA, were used experimental data presented in previous works of the MDIF s research group. The experimental data and those obtained with the model was assessed regarding its normality by means of the Shapiro-Wilk s test, and validated against the experimental results with the Student s t test and the Kruskal-Wallis s test, depending on the result. The results showed satisfactory performance of the model AMADDA in the evaluation of the MDIF s separation efficiency, being possible to determinate that at 1% significance level the calculated results are equivalent to those determinated experimentally in the reference works

The succession dynamics of a macroalgal community after a flood disturbance in a tropical stream from São Paulo State, southeastern Brazil

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Interaction of small amounts of bovine serum albumin with phospholipid monolayers investigated by surface pressure and atomic force microscopy

The influence of small amounts of bovine serum albumin (BSA) (nM concentration) on the lateral organization of phospholipid monolayers at the air-water interface and transferred onto solid substrates as one-layer Langmuir-Blodgett (LB) films was investigated. The kinetics of adsorption of BSA onto the phospholipid monolayers was monitored with surface pressure isotherms in a Langmuir trough, for the zwitterionic dipalmitoylphosphatidyl ethanolamine (N,N-dimethyl-PE) and the anionic dimyristoylphosphatidic acid (DMPA). A monolayer of N,N-dimethyl-PE or DMPA incorporating BSA was transferred onto a solid substrate using the Langmuir-Blodgett technique. Atomic force microscopy (AFM) images of one-layer LB films displayed protein-phospholipid domains, whose morphology was characterized using dynamic scaling theories to calculate roughness exponents. For DMPA-BSA films the surface is characteristic of self-affine fractals, which may be described with the Kardar-Parisi-Zhang (KPZ) equation. on the other hand, for N,N-dimethyl-PE-BSA films, the results indicate a relatively flat surface within the globule. The height profile and the number and size of globules varied with the type of phospholipid. The overall results, from kinetics of adsorption on Langmuir monolayers and surface morphology in LB films, could be interpreted in terms of the higher affinity of BSA to the anionic DMPA than to the zwitterionic N,N-dimethyl-PE. Furthermore, the effects from such small amounts of BSA in the monolayer point to a cooperative response of DMPA and N,N-dimethyl-PE monolayers to the protein. (c) 2005 Elsevier B.V. All rights reserved.

Bifurcation analysis of a new Lorenz-like chaotic system

In this paper we study the local codimension one and two bifurcations which occur in a family of three-dimensional vector fields depending on three parameters. An equivalent family, depending on five parameters, was recently proposed as a new chaotic system with a Lorenz-like butterfly shaped attractor and was studied mainly from a numerical point of view, for particular values of the parameters, for which computational evidences of the chaotic attractor was shown. In order to contribute to the understand of this new system we present an analytical study and the bifurcation diagrams of an equivalent three parameter system, showing the qualitative changes in the dynamics of its solutions, for different values of the parameters. (C) 2007 Elsevier Ltd. All rights reserved.

Aplicações da teoria da percolação à modelagem e simulação de reservatórios de petróleo

In this thesis we study some problems related to petroleum reservoirs using methods and concepts of Statistical Physics. The thesis could be divided percolation problem in random multifractal support motivated by its potential application in modelling oil reservoirs. We develped an heterogeneous and anisotropic grid that followin two parts. The first one introduce a study of the percolations a random multifractal distribution of its sites. After, we determine the percolation threshold for this grid, the fractal dimension of the percolating cluster and the critical exponents ß and v. In the second part, we propose an alternative systematic of modelling and simulating oil reservoirs. We introduce a statistical model based in a stochastic formulation do Darcy Law. In this model, the distribution of permeabilities is localy equivalent to the basic model of bond percolation