Uma contribuição à solução do problema dos k-servos usando aprendizagem por reforço

Neste trabalho é proposto um novo algoritmo online para o resolver o Problema dos k-Servos (PKS). O desempenho desta solução é comparado com o de outros algoritmos existentes na literatura, a saber, os algoritmos Harmonic e Work Function, que mostraram ser competitivos, tornando-os parâmetros de comparação significativos. Um algoritmo que apresente desempenho eficiente em relação aos mesmos tende a ser competitivo também, devendo, obviamente, se provar o referido fato. Tal prova, entretanto, foge aos objetivos do presente trabalho. O algoritmo apresentado para a solução do PKS é baseado em técnicas de aprendizagem por reforço. Para tanto, o problema foi modelado como um processo de decisão em múltiplas etapas, ao qual é aplicado o algoritmo Q-Learning, um dos métodos de solução mais populares para o estabelecimento de políticas ótimas neste tipo de problema de decisão. Entretanto, deve-se observar que a dimensão da estrutura de armazenamento utilizada pela aprendizagem por reforço para se obter a política ótima cresce em função do número de estados e de ações, que por sua vez é proporcional ao número n de nós e k de servos. Ao se analisar esse crescimento (matematicamente, ) percebe-se que o mesmo ocorre de maneira exponencial, limitando a aplicação do método a problemas de menor porte, onde o número de nós e de servos é reduzido. Este problema, denominado maldição da dimensionalidade, foi introduzido por Belmann e implica na impossibilidade de execução de um algoritmo para certas instâncias de um problema pelo esgotamento de recursos computacionais para obtenção de sua saída. De modo a evitar que a solução proposta, baseada exclusivamente na aprendizagem por reforço, seja restrita a aplicações de menor porte, propõe-se uma solução alternativa para problemas mais realistas, que envolvam um número maior de nós e de servos. Esta solução alternativa é hierarquizada e utiliza dois métodos de solução do PKS: a aprendizagem por reforço, aplicada a um número reduzido de nós obtidos a partir de um processo de agregação, e um método guloso, aplicado aos subconjuntos de nós resultantes do processo de agregação, onde o critério de escolha do agendamento dos servos é baseado na menor distância ao local de demanda

Classificação de padrões através de um comitê de máquinas aprimorado por aprendizagem por reforço

Reinforcement learning is a machine learning technique that, although finding a large number of applications, maybe is yet to reach its full potential. One of the inadequately tested possibilities is the use of reinforcement learning in combination with other methods for the solution of pattern classification problems. It is well documented in the literature the problems that support vector machine ensembles face in terms of generalization capacity. Algorithms such as Adaboost do not deal appropriately with the imbalances that arise in those situations. Several alternatives have been proposed, with varying degrees of success. This dissertation presents a new approach to building committees of support vector machines. The presented algorithm combines Adaboost algorithm with a layer of reinforcement learning to adjust committee parameters in order to avoid that imbalances on the committee components affect the generalization performance of the final hypothesis. Comparisons were made with ensembles using and not using the reinforcement learning layer, testing benchmark data sets widely known in area of pattern classification

Roteamento em Redes de Sensores Sem Fios Com Base Em Aprendizagem Por Reforço

The use of wireless sensor and actuator networks in industry has been increasing past few years, bringing multiple benefits compared to wired systems, like network flexibility and manageability. Such networks consists of a possibly large number of small and autonomous sensor and actuator devices with wireless communication capabilities. The data collected by sensors are sent directly or through intermediary nodes along the network to a base station called sink node. The data routing in this environment is an essential matter since it is strictly bounded to the energy efficiency, thus the network lifetime. This work investigates the application of a routing technique based on Reinforcement Learning s Q-Learning algorithm to a wireless sensor network by using an NS-2 simulated environment. Several metrics like energy consumption, data packet delivery rates and delays are used to validate de proposal comparing it with another solutions existing in the literature

Equalização neural aplicada a sistemas com modulação bidimensional em fibra óptica

Nowadays, optic fiber is one of the most used communication methods, mainly due to the fact that the data transmission rates of those systems exceed all of the other means of digital communication. Despite the great advantage, there are problems that prevent full utilization of the optical channel: by increasing the transmission speed and the distances involved, the data is subjected to non-linear inter symbolic interference caused by the dispersion phenomena in the fiber. Adaptive equalizers can be used to solve this problem, they compensate non-ideal responses of the channel in order to restore the signal that was transmitted. This work proposes an equalizer based on artificial neural networks and evaluates its performance in optical communication systems. The proposal is validated through a simulated optic channel and the comparison with other adaptive equalization techniques

Compósitos híbridos: desenvolvimento de configuração e efeitos de umidificação

The advantage in using vegetable fibres in place of synthetic fibres such as glass fibre, for reinforcements in composites are: biodegradability, low cost, low density, good tenacity, good thermal properties, low energy content and reduced use of instruments for its treatment or processing. Even though, problems related to low mechanical performance of some of the natural fibres, has caused difficulty in their direct application in structural elements. The use of alternative materials like hybrid composites has been encouraged, thus trying to better the structural performance of the composites with natural fibres. This work presents a comparative study of the strength and stiffness of hybrid composites with orthopthalic polyester matrix reinforced with E-fibre glass, jute and curauá. The experimental part includes uniaxial tension and three point bending tests to determine the mechanical properties of the final product. The hybrid composite was manufactured in a local industry and was in the form of laminates. All the samples were projected to withstand the possible structural applications as reservoirs and pipes. CH (laminated hybrid composite with glass and curauá fibres). The results obtained show clearly the influence of the hybridization in all the types tested and indicate a good mechanical performance of the composite with glass/curauá fibres in relation to the composite with glass fibres. Aspects in relation to the interfaces glass/curauá composites besides the fracture characteristics for all loading types were also analysed

Efeitos do envelhecimento ambiental acelerado em compósitos poliméricos

The advantages of the use of vegetable fibers on the synthetic fibers, such as glass fibers, in the reinforcements in composites are: low cost, low density, good tenacity, good thermal properties and reduced use of instruments for their treatment or processing. However, problems related to poor performance of some mechanical natural fibers, have hindered its direct use in structural elements. In this sense, the emergence of alternative materials such as hybrids composites, involving natural and synthetic fibers, has been encouraged by seeking to improve the performance of structural composites based only on natural fibers. The differences between the physical, chemical and mechanical properties of these fibers, especially facing the adverse environmental conditions such as the presence of moisture and ultraviolet radiation, is also becoming a concern in the final response of these composites. This piece of research presents a comparative study of the strength and stiffness between two composite, both of ortoftalic polyester matrix, one reinforced with fibers of glass-E (CV) and other hybrid reinforced with natural fibers of curauá and fiberglass-E (CH). All the comparative study is based on the influence of exposure to UV rays and steam heated water in composites, simulating the aging environment. The conditions for the tests are accelerated through the use of the aging chamber. The composites will be evaluated through tests of uniaxial static mechanical traction and bending on three points. The composite of glass fiber and hybrid manufacturing industry are using the rolling manual (hand lay-up) and have been developed in the form of composites. All were designed to meet possible structural applications such as tanks and pipes. The reinforcements used in composites were in the forms of short fiber glass-E quilts (450g/m2 - 5cm) of continuous wires and fuses (whose title was of 0.9 dtex) for the curauá fibers. The results clearly show the influence of aging on the environmental mechanical performance of the composite CV and CH. The issues concerning the final characteristics of the fracture for all types of cargoes studied were also analyzed

Capacidade de absorção de energia de tubos de compósitos submetidos à compressão

Structures capable of absorbing large amounts of energy are of great interest, particularly for the automotive and aviation industries, to reduce tbe impact on passengers in the case of a collision. The energy absorption properties of composite materials structures can be tailored, thus making these structures an appealing option a substitute of more traditional structures in applications where energy absorption is crucial. ln this research, the influence of some parameters, which affect the energy absorption capacity of composite material tubes, was investigated. The tubes were fabricated by hand lay-up, using orthophthalic polyester resin and a plain weave E-glass fabric Test specimens were prepared and tested under compression load. The ínfluence of the following parameters on the specific energy absorption capacity of the tubes was studied: fiber configuration (0/90º or ± 45°), tube cross-section (circular or square), and processing conditions (with or without vacuum). The results indicated that circular cross-section tubes with fibers oriented at 0/90º presented the highest level of specific energy absorbed. Further, specimens from tubes fabricated under vacuum displayed higher energy absorption capacity, when compared with specimens from tubes fabricated without vacuum. Thus, it can be concluded that the fabrication process with vacuum produce composite structures with better energy absorption capacity

Análise do desempenho térmico de sistema constutivo de concreto com EPS como agregado graúdo

In the last decades there was a concentrate effort of researchers in the search for options to the problem of the continuity of city development and environmental preservation. The recycling and reuse of materials in industry have been considerate as the best option to sustainable development. One of the relevant aspects in this case refers to the rational use of electrical energy. At this point, the role of engineering is to conceive new processes and materials, with the objective of reducing energy consumption and maintaining, at the same time the benefits of the technology. In this context, the objective of the present research is to analyze quantitatively the thermal behavior of walls constructed with concrete blocks which composition aggregates the expanded polystyrene (EPS) reused in the shape of flakes and in the shape of a board, resulting in a “light concrete”. Experiments were conducted, systematically, with a wall (considerate as a standard) constructed with blocks of ordinary concrete; two walls constructed with blocks of light concrete, distinct by the proportion of EPS/sand; a wall of ceramic bricks (“eight holes” type) and a wall with ordinary blocks of cement, in a way to obtain a comparative analysis of the thermal behavior of the systems. Others tests conducted with the blocks were: stress analysis and thermal properties analysis (ρ, cp e k). Based on the results, it was possible to establish quantitative relationship between the concentration (density) of EPS in the constructive elements and the decreasing of the heat transfer rate, that also changes the others thermal properties of the material, as was proved. It was observed that the walls of light concrete presents better thermal behavior compared with the other two constructive systems world wide used. Based in the results of the investigation, there was shown the viability of the use of EPS as aggregate (raw material) in the composition of the concrete, with the objective of the fabrication of blocks to non-structural masonry that works as a thermal insulation in buildings. A direct consequence of this result is the possibility of reduction of the consume of the electrical energy used to climatization of buildings. Other aspect of the investigation that must be pointed was the reuse of the EPS as a raw material to civil construction, with a clear benefit to reducing of environmental problems

Desenvolvimento e caracterização de nanocompósito de resina epóxi com nanopartículas de sílica para revestimento de dutos para transporte de petróleo

The use of polymer based coatings is a promising approach to reduce the corrosion problem in carbon steel pipes used for the transport of oil and gas in the oil industry. However, conventional polymer coatings offer limited properties, which often cannot meet design requirements for this type of application, particularly in regard to use temperature and wear resistance. Polymer nanocomposites are known to exhibit superior properties and, therefore, offer great potential for this type of application. Nevertheless, the degree of enhancement of a particular property is greatly dependent upon the matrix/nanoparticle material system used, the matrix/nanoparticle interfacial bonding and also the state of dispersion of the nanoparticle in the polymer matrix. The objective of the present research is to develop and characterize polymer based nanocomposites to be used as coatings in metallic pipelines for the transportation of oil and natural gas. Epoxy/SiO2 nanocomposites with nanoparticle contents of 2, 4, and 8 wt % were processed using a high-energy mill. Modifications of the SiO2 nanoparticles‟ surfaces with two different silane agents were carried out and their effect on the material properties were investigated. The state of dispersion of the materials processed was studied using Scanning and Transmission Electron Microscopy (SEM and TEM) micrographs. Thermogravimetric analysis (TG) were also conducted to determine the thermal stability of the nanocomposites. In addition, the processed nanocomposites were characterized by dynamic mechanical analysis (DMA) to investigate the effect of nanoparticles content and silane treatment on the viscoelastic properties and on the glass transition temperature. Finally, wear tests of the pin-on-disc type were carried out to determine the effects of the nanoparticles and the silane treatments studied. According to the results, the addition of SiO2 nanoparticles treated with silane increased the thermal stability, the storage modulus and Tg of the epoxy resin and decreased wear rate. This confirms that the interaction between the nanoparticles and the polymer chains plays a critical role on the properties of the nanocomposites

Análise de viabilidade técnica de utilização da Fibra de bananeira com resina sintética em Compósitos

This paper aims to present the feasibility of using a composite using discarded material from the cultivation of banana tree (pseudostem), which is fibrillated together with synthetic resin replacing glass fiber to be used in structural elements that do not demand large mechanical stress such as reservoirs, troughs, domes, sewage pipes etc.. For this, there were studies about the mechanical properties of a composite made with polyester resin and fiber of banana tree (Musa sp, musac), in which the splints were removed from the pseudostem, being made fibrillation by hand, with the aid of a brush steel, followed by natural drying. After treatment for cleaning and removal of wax, the fiber was cut into pieces of approximately 60 mm to 100 mm, for, together with synthetic resin, make cards of a features fiber composite with random orientation relative to the weight of the resin. We used three different percentages of fiber (3%, 6% and 9%), in order to make a comparative study between them and what would be the one with the best performance. Were manufactured specimens of each material and then subjected to uniaxial tensile tests, three point bending, moisture absorption and thermal characteristics. The results show that, in general, the use of banana tree fiber is feasible simply by an improvement in the production process (machining of the procedure) and greater care in the manufacture of parts

Desenvolvimento de um laminado compósito híbrido a base de fibras de vidro/licurí

This research work is based, in search of reinforcement s vegetable alternative to polymer composites. The idealization of making a hybrid composite reinforced with vegetable fibers licuri with synthetic fibers is a pioneer in this area. Thus was conceived a hybrid composite laminate consisting of 05 (five) layers being 03 (three) webs of synthetic fibers of glass and E-02 (two) unidirectional fabrics of vegetable fibers licuri. In the configuration of the laminate layers have alternating distribution. The composite laminate was manufactured in Tecniplas Commerce & Industry LTD, in the form of a card through the manufacturing process of hand lay up. Licuri fibers used in making the foil were the City of Mare Island in the state of Bahia. After cooking and the idealization of the hybrid composite laminate, the objective of this research work has focused on evaluating the performance of the mechanical properties (ultimate strength, stiffness and elongation at break) through uniaxial tensile tests and three point bending. Comparative studies of the mechanical properties and as well as among other types of laminated hybrid composites studied previously, were performed. Promising results were found with respect to the mechanical properties of strength and stiffness to the hybridization process idealized here. To complement the entire study were analyzed in terms of macroscopic and microscopic characteristics of the fracture for all tests.

Avaliação do comportamento tribológico de compósitos poliéster/fibra de vidro utilizando resíduos de poliéster/fibra de vidro e carbonato de cálcio como carga

The use of composite materials and alternative is being increased every day, as it becomes more widespread awareness that the use of renewable and not harmful to the environment is part of a new environmentally friendly model. Since its waste (primarily fiberglass) can not be easily recycled by the difficulty that still exists in this process, since they have two phases mixed, a polymeric matrix thermoset difficult to recycle because it is infusible and phase of fiber reinforcements. Thermoset matrix composites like Polyester + fiberglass pose a threat due to excessive discharge. Aiming to minimize this problem, aimed to reuse the composite Polyester + fiber glass, through the wastes obtained by the grinding of knifes and balls. These residues were incorporated into the new composite Polyester/Fiberglass for hot compression mold and compared tribological to composites with filler CaCO3, generally used as filler, targeting a partial replacement of CaCO3 by such waste. The composites were characterized by thermal analysis (TGA, DSC and DMA), by the surface integrity (roughness determination, contact angle and surface energy), mechanical properties (hardness) and tribological tests (wear and coefficient of dynamic friction) in order to evaluate the effect of loads and characterize these materials for applications that can take, in the tribological point of view since waste Polyester + fiberglass has great potential for replacement of CaCO3

Estudo da flamabilidade e resistência à chama de compósito de poliéster insaturado e fibra da folha do abacaxizeiro (PALF)

In the present research work, composites were prepared using pine apple leaf fibres (PALF) as reinforcement with unsaturated polyester resin as matrix, incorporating with fire retardant at different compositions. The PALF was obtained from the decortication of pine apple leaves obtained from Ramada 4 from Ielmo Marinho in the State of Rio Grande do Norte. The unsaturated polyester resin and the catalyzer were bought from the local establishment. The fire retardant, aluminium tri-hydroxide - Al(OH)3 was donated by Alcoa Alumínio S.A and was used in the proportions of 20%, 40% and 60% w/w. Initially the fibres were treated with 2% NaOH for 1 hour, to remove any impurities present on the fibre surface, such as wax, fat, pectin and pectate, in order to have a better adsorption of the fibres with the matrix as well as the flame retardant. The fibre mat was prepared in a mat preparator by immersion, developed in the Textile Engineering Laboratory, at the UFRN. The composites (300x300x3 mm) were prepared by compression molding and the samples (150x25x3 mm) for analysis of the properties were cut randomly using a laser cutter. Some of the cut samples were used to measure the smoke emission and fire resistance using UL94 standard. Mechanical tension-extension and flexural properties were carried in CTGás RN and the Laboratório de Metais e Ensaios Mecânicos Engenharia de Materiais UFRN , as well as SEM studies were carried out at Núcleo de Estudos em Petróleo e Gás Natural - UFRN . From the observed results, it was noted that, there was no marked influence of the fire retardant on the mechanical properties. Also in the water absorption test, the quantity of water absorbed was less in the sample with higher concentration of fire retardant. It was also observed that the increase in the proportion of the fire retardant increased the time of burning, may be due to the compactness of the composite due to the presence of fire retardant as a filling material even though it was meant to reduce the rate of inflammability of the composite

Desenvolvimento de uma desfribadeira para obtenção da fibra da folha do abacaxi

In 1998 the first decorticator was developed in the Textile Engineering Laboratory and patented for the purpose of extracting fibres from pineapple leaves, with the financial help from CNPq and BNB. The objective of the present work was to develop an automatic decorticator different from the first one with a semiautomatic system of decortication with automatic feeding of the leaves and collection of the extracted fibres. The system is started through a command system that passes information to two engines, one for starting the beater cylinder and the other for the feeding of the leaves as well as the extraction of the decorticated fibres automatically. This in turn introduces the leaves between a knife and a beater cylinder with twenty blades (the previous one had only 8 blades). These blades are supported by equidistant flanges with a central transmission axis that would help in increasing the number of beatings of the leaves. In the present system the operator has to place the leaves on the rotating endless feeding belt and collect the extracted leaves that are being carried out through another endless belt. The pulp resulted form the extraction is collected in a tray through a collector. The feeding of the leaves as well as the extraction of the fibres is controlled automatically by varying the velocity of the cylinders. The semi-automatic decorticator basically composed of a chassis made out of iron bars (profile L) with 200cm length, 91 cm of height 68 cm of width. The decorticator weighs around 300Kg. It was observed that the increase in the number of blades from 8 to twenty in the beater cylinder reduced the turbulence inside the decorticator, which helped to improve the removal of the fibres without any problems as well as the quality of the fibres. From the studies carried out, from each leaf 2,8 to 4,5% of fibres can be extracted. This gives around 4 to 5 tons of fibres per hectare, which is more than that of cotton production per hectare. This quantity with no doubt could generate jobs to the people not only on the production of the fibres but also on their application in different areas

Desenvolvimento e análise de material compósito a base de penas de frango (fibras de queratina-KF) e matriz de poliéster insaturado

Use of natural fibres as a reinforcement material in the manufacture of composites show a series of advantages: availability, biodegradability, low weight and regeneration in relation to synthetic fibres, thus justifying its utilization. In the present research work, composites were developed with chicken feathers (KF), using unsaturated polyester resin as matrix, for diversified applications, mainly in the furniture/timber industry.At present, in Brazil the chicken feathers are used as part of the animal feed, even though this material possesses low aggregated value. The chicken feathers are hollow, light and resistant. After washing with water at room temperature, a part of the chicken feathers were treated with 2% NaOH. Composites were manufactured using treated and untreated chicken feathers with unsaturated orthothalic polyester resin and 1% peroxide as catalyser, obtained in the commerce. Samples with size 150x25x3 mm for mechanical tests were cut by laser in the composite plate. Mechanical analyses were carried out in the Laboratório de Metais e Ensaios Mecânicos UFRN. All the analyses were in accordance with ASTM standards. SEM analyses were also carried out on the samples.In the analyses of the results obtained, it was observed that the composites made with untreated chicken feathers showed better results (Traction 11.406 MPa and 9.107 MPa Bending 34.947 and 20.918 MPa for samples with and without treatment respectively) compared to the composite with treated feathers. Very low values of the water absorption results, evidenced the impermeability characteristic of the feathers. From the SEM images, the structure, fracture and the fibre/matrix adsorption can be evidenced. In the flammability test, it was observed that despite the feathers having sulfur as a constituent, natural inhibitor of flame, no burning support of the composites, because the manufacturing process of the composite