Aprendizado por Reforço com Valores deInfluência em Sistemas Multi-Agente

We propose a new paradigm for collective learning in multi-agent systems (MAS) as a solution to the problem in which several agents acting over the same environment must learn how to perform tasks, simultaneously, based on feedbacks given by each one of the other agents. We introduce the proposed paradigm in the form of a reinforcement learning algorithm, nominating it as reinforcement learning with influence values. While learning by rewards, each agent evaluates the relation between the current state and/or action executed at this state (actual believe) together with the reward obtained after all agents that are interacting perform their actions. The reward is a result of the interference of others. The agent considers the opinions of all its colleagues in order to attempt to change the values of its states and/or actions. The idea is that the system, as a whole, must reach an equilibrium, where all agents get satisfied with the obtained results. This means that the values of the state/actions pairs match the reward obtained by each agent. This dynamical way of setting the values for states and/or actions makes this new reinforcement learning paradigm the first to include, naturally, the fact that the presence of other agents in the environment turns it a dynamical model. As a direct result, we implicitly include the internal state, the actions and the rewards obtained by all the other agents in the internal state of each agent. This makes our proposal the first complete solution to the conceptual problem that rises when applying reinforcement learning in multi-agent systems, which is caused by the difference existent between the environment and agent models. With basis on the proposed model, we create the IVQ-learning algorithm that is exhaustive tested in repetitive games with two, three and four agents and in stochastic games that need cooperation and in games that need collaboration. This algorithm shows to be a good option for obtaining solutions that guarantee convergence to the Nash optimum equilibrium in cooperative problems. Experiments performed clear shows that the proposed paradigm is theoretical and experimentally superior to the traditional approaches. Yet, with the creation of this new paradigm the set of reinforcement learning applications in MAS grows up. That is, besides the possibility of applying the algorithm in traditional learning problems in MAS, as for example coordination of tasks in multi-robot systems, it is possible to apply reinforcement learning in problems that are essentially collaborative

Modelagem e controle de um microveículo aéreo: uma aplicação de estabilidade robusta com a técnica backstepping em uma estrutura hexarrotor

In this Thesis, the development of the dynamic model of multirotor unmanned aerial vehicle with vertical takeoff and landing characteristics, considering input nonlinearities and a full state robust backstepping controller are presented. The dynamic model is expressed using the Newton-Euler laws, aiming to obtain a better mathematical representation of the mechanical system for system analysis and control design, not only when it is hovering, but also when it is taking-off, or landing, or flying to perform a task. The input nonlinearities are the deadzone and saturation, where the gravitational effect and the inherent physical constrains of the rotors are related and addressed. The experimental multirotor aerial vehicle is equipped with an inertial measurement unit and a sonar sensor, which appropriately provides measurements of attitude and altitude. A real-time attitude estimation scheme based on the extended Kalman filter using quaternions was developed. Then, for robustness analysis, sensors were modeled as the ideal value with addition of an unknown bias and unknown white noise. The bounded robust attitude/altitude controller were derived based on globally uniformly practically asymptotically stable for real systems, that remains globally uniformly asymptotically stable if and only if their solutions are globally uniformly bounded, dealing with convergence and stability into a ball of the state space with non-null radius, under some assumptions. The Lyapunov analysis technique was used to prove the stability of the closed-loop system, compute bounds on control gains and guaranteeing desired bounds on attitude dynamics tracking errors in the presence of measurement disturbances. The controller laws were tested in numerical simulations and in an experimental hexarotor, developed at the UFRN Robotics Laboratory

Motor de indução trifásico sem mancais com bobinado dividido: otimização do sistema de posicionamento radial

This dissertation dea1s with the active magnetic suspension controI system of an induction bearingIess motor configured with split windings. It analyses a dynamic modeI for the radial magnetic forces actuating on the rotor. From that, it proposes a new approach for the composition of the currents imposed to the machine's stator. It shows the tests accomplished with a prototype, proving the usefulness of the new actuating structure for the radial positioning controI. Finnaly, it points out the importance of adapting this structure to well-known rotational controI techniques, continuing this kind of equipment research, which is carried out at Federal University of Rio Grande do Norte since 2000

Identificação em tempo real de modelo dinâmico de robô móvel com acionamento diferencial e zona morta

Several mobile robots show non-linear behavior, mainly due friction phenomena between the mechanical parts of the robot or between the robot and the ground. Linear models are efficient in some cases, but it is necessary take the robot non-linearity in consideration when precise displacement and positioning are desired. In this work a parametric model identification procedure for a mobile robot with differential drive that considers the dead-zone in the robot actuators is proposed. The method consists in dividing the system into Hammerstein systems and then uses the key-term separation principle to present the input-output relations which shows the parameters from both linear and non-linear blocks. The parameters are then simultaneously estimated through a recursive least squares algorithm. The results shows that is possible to identify the dead-zone thresholds together with the linear parameters

Controle dinâmico de robôs móveis com acionamento diferencial

This work addresses the dynamic control problem of two-wheeled differentially driven non-holonomic mobile robot. Strategies for robot positioning control and robot orientating control are presented. Such strategies just require information about the robot con¯guration (x, y and teta), which can be collected by an absolute positioning system. The strategies development is related to a change on the controlled variables for such systems, from x, y and teta to s (denoting the robot linear displacement) and teta, and makes use of the polar coordinates representation for the robot kinematic model. Thus, it is possible to obtain a linear representation for the mobile robot dynamic model and to develop such strategies. It is also presented that such strategies allow the use of linear controllers to solve the control problem. It is shown that there is flexibility to choice the linear controller (P, PI, PID, Model Matching techniques, others) to be implemented. This work presents an introduction to mobile robotics and their characteristics followed by the control strategies development and controllers design. Finally, simulated and experimental results are presented and commented

Sistema Especialista para Supressão Online de Alarmes em Processos Industriais

Operating industrial processes is becoming more complex each day, and one of the factors that contribute to this growth in complexity is the integration of new technologies and smart solutions employed in the industry, such as the decision support systems. In this regard, this dissertation aims to develop a decision support system based on an computational tool called expert system. The main goal is to turn operation more reliable and secure while maximizing the amount of relevant information to each situation by using an expert system based on rules designed for a particular area of expertise. For the modeling of such rules has been proposed a high-level environment, which allows the creation and manipulation of rules in an easier way through visual programming. Despite its wide range of possible applications, this dissertation focuses only in the context of real-time filtering of alarms during the operation, properly validated in a case study based on a real scenario occurred in an industrial plant of an oil and gas refinery

Controle fuzzy espacialmente diferenciado para um sistema de irrigação

Traditional irrigation projects do not locally determine the water availability in the soil. Then, irregular irrigation cycles may occur: some with insufficient amount that leads to water deficit, other with excessive watering that causes lack of oxygen in plants. Due to the nonlinear nature of this problem and the multivariable context of irrigation processes, fuzzy logic is suggested to replace commercial ON-OFF irrigation system with predefined timing. Other limitation of commercial solutions is that irrigation processes either consider the different watering needs throughout plant growth cycles or the climate changes. In order to fulfill location based agricultural needs, it is indicated to monitor environmental data using wireless sensors connected to an intelligent control system. This is more evident in applications as precision agriculture. This work presents the theoretical and experimental development of a fuzzy system to implement a spatially differentiated control of an irrigation system, based on soil moisture measurement with wireless sensor nodes. The control system architecture is modular: a fuzzy supervisor determines the soil moisture set point of each sensor node area (according to the soil-plant set) and another fuzzy system, embedded in the sensor node, does the local control and actuates in the irrigation system. The fuzzy control system was simulated with SIMULINK® programming tool and was experimentally built embedded in mobile device SunSPOTTM operating in ZigBee. Controller models were designed and evaluated in different combinations of input variables and inference rules base

Método dinâmico aplicado para antenas cilíndricas

Nowadays there has been a major breakthrough in the aerospace area, with regard to rocket launches to research, experiments, telemetry system, remote sensing, radar system (tracking and monitoring), satellite communications system and insertion of satellites in orbit. This work aims at the application of a circular cylindrical microstrip antenna, ring type, and other cylindrical rectangular in structure of a rocket or missile to obtain telemetry data, operating in the range of 2 to 4 GHz, in S-band. Throughout this was developed just the theoretical analysis of the Transverse transmission line method which is a method of rigorous analysis in spectral domain, for use in rockets and missiles. This analyzes the spread in the direction "ρ" , transverse to dielectric interfaces "z" and "φ", for cylindrical coordinates, thus taking the general equations of electromagnetic fields in function of e [1]. It is worth mentioning that in order to obtain results, simulations and analysis of the structure under study was used HFSS program (High Frequency Structural Simulator) that uses the finite element method. With the theory developed computational resources were used to obtain the numerical calculations, using Fortran Power Station, Scilab and Wolfram Mathematica ®. The prototype was built using, as a substrate, the ULTRALAM ® 3850, of Rogers Corporation, and an aluminum plate as a cylindrical structure used to support. The agreement between the measured and simulated results validate the established processes. Conclusions and suggestions are presented for continuing this work

Análise do desempenho térmico de sistema constutivo de concreto com EPS como agregado graúdo

In the last decades there was a concentrate effort of researchers in the search for options to the problem of the continuity of city development and environmental preservation. The recycling and reuse of materials in industry have been considerate as the best option to sustainable development. One of the relevant aspects in this case refers to the rational use of electrical energy. At this point, the role of engineering is to conceive new processes and materials, with the objective of reducing energy consumption and maintaining, at the same time the benefits of the technology. In this context, the objective of the present research is to analyze quantitatively the thermal behavior of walls constructed with concrete blocks which composition aggregates the expanded polystyrene (EPS) reused in the shape of flakes and in the shape of a board, resulting in a “light concrete”. Experiments were conducted, systematically, with a wall (considerate as a standard) constructed with blocks of ordinary concrete; two walls constructed with blocks of light concrete, distinct by the proportion of EPS/sand; a wall of ceramic bricks (“eight holes” type) and a wall with ordinary blocks of cement, in a way to obtain a comparative analysis of the thermal behavior of the systems. Others tests conducted with the blocks were: stress analysis and thermal properties analysis (ρ, cp e k). Based on the results, it was possible to establish quantitative relationship between the concentration (density) of EPS in the constructive elements and the decreasing of the heat transfer rate, that also changes the others thermal properties of the material, as was proved. It was observed that the walls of light concrete presents better thermal behavior compared with the other two constructive systems world wide used. Based in the results of the investigation, there was shown the viability of the use of EPS as aggregate (raw material) in the composition of the concrete, with the objective of the fabrication of blocks to non-structural masonry that works as a thermal insulation in buildings. A direct consequence of this result is the possibility of reduction of the consume of the electrical energy used to climatization of buildings. Other aspect of the investigation that must be pointed was the reuse of the EPS as a raw material to civil construction, with a clear benefit to reducing of environmental problems

Modelagem dinâmica do escoamento de um sistema de elevação por plunger lift

A critical problem in mature gas wells is the liquid loading. As the reservoir pressure decreases, gas superficial velocities decreases and the drag exerted on the liquid phase may become insufficient to bring all the liquid to the surface. Liquid starts to drain downward, flooding the well and increasing the backpressure which decreases the gas superficial velocity and so on. A popular method to remedy this problem is the Plunger Lift. This method consists of dropping the "plunger"to the bottom of the tubing well with the main production valve closed. When the plunger reaches the well bottom the production valve is opened and the plunger carry the liquid to the surface. However, models presented in literature for predicting the behavior in plunger lift are simplistic, in many cases static (not considering the transient effects). Therefore work presents the development and validation of a numerical algorithm to solve one-dimensional compressible in gas wells using the Finite Volume Method and PRIME techniques for treating coupling of pressure and velocity fields. The code will be then used to develop a dynamic model for the plunger lift which includes the transient compressible flow within the well

Otimização de sistema de bombeamento com energia eólica: sistema de bombeamento de São Gabriel/BA

After the Protocol of Kyoto and of the ECHO 92 - Rio de Janeiro, the attentions of the world focus to the preservation of the environment and of the maintainable use of the natural resources. People were looking for preserving environment for the future generations. Ever since, solutions are looked for the supply of energy in its more acquaintances forms and the substitution of the use of fossil fuels for the such alternative forms as: Photovoltaics, solar heat systems for water, wind , bio-diesel, etc. and in this context the Company of Engineering of the State of Bahia - Cerb changed a diesel pumping system by an wind one, It´s the first community system of this nature in Bahia. Facing problems with the model, a Cerb involved the academic segment of the Federal Center of Technological Education of Bahia Cefetba looking for a solution. This work intends to demonstrate the possibilities of optimization of the pumping communit system that supply water to approximately 50 people in the place of Romão, municipal district of São Gabriel-Ba. Technical reports were published in AGRENERGD2004-Unicamp SP and Scientific Magazine ETC,Cefetba, 2005. A simulation of the increase of energy is presented for heights of 15 and 20m, considering the eletromecanical balance from the pumping energy to the wind turbine. From the accomplished bibliographical revision, we emphasized the mechanical aspects of the engineering once in UFRN, those studies concentrate on the Department of Mechanical Engineering while, in others eletroelectronic are more emphasized. Finally, documents that we judged important were enclosed for the perfect understanding of this work

Síntese e caracterização de tensoativos e estudo de equilíbrio de fase dos sistemas CnH2n+2/H2O/CnH2n+1COONa/But-OH

The growing utilization of surfactants in several different areas of industry has led to an increase on the studies involving solutions containing this type of molecules. Due to its amphiphilic nature, its molecule presents one polar part and one nonpolar end, which easily interacts with other molecules, being able to modify the media properties. When the concentration in which its monomers are saturated, the airliquid system interface is reached, causing a decrease in interfacial tension. The surfactants from pure fatty acids containing C8, C12 and C16 carbonic chains were synthesized in an alcoholic media using sodium hydroxide. They were characterized via thermal analysis (DTA and DTG) and via infrared spectroscopy, with the intention of observing their purity. Physical and chemical properties such as superficial tension, critical micelle concentration (c.m.c), surfactant excess on surface and Gibbs free energy of micellization were determined in order to understand the behaviour of these molecules with an aqueous media. Pseudo-ternary phase diagrams were obtained aiming to limit the Windsor equilibria conditions so it could be possible to understand how the surfactants carbonic chain size contributes to the microemulsion region. Solutions with known concentrations were prepared to study how the surfactants can influence the dynamic light scattering spectroscopy (DLS) and how the diffusion coefficient is influenced when the media concentration is altered. The results showed the variation on the chain size of the studied surfactant lipophilic part allows the conception of surfactants with similar interfacial properties, but dependent on the size of the lipophilic part of the surfactant. This variation causes the surfactant to have less tendency of microemulsionate oil in water. Another observed result is that the n-alcanes molecule size promoted a decrease on the microemulsion region on the obtained phase diagrams

Otimização de um sistema de distribuição predial de água fria: estudo de caso

This dissertation presents a methodology to the optimization of a predial system of cold water distribution. It s about a study of a case applied to the Tropical Buzios Residential Condominium, located in the Búzio s Beach, Nísia Floresta city, the east coast of the Rio Grande do Norte state, twenty kilometers far from Natal. The design of cold water distribution networks according to Norm NBR 5626 of the ABNT - Brazilian Association of Techniques Norms, does not guarantee that the joined solution is the optimal solution of less cost. It s necessary the use of an optimization methodology, that supplies us, between all the possible solutions, the minimum cost solution. In the optimization process of the predial system of water distribution of the Tropical Búzios Condominium, is used Method Granados, that is an iterative algorithm of optimization, based on the Dynamic Programming, that supplies the minimum cost s network, in function of the piezometric quota of the reservoir. For the application of this Method in ramifies networks, is used a program of computer in C language. This process is divided in two stages: attainment of the previous solution and reduction of the piezometric quota of headboard. In the attainment of the previous solution, the minors possible diameters are used that guarantee the limit of maximum speed and the requirements of minimum pressures. The piezometric quota of headboard is raised to guarantee these requirements. In the second stage of the Granados Method, an iterative process is used and it objective is to reduce the quota of headboard gradually, considering the substitution of stretches of the network pipes for the subsequent diameters, considering a minimum addition of the network cost. The diameter change is made in the optimal stretch that presents the lesser Exchange Gradient. The process is locked up when the headboard quota of desired is reached. The optimized network s material costs are calculated, and is made the analysis of the same ones, through the comparison with the conventional network s costs

Adsorção de Mn (II) e Zn (II) em soluções aquosas usando perlita expandida revestida com quitosana

In this work, chitosan was used as a coating of pure perlite in order to increase the accessibility of the groups OH- e NH2+the adsorptionof ions Mn2+ e Zn2+.The characterization results of the expanded perlite classified as microporous and whose surface area 3,176 m2 g-1after the change resulted in 4,664 m2g-1.From the thermogravimetry(TG) it was found that the percentage of coating was34,3%.The infrared analysis can prove the presence of groups Si-OH, Si-O e Al-O-Siresulting from the perlite and C=O, NH2and OH characterization of chitosan. The experiments on experiments on the adsorption of Mn and Zn were performed in the concentration range of10 a 50 mgL-1and the adsorption capacity inpH 5,8 e 5,2 was 19,49 and 23,09 mgg-1to 25 oC,respectively.The adsorption data were best fitted to Langmuir adsorption model to Langmuir adsorption model for both metalionsisindicative of monolayer adsorption. The kinetics of adsorption were calculated from the equation of Lagergren fitting the model pseudo-second-order for all initial concentrations, suggesting that adsorption of ions Mn2+ and Zn2+ follows the kinetics of pseudo-second-order and whose constant Speedk2(g/mg.min) are 0,105 e 3,98 and capacity and maximum removal qe 4,326 e 3,348,respectively.In this study we used a square wave voltammetry cathodic stripping voltammetry to quantify the adsorbed ions, and the working electrode glassy carbon, reference electrode silver / silver chloride and a platinum auxiliary electrode. The attainment of the peaks corresponding to ions Mn2+ and Zn2+ was evaluated in and electrochemical cell with a capacity of 30 mL using a buffer system (Na2HPO4/NaH2PO4)at pH 4 and was adjusted with solutionsH3PO4 0,1molL-1and NaOH 0,1 molL-1and addition of the analyte has been a cathodic peak in- 0,873 Vand detection limit of2,55x10-6molL-1para Zn.The dough used for obtaining the adsorption isotherm was 150 mg and reached in 120 min time of equilibrium for both metal ions.The maximum adsorption for 120 min with Mn concentration 20 mgL-1 and Zn 10 mgL-1,was91, 09 e 94, 34%, respectively

Sistema microemulsionado: caracterização e aplicação na indústria de petróleo

Alkyl polyethoxylates are surfactants widely used in vastly different fields, from oil exploitation to pharmaceutical applications. One of the most interesting characteristics of these surfactants is their ability to form micellar systems with specific geometry, the so-called wormlike micelle. In this work, microemulsions with three distinct compositions (C/T = 40 %, 30 % and 25 %) was used with contain UNITOL / butanol / water / xylene, cosurfactant / surfactante (C/S) ratio equal to 0,5. The microemulsion was characterized by dynamic light scattering (DLS), capillary viscometry, torque rheometry and surface tensiometry experiments carried out with systems based on xylene, water, butanol (cosurfactant) and nonaethyleneglycolmonododecyl ether (surfactant), with fixed surfactant:cosurfactant:oil composition (with and without oil phase) and varying the overall concentration of the microemulsion. The results showed that a transition from wormlike micelles to nanodrops was characterized by maximum relative viscosity (depending on how relative viscosity was defined), which was connected to maximum effective diameter, determined by DLS. Surface tension suggested that adsorption at the air water interface had a Langmuir character and that the limiting value of the surfactant surface excess was independent of the presence of cosurfactant and xylene. The results of the solubilization of oil sludge and oil recovery with the microemulsion: C/S = 40%, 30% and 25% proved to be quite effective in solubilization of oil sludge, with the percentage of solubilization (%solubilization) as high as 92.37% and enhanced oil recovery rates up to 90.22% for the point with the highest concentration of active material (surfactant), that is, 40%.