Experience generalization for multi-agent reinforcement learning

On-line learning methods have been applied successfully in multi-agent systems to achieve coordination among agents. Learning in multi-agent systems implies in a non-stationary scenario perceived by the agents, since the behavior of other agents may change as they simultaneously learn how to improve their actions. Non-stationary scenarios can be modeled as Markov Games, which can be solved using the Minimax-Q algorithm a combination of Q-learning (a Reinforcement Learning (RL) algorithm which directly learns an optimal control policy) and the Minimax algorithm. However, finding optimal control policies using any RL algorithm (Q-learning and Minimax-Q included) can be very time consuming. Trying to improve the learning time of Q-learning, we considered the QS-algorithm. in which a single experience can update more than a single action value by using a spreading function. In this paper, we contribute a Minimax-QS algorithm which combines the Minimax-Q algorithm and the QS-algorithm. We conduct a series of empirical evaluation of the algorithm in a simplified simulator of the soccer domain. We show that even using a very simple domain-dependent spreading function, the performance of the learning algorithm can be improved.

Multi-Agent Simulation of Urban Social Dynamics for Spatial Load Forecasting

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Multi-Agent Framework for Spatial Load Forecasting

A multi-agent framework for spatial electric load forecasting, especially suited to simulate the different dynamics involved on distribution systems, is presented. The service zone is divided into several sub-zones, each subzone is considered as an independent agent identified with a corresponding load level, and their relationships with the neighbor zones are represented as development probabilities. With this setting, different kind of agents can be developed to simulate the growth pattern of the loads in distribution systems. This paper presents two different kinds of agents to simulate different situations, presenting some promissory results.

A multi-agent architecture for intelligent data analysis

The present study introduces a multi-agent architecture designed for doing automation process of data integration and intelligent data analysis. Different from other approaches the multi-agent architecture was designed using a multi-agent based methodology. Tropos, an agent based methodology was used for design. Based on the proposed architecture, we describe a Web based application where the agents are responsible to analyse petroleum well drilling data to identify possible abnormalities occurrence. The intelligent data analysis methods used was the Neural Network.

A multi-agent system with a percolation approach to simulate the driving pattern of Plug-In ELectric Vehicles

A multi-agent system with a percolation approach to simulate the driving pattern of Plug-In Electric Vehicle (PEV), especially suited to simulate the PEVs behavior on any distribution systems, is presented. This tool intends to complement information about the driving patterns database on systems where that kind of information is not available. So, this paper aims to provide a framework that is able to work with any kind of technology and load generated of PEVs. The service zone is divided into several sub-zones, each subzone is considered as an independent agent identified with corresponding load level, and their relationships with the neighboring zones are represented as network probabilities. A percolation approach is used to characterize the autonomy of the battery of the PVEs to move through the city. The methodology is tested with data from a mid-size city real distribution system. The result shows the sub-area where the battery of PEVs will need to be recharge and gives the planners of distribution systems the necessary input for a medium to long term network planning in a smart grid environment. © 2012 IEEE.

Energy Management and Control for Islanded Microgrid Using Multi-Agents

This paper presents a multi-agent system for real-time operation of simulated microgrid using the Smart-Grid Test Bed at Washington State University. The multi-agent system (MAS) was developed in JADE (Java Agent DEvelopment Framework) which is a Foundation for Intelligent Physical Agents (FIPA) compliant open source multi-agent platform. The proposed operational strategy is mainly focused on using an appropriate energy management and control strategies to improve the operation of an islanded microgrid, formed by photovoltaic (PV) solar energy, batteries and resistive and rotating machines loads. The focus is on resource management and to avoid impact on loads from abrupt variations or interruption that changes the operating conditions. The management and control of the PV system is performed in JADE, while the microgrid model is simulated in RSCAD/RTDS (Real-Time Digital Simulator). Finally, the outcome of simulation studies demonstrated the feasibility of the proposed multi-agent approach for real-time operation of a microgrid.

A multiagent architecture for supervisory and control system

Supervising and controlling the many processes involved in petroleum production is both dangerous and complex. Herein, we propose a multiagent supervisory and control system for handle continuous processes like those in chemical and petroleum industries In its architeture, there are agents responsible for managing data production and analysis, and also the production equipments. Fuzzy controllers were used as control agents. The application of a fuzzy control system to managing an off-shore installation for petroleum production onto a submarine separation process is described. © 2008 IEEE.

Spatial electric load forecasting using a local movement approach

An agent based model for spatial electric load forecasting using a local movement approach for the spatiotemporal allocation of the new loads in the service zone is presented. The density of electrical load for each of the major consumer classes in each sub-zone is used as the current state of the agents. The spatial growth is simulated with a walking agent who starts his path in one of the activity centers of the city and goes to the limits of the city following a radial path depending on the different load levels. A series of update rules are established to simulate the S growth behavior and the complementarity between classes. The results are presented in future load density maps. The tests in a real system from a mid-size city show a high rate of success when compared with other techniques. The most important features of this methodology are the need for few data and the simplicity of the algorithm, allowing for future scalability. © 2009 IEEE.

Spatial load forecasting using a demand propagation approach

A method for spatial electric load forecasting using multi-agent systems, especially suited to simulate the local effect of special loads in distribution systems is presented. The method based on multi-agent systems uses two kinds of agents: reactive and proactive. The reactive agents represent each sub-zone in the service zone, characterizing each one with their corresponding load level, represented in a real number, and their relationships with other sub-zones represented in development probabilities. The proactive agent carry the new load expected to be allocated because of the new special load, this agent distribute the new load in a propagation pattern. The results are presented with maps of future expected load levels in the service zone. The method is tested with data from a mid-size city real distribution system, simulating the effect of a load with attraction and repulsion attributes. The method presents good results and performance. © 2011 IEEE.

Considering Urban Dynamics in spatial electric load forecasting

When dealing with spatio-temporal simulations of load growth inside a service zone, one of the most important problems faced by a Distribution Utility is how to represent the different relationships among different areas. A new load in a certain part of the city could modify the load growth in other parts of the city, even outside of its radius of influence. These interactions are called Urban Dynamics. This work aims to discuss how to implement Urban Dynamics considerations into the spatial electric load forecasting simulations using multi-agent simulations. To explain the approach, three examples are introduced, including the effect of an attraction load, the effect of a repulsive load, and the effect of several attraction/repulsive loads at the same time when considering the natural load growth. © 2012 IEEE.

Prognostic factors associated with time to hCG remission in patients with low-risk postmolar gestational trophoblastic neoplasia

Objective The purpose of this study was to identify the clinical factors associated with time to hCG remission among women with low-risk postmolar GTN. Methods This study included a non-concurrent cohort of 328 patients diagnosed with low-risk postmolar GTN according to FIGO 2002 criteria. Associations of time to hCG remission with history of prior mole, molar histology, time to persistence, use of D&C at persistence, presence of metastatic disease, FIGO score, hCG values at persistence, type of first line therapy and use of multiagent chemotherapy were investigated with both univariate and multivariate analyses. Results Overall median time to remission was 46 days. Ten percent of the patients required multi-agent chemotherapy to achieve hCG remission. Multivariate analysis incorporating the variables significant on univariate analysis confirmed that complete molar histology (HR 1.45), metastatic disease (HR 1.66), use of multi-agent therapy (HR 2.00) and FIGO score (HR 1.82) were associated with longer time to remission. There was a linear relationship between FIGO score and time to hCG remission. Each 1-point increment in FIGO score was associated with an average 17-day increase in hCG remission time (95% CI: 12.5-21.6). Conclusions Complete mole histology prior to GTN, presence of metastatic disease, use of multi-agent therapy and higher FIGO score were independent factors associated with longer time to hCG remission in low-risk GTN. Identifying the prognostic factors associated with time to remission and effective counseling may help improve treatment planning and reduce anxiety in patients and their families. © 2013 Elsevier Inc. All rights reserved.

Aplicações de sistemas multiagentes na previsão espacial de demanda elétrica em sistemas de distribuição

Pós-graduação em Engenharia Elétrica - FEIS

Carregamento ótimo de veículos elétricos considerando as restrições das redes de distribuição de média tensão

Pós-graduação em Engenharia Elétrica - FEIS

Comportamento do pedestre e ambiente térmico urbano

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Uma arquitetura multiagentes para sistema educacional baseado na web semântica

Pós-graduação em Ciência da Computação - IBILCE