Experience generalization for multi-agent reinforcement learning

On-line learning methods have been applied successfully in multi-agent systems to achieve coordination among agents. Learning in multi-agent systems implies in a non-stationary scenario perceived by the agents, since the behavior of other agents may change as they simultaneously learn how to improve their actions. Non-stationary scenarios can be modeled as Markov Games, which can be solved using the Minimax-Q algorithm a combination of Q-learning (a Reinforcement Learning (RL) algorithm which directly learns an optimal control policy) and the Minimax algorithm. However, finding optimal control policies using any RL algorithm (Q-learning and Minimax-Q included) can be very time consuming. Trying to improve the learning time of Q-learning, we considered the QS-algorithm. in which a single experience can update more than a single action value by using a spreading function. In this paper, we contribute a Minimax-QS algorithm which combines the Minimax-Q algorithm and the QS-algorithm. We conduct a series of empirical evaluation of the algorithm in a simplified simulator of the soccer domain. We show that even using a very simple domain-dependent spreading function, the performance of the learning algorithm can be improved.

A multi-agent system with a percolation approach to simulate the driving pattern of Plug-In ELectric Vehicles

A multi-agent system with a percolation approach to simulate the driving pattern of Plug-In Electric Vehicle (PEV), especially suited to simulate the PEVs behavior on any distribution systems, is presented. This tool intends to complement information about the driving patterns database on systems where that kind of information is not available. So, this paper aims to provide a framework that is able to work with any kind of technology and load generated of PEVs. The service zone is divided into several sub-zones, each subzone is considered as an independent agent identified with corresponding load level, and their relationships with the neighboring zones are represented as network probabilities. A percolation approach is used to characterize the autonomy of the battery of the PVEs to move through the city. The methodology is tested with data from a mid-size city real distribution system. The result shows the sub-area where the battery of PEVs will need to be recharge and gives the planners of distribution systems the necessary input for a medium to long term network planning in a smart grid environment. © 2012 IEEE.

Multi-Agent Framework for Spatial Load Forecasting

A multi-agent framework for spatial electric load forecasting, especially suited to simulate the different dynamics involved on distribution systems, is presented. The service zone is divided into several sub-zones, each subzone is considered as an independent agent identified with a corresponding load level, and their relationships with the neighbor zones are represented as development probabilities. With this setting, different kind of agents can be developed to simulate the growth pattern of the loads in distribution systems. This paper presents two different kinds of agents to simulate different situations, presenting some promissory results.

Multi-Agent Simulation of Urban Social Dynamics for Spatial Load Forecasting

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

A framework for learning in humanoid simulated robots

One of the most important characteristics of intelligent activity is the ability to change behaviour according to many forms of feedback. Through learning an agent can interact with its environment to improve its performance over time. However, most of the techniques known that involves learning are time expensive, i.e., once the agent is supposed to learn over time by experimentation, the task has to be executed many times. Hence, high fidelity simulators can save a lot of time. In this context, this paper describes the framework designed to allow a team of real RoboNova-I humanoids robots to be simulated under USARSim environment. Details about the complete process of modeling and programming the robot are given, as well as the learning methodology proposed to improve robot's performance. Due to the use of a high fidelity model, the learning algorithms can be widely explored in simulation before adapted to real robots. © 2008 Springer-Verlag Berlin Heidelberg.

Robust fault diagnosis in power distribution systems based on fuzzy ARTMAP neural network-aided evidence theory

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Determination of the antimutagenicity of an aqueous extract of Rhizophora mangle L. (Rhizophoraceae), using in vivo and in vitro test systems

An aqueous extract of Rhizophora mangle L. bark is used as raw material in pottery making in the State of Espirito Santo, Brazil. This extract presents large quantities of tannins, compounds possessing antioxidant properties. Tannin antioxidant activity, as a plant chemical defense mechanism in the process of stabilizing free radicals, has been an incentive to studies on anti-mutagenicity. The present work aimed to evaluate possible antimutagenic activity of a R. mangle aqueous extract, using the Allium cepa test-system and micronuclear (MN) assay with blockage of cytokinesis in Chinese hamster ovary cells (CHO-K1). The Allium cepa test-system indicated antimutagenic activity against the damage induced by the mutagenic agent methyl methanesulfonate. A reduction in both MN cell frequency and chromosome breaks occurred in both the pre and post-treatment protocols. The MN testing of CHO-K1 cells revealed anti-mutagenic activity of the R. mangle extract against methyl methanesulfonate and doxorubicin in pre, simultaneous and post-treatment protocols. These results suggest the presence of phyto-constituents in the extract presenting demutagenic and bio-antimutagenic activities. Since the chemical constitution of Rhizophora mangle species presents elevated tannin content, it is highly probable that these compounds are the antimutagenic promoters themselves.

High current-density anodic electro-dissolution in flow-injection systems for the determination of aluminium, copper and zinc in non-ferroalloys by flame atomic absorption spectrometry

An automatic Procedure with a high current-density anodic electrodissolution unit (HDAE) is proposed for the determination of aluminium, copper and zinc in non-ferroalloys by flame atonic absorption spectrometry, based on the direct solid analysis. It consists of solenoid valve-based commutation in a flow-injection system for on-line sample electro-dissolution and calibration with one multi-element standard, an electrolytic cell equipped with two electrodes (a silver needle acts as cathode, and sample as anode), and an intelligent unit. The latter is assembled in a PC-compatible microcomputer for instrument control, and far data acquisition and processing. General management of the process is achieved by use of software written in Pascal. Electrolyte compositions, flow rates, commutation times, applied current and electrolysis time mere investigated. A 0.5 mol l(-1) HNO3 solution was elected as electrolyte and 300 A/cm(2) as the continuous current pulse. The performance of the proposed system was evaluated by analysing aluminium in Al-allay samples, and copper/zinc in brass and bronze samples, respectively. The system handles about 50 samples per hour. Results are precise (R.S.D < 2%) and in agreement with those obtained by ICP-AES and spectrophotometry at a 95% confidence level.

Minimization of lead and copper interferences on spectrophotometric determination of cadmium using electrolytic deposition and ion-exchange in multi-commutation flow system

A new strategy for minimization of Cu2+ and Pb2+ interferences on the spectrophotometric determination of Cd2+ by the Malachite green (MG)-iodide reaction using electrolytic deposition of interfering species and solid phase extraction of Cd2+ in flow system is proposed. The electrolytic cell comprises two coiled Pt electrodes concentrically assembled. When the sample solution is electrolyzed in a mixed solution containing 5% (v/v) HNO3, 0.1% (v/v) H2SO4 and 0.5 M NaCl, Cu2+ is deposited as Cu on the cathode, Pb2+ is deposited as PbO2 on the anode while Cd2+ is kept in solution. After electrolysis, the remaining solution passes through an AG1-X8 resin (chloride form) packed minicolumn in which Cd2+ is extracted as CdCl4/2-. Electrolyte compositions, flow rates, timing, applied current, and electrolysis time was investigated. With 60 s electrolysis time, 0.25 A applied current, Pb2+ and Cu2+ levels up to 50 and 250 mg 1-1, respectively, can be tolerated without interference. For 90 s resin loading time, a linear relationship between absorbance and analyte concentration in the 5.00-50.0 μg Cd 1-1 range (r2 = 0.9996) is obtained. A throughput of 20 samples per h is achieved, corresponding to about 0.7 mg MG and 500 mg KI and 5 ml sample consumed per determination. The detection limit is 0.23 μg Cd 1-1. The accuracy was checked for cadmium determination in standard reference materials, vegetables and tap water. Results were in agreement with certified values of standard reference materials and with those obtained by graphite furnace atomic absorption spectrometry at 95% confidence level. The R.S.D. for plant digests and water containing 13.0 μg Cd 1-1 was 3.85% (n = 12). The recoveries of analyte spikes added to the water and vegetable samples ranged from 94 to 104%. (C) 2000 Elsevier Science B.V.

A multi-agent architecture for intelligent data analysis

The present study introduces a multi-agent architecture designed for doing automation process of data integration and intelligent data analysis. Different from other approaches the multi-agent architecture was designed using a multi-agent based methodology. Tropos, an agent based methodology was used for design. Based on the proposed architecture, we describe a Web based application where the agents are responsible to analyse petroleum well drilling data to identify possible abnormalities occurrence. The intelligent data analysis methods used was the Neural Network.

A mixed-integer LP model for the optimal allocation of voltage regulators and capacitors in radial distribution systems

This paper presents a mixed-integer linear programming model to solve the problem of allocating voltage regulators and fixed or switched capacitors (VRCs) in radial distribution systems. The use of a mixed-integer linear model guarantees convergence to optimality using existing optimization software. In the proposed model, the steady-state operation of the radial distribution system is modeled through linear expressions. The results of one test system and one real distribution system are presented in order to show the accuracy as well as the efficiency of the proposed solution technique. An heuristic to obtain the Pareto front for the multiobjective VRCs allocation problem is also presented. © 2012 Elsevier Ltd. All rights reserved.

Bilevel approach for optimal location and contract pricing of distributed generation in radial distribution systems using mixed-integer linear programming

In this study, a novel approach for the optimal location and contract pricing of distributed generation (DG) is presented. Such an approach is designed for a market environment in which the distribution company (DisCo) can buy energy either from the wholesale energy market or from the DG units within its network. The location and contract pricing of DG is determined by the interaction between the DisCo and the owner of the distributed generators. The DisCo intends to minimise the payments incurred in meeting the expected demand, whereas the owner of the DG intends to maximise the profits obtained from the energy sold to the DisCo. This two-agent relationship is modelled in a bilevel scheme. The upper-level optimisation is for determining the allocation and contract prices of the DG units, whereas the lower-level optimisation is for modelling the reaction of the DisCo. The bilevel programming problem is turned into an equivalent single-level mixed-integer linear optimisation problem using duality properties, which is then solved using commercially available software. Results show the robustness and efficiency of the proposed model compared with other existing models. As regards to contract pricing, the proposed approach allowed to find better solutions than those reported in previous works. © The Institution of Engineering and Technology 2013.