Interaction among agents that plan

The development of practical agent languages has progressed significantly over recent years, but this has largely been independent of distinct developments in aspects of multiagent cooperation and planning. For example, while the popular AgentSpeak(L) has had various extensions and improvements proposed, it still essentially a single-agent language. In response, in this paper, we describe a simple, yet effective, technique for multiagent planning that enables an agent to take advantage of cooperating agents in a society. In particular, we build on a technique that enables new plans to be added to a plan library through the invocation of an external planning component, and extend it to include the construction of plans involving the chaining of subplans of others. Our mechanism makes use of plan patterns that insulate the planning process from the resulting distributed aspects of plan execution through local proxy plans that encode information about the preconditions and effects of the external plans provided by agents willing to cooperate. In this way, we allow an agent to discover new ways of achieving its goals through local planning and the delegation of tasks for execution by others, allowing it to overcome individual limitations.

Aprendizado por Reforço com Valores deInfluência em Sistemas Multi-Agente

We propose a new paradigm for collective learning in multi-agent systems (MAS) as a solution to the problem in which several agents acting over the same environment must learn how to perform tasks, simultaneously, based on feedbacks given by each one of the other agents. We introduce the proposed paradigm in the form of a reinforcement learning algorithm, nominating it as reinforcement learning with influence values. While learning by rewards, each agent evaluates the relation between the current state and/or action executed at this state (actual believe) together with the reward obtained after all agents that are interacting perform their actions. The reward is a result of the interference of others. The agent considers the opinions of all its colleagues in order to attempt to change the values of its states and/or actions. The idea is that the system, as a whole, must reach an equilibrium, where all agents get satisfied with the obtained results. This means that the values of the state/actions pairs match the reward obtained by each agent. This dynamical way of setting the values for states and/or actions makes this new reinforcement learning paradigm the first to include, naturally, the fact that the presence of other agents in the environment turns it a dynamical model. As a direct result, we implicitly include the internal state, the actions and the rewards obtained by all the other agents in the internal state of each agent. This makes our proposal the first complete solution to the conceptual problem that rises when applying reinforcement learning in multi-agent systems, which is caused by the difference existent between the environment and agent models. With basis on the proposed model, we create the IVQ-learning algorithm that is exhaustive tested in repetitive games with two, three and four agents and in stochastic games that need cooperation and in games that need collaboration. This algorithm shows to be a good option for obtaining solutions that guarantee convergence to the Nash optimum equilibrium in cooperative problems. Experiments performed clear shows that the proposed paradigm is theoretical and experimentally superior to the traditional approaches. Yet, with the creation of this new paradigm the set of reinforcement learning applications in MAS grows up. That is, besides the possibility of applying the algorithm in traditional learning problems in MAS, as for example coordination of tasks in multi-robot systems, it is possible to apply reinforcement learning in problems that are essentially collaborative

Experience generalization for multi-agent reinforcement learning

On-line learning methods have been applied successfully in multi-agent systems to achieve coordination among agents. Learning in multi-agent systems implies in a non-stationary scenario perceived by the agents, since the behavior of other agents may change as they simultaneously learn how to improve their actions. Non-stationary scenarios can be modeled as Markov Games, which can be solved using the Minimax-Q algorithm a combination of Q-learning (a Reinforcement Learning (RL) algorithm which directly learns an optimal control policy) and the Minimax algorithm. However, finding optimal control policies using any RL algorithm (Q-learning and Minimax-Q included) can be very time consuming. Trying to improve the learning time of Q-learning, we considered the QS-algorithm. in which a single experience can update more than a single action value by using a spreading function. In this paper, we contribute a Minimax-QS algorithm which combines the Minimax-Q algorithm and the QS-algorithm. We conduct a series of empirical evaluation of the algorithm in a simplified simulator of the soccer domain. We show that even using a very simple domain-dependent spreading function, the performance of the learning algorithm can be improved.

Multi-Agent Simulation of Urban Social Dynamics for Spatial Load Forecasting

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Multi-Agent Framework for Spatial Load Forecasting

A multi-agent framework for spatial electric load forecasting, especially suited to simulate the different dynamics involved on distribution systems, is presented. The service zone is divided into several sub-zones, each subzone is considered as an independent agent identified with a corresponding load level, and their relationships with the neighbor zones are represented as development probabilities. With this setting, different kind of agents can be developed to simulate the growth pattern of the loads in distribution systems. This paper presents two different kinds of agents to simulate different situations, presenting some promissory results.

A multi-agent architecture for intelligent data analysis

The present study introduces a multi-agent architecture designed for doing automation process of data integration and intelligent data analysis. Different from other approaches the multi-agent architecture was designed using a multi-agent based methodology. Tropos, an agent based methodology was used for design. Based on the proposed architecture, we describe a Web based application where the agents are responsible to analyse petroleum well drilling data to identify possible abnormalities occurrence. The intelligent data analysis methods used was the Neural Network.

A multi-agent system with a percolation approach to simulate the driving pattern of Plug-In ELectric Vehicles

A multi-agent system with a percolation approach to simulate the driving pattern of Plug-In Electric Vehicle (PEV), especially suited to simulate the PEVs behavior on any distribution systems, is presented. This tool intends to complement information about the driving patterns database on systems where that kind of information is not available. So, this paper aims to provide a framework that is able to work with any kind of technology and load generated of PEVs. The service zone is divided into several sub-zones, each subzone is considered as an independent agent identified with corresponding load level, and their relationships with the neighboring zones are represented as network probabilities. A percolation approach is used to characterize the autonomy of the battery of the PVEs to move through the city. The methodology is tested with data from a mid-size city real distribution system. The result shows the sub-area where the battery of PEVs will need to be recharge and gives the planners of distribution systems the necessary input for a medium to long term network planning in a smart grid environment. © 2012 IEEE.

A cidade poli(multi) nucleada: a reestruturação do espaço urbano em Salvador

Pós-graduação em Geografia - FCT

Um modelo bi-nível para o investimento multi-etapa em capacidade de transmissão e geração de energia elétrica em mercados competitivos

Pós-graduação em Engenharia Elétrica - FEIS

Adesivos multi-uso-avaliação da resistência adesiva a dentina frente a teste de cisalhamento, estereomicroscopia e microscopia eletrônica de varredura

Recent bonding systems have been advocated as multi-purpose bonding agents. The aim of this study was to determine if some of these bonding systems could be associated to composite resins from different manufacturers. This investigation was conducted to test lhe shear bond strength of three bonding systems: Scotchbond Multi-Purpose (3M Dental Products), Optibond Light Cure (Kerr) and Optibond Dual Cure (Kerr), when each of them was associated to lhe composite resins: Z1 00 (3M Dental Products), Prisma - APH (Dentsply) and Herculite XRV (Kerr). Seventy-two flat dentin bonding sites were prepared to 600 grit on human premolars mounted using acrilic resins. The teeth were assigned at random to 9 groups of 8 samples each. A split die with a 3mm diameter was placed over lhe surface of lhe dentin treated with one of lhe adhesive systems, and lhe selected composite resin was inserted and light cured. The split mold was removed and all samples were termocycled and stored in 37ºC water for 24 hours before testing. Shear bond strength was determined using an lnstron Universal testing machine. Some failures were examined under lhe S.E.M. Data was analysed by one-way analysis of variance, that demonstrated a significant difference (p<0,05) in the mean shear bond strength among Optibond Light Cure (15,446 MPa), Scotchbond Multi-Purpose (13,339 MPa) and Optibond Dual Cure (10,019 MPa). These values did not depend on the composite resin used. The association between bonding system/composite resin was statistycally significant (p<0,05) and the best results were obtained when the composite resins Z100 and Herculite were used with the adhesive system Optibond Light Cure, and when the composite resin APH was used with the adhesive system Scotchbond Multi-Purpose

Wildlife Master Volunteers: A Multi-County Approach to Resolving Human-Wildlife Conflicts

The Wildlife Master (WM) Program in Colorado was modeled after the highly successful Master Gardener volunteer program. In 10 highly populated suburban counties with large rural areas surrounding the Denver Metro Area, Colorado State University (CSU) Cooperative Extension Natural Resources agents train, supervise and manage these volunteers in the identification, referral, and resolution of wildlife damage issues. High quality, research-based training is provided by university faculty and other professionals in public health, animal damage control, wildlife management and animal behavior. Inquiries are responded to mainly via telephone. Calls by concerned residents are forwarded to WMs who provide general information about human-wildlife conflicts and possible ways to resolve complaints. Each volunteer serves a minimum of 14 days on phone duty annually, calling in from a remote location to a voice mail system from which phone messages can be conveniently retrieved. Response time per call is generally less than 24 hours. During 2004, more than 2,000 phone calls, e-mail messages and walk-in requests for assistance were fielded by 100 cooperative extension WMs. Calls fielded by volunteers in one county increased five-fold during the past five years, from 100 calls to over 500 calls annually. Valued at the rate of approximately $18.00 per volunteer hour, the leveraged value of each WM was about $450 in 2005, based on 25 hours of service and training. The estimated value of the program to Colorado in 2004 was over $45,000 of in-kind service, or about one full-time equivalent faculty member. This paper describes components of Colorado’s WM Program, with guides to the set-up of similar programs in other states.

A communication infrastructure to support knowledge level agents on the web

Agent Communication Languages (ACLs) have been developed to provide a way for agents to communicate with each other supporting cooperation in Multi-Agent Systems. In the past few years many ACLs have been proposed for Multi-Agent Systems, such as KQML and FIPA-ACL. The goal of these languages is to support high-level, human like communication among agents, exploiting Knowledge Level features rather than symbol level ones. Adopting these ACLs, and mainly the FIPA-ACL specifications, many agent platforms and prototypes have been developed. Despite these efforts, an important issue in the research on ACLs is still open and concerns how these languages should deal (at the Knowledge Level) with possible failures of agents. Indeed, the notion of Knowledge Level cannot be straightforwardly extended to a distributed framework such as MASs, because problems concerning communication and concurrency may arise when several Knowledge Level agents interact (for example deadlock or starvation). The main contribution of this Thesis is the design and the implementation of NOWHERE, a platform to support Knowledge Level Agents on the Web. NOWHERE exploits an advanced Agent Communication Language, FT-ACL, which provides high-level fault-tolerant communication primitives and satisfies a set of well defined Knowledge Level programming requirements. NOWHERE is well integrated with current technologies, for example providing full integration for Web services. Supporting different middleware used to send messages, it can be adapted to various scenarios. In this Thesis we present the design and the implementation of the architecture, together with a discussion of the most interesting details and a comparison with other emerging agent platforms. We also present several case studies where we discuss the benefits of programming agents using the NOWHERE architecture, comparing the results with other solutions. Finally, the complete source code of the basic examples can be found in appendix.

New Synthetic Polyamines as Multi-Target-Directed Ligands for the Treatment of Alzheimer's Disease and Cancer: Design, Synthesis and Biological Evaluation

Alzheimer's disease (AD) and cancer represent two of the main causes of death worldwide. They are complex multifactorial diseases and several biochemical targets have been recognized to play a fundamental role in their development. Basing on their complex nature, a promising therapeutical approach could be represented by the so-called "Multi-Target-Directed Ligand" approach. This new strategy is based on the assumption that a single molecule could hit several targets responsible for the onset and/or progression of the pathology. In particular in AD, most currently prescribed drugs aim to increase the level of acetylcholine in the brain by inhibiting the enzyme acetylcholinesterase (AChE). However, clinical experience shows that AChE inhibition is a palliative treatment, and the simple modulation of a single target does not address AD aetiology. Research into newer and more potent anti-AD agents is thus focused on compounds whose properties go beyond AChE inhibition (such as inhibition of the enzyme β-secretase and inhibition of the aggregation of beta-amyloid). Therefore, the MTDL strategy seems a more appropriate approach for addressing the complexity of AD and may provide new drugs for tackling its multifactorial nature. In this thesis, it is described the design of new MTDLs able to tackle the multifactorial nature of AD. Such new MTDLs designed are less flexible analogues of Caproctamine, one of the first MTDL owing biological properties useful for the AD treatment. These new compounds are able to inhibit the enzymes AChE, beta-secretase and to inhibit both AChE-induced and self-induced beta-amyloid aggregation. In particular, the most potent compound of the series is able to inhibit AChE in subnanomolar range, to inhibit β-secretase in micromolar concentration and to inhibit both AChE-induced and self-induced beta-amyloid aggregation in micromolar concentration. Cancer, as AD, is a very complex pathology and many different therapeutical approaches are currently use for the treatment of such pathology. However, due to its multifactorial nature the MTDL approach could be, in principle, apply also to this pathology. Aim of this thesis has been the development of new molecules owing different structural motifs able to simultaneously interact with some of the multitude of targets responsible for the pathology. The designed compounds displayed cytotoxic activity in different cancer cell lines. In particular, the most potent compounds of the series have been further evaluated and they were able to bind DNA resulting 100-fold more potent than the reference compound Mitonafide. Furthermore, these compounds were able to trigger apoptosis through caspases activation and to inhibit PIN1 (preliminary result). This last protein is a very promising target because it is overexpressed in many human cancers, it functions as critical catalyst for multiple oncogenic pathways and in several cancer cell lines depletion of PIN1 determines arrest of mitosis followed by apoptosis induction. In conclusion, this study may represent a promising starting pint for the development of new MTDLs hopefully useful for cancer and AD treatment.