A Two-Stage Online Approach for Collaborative Multi-Agent Planning under Uncertainty

In a team of multiple agents, the pursuance of a common goal is a defining characteristic. Since agents may have different capabilities, and effects of actions may be uncertain, a common goal can generally only be achieved through a careful cooperation between the different agents. In this work, we propose a novel two-stage planner that combines online planning at both team level and individual level through a subgoal delegation scheme. The proposal brings the advantages of online planning approaches to the multi-agent setting. A number of modifications are made to a classical UCT approximate algorithm to (i) adapt it to the application domains considered, (ii) reduce the branching factor in the underlying search process, and (iii) effectively manage uncertain information of action effects by using information fusion mechanisms. The proposed online multi-agent planner reduces the cost of planning and decreases the temporal cost of reaching a goal, while significantly increasing the chance of success of achieving the common goal. 

Systems of exchange values as tools for multi-agent organizations

This paper introduces systems of exchange values as tools for the organization of multi-agent systems. Systems of exchange values are defined on the basis of the theory of social exchanges, developed by Piaget and Homans. A model of social organization is proposed, where social relations are construed as social exchanges and exchange values are put into use in the support of the continuity of the performance of social exchanges. The dynamics of social organizations is formulated in terms of the regulation of exchanges of values, so that social equilibrium is connected to the continuity of the interactions. The concept of supervisor of social equilibrium is introduced as a centralized mechanism for solving the problem of the equilibrium of the organization The equilibrium supervisor solves such problem making use of a qualitative Markov Decision Process that uses numerical intervals for the representation of exchange values.

A Multi-Agent Simulation of Retail Management Practices

We apply Agent-Based Modeling and Simulation (ABMS) to investigate a set of problems in a retail context. Specifically, we are working to understand the relationship between human resource management practices and retail productivity. Despite the fact we are working within a relatively novel and complex domain, it is clear that intelligent agents do offer potential for developing organizational capabilities in the future. Our multi-disciplinary research team has worked with a UK department store to collect data and capture perceptions about operations from actors within departments. Based on this case study work, we have built a simulator that we present in this paper. We then use the simulator to gather empirical evidence regarding two specific management practices: empowerment and employee development.

Multi-Agent Simulation and Management Practices

Intelligent agents offer a new and exciting way of understanding the world of work. Agent-Based Simulation (ABS), one way of using intelligent agents, carries great potential for progressing our understanding of management practices and how they link to retail performance. We have developed simulation models based on research by a multi-disciplinary team of economists, work psychologists and computer scientists. We will discuss our experiences of implementing these concepts working with a well-known retail department store. There is no doubt that management practices are linked to the performance of an organisation (Reynolds et al., 2005; Wall & Wood, 2005). Best practices have been developed, but when it comes down to the actual application of these guidelines considerable ambiguity remains regarding their effectiveness within particular contexts (Siebers et al., forthcoming a). Most Operational Research (OR) methods can only be used as analysis tools once management practices have been implemented. Often they are not very useful for giving answers to speculative ‘what-if’ questions, particularly when one is interested in the development of the system over time rather than just the state of the system at a certain point in time. Simulation can be used to analyse the operation of dynamic and stochastic systems. ABS is particularly useful when complex interactions between system entities exist, such as autonomous decision making or negotiation. In an ABS model the researcher explicitly describes the decision process of simulated actors at the micro level. Structures emerge at the macro level as a result of the actions of the agents and their interactions with other agents and the environment. We will show how ABS experiments can deal with testing and optimising management practices such as training, empowerment or teamwork. Hence, questions such as “will staff setting their own break times improve performance?” can be investigated.

Introduction to Multi-Agent Simulation

When designing systems that are complex, dynamic and stochastic in nature, simulation is generally recognised as one of the best design support technologies, and a valuable aid in the strategic and tactical decision making process. A simulation model consists of a set of rules that define how a system changes over time, given its current state. Unlike analytical models, a simulation model is not solved but is run and the changes of system states can be observed at any point in time. This provides an insight into system dynamics rather than just predicting the output of a system based on specific inputs. Simulation is not a decision making tool but a decision support tool, allowing better informed decisions to be made. Due to the complexity of the real world, a simulation model can only be an approximation of the target system. The essence of the art of simulation modelling is abstraction and simplification. Only those characteristics that are important for the study and analysis of the target system should be included in the simulation model. The purpose of simulation is either to better understand the operation of a target system, or to make predictions about a target system’s performance. It can be viewed as an artificial white-room which allows one to gain insight but also to test new theories and practices without disrupting the daily routine of the focal organisation. What you can expect to gain from a simulation study is very well summarised by FIRMA (2000). His idea is that if the theory that has been framed about the target system holds, and if this theory has been adequately translated into a computer model this would allow you to answer some of the following questions: · Which kind of behaviour can be expected under arbitrarily given parameter combinations and initial conditions? · Which kind of behaviour will a given target system display in the future? · Which state will the target system reach in the future? The required accuracy of the simulation model very much depends on the type of question one is trying to answer. In order to be able to respond to the first question the simulation model needs to be an explanatory model. This requires less data accuracy. In comparison, the simulation model required to answer the latter two questions has to be predictive in nature and therefore needs highly accurate input data to achieve credible outputs. These predictions involve showing trends, rather than giving precise and absolute predictions of the target system performance. The numerical results of a simulation experiment on their own are most often not very useful and need to be rigorously analysed with statistical methods. These results then need to be considered in the context of the real system and interpreted in a qualitative way to make meaningful recommendations or compile best practice guidelines. One needs a good working knowledge about the behaviour of the real system to be able to fully exploit the understanding gained from simulation experiments. The goal of this chapter is to brace the newcomer to the topic of what we think is a valuable asset to the toolset of analysts and decision makers. We will give you a summary of information we have gathered from the literature and of the experiences that we have made first hand during the last five years, whilst obtaining a better understanding of this exciting technology. We hope that this will help you to avoid some pitfalls that we have unwittingly encountered. Section 2 is an introduction to the different types of simulation used in Operational Research and Management Science with a clear focus on agent-based simulation. In Section 3 we outline the theoretical background of multi-agent systems and their elements to prepare you for Section 4 where we discuss how to develop a multi-agent simulation model. Section 5 outlines a simple example of a multi-agent system. Section 6 provides a collection of resources for further studies and finally in Section 7 we will conclude the chapter with a short summary.

Robust Adaptive Control with Applications to Multi-Agent Systems and Mobile Robotics

This thesis deals with robust adaptive control and its applications, and it is divided into three main parts. The first part is about the design of robust estimation algorithms based on recursive least squares. First, we present an estimator for the frequencies of biased multi-harmonic signals, and then an algorithm for distributed estimation of an unknown parameter over a network of adaptive agents. In the second part of this thesis, we consider a cooperative control problem over uncertain networks of linear systems and Kuramoto systems, in which the agents have to track the reference generated by a leader exosystem. Since the reference signal is not available to each network node, novel distributed observers are designed so as to reconstruct the reference signal locally for each agent, and therefore decentralizing the problem. In the third and final part of this thesis, we consider robust estimation tasks for mobile robotics applications. In particular, we first consider the problem of slip estimation for agricultural tracked vehicles. Then, we consider a search and rescue application in which we need to drive an unmanned aerial vehicle as close as possible to the unknown (and to be estimated) position of a victim, who is buried under the snow after an avalanche event. In this thesis, robustness is intended as an input-to-state stability property of the proposed identifiers (sometimes referred to as adaptive laws), with respect to additive disturbances, and relative to a steady-state trajectory that is associated with a correct estimation of the unknown parameter to be found.

Modelling, simulation and control of rotary-wing aircraft in multi-agent scenario

The topic of this thesis is the design and the implementation of mathematical models and control system algorithms for rotary-wing unmanned aerial vehicles to be used in cooperative scenarios. The use of rotorcrafts has many attractive advantages, since these vehicles have the capability to take-off and land vertically, to hover and to move backward and laterally. Rotary-wing aircraft missions require precise control characteristics due to their unstable and heavy coupling aspects. As a matter of fact, flight test is the most accurate way to evaluate flying qualities and to test control systems. However, it may be very expensive and/or not feasible in case of early stage design and prototyping. A good compromise is made by a preliminary assessment performed by means of simulations and a reduced flight testing campaign. Consequently, having an analytical framework represents an important stage for simulations and control algorithm design. In this work mathematical models for various helicopter configurations are implemented. Different flight control techniques for helicopters are presented with theoretical background and tested via simulations and experimental flight tests on a small-scale unmanned helicopter. The same platform is used also in a cooperative scenario with a rover. Control strategies, algorithms and their implementation to perform missions are presented for two main scenarios. One of the main contributions of this thesis is to propose a suitable control system made by a classical PID baseline controller augmented with L1 adaptive contribution. In addition a complete analytical framework and the study of the dynamics and the stability of a synch-rotor are provided. At last, the implementation of cooperative control strategies for two main scenarios that include a small-scale unmanned helicopter and a rover.

Agents as multi-threaded logical objects

In this paper we describe a distributed object oriented logic programming language in which an object is a collection of threads deductively accessing and updating a shared logic program. The key features of the language, such as static and dynamic object methods and multiple inheritance, are illustrated through a series of small examples. We show how we can implement object servers, allowing remote spawning of objects, which we can use as staging posts for mobile agents. We give as an example an information gathering mobile agent that can be queried about the information it has so far gathered whilst it is gathering new information. Finally we define a class of co-operative reasoning agents that can do resource bounded inference for full first order predicate logic, handling multiple queries and information updates concurrently. We believe that the combination of the concurrent OO and the LP programming paradigms produces a powerful tool for quickly implementing rational multi-agent applications on the internet.

Estratégias para negociação de contratos bilaterais em mercados multi-agente de energia eléctrica

A energia eléctrica é um bem essencial para a maioria das sociedades. O seu fornecimento tem sido encarado como um serviço público, da responsabilidade dos governos, através de empresas monopolistas, públicas e privadas. O Mercado Ibérico de Electricidade (MIBEL) surge com o objectivo da integração e cooperação do sector eléctrico Português e Espanhol, no qual é possível negociar preços e volumes de energia. Actualmente, as entidades podem negociar através de um mercado bolsista ou num mercado de contratos bilaterais. Uma análise dos mercados de electricidade existentes mostra que estes estão longe de estarem liberalizados. As tarifas não reflectem o efeito da competitividade. Além disso, o recurso a contratos bilaterais limita frequentemente os clientes a um único fornecedor de energia eléctrica. Nos últimos anos, têm surgido uma série de ferramentas computacionais que permitem simular, parte ou a totalidade, dos mercados de electricidade. Contudo, apesar das suas potencialidades, muitos simuladores carecem de flexibilidade e generalidade. Nesta perspectiva, esta dissertação tem como principal objectivo o desenvolvimento de um simulador de mercados de energia eléctrica que possibilite lidar com as dificuldades inerentes a este novo modelo de mercado, recorrendo a agentes computacionais autónomos. A dissertação descreve o desenho e a implementação de um simulador simplificado para negociação de contratos bilaterais em mercados de energia, com particular incidência para o desenho das estratégias a utilizar pelas partes negociais. Além disso, efectua-se a descrição de um caso prático, com dados do MIBEL. Descrevem-se também várias simulações computacionais, envolvendo retalhistas e consumidores de energia eléctrica, que utilizam diferentes estratégias negociais. Efectua-se a análise detalhada dos resultados obtidos. De forma sucinta, os resultados permitem concluir que as melhores estratégias para cada entidade, no caso prático estudado, são: a estratégia de concessões fixas, para o retalhista, e a estratégia de concessões baseada no volume de energia, para o consumidor.

Contratos bilaterais em mercados multi-agente de energia eléctrica: protocolo de ofertas alternadas

A liberalização do sector eléctrico, e a consequente criação de mercados de energia eléctrica regulados e liberalizados, mudou a forma de comercialização da electricidade. Em particular, permitiu a entrada de empresas nas actividades de produção e comercialização, aumentando a competitividade e assegurando a liberdade de escolha dos consumidores, para decidir o fornecedor de electricidade que pretenderem. A competitividade no sector eléctrico aumentou a necessidade das empresas que o integram a proporem preços mais aliciantes (do que os preços propostos pelos concorrentes), e contribuiu para o desenvolvimento de estratégias de mercado que atraiam mais clientes e aumentem a eficiência energética e económica. A comercialização de electricidade pode ser realizada em mercados organizados ou através de contratação directa entre comercializadores e consumidores, utilizando os contratos bilaterais físicos. Estes contratos permitem a negociação dos preços de electricidade entre os comercializadores e os consumidores. Actualmente, existem várias ferramentas computacionais para fazer a simulação de mercados de energia eléctrica. Os simuladores existentes permitem simulações de transacções em bolsas de energia, negociação de preços através de contratos bilaterais, e análises técnicas a redes de energia. No entanto, devido à complexidade dos sistemas eléctricos, esses simuladores apresentam algumas limitações. Esta dissertação apresenta um simulador de contratos bilaterais em mercados de energia eléctrica, sendo dando ênfase a um protocolo de ofertas alternadas, desenvolvido através da tecnologia multi-agente. Em termos sucintos, um protocolo de ofertas alternadas é um protocolo de interacção que define as regras da negociação entre um agente vendedor (por exemplo um retalhista) e um agente comprador (por exemplo um consumidor final). Aplicou-se o simulador na resolução de um caso prático, baseado em dados reais. Os resultados obtidos permitem concluir que o simulador, apesar de simplificado, pode ser uma ferramenta importante na ajuda à tomada de decisões inerentes à negociação de contratos bilaterais em mercados de electricidade.

MASCEM improvement for agent based coalitions

This paper presents MASCEM - Multi-Agent Simulator for Electricity Markets improvement towards an enlarged model for Seller Agents coalitions. The simulator has been improved, both regarding its user interface and internal structure. The OOA, used as development platform, version was updated and the multi-agent model was adjusted for implementing and testing several negotiations regarding Seller agents’ coalitions. Seller coalitions are a very important subject regarding the increased relevance of Distributed Generation under liberalised electricity markets.

Agent-based simulation of electronic marketplaces with decision support

This paper presents a Multi-Agent Market simulator designed for analyzing agent market strategies based on a complete understanding of buyer and seller behaviors, preference models and pricing algorithms, considering user risk preferences and game theory for scenario analysis. The system includes agents that are capable of improving their performance with their own experience, by adapting to the market conditions, and capable of considering other agents reactions.

Modeling group decision meeting participants with an Agent-based approach

This paper aims to present a multi-agent model for a simulation, whose goal is to help one specific participant of multi-criteria group decision making process.This model has five main intervenient types: the human participant, who is using the simulation and argumentation support system; the participant agents, one associated to the human participant and the others simulating the others human members of the decision meeting group; the directory agent; the proposal agents, representing the different alternatives for a decision (the alternatives are evaluated based on criteria); and the voting agent responsiblefor all voting machanisms.At this stage it is proposed a two phse algorithm. In the first phase each participantagent makes his own evaluation of the proposals under discussion, and the voting agent proposes a simulation of a voting process.In the second phase, after the dissemination of the voting results,each one ofthe partcipan agents will argue to convince the others to choose one of the possible alternatives. The arguments used to convince a specific participant are dependent on agent knowledge about that participant. This two-phase algorithm is applied iteratively.

Agent based simulation for group formation

Group decision making plays an important role in today’s organisations. The impact of decision making is so high and complex, that rarely the decision making process is made just by one individual. The simulation of group decision making through a Multi-Agent System is a very interesting research topic. The purpose of this paper it to specify the actors involved in the simulation of a group decision, to present a model to the process of group formation and to describe the approach made to implement that model. In the group formation model it is considered the existence of incomplete and negative information, which was identified as crucial to make the simulation closer to the reality.

Sistema multi-agente para gestão de tráfego em centros urbanos

Mestrado em Engenharia Informática