MASCEM: electricity markets simulation with strategic agents

Electricity markets are complex environments, involving numerous entities trying to obtain the best advantages and profits while limited by power-network characteristics and constraints.1 The restructuring and consequent deregulation of electricity markets introduced a new economic dimension to the power industry. Some observers have criticized the restructuring process, however, because it has failed to improve market efficiency and has complicated the assurance of reliability and fairness of operations. To study and understand this type of market, we developed the Multiagent Simulator of Competitive Electricity Markets (MASCEM) platform based on multiagent simulation. The MASCEM multiagent model includes players with strategies for bid definition, acting in forward, day-ahead, and balancing markets and considering both simple and complex bids. Our goal with MASCEM was to simulate as many market models and player types as possible. This approach makes MASCEM both a short- and mediumterm simulation as well as a tool to support long-term decisions, such as those taken by regulators. This article proposes a new methodology integrated in MASCEM for bid definition in electricity markets. This methodology uses reinforcement learning algorithms to let players perceive changes in the environment, thus helping them react to the dynamic environment and adapt their bids accordingly.

Adaptive learning in agents behaviour: a framework for electricity markets simulation

Electricity markets are complex environments with very particular characteristics. A critical issue regarding these specific characteristics concerns the constant changes they are subject to. This is a result of the electricity markets’ restructuring, which was performed so that the competitiveness could be increased, but it also had exponential implications in the increase of the complexity and unpredictability in those markets scope. The constant growth in markets unpredictability resulted in an amplified need for market intervenient entities in foreseeing market behaviour. The need for understanding the market mechanisms and how the involved players’ interaction affects the outcomes of the markets, contributed to the growth of usage of simulation tools. Multi-agent based software is particularly well fitted to analyze dynamic and adaptive systems with complex interactions among its constituents, such as electricity markets. This dissertation presents ALBidS – Adaptive Learning strategic Bidding System, a multiagent system created to provide decision support to market negotiating players. This system is integrated with the MASCEM electricity market simulator, so that its advantage in supporting a market player can be tested using cases based on real markets’ data. ALBidS considers several different methodologies based on very distinct approaches, to provide alternative suggestions of which are the best actions for the supported player to perform. The approach chosen as the players’ actual action is selected by the employment of reinforcement learning algorithms, which for each different situation, simulation circumstances and context, decides which proposed action is the one with higher possibility of achieving the most success. Some of the considered approaches are supported by a mechanism that creates profiles of competitor players. These profiles are built accordingly to their observed past actions and reactions when faced with specific situations, such as success and failure. The system’s context awareness and simulation circumstances analysis, both in terms of results performance and execution time adaptation, are complementary mechanisms, which endow ALBidS with further adaptation and learning capabilities.

Adaptive learning in agents behaviour: A framework for electricity markets simulation

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM (Multi-Agent System for Competitive Electricity Markets) is a multi-agent electricity market simulator that models market players and simulates their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. This paper presents a methodology to provide decision support to electricity market negotiating players. This model allows integrating different strategic approaches for electricity market negotiations, and choosing the most appropriate one at each time, for each different negotiation context. This methodology is integrated in ALBidS (Adaptive Learning strategic Bidding System) – a multiagent system that provides decision support to MASCEM's negotiating agents so that they can properly achieve their goals. ALBidS uses artificial intelligence methodologies and data analysis algorithms to provide effective adaptive learning capabilities to such negotiating entities. The main contribution is provided by a methodology that combines several distinct strategies to build actions proposals, so that the best can be chosen at each time, depending on the context and simulation circumstances. The choosing process includes reinforcement learning algorithms, a mechanism for negotiating contexts analysis, a mechanism for the management of the efficiency/effectiveness balance of the system, and a mechanism for competitor players' profiles definition.

Intelligent decision making in electricity markets: simulated annealing Q-Learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM is integrated with ALBidS, a system that provides several dynamic strategies for agents’ behavior. This paper presents a method that aims at enhancing ALBidS competence in endowing market players with adequate strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible actions. These actions are defined accordingly to the most probable points of bidding success. With the purpose of accelerating the convergence process, a simulated annealing based algorithm is included.

Balancing market integration in MASCEM electricity market simulator

With the restructuring of the energy sector in industrialized countries there is an increased complexity in market players’ interactions along with emerging problems and new issues to be addressed. Decision support tools that facilitate the study and understanding of these markets are extremely useful to provide players with competitive advantage. In this context arises MASCEM, a multi-agent simulator for competitive electricity markets. It is essential to reinforce MASCEM with the ability to recreate electricity markets reality in the fullest possible extent, making it able to simulate as many types of markets models and players as possible. This paper presents the development of the Balancing Market in MASCEM. A key module to the study of competitive electricity markets, as it has well defined and distinct characteristics previously implemented.

VPP´s multi-level negotiation in smart grids and competitive electricity markets

The increase of distributed generation (DG) has brought about new challenges in electrical networks electricity markets and in DG units operation and management. Several approaches are being developed to manage the emerging potential of DG, such as Virtual Power Players (VPPs), which aggregate DG plants; and Smart Grids, an approach that views generation and associated loads as a subsystem. This paper presents a multi-level negotiation mechanism for Smart Grids optimal operation and negotiation in the electricity markets, considering the advantages of VPPs’ management. The proposed methodology is implemented and tested in MASCEM – a multiagent electricity market simulator, developed to allow deep studies of the interactions between the players that take part in the electricity market negotiations.

Cost dependent strategy for electricity markets bidding based on adaptive reinforcement learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM provides several dynamic strategies for agents’ behavior. This paper presents a method that aims to provide market players with strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible bids. These bids are defined accordingly to the cost function that each producer presents.

Adapting Meeting Tools to Agent Decision

Electricity markets are complex environments comprising several negotiation mechanisms. MASCEM (Multi- Agent System for Competitive Electricity Markets) is a simulator developed to allow deep studies of the interactions between the players that take part in the electricity market negotiations. ALBidS (Adaptive Learning Strategic Bidding System) is a multiagent system created to provide decision support to market negotiating players. Fully integrated with MASCEM it considers several different methodologies based on very distinct approaches. The Six Thinking Hats is a powerful technique used to look at decisions from different perspectives. This paper aims to complement ALBidS strategies usage by MASCEM players, providing, through the Six Thinking Hats group decision technique, a means to combine them and take advantages from their different perspectives. The combination of the different proposals resulting from ALBidS’ strategies is performed through the application of a Genetic Algorithm, resulting in an evolutionary learning approach.

Fibrous Aerial Robotics: study of spiderweb strategies for the design of architectural envelopes using swarms of drones and inflatable formworks

La tesi approccia in modo transdisciplinare biologia, architettura e robotica, con la finalità di indagare e applicare principi costruttivi attraverso l’interazione tra sciami di droni che depositano materiale fibroso su strutture gonfiabili di supporto. L’attenzione principale è nello sviluppo (attraverso un workflow computazionale che gestisce sciami di agenti costruttori) di una tettonica che integra struttura, spazio e ornamento all’interno dello stesso processo progettuale, il quale si sviluppa coerentemente dall’ideazione fino alla fabbricazione. Sono stati studiati modelli biologici quali le colonie di ragni sociali, i quali costruiscono artefatti di grandi dimensioni relativamente a quelle del singolo individuo grazie ad un’organizzazione coordinata ed emergente e alle proprietà dei sistemi fibrosi. L’auto-organizzazione e la decentralizzazione, insieme alle caratteristiche del sistema materiale, sono stati elementi indispensabili nell’estrapolazione prima e nella codificazione poi di un insieme di regole adatte allo sviluppo del sistema costruttivo. Parallelamente alla simulazione digitale si è andati a sviluppare anche un processo fisico di fabbricazione che, pur tenendo conto dei vincoli economici e tecnici, potesse dimostrarsi una prova di concetto e fattibilità del sistema costruttivo. Sono state investigate le possibilità che un drone offre nel campo della fabbricazione architettonica mediante il rilascio di fili su elementi gonfiabili in pressione. Il processo può risultare vantaggioso in scenari in cui non è possibile allestire infrastrutture costruttive tradizionali (es. gole alpine, foreste). Tendendo conto dei vincoli e delle caratteristiche del sistema di fabbricazione proposto, sono state esplorate potenzialità e criticità del sistema studiato.

Broadcast Control Mechanism for Positional Consensus in Multiagent Systems

In this paper, a strategy for controlling a group of agents to achieve positional consensus is presented. The problem is constrained by the requirement that every agent must be given the same control input through a broadcast communication mechanism. Although the control command is computed using state information in a global framework, the control input is implemented by the agents in a local coordinate frame. We propose a novel linear programming (LP) formulation that is computationally less intensive than earlier proposed methods. Moreover, a random perturbation input in the control command that helps the agents to come close to each other even for a large number of agents, which was not possible with an existing strategy in the literature, is introduced. The method is extended to achieve positional consensus at a prespecified location. The effectiveness of the approach is illustrated through simulation results. A comparison between the LP approach and the existing second-order cone programming-based approach is also presented. The algorithm was successfully implemented on a robotic platform with three robots.

Agent-based simulation of lock-in dynamics in a duopoly

Lock-in is observed in real world markets of experience goods; experience goods are goods whose characteristics are difficult to determine in advance, but ascertained upon consumption. We create an agent-based simulation of consumers choosing between two experience goods available in a virtual market. We model consumers in a grid representing the spatial network of the consumers. Utilising simple assumptions, including identical distributions of product experience and consumers having a degree of follower tendency, we explore the dynamics of the model through simulations. We conduct simulations to create a lock-in before testing several hypotheses upon how to break an existing lock-in; these include the effect of advertising and free give-away. Our experiments show that the key to successfully breaking a lock-in required the creation of regions in a consumer population. Regions arise due to the degree of local conformity between agents within the regions, which spread throughout the population when a mildly superior competitor was available. These regions may be likened to a niche in a market, which gains in popularity to transition into the mainstream.

Bi-directional double auction for financial market simulation

Typical Double Auction (DA) models assume that trading agents are one-way traders. With this limitation, they cannot directly reflect the fact individual traders in financial markets (the most popular application of double auction) choose their trading directions dynamically. To address this issue, we introduce the Bi-directional Double Auction (BDA) market which is populated by two-way traders. Based on experiments under both static and dynamic settings, we find that the allocative efficiency of a static continuous BDA market comes from rational selection of trading directions and is negatively related to the intelligence of trading strategies. Moreover, we introduce Kernel trading strategy designed based on probability density estimation for general DA market. Our experiments show it outperforms some intelligent DA market trading strategies. Copyright © 2013, International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.

Optimizing last mile delivery using public transport with multiagent based control

The majority of research work carried out in the field of Operations-Research uses methods and algorithms to optimize the pick-up and delivery problem. Most studies aim to solve the vehicle routing problem, to accommodate optimum delivery orders, vehicles etc. This paper focuses on green logistics approach, where existing Public Transport infrastructure capability of a city is used for the delivery of small and medium sized packaged goods thus, helping improve the situation of urban congestion and greenhouse gas emissions reduction. It carried out a study to investigate the feasibility of the proposed multi-agent based simulation model, for efficiency of cost, time and energy consumption. Multimodal Dijkstra Shortest Path algorithm and Nested Monte Carlo Search have been employed for a two-phase algorithmic approach used for generation of time based cost matrix. The quality of the tour is dependent on the efficiency of the search algorithm implemented for plan generation and route planning. The results reveal a definite advantage of using Public Transportation over existing delivery approaches in terms of energy efficiency.