Adaptive learning in agents behaviour: a framework for electricity markets simulation

Electricity markets are complex environments with very particular characteristics. A critical issue regarding these specific characteristics concerns the constant changes they are subject to. This is a result of the electricity markets’ restructuring, which was performed so that the competitiveness could be increased, but it also had exponential implications in the increase of the complexity and unpredictability in those markets scope. The constant growth in markets unpredictability resulted in an amplified need for market intervenient entities in foreseeing market behaviour. The need for understanding the market mechanisms and how the involved players’ interaction affects the outcomes of the markets, contributed to the growth of usage of simulation tools. Multi-agent based software is particularly well fitted to analyze dynamic and adaptive systems with complex interactions among its constituents, such as electricity markets. This dissertation presents ALBidS – Adaptive Learning strategic Bidding System, a multiagent system created to provide decision support to market negotiating players. This system is integrated with the MASCEM electricity market simulator, so that its advantage in supporting a market player can be tested using cases based on real markets’ data. ALBidS considers several different methodologies based on very distinct approaches, to provide alternative suggestions of which are the best actions for the supported player to perform. The approach chosen as the players’ actual action is selected by the employment of reinforcement learning algorithms, which for each different situation, simulation circumstances and context, decides which proposed action is the one with higher possibility of achieving the most success. Some of the considered approaches are supported by a mechanism that creates profiles of competitor players. These profiles are built accordingly to their observed past actions and reactions when faced with specific situations, such as success and failure. The system’s context awareness and simulation circumstances analysis, both in terms of results performance and execution time adaptation, are complementary mechanisms, which endow ALBidS with further adaptation and learning capabilities.

Multi-agent Simulation of Bilateral Contracting in Competitive Electricity Markets

Traditional vertically integrated power utilities around the world have evolved from monopoly structures to open markets that promote competition among suppliers and provide consumers with a choice of services. Market forces drive the price of electricity and reduce the net cost through increased competition. Electricity can be traded in both organized markets or using forward bilateral contracts. This article focuses on bilateral contracts and describes some important features of an agent-based system for bilateral trading in competitive markets. Special attention is devoted to the negotiation process, demand response in bilateral contracting, and risk management. The article also presents a case study on forward bilateral contracting: a retailer agent and a customer agent negotiate a 24h-rate tariff.

Distributed Intelligent Management of Microgrids using a Multi-Agent Simulation Platform

Multi-agent approaches have been widely used to model complex systems of distributed nature with a large amount of interactions between the involved entities. Power systems are a reference case, mainly due to the increasing use of distributed energy sources, largely based on renewable sources, which have potentiated huge changes in the power systems’ sector. Dealing with such a large scale integration of intermittent generation sources led to the emergence of several new players, as well as the development of new paradigms, such as the microgrid concept, and the evolution of demand response programs, which potentiate the active participation of consumers. This paper presents a multi-agent based simulation platform which models a microgrid environment, considering several different types of simulated players. These players interact with real physical installations, creating a realistic simulation environment with results that can be observed directly in the reality. A case study is presented considering players’ responses to a demand response event, resulting in an intelligent increase of consumption in order to face the wind generation surplus.

Enhancing Agent Mediated Electronic Markets with Ontology Matching Services and Social Network Support

Os Mercados Eletrónicos atingiram uma complexidade e nível de sofisticação tão elevados, que tornaram inadequados os modelos de software convencionais. Estes mercados são caracterizados por serem abertos, dinâmicos e competitivos, e constituídos por várias entidades independentes e heterogéneas. Tais entidades desempenham os seus papéis de forma autónoma, seguindo os seus objetivos, reagindo às ocorrências do ambiente em que se inserem e interagindo umas com as outras. Esta realidade levou a que existisse por parte da comunidade científica um especial interesse no estudo da negociação automática executada por agentes de software [Zhang et al., 2011]. No entanto, a diversidade dos atores envolvidos pode levar à existência de diferentes conceptualizações das suas necessidades e capacidades dando origem a incompatibilidades semânticas, que podem prejudicar a negociação e impedir a ocorrência de transações que satisfaçam as partes envolvidas. Os novos mercados devem, assim, possuir mecanismos que lhes permitam exibir novas capacidades, nomeadamente a capacidade de auxiliar na comunicação entre os diferentes agentes. Pelo que, é defendido neste trabalho que os mercados devem oferecer serviços de ontologias que permitam facilitar a interoperabilidade entre os agentes. No entanto, os humanos tendem a ser relutantes em aceitar a conceptualização de outros, a não ser que sejam convencidos de que poderão conseguir um bom negócio. Neste contexto, a aplicação e exploração de relações capturadas em redes sociais pode resultar no estabelecimento de relações de confiança entre vendedores e consumidores, e ao mesmo tempo, conduzir a um aumento da eficiência da negociação e consequentemente na satisfação das partes envolvidas. O sistema AEMOS é uma plataforma de comércio eletrónico baseada em agentes que inclui serviços de ontologias, mais especificamente, serviços de alinhamento de ontologias, incluindo a recomendação de possíveis alinhamentos entre as ontologias dos parceiros de negociação. Este sistema inclui também uma componente baseada numa rede social, que é construída aplicando técnicas de análise de redes socias sobre informação recolhida pelo mercado, e que permite melhorar a recomendação de alinhamentos e auxiliar os agentes na sua escolha. Neste trabalho são apresentados o desenvolvimento e implementação do sistema AEMOS, mais concretamente: • É proposto um novo modelo para comércio eletrónico baseado em agentes que disponibiliza serviços de ontologias; • Adicionalmente propõem-se o uso de redes sociais emergentes para captar e explorar informação sobre relações entre os diferentes parceiros de negócio; • É definida e implementada uma componente de serviços de ontologias que é capaz de: • o Sugerir alinhamentos entre ontologias para pares de agentes; • o Traduzir mensagens escritas de acordo com uma ontologia em mensagens escritas de acordo com outra, utilizando alinhamentos previamente aprovados; • o Melhorar os seus próprios serviços recorrendo às funcionalidades disponibilizadas pela componente de redes sociais; • É definida e implementada uma componente de redes sociais que: • o É capaz de construir e gerir um grafo de relações de proximidade entre agentes, e de relações de adequação de alinhamentos a agentes, tendo em conta os perfis, comportamento e interação dos agentes, bem como a cobertura e utilização dos alinhamentos; • o Explora e adapta técnicas e algoritmos de análise de redes sociais às várias fases dos processos do mercado eletrónico. A implementação e experimentação do modelo proposto demonstra como a colaboração entre os diferentes agentes pode ser vantajosa na melhoria do desempenho do sistema e como a inclusão e combinação de serviços de ontologias e redes sociais se reflete na eficiência da negociação de transações e na dinâmica do mercado como um todo.

Virtual power producers market strategies

This paper presents MASCEM - a multi-agent based electricity market simulator. MASCEM uses game theory, machine learning techniques, scenario analysis and optimisation techniques to model market agents and to provide them with decision-support. This paper mainly focus on the MASCEM ability to provide the means to model and simulate Virtual Power Producers (VPP). VPPs are represented as a coalition of agents, with specific characteristics and goals. The paper detail some of the most important aspects considered in VPP formation and in the aggregation of new producers and includes a case study.

MASCEM - an electricity market simulator providing coalition support for virtual power players

This paper presents MASCEM - a multi-agent based electricity market simulator. MASCEM uses game theory, machine learning techniques, scenario analysis and optimization techniques to model market agents and to provide them with decision-support. This paper mainly focus on the MASCEM ability to provide the means to model and simulate Virtual Power Players (VPP). VPPs are represented as a coalition of agents, with specific characteristics and goals. The paper details some of the most important aspects considered in VPP formation and in the aggregation of new producers and includes a case study based on real data.

Dgenco market balancing - cost repartition under light regulation markets

In this paper we present a new methodology, based in game theory, to obtain the market balancing between Distribution Generation Companies (DGENCO), in liberalized electricity markets. The new contribution of this methodology is the verification of the participation rate of each agent based in Nucléolo Balancing and in Shapley Value. To validate the results we use the Zaragoza Distribution Network with 42 Bus and 5 DGENCO.

Electricity Markets Portfolio Optimization using a Particle Swarm Approach

Energy systems worldwide are complex and challenging environments. Multi-agent based simulation platforms are increasing at a high rate, as they show to be a good option to study many issues related to these systems, as well as the involved players at act in this domain. In this scope the authors’ research group has developed a multi-agent system: MASCEM (Multi-Agent System for Competitive Electricity Markets), which simulates the electricity markets. MASCEM is integrated with ALBidS (Adaptive Learning Strategic Bidding System) that works as a decision support system for market players. The ALBidS system allows MASCEM market negotiating players to take the best possible advantages from the market context. However, it is still necessary to adequately optimize the player’s portfolio investment. For this purpose, this paper proposes a market portfolio optimization method, based on particle swarm optimization, which provides the best investment profile for a market player, considering the different markets the player is acting on in each moment, and depending on different contexts of negotiation, such as the peak and offpeak periods of the day, and the type of day (business day, weekend, holiday, etc.). The proposed approach is tested and validated using real electricity markets data from the Iberian operator – OMIE.

Cooperation mechanism for team-work based multi-agent system in dynamic scheduling through meta-heuristics

This paper describes a Multi-agent Scheduling System that assumes the existence of several Machines Agents (which are decision-making entities) distributed inside the Manufacturing System that interact and cooperate with other agents in order to obtain optimal or near-optimal global performances. Agents have to manage their internal behaviors and their relationships with other agents via cooperative negotiation in accordance with business policies defined by the user manager. Some Multi Agent Systems (MAS) organizational aspects are considered. An original Cooperation Mechanism for a Team-work based Architecture is proposed to address dynamic scheduling using Meta-Heuristics.

Multi-agent system for personalization of location-based services

The current ubiquitous network access and increase in network bandwidth are driving the sales of mobile location-aware user devices and, consequently, the development of context-aware applications, namely location-based services. The goal of this project is to provide consumers of location-based services with a richer end-user experience by means of service composition, personalization, device adaptation and continuity of service. Our approach relies on a multi-agent system composed of proxy agents that act as mediators and providers of personalization meta-services, device adaptation and continuity of service for consumers of pre-existing location-based services. These proxy agents, which have Web services interfaces to ensure a high level of interoperability, perform service composition and take in consideration the preferences of the users, the limitations of the user devices, making the usage of different types of devices seamless for the end-user. To validate and evaluate the performance of this approach, use cases were defined, tests were conducted and results gathered which demonstrated that the initial goals were successfully fulfilled.

Adapting Meeting Tools to Agent Decision

Electricity markets are complex environments comprising several negotiation mechanisms. MASCEM (Multi- Agent System for Competitive Electricity Markets) is a simulator developed to allow deep studies of the interactions between the players that take part in the electricity market negotiations. ALBidS (Adaptive Learning Strategic Bidding System) is a multiagent system created to provide decision support to market negotiating players. Fully integrated with MASCEM it considers several different methodologies based on very distinct approaches. The Six Thinking Hats is a powerful technique used to look at decisions from different perspectives. This paper aims to complement ALBidS strategies usage by MASCEM players, providing, through the Six Thinking Hats group decision technique, a means to combine them and take advantages from their different perspectives. The combination of the different proposals resulting from ALBidS’ strategies is performed through the application of a Genetic Algorithm, resulting in an evolutionary learning approach.

Scenarios Generation for Multi-Agent simulation of Electricity Markets based on Intelligent Data Analysis

This document presents a tool able to automatically gather data provided by real energy markets and to generate scenarios, capture and improve market players’ profiles and strategies by using knowledge discovery processes in databases supported by artificial intelligence techniques, data mining algorithms and machine learning methods. It provides the means for generating scenarios with different dimensions and characteristics, ensuring the representation of real and adapted markets, and their participating entities. The scenarios generator module enhances the MASCEM (Multi-Agent Simulator of Competitive Electricity Markets) simulator, endowing a more effective tool for decision support. The achievements from the implementation of the proposed module enables researchers and electricity markets’ participating entities to analyze data, create real scenarios and make experiments with them. On the other hand, applying knowledge discovery techniques to real data also allows the improvement of MASCEM agents’ profiles and strategies resulting in a better representation of real market players’ behavior. This work aims to improve the comprehension of electricity markets and the interactions among the involved entities through adequate multi-agent simulation.

ANN-Based LMP forecasting in a distribution network with large penetration of DG

In recent years, power systems have experienced many changes in their paradigm. The introduction of new players in the management of distributed generation leads to the decentralization of control and decision-making, so that each player is able to play in the market environment. In the new context, it will be very relevant that aggregator players allow midsize, small and micro players to act in a competitive environment. In order to achieve their objectives, virtual power players and single players are required to optimize their energy resource management process. To achieve this, it is essential to have financial resources capable of providing access to appropriate decision support tools. As small players have difficulties in having access to such tools, it is necessary that these players can benefit from alternative methodologies to support their decisions. This paper presents a methodology, based on Artificial Neural Networks (ANN), and intended to support smaller players. In this case the present methodology uses a training set that is created using energy resource scheduling solutions obtained using a mixed-integer linear programming (MIP) approach as the reference optimization methodology. The trained network is used to obtain locational marginal prices in a distribution network. The main goal of the paper is to verify the accuracy of the ANN based approach. Moreover, the use of a single ANN is compared with the use of two or more ANN to forecast the locational marginal price.

Multi-agent simulation of continental, regional, and micro electricity markets

Renewable based power generation has significantly increased over the last years. However, this process has evolved separately from electricity markets, leading to an inadequacy of the present market models to cope with huge quantities of renewable energy resources, and to take full advantage of the presently existing and the increasing envisaged renewable based and distributed energy resources. This paper proposes the modelling of electricity markets at several levels (continental, regional and micro), taking into account the specific characteristics of the players and resources involved in each level and ensuring that the proposed models accommodate adequate business models able to support the contribution of all the resources in the system, from the largest to the smaller ones. The proposed market models are integrated in MASCEM (Multi- Agent Simulator of Competitive Electricity Markets), using the multi agent approach advantages for overcoming the current inadequacy and significant limitations of the presently existing electricity market simulators to deal with the complex electricity market models that must be adopted.