Cost dependent strategy for electricity markets bidding based on adaptive reinforcement learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM provides several dynamic strategies for agents’ behavior. This paper presents a method that aims to provide market players with strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible bids. These bids are defined accordingly to the cost function that each producer presents.

MASCEM: electricity markets simulation with strategic agents

Electricity markets are complex environments, involving numerous entities trying to obtain the best advantages and profits while limited by power-network characteristics and constraints.1 The restructuring and consequent deregulation of electricity markets introduced a new economic dimension to the power industry. Some observers have criticized the restructuring process, however, because it has failed to improve market efficiency and has complicated the assurance of reliability and fairness of operations. To study and understand this type of market, we developed the Multiagent Simulator of Competitive Electricity Markets (MASCEM) platform based on multiagent simulation. The MASCEM multiagent model includes players with strategies for bid definition, acting in forward, day-ahead, and balancing markets and considering both simple and complex bids. Our goal with MASCEM was to simulate as many market models and player types as possible. This approach makes MASCEM both a short- and mediumterm simulation as well as a tool to support long-term decisions, such as those taken by regulators. This article proposes a new methodology integrated in MASCEM for bid definition in electricity markets. This methodology uses reinforcement learning algorithms to let players perceive changes in the environment, thus helping them react to the dynamic environment and adapt their bids accordingly.

A new approach for multi-agent coalition formation and management in the scope of electricity markets

This paper presents a new methodology for the creation and management of coalitions in Electricity Markets. This approach is tested using the multi-agent market simulator MASCEM, taking advantage of its ability to provide the means to model and simulate VPP (Virtual Power Producers). VPPs are represented as coalitions of agents, with the capability of negotiating both in the market, and internally, with their members, in order to combine and manage their individual specific characteristics and goals, with the strategy and objectives of the VPP itself. The new features include the development of particular individual facilitators to manage the communications amongst the members of each coalition independently from the rest of the simulation, and also the mechanisms for the classification of the agents that are candidates to join the coalition. In addition, a global study on the results of the Iberian Electricity Market is performed, to compare and analyze different approaches for defining consistent and adequate strategies to integrate into the agents of MASCEM. This, combined with the application of learning and prediction techniques provide the agents with the ability to learn and adapt themselves, by adjusting their actions to the continued evolving states of the world they are playing in.

Renewable-based generation integration in electricity markets with virtual power producers

The development of renewable energy sources and Distributed Generation (DG) of electricity is of main importance in the way towards a sustainable development. However, the management, in large scale, of these technologies is complicated because of the intermittency of primary resources (wind, sunshine, etc.) and small scale of some plants. The aggregation of DG plants gives place to a new concept: the Virtual Power Producer (VPP). VPPs can reinforce the importance of these generation technologies making them valuable in electricity markets. VPPs can ensure a secure, environmentally friendly generation and optimal management of heat, electricity and cold as well as optimal operation and maintenance of electrical equipment, including the sale of electricity in the energy market. For attaining these goals, there are important issues to deal with, such as reserve management strategies, strategies for bids formulation, the producers’ remuneration, and the producers’ characterization for coalition formation. This chapter presents the most important concepts related with renewable-based generation integration in electricity markets, using VPP paradigm. The presented case studies make use of two main computer applications:ViProd and MASCEM. ViProd simulates VPP operation, including the management of plants in operation. MASCEM is a multi-agent based electricity market simulator that supports the inclusion of VPPs in the players set.

Bid definition method for electricity markets based on an adaptive multiagent system

Electricity markets are complex environments with very particular characteristics. MASCEM is a market simulator developed to allow deep studies of the interactions between the players that take part in the electricity market negotiations. This paper presents a new proposal for the definition of MASCEM players’ strategies to negotiate in the market. The proposed methodology is multiagent based, using reinforcement learning algorithms to provide players with the capabilities to perceive the changes in the environment, while adapting their bids formulation according to their needs, using a set of different techniques that are at their disposal. Each agent has the knowledge about a different method for defining a strategy for playing in the market, the main agent chooses the best among all those, and provides it to the market player that requests, to be used in the market. This paper also presents a methodology to manage the efficiency/effectiveness balance of this method, to guarantee that the degradation of the simulator processing times takes the correct measure.

Strategic bidding methodology for electricity markets using adaptive learning

The very particular characteristics of electricity markets, require deep studies of the interactions between the involved players. MASCEM is a market simulator developed to allow studying electricity market negotiations. This paper presents a new proposal for the definition of MASCEM players’ strategies to negotiate in the market. The proposed methodology is implemented as a multiagent system, using reinforcement learning algorithms to provide players with the capabilities to perceive the changes in the environment, while adapting their bids formulation according to their needs, using a set of different techniques that are at their disposal. This paper also presents a methodology to define players’ models based on the historic of their past actions, interpreting how their choices are affected by past experience, and competition.

Multi-agent negotiation for coalition formation and management in electricity markets

Negotiation is a fundamental tool for reaching understandings that allow each involved party to gain an advantage for themselves by the end of the process. In recent years, with the increasing of compe-titiveness in most sectors, negotiation procedures become present in practically all of them. One particular environment in which the competitiveness has been increasing exponentially is the electricity markets sector. This work is directed to the study of electricity markets’ partici-pating entities interaction, namely in what concerns the formation, management and operation of aggregating entities – Virtual Power Players (VPPs). VPPs are responsible for managing coalitions of market players with small market negotiating influence, which take strategic advantage in entering such aggregations, to increase their negotiating power. This chapter presents a negotiation methodology for the creation and management of coalitions in Electricity Markets. This approach is tested using MASCEM, taking advantage of its ability to provide the means to model and simulate VPPs. VPPs are represented as coalitions of agents, with the capability of negotiating both in the market, and internally, with their members, in order to combine and manage their individual specific characteristics and goals, with the strategy and objectives of the VPP itself.

Virtual power producers simulation - negotiating renewable distributed generation in competitive electricity markets

Sustainable development concerns made renewable energy sources to be increasingly used for electricity distributed generation. However, this is mainly due to incentives or mandatory targets determined by energy policies as in European Union. Assuring a sustainable future requires distributed generation to be able to participate in competitive electricity markets. To get more negotiation power in the market and to get advantages of scale economy, distributed generators can be aggregated giving place to a new concept: the Virtual Power Producer (VPP). VPPs are multi-technology and multisite heterogeneous entities that should adopt organization and management methodologies so that they can make distributed generation a really profitable activity, able to participate in the market. This paper presents ViProd, a simulation tool that allows simulating VPPs operation, in the context of MASCEM, a multi-agent based eletricity market simulator.

Transmission tariffs in competitive electricity markets

In context of electricity market, the transmission price is an important tool to an efficient development of the electricity system. The electricity market is influenced by several factors; however the transmission network management is one of the most important aspects, because the network is a natural monopoly. The transmission tariffs can help to regulate the market, for that reason evaluate tariff must have strict criterions. This paper explains several methodologies to tariff the use of transmission network by transmission network users. The methods presented are: Post-Stamp Method; MW-Mile Method; Distribution Factors Methods; Tracing Methodology; Bialek’s Tracing Method and Locational Marginal Price.

Seller, buyer and VPP agents behavior on a simulated electricity market

Competitive electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is an electricity market simulator able to model market players and simulate their operation in the market. As market players are complex entities, having their characteristics and objectives, making their decisions and interacting with other players, a multi-agent architecture is used and proved to be adequate. MASCEM players have learning capabilities and different risk preferences. They are able to refine their strategies according to their past experience (both real and simulated) and considering other agents’ behavior. Agents’ behavior is also subject to its risk preferences.

Nodal price simulation in competitive electricity markets

Locational Marginal Prices (LMP) are important pricing signals for the participants of competitive electricity markets, as the effects of transmission losses and binding constraints are embedded in LMPs [1],[2]. This paper presents a software tool that evaluates the nodal marginal prices considering losses and congestion. The initial dispatch is based on all the electricity transactions negotiated in the pool and in bilateral contracts. It must be checked if the proposed initial dispatch leads to congestion problems; if a congestion situation is detected, it must be solved. An AC power flow is used to verify if there are congestion situations in the initial dispatch. Whenever congestion situations are detected, they are solved and a feasible dispatch (re-dispatch) is obtained. After solving the congestion problems, the simulator evaluates LMP. The paper presents a case study based on the the 118 IEEE bus test network.

Multi-agent based electricity market simulator with VPP: Conceptual and implementation issues

This paper presents a new architecture for the MASCEM, a multi-agent electricity market simulator. This is implemented in a Prolog which is integrated in the JAVA program by using the LPA Win-Prolog Intelligence Server (IS) provides a DLL interface between Win-Prolog and other applications. This paper mainly focus on the MASCEM ability to provide the means to model and simulate Virtual Power Producers (VPP). VPPs are represented as a coalition of agents, with specific characteristics and goals. VPPs can reinforce the importance of these generation technologies making them valuable in electricity markets.

MASCEM - an electricity market simulator providing coalition support for virtual power players

This paper presents MASCEM - a multi-agent based electricity market simulator. MASCEM uses game theory, machine learning techniques, scenario analysis and optimization techniques to model market agents and to provide them with decision-support. This paper mainly focus on the MASCEM ability to provide the means to model and simulate Virtual Power Players (VPP). VPPs are represented as a coalition of agents, with specific characteristics and goals. The paper details some of the most important aspects considered in VPP formation and in the aggregation of new producers and includes a case study based on real data.

ANN based day-ahead spinning reserve forecast for electricity market simulation

Electricity market players operating in a liberalized environment require adequate decision support tools, allowing them to consider all the business opportunities and take strategic decisions. Ancillary services represent a good negotiation opportunity that must be considered by market players. This paper deals with short-term predication of day-ahead spinning reserve (SR) requirement that helps the ISO to make effective and timely decisions. Based on these forecasted information, market participants can use strategic bidding for day-ahead SR market. The proposed concepts and methodologies are implemented in MASCEM, a multi-agent based electricity market simulator. A case study based on California ISO (CAISO) data is included; the forecasted results are presented and compared with CAISO published forecast.

Electricity markets: transmission prices methods

In the context of electricity markets, transmission pricing is an important tool to achieve an efficient operation of the electricity system. The electricity market is influenced by several factors; however the transmission network management is one of the most important aspects, because the network is a natural monopoly. The transmission tariffs can help to regulate the market, for this reason transmission tariffs must follow strict criteria. This paper presents the following methods to tariff the use of transmission networks by electricity market players: Post-Stamp Method; MW-Mile Method Distribution Factors Methods; Tracing Methodology; Bialek’s Tracing Method and Locational Marginal Price. A nine bus transmission network is used to illustrate the application of the tariff methods.