Cost dependent strategy for electricity markets bidding based on adaptive reinforcement learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM provides several dynamic strategies for agents’ behavior. This paper presents a method that aims to provide market players with strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible bids. These bids are defined accordingly to the cost function that each producer presents.

MASCEM: electricity markets simulation with strategic agents

Electricity markets are complex environments, involving numerous entities trying to obtain the best advantages and profits while limited by power-network characteristics and constraints.1 The restructuring and consequent deregulation of electricity markets introduced a new economic dimension to the power industry. Some observers have criticized the restructuring process, however, because it has failed to improve market efficiency and has complicated the assurance of reliability and fairness of operations. To study and understand this type of market, we developed the Multiagent Simulator of Competitive Electricity Markets (MASCEM) platform based on multiagent simulation. The MASCEM multiagent model includes players with strategies for bid definition, acting in forward, day-ahead, and balancing markets and considering both simple and complex bids. Our goal with MASCEM was to simulate as many market models and player types as possible. This approach makes MASCEM both a short- and mediumterm simulation as well as a tool to support long-term decisions, such as those taken by regulators. This article proposes a new methodology integrated in MASCEM for bid definition in electricity markets. This methodology uses reinforcement learning algorithms to let players perceive changes in the environment, thus helping them react to the dynamic environment and adapt their bids accordingly.

A new approach for multi-agent coalition formation and management in the scope of electricity markets

This paper presents a new methodology for the creation and management of coalitions in Electricity Markets. This approach is tested using the multi-agent market simulator MASCEM, taking advantage of its ability to provide the means to model and simulate VPP (Virtual Power Producers). VPPs are represented as coalitions of agents, with the capability of negotiating both in the market, and internally, with their members, in order to combine and manage their individual specific characteristics and goals, with the strategy and objectives of the VPP itself. The new features include the development of particular individual facilitators to manage the communications amongst the members of each coalition independently from the rest of the simulation, and also the mechanisms for the classification of the agents that are candidates to join the coalition. In addition, a global study on the results of the Iberian Electricity Market is performed, to compare and analyze different approaches for defining consistent and adequate strategies to integrate into the agents of MASCEM. This, combined with the application of learning and prediction techniques provide the agents with the ability to learn and adapt themselves, by adjusting their actions to the continued evolving states of the world they are playing in.

Virtual power producers simulation - negotiating renewable distributed generation in competitive electricity markets

Sustainable development concerns made renewable energy sources to be increasingly used for electricity distributed generation. However, this is mainly due to incentives or mandatory targets determined by energy policies as in European Union. Assuring a sustainable future requires distributed generation to be able to participate in competitive electricity markets. To get more negotiation power in the market and to get advantages of scale economy, distributed generators can be aggregated giving place to a new concept: the Virtual Power Producer (VPP). VPPs are multi-technology and multisite heterogeneous entities that should adopt organization and management methodologies so that they can make distributed generation a really profitable activity, able to participate in the market. This paper presents ViProd, a simulation tool that allows simulating VPPs operation, in the context of MASCEM, a multi-agent based eletricity market simulator.

Seller, buyer and VPP agents behavior on a simulated electricity market

Competitive electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is an electricity market simulator able to model market players and simulate their operation in the market. As market players are complex entities, having their characteristics and objectives, making their decisions and interacting with other players, a multi-agent architecture is used and proved to be adequate. MASCEM players have learning capabilities and different risk preferences. They are able to refine their strategies according to their past experience (both real and simulated) and considering other agents’ behavior. Agents’ behavior is also subject to its risk preferences.

Multi-agent based electricity market simulator with VPP: Conceptual and implementation issues

This paper presents a new architecture for the MASCEM, a multi-agent electricity market simulator. This is implemented in a Prolog which is integrated in the JAVA program by using the LPA Win-Prolog Intelligence Server (IS) provides a DLL interface between Win-Prolog and other applications. This paper mainly focus on the MASCEM ability to provide the means to model and simulate Virtual Power Producers (VPP). VPPs are represented as a coalition of agents, with specific characteristics and goals. VPPs can reinforce the importance of these generation technologies making them valuable in electricity markets.

MASCEM - an electricity market simulator providing coalition support for virtual power players

This paper presents MASCEM - a multi-agent based electricity market simulator. MASCEM uses game theory, machine learning techniques, scenario analysis and optimization techniques to model market agents and to provide them with decision-support. This paper mainly focus on the MASCEM ability to provide the means to model and simulate Virtual Power Players (VPP). VPPs are represented as a coalition of agents, with specific characteristics and goals. The paper details some of the most important aspects considered in VPP formation and in the aggregation of new producers and includes a case study based on real data.

ANN based day-ahead spinning reserve forecast for electricity market simulation

Electricity market players operating in a liberalized environment require adequate decision support tools, allowing them to consider all the business opportunities and take strategic decisions. Ancillary services represent a good negotiation opportunity that must be considered by market players. This paper deals with short-term predication of day-ahead spinning reserve (SR) requirement that helps the ISO to make effective and timely decisions. Based on these forecasted information, market participants can use strategic bidding for day-ahead SR market. The proposed concepts and methodologies are implemented in MASCEM, a multi-agent based electricity market simulator. A case study based on California ISO (CAISO) data is included; the forecasted results are presented and compared with CAISO published forecast.

Auction of financial transmission rights in electricity market environment

An auction model is used to increase the individual profits for market players with products they do not use. A Financial Transmission Rights Auction has the goal of trade transmission rights between Bidders and helps them raise their own profits. The ISO plays a major rule on keep the system in technical limits without interfere on the auctions offers. In some auction models the ISO decide want bids are implemented on the network, always with the objective maximize the individual profits for all bidders in the auction. This paper proposes a methodology for a Financial Transmission Rights Auction and an informatics application. The application receives offers from the purchase and sale side and considers bilateral contracts as Base Case. This goal is maximize the individual profits within the system in their technical limits. The paper includes a case study for the 30 bus IEEE test case.

ANN based day-ahead ancillary services forecast for electricity market simulation

Adequate decision support tools are required by electricity market players operating in a liberalized environment, allowing them to consider all the business opportunities and take strategic decisions. Ancillary services (AS) represent a good negotiation opportunity that must be considered by market players. Based on the ancillary services forecasting, market participants can use strategic bidding for day-ahead ancillary services markets. For this reason, ancillary services market simulation is being included in MASCEM, a multi-agent based electricity market simulator that can be used by market players to test and enhance their bidding strategies. The paper presents the methodology used to undertake ancillary services forecasting, based on an Artificial Neural Network (ANN) approach. ANNs are used to day-ahead prediction of non-spinning reserve (NS), regulation-up (RU), and regulation down (RD). Spinning reserve (SR) is mentioned as past work for comparative analysis. A case study based on California ISO (CAISO) data is included; the forecasted results are presented and compared with CAISO published forecast.

A long-term risk management tool for electricity markets using swarm intelligence

This paper addresses the optimal involvement in derivatives electricity markets of a power producer to hedge against the pool price volatility. To achieve this aim, a swarm intelligence meta-heuristic optimization technique for long-term risk management tool is proposed. This tool investigates the long-term opportunities for risk hedging available for electric power producers through the use of contracts with physical (spot and forward contracts) and financial (options contracts) settlement. The producer risk preference is formulated as a utility function (U) expressing the trade-off between the expectation and the variance of the return. Variance of return and the expectation are based on a forecasted scenario interval determined by a long-term price range forecasting model. This model also makes use of particle swarm optimization (PSO) to find the best parameters allow to achieve better forecasting results. On the other hand, the price estimation depends on load forecasting. This work also presents a regressive long-term load forecast model that make use of PSO to find the best parameters as well as in price estimation. The PSO technique performance has been evaluated by comparison with a Genetic Algorithm (GA) based approach. A case study is presented and the results are discussed taking into account the real price and load historical data from mainland Spanish electricity market demonstrating the effectiveness of the methodology handling this type of problems. Finally, conclusions are dully drawn.

Adaptive learning in agents behaviour: a framework for electricity markets simulation

Electricity markets are complex environments with very particular characteristics. A critical issue regarding these specific characteristics concerns the constant changes they are subject to. This is a result of the electricity markets’ restructuring, which was performed so that the competitiveness could be increased, but it also had exponential implications in the increase of the complexity and unpredictability in those markets scope. The constant growth in markets unpredictability resulted in an amplified need for market intervenient entities in foreseeing market behaviour. The need for understanding the market mechanisms and how the involved players’ interaction affects the outcomes of the markets, contributed to the growth of usage of simulation tools. Multi-agent based software is particularly well fitted to analyze dynamic and adaptive systems with complex interactions among its constituents, such as electricity markets. This dissertation presents ALBidS – Adaptive Learning strategic Bidding System, a multiagent system created to provide decision support to market negotiating players. This system is integrated with the MASCEM electricity market simulator, so that its advantage in supporting a market player can be tested using cases based on real markets’ data. ALBidS considers several different methodologies based on very distinct approaches, to provide alternative suggestions of which are the best actions for the supported player to perform. The approach chosen as the players’ actual action is selected by the employment of reinforcement learning algorithms, which for each different situation, simulation circumstances and context, decides which proposed action is the one with higher possibility of achieving the most success. Some of the considered approaches are supported by a mechanism that creates profiles of competitor players. These profiles are built accordingly to their observed past actions and reactions when faced with specific situations, such as success and failure. The system’s context awareness and simulation circumstances analysis, both in terms of results performance and execution time adaptation, are complementary mechanisms, which endow ALBidS with further adaptation and learning capabilities.

Stochastic short-term maintenance scheduling of GENCOs in an oligopolistic electricity market

In the proposed model, the independent system operator (ISO) provides the opportunity for maintenance outage rescheduling of generating units before each short-term (ST) time interval. Long-term (LT) scheduling for 1 or 2 years in advance is essential for the ISO and the generation companies (GENCOs) to decide their LT strategies; however, it is not possible to be exactly followed and requires slight adjustments. The Cournot-Nash equilibrium is used to characterize the decision-making procedure of an individual GENCO for ST intervals considering the effective coordination with LT plans. Random inputs, such as parameters of the demand function of loads, hourly demand during the following ST time interval and the expected generation pattern of the rivals, are included as scenarios in the stochastic mixed integer program defined to model the payoff-maximizing objective of a GENCO. Scenario reduction algorithms are used to deal with the computational burden. Two reliability test systems were chosen to illustrate the effectiveness of the proposed model for the ST decision-making process for future planned outages from the point of view of a GENCO.

Mixed integer non linear programming and artificial neural network based approach to ancillary services dispatch in competitive electricity markets

Ancillary services represent a good business opportunity that must be considered by market players. This paper presents a new methodology for ancillary services market dispatch. The method considers the bids submitted to the market and includes a market clearing mechanism based on deterministic optimization. An Artificial Neural Network is used for day-ahead prediction of Regulation Down, regulation-up, Spin Reserve and Non-Spin Reserve requirements. Two test cases based on California Independent System Operator data concerning dispatch of Regulation Down, Regulation Up, Spin Reserve and Non-Spin Reserve services are included in this paper to illustrate the application of the proposed method: (1) dispatch considering simple bids; (2) dispatch considering complex bids.

Evaluation of dynamic pass-through of carbon prices into electricity prices – a cointegrated VECM analysis

This paper addresses the impact of the CO2 opportunity cost on the wholesale electricity price in the context of the Iberian electricity market (MIBEL), namely on the Portuguese system, for the period corresponding to the Phase II of the European Union Emission Trading Scheme (EU ETS). In the econometric analysis a vector error correction model (VECM) is specified to estimate both long–run equilibrium relations and short–run interactions between the electricity price and the fuel (natural gas and coal) and carbon prices. The model is estimated using daily spot market prices and the four commodities prices are jointly modelled as endogenous variables. Moreover, a set of exogenous variables is incorporated in order to account for the electricity demand conditions (temperature) and the electricity generation mix (quantity of electricity traded according the technology used). The outcomes for the Portuguese electricity system suggest that the dynamic pass–through of carbon prices into electricity prices is strongly significant and a long–run elasticity was estimated (equilibrium relation) that is aligned with studies that have been conducted for other markets.