Cost dependent strategy for electricity markets bidding based on adaptive reinforcement learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM provides several dynamic strategies for agents’ behavior. This paper presents a method that aims to provide market players with strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible bids. These bids are defined accordingly to the cost function that each producer presents.

MASCEM: electricity markets simulation with strategic agents

Electricity markets are complex environments, involving numerous entities trying to obtain the best advantages and profits while limited by power-network characteristics and constraints.1 The restructuring and consequent deregulation of electricity markets introduced a new economic dimension to the power industry. Some observers have criticized the restructuring process, however, because it has failed to improve market efficiency and has complicated the assurance of reliability and fairness of operations. To study and understand this type of market, we developed the Multiagent Simulator of Competitive Electricity Markets (MASCEM) platform based on multiagent simulation. The MASCEM multiagent model includes players with strategies for bid definition, acting in forward, day-ahead, and balancing markets and considering both simple and complex bids. Our goal with MASCEM was to simulate as many market models and player types as possible. This approach makes MASCEM both a short- and mediumterm simulation as well as a tool to support long-term decisions, such as those taken by regulators. This article proposes a new methodology integrated in MASCEM for bid definition in electricity markets. This methodology uses reinforcement learning algorithms to let players perceive changes in the environment, thus helping them react to the dynamic environment and adapt their bids accordingly.

A new approach for multi-agent coalition formation and management in the scope of electricity markets

This paper presents a new methodology for the creation and management of coalitions in Electricity Markets. This approach is tested using the multi-agent market simulator MASCEM, taking advantage of its ability to provide the means to model and simulate VPP (Virtual Power Producers). VPPs are represented as coalitions of agents, with the capability of negotiating both in the market, and internally, with their members, in order to combine and manage their individual specific characteristics and goals, with the strategy and objectives of the VPP itself. The new features include the development of particular individual facilitators to manage the communications amongst the members of each coalition independently from the rest of the simulation, and also the mechanisms for the classification of the agents that are candidates to join the coalition. In addition, a global study on the results of the Iberian Electricity Market is performed, to compare and analyze different approaches for defining consistent and adequate strategies to integrate into the agents of MASCEM. This, combined with the application of learning and prediction techniques provide the agents with the ability to learn and adapt themselves, by adjusting their actions to the continued evolving states of the world they are playing in.

Renewable-based generation integration in electricity markets with virtual power producers

The development of renewable energy sources and Distributed Generation (DG) of electricity is of main importance in the way towards a sustainable development. However, the management, in large scale, of these technologies is complicated because of the intermittency of primary resources (wind, sunshine, etc.) and small scale of some plants. The aggregation of DG plants gives place to a new concept: the Virtual Power Producer (VPP). VPPs can reinforce the importance of these generation technologies making them valuable in electricity markets. VPPs can ensure a secure, environmentally friendly generation and optimal management of heat, electricity and cold as well as optimal operation and maintenance of electrical equipment, including the sale of electricity in the energy market. For attaining these goals, there are important issues to deal with, such as reserve management strategies, strategies for bids formulation, the producers’ remuneration, and the producers’ characterization for coalition formation. This chapter presents the most important concepts related with renewable-based generation integration in electricity markets, using VPP paradigm. The presented case studies make use of two main computer applications:ViProd and MASCEM. ViProd simulates VPP operation, including the management of plants in operation. MASCEM is a multi-agent based electricity market simulator that supports the inclusion of VPPs in the players set.

Bid definition method for electricity markets based on an adaptive multiagent system

Electricity markets are complex environments with very particular characteristics. MASCEM is a market simulator developed to allow deep studies of the interactions between the players that take part in the electricity market negotiations. This paper presents a new proposal for the definition of MASCEM players’ strategies to negotiate in the market. The proposed methodology is multiagent based, using reinforcement learning algorithms to provide players with the capabilities to perceive the changes in the environment, while adapting their bids formulation according to their needs, using a set of different techniques that are at their disposal. Each agent has the knowledge about a different method for defining a strategy for playing in the market, the main agent chooses the best among all those, and provides it to the market player that requests, to be used in the market. This paper also presents a methodology to manage the efficiency/effectiveness balance of this method, to guarantee that the degradation of the simulator processing times takes the correct measure.

Strategic bidding methodology for electricity markets using adaptive learning

The very particular characteristics of electricity markets, require deep studies of the interactions between the involved players. MASCEM is a market simulator developed to allow studying electricity market negotiations. This paper presents a new proposal for the definition of MASCEM players’ strategies to negotiate in the market. The proposed methodology is implemented as a multiagent system, using reinforcement learning algorithms to provide players with the capabilities to perceive the changes in the environment, while adapting their bids formulation according to their needs, using a set of different techniques that are at their disposal. This paper also presents a methodology to define players’ models based on the historic of their past actions, interpreting how their choices are affected by past experience, and competition.

Multi-agent negotiation for coalition formation and management in electricity markets

Negotiation is a fundamental tool for reaching understandings that allow each involved party to gain an advantage for themselves by the end of the process. In recent years, with the increasing of compe-titiveness in most sectors, negotiation procedures become present in practically all of them. One particular environment in which the competitiveness has been increasing exponentially is the electricity markets sector. This work is directed to the study of electricity markets’ partici-pating entities interaction, namely in what concerns the formation, management and operation of aggregating entities – Virtual Power Players (VPPs). VPPs are responsible for managing coalitions of market players with small market negotiating influence, which take strategic advantage in entering such aggregations, to increase their negotiating power. This chapter presents a negotiation methodology for the creation and management of coalitions in Electricity Markets. This approach is tested using MASCEM, taking advantage of its ability to provide the means to model and simulate VPPs. VPPs are represented as coalitions of agents, with the capability of negotiating both in the market, and internally, with their members, in order to combine and manage their individual specific characteristics and goals, with the strategy and objectives of the VPP itself.

Transmission tariffs in competitive electricity markets

In context of electricity market, the transmission price is an important tool to an efficient development of the electricity system. The electricity market is influenced by several factors; however the transmission network management is one of the most important aspects, because the network is a natural monopoly. The transmission tariffs can help to regulate the market, for that reason evaluate tariff must have strict criterions. This paper explains several methodologies to tariff the use of transmission network by transmission network users. The methods presented are: Post-Stamp Method; MW-Mile Method; Distribution Factors Methods; Tracing Methodology; Bialek’s Tracing Method and Locational Marginal Price.

Nodal price simulation in competitive electricity markets

Locational Marginal Prices (LMP) are important pricing signals for the participants of competitive electricity markets, as the effects of transmission losses and binding constraints are embedded in LMPs [1],[2]. This paper presents a software tool that evaluates the nodal marginal prices considering losses and congestion. The initial dispatch is based on all the electricity transactions negotiated in the pool and in bilateral contracts. It must be checked if the proposed initial dispatch leads to congestion problems; if a congestion situation is detected, it must be solved. An AC power flow is used to verify if there are congestion situations in the initial dispatch. Whenever congestion situations are detected, they are solved and a feasible dispatch (re-dispatch) is obtained. After solving the congestion problems, the simulator evaluates LMP. The paper presents a case study based on the the 118 IEEE bus test network.

Dgenco market balancing - cost repartition under light regulation markets

In this paper we present a new methodology, based in game theory, to obtain the market balancing between Distribution Generation Companies (DGENCO), in liberalized electricity markets. The new contribution of this methodology is the verification of the participation rate of each agent based in Nucléolo Balancing and in Shapley Value. To validate the results we use the Zaragoza Distribution Network with 42 Bus and 5 DGENCO.

A long-term swarm intelligence hedging tool applied to electricity markets

This paper proposes a swarm intelligence long-term hedging tool to support electricity producers in competitive electricity markets. This tool investigates the long-term hedging opportunities available to electric power producers through the use of contracts with physical (spot and forward) and financial (options) settlement. To find the optimal portfolio the producer risk preference is stated by a utility function (U) expressing the trade-off between the expectation and the variance of the return. Variance estimation and the expected return are based on a forecasted scenario interval determined by a long-term price range forecast model, developed by the authors, whose explanation is outside the scope of this paper. The proposed tool makes use of Particle Swarm Optimization (PSO) and its performance has been evaluated by comparing it with a Genetic Algorithm (GA) based approach. To validate the risk management tool a case study, using real price historical data for mainland Spanish market, is presented to demonstrate the effectiveness of the proposed methodology.

Electricity markets: transmission prices methods

In the context of electricity markets, transmission pricing is an important tool to achieve an efficient operation of the electricity system. The electricity market is influenced by several factors; however the transmission network management is one of the most important aspects, because the network is a natural monopoly. The transmission tariffs can help to regulate the market, for this reason transmission tariffs must follow strict criteria. This paper presents the following methods to tariff the use of transmission networks by electricity market players: Post-Stamp Method; MW-Mile Method Distribution Factors Methods; Tracing Methodology; Bialek’s Tracing Method and Locational Marginal Price. A nine bus transmission network is used to illustrate the application of the tariff methods.

A long-term risk management tool for electricity markets using swarm intelligence

This paper addresses the optimal involvement in derivatives electricity markets of a power producer to hedge against the pool price volatility. To achieve this aim, a swarm intelligence meta-heuristic optimization technique for long-term risk management tool is proposed. This tool investigates the long-term opportunities for risk hedging available for electric power producers through the use of contracts with physical (spot and forward contracts) and financial (options contracts) settlement. The producer risk preference is formulated as a utility function (U) expressing the trade-off between the expectation and the variance of the return. Variance of return and the expectation are based on a forecasted scenario interval determined by a long-term price range forecasting model. This model also makes use of particle swarm optimization (PSO) to find the best parameters allow to achieve better forecasting results. On the other hand, the price estimation depends on load forecasting. This work also presents a regressive long-term load forecast model that make use of PSO to find the best parameters as well as in price estimation. The PSO technique performance has been evaluated by comparison with a Genetic Algorithm (GA) based approach. A case study is presented and the results are discussed taking into account the real price and load historical data from mainland Spanish electricity market demonstrating the effectiveness of the methodology handling this type of problems. Finally, conclusions are dully drawn.

Impact of maritime air mass trajectories on the Western European coast urban aerosol

Lisbon is the largest urban area in the Western European coast. Due to this geographical position the Atlantic Ocean serves as an important source of particles and plays an important role in many atmospheric processes. The main objectives of this study were to (1) perform a chemical characterization of particulate matter (PM2.5) sampled in Lisbon, (2) identify the main sources of particles, (3) determine PM contribution to this urban area, and (4) assess the impact of maritime air mass trajectories on concentration and composition of respirable PM sampled in Lisbon. During 2007, PM2.5 was collected on a daily basis in the center of Lisbon with a Partisol sampler. The exposed Teflon filters were measured by gravimetry and cut into two parts: one for analysis by instrumental neutron activation analysis (INAA) and the other by ion chromatography (IC). Principal component analysis (PCA) and multilinear regression analysis (MLRA) were used to identify possible sources of PM2.5 and determine mass contribution. Five main groups of sources were identified: secondary aerosols, traffic, calcium, soil, and sea. Four-day backtracking trajectories ending in Lisbon at the starting sampling time were calculated using the HYSPLIT model. Results showed that maritime transport scenarios were frequent. These episodes were characterized by a significant decrease of anthropogenic aerosol concentrations and exerted a significant role on air quality in this urban area.

European information users: topics from a comparative analysis within the European Union

Purpose – The aim of this article is to present some results from research undertaken into the information behaviour of European Documentation Centre (EDC) users. It will reflect on the practices of a group of 234 users of 55 EDCs covering 21 Member States of the European Union (EU), used to access European information. Design/methodology/approach – In order to collect the data presented here, five questionnaires were sent to users in all the EDCs in Finland, Ireland, Hungary and Portugal. In the remaining EU countries, five questionnaires were sent to two EDCs chosen at random. The questionnaires were sent by post, following telephone contact with the EDC managers. Findings – Factors determining access to information on the European Union and the frequency of this access are identified. The information providers most commonly used to access European information and the information sources considered the most reliable by respondents will also be analysed. Another area of analysis concerns the factors cited by respondents as facilitating access to information on Europe or, conversely, making it more difficult to access. Parallel to this, the aspects of accessing information on EU that are valued most by users will also be assessed. Research limitations/implications – Questionnaires had to be used, as the intention was to cover a very extensive geographical area. However, in opting for closed questions, it is acknowledged that standard responses have been obtained with no scope for capturing the individual circumstances of each respondent, thus making a qualitative approach difficult. Practical implications – The results provide an overall picture of certain aspects of the information behaviour of EDC users. They may serve as a starting point for planning training sessions designed to develop the skills required to search, access, evaluate and apply European information within an academic context. From a broader perspective, they also constitute factors which the European Commission should take into consideration when formulating its information and communication policy. Originality/value – This is the first piece of academic research into the EDCs and their users, which aimed to cover all Members State of the EU.