An automatic tool to Extract, Transform and Load data from real electricity markets

The study of Electricity Markets operation has been gaining an increasing importance in the last years, as result of the new challenges that the restructuring produced. Currently, lots of information concerning Electricity Markets is available, as market operators provide, after a period of confidentiality, data regarding market proposals and transactions. These data can be used as source of knowledge, to define realistic scenarios, essential for understanding and forecast Electricity Markets behaviour. The development of tools able to extract, transform, store and dynamically update data, is of great importance to go a step further into the comprehension of Electricity Markets and the behaviour of the involved entities. In this paper we present an adaptable tool capable of downloading, parsing and storing data from market operators’ websites, assuring actualization and reliability of stored data.

Multi-agent Simulation of Bilateral Contracting in Competitive Electricity Markets

Traditional vertically integrated power utilities around the world have evolved from monopoly structures to open markets that promote competition among suppliers and provide consumers with a choice of services. Market forces drive the price of electricity and reduce the net cost through increased competition. Electricity can be traded in both organized markets or using forward bilateral contracts. This article focuses on bilateral contracts and describes some important features of an agent-based system for bilateral trading in competitive markets. Special attention is devoted to the negotiation process, demand response in bilateral contracting, and risk management. The article also presents a case study on forward bilateral contracting: a retailer agent and a customer agent negotiate a 24h-rate tariff.

Realistic Multi-Agent Simulation of Competitive Electricity Markets

The dynamism and ongoing changes that the electricity markets sector is constantly suffering, enhanced by the huge increase in competitiveness, create the need of using simulation platforms to support operators, regulators, and the involved players in understanding and dealing with this complex environment. This paper presents an enhanced electricity market simulator, based on multi-agent technology, which provides an advanced simulation framework for the study of real electricity markets operation, and the interactions between the involved players. MASCEM (Multi-Agent Simulator of Competitive Electricity Markets) uses real data for the creation of realistic simulation scenarios, which allow the study of the impacts and implications that electricity markets transformations bring to different countries. Also, the development of an upper-ontology to support the communication between participating agents, provides the means for the integration of this simulator with other frameworks, such as MAN-REM (Multi-Agent Negotiation and Risk Management in Electricity Markets). A case study using the enhanced simulation platform that results from the integration of several systems and different tools is presented, with a scenario based on real data, simulating the MIBEL electricity market environment, and comparing the simulation performance with the real electricity market results.

Data Mining Approach to support the Generation of Realistic Scenarios for Multi-Agent simulation of Electricity Markets

This paper presents the Realistic Scenarios Generator (RealScen), a tool that processes data from real electricity markets to generate realistic scenarios that enable the modeling of electricity market players’ characteristics and strategic behavior. The proposed tool provides significant advantages to the decision making process in an electricity market environment, especially when coupled with a multi-agent electricity markets simulator. The generation of realistic scenarios is performed using mechanisms for intelligent data analysis, which are based on artificial intelligence and data mining algorithms. These techniques allow the study of realistic scenarios, adapted to the existing markets, and improve the representation of market entities as software agents, enabling a detailed modeling of their profiles and strategies. This work contributes significantly to the understanding of the interactions between the entities acting in electricity markets by increasing the capability and realism of market simulations.

Electricity markets modeling considering complex contracts and aggregators

All over the world, the liberalization of electricity markets, which follows different paradigms, has created new challenges for those involved in this sector. In order to respond to these challenges, electric power systems suffered a significant restructuring in its mode of operation and planning. This restructuring resulted in a considerable increase of the electric sector competitiveness. Particularly, the Ancillary Services (AS) market has been target of constant renovations in its operation mode as it is a targeted market for the trading of services, which have as main objective to ensure the operation of electric power systems with appropriate levels of stability, safety, quality, equity and competitiveness. In this way, with the increasing penetration of distributed energy resources including distributed generation, demand response, storage units and electric vehicles, it is essential to develop new smarter and hierarchical methods of operation of electric power systems. As these resources are mostly connected to the distribution network, it is important to consider the introduction of this kind of resources in AS delivery in order to achieve greater reliability and cost efficiency of electrical power systems operation. The main contribution of this work is the design and development of mechanisms and methodologies of AS market and for energy and AS joint market, considering different management entities of transmission and distribution networks. Several models developed in this work consider the most common AS in the liberalized market environment: Regulation Down; Regulation Up; Spinning Reserve and Non-Spinning Reserve. The presented models consider different rules and ways of operation, such as the division of market by network areas, which allows the congestion management of interconnections between areas; or the ancillary service cascading process, which allows the replacement of AS of superior quality by lower quality of AS, ensuring a better economic performance of the market. A major contribution of this work is the development an innovative methodology of market clearing process to be used in the energy and AS joint market, able to ensure viable and feasible solutions in markets, where there are technical constraints in the transmission network involving its division into areas or regions. The proposed method is based on the determination of Bialek topological factors and considers the contribution of the dispatch for all services of increase of generation (energy, Regulation Up, Spinning and Non-Spinning reserves) in network congestion. The use of Bialek factors in each iteration of the proposed methodology allows limiting the bids in the market while ensuring that the solution is feasible in any context of system operation. Another important contribution of this work is the model of the contribution of distributed energy resources in the ancillary services. In this way, a Virtual Power Player (VPP) is considered in order to aggregate, manage and interact with distributed energy resources. The VPP manages all the agents aggregated, being able to supply AS to the system operator, with the main purpose of participation in electricity market. In order to ensure their participation in the AS, the VPP should have a set of contracts with the agents that include a set of diversified and adapted rules to each kind of distributed resource. All methodologies developed and implemented in this work have been integrated into the MASCEM simulator, which is a simulator based on a multi-agent system that allows to study complex operation of electricity markets. In this way, the developed methodologies allow the simulator to cover more operation contexts of the present and future of the electricity market. In this way, this dissertation offers a huge contribution to the AS market simulation, based on models and mechanisms currently used in several real markets, as well as the introduction of innovative methodologies of market clearing process on the energy and AS joint market. This dissertation presents five case studies; each one consists of multiple scenarios. The first case study illustrates the application of AS market simulation considering several bids of market players. The energy and ancillary services joint market simulation is exposed in the second case study. In the third case study it is developed a comparison between the simulation of the joint market methodology, in which the player bids to the ancillary services is considered by network areas and a reference methodology. The fourth case study presents the simulation of joint market methodology based on Bialek topological distribution factors applied to transmission network with 7 buses managed by a TSO. The last case study presents a joint market model simulation which considers the aggregation of small players to a VPP, as well as complex contracts related to these entities. The case study comprises a distribution network with 33 buses managed by VPP, which comprises several kinds of distributed resources, such as photovoltaic, CHP, fuel cells, wind turbines, biomass, small hydro, municipal solid waste, demand response, and storage units.

Negotiation context analysis in electricity markets

Contextualization is critical in every decision making process. Adequate responses to problems depend not only on the variables with direct influence on the outcomes, but also on a correct contextualization of the problem regarding the surrounding environment. Electricity markets are dynamic environments with increasing complexity, potentiated by the last decades' restructuring process. Dealing with the growing complexity and competitiveness in this sector brought the need for using decision support tools. A solid example is MASCEM (Multi-Agent Simulator of Competitive Electricity Markets), whose players' decisions are supported by another multiagent system – ALBidS (Adaptive Learning strategic Bidding System). ALBidS uses artificial intelligence techniques to endow market players with adaptive learning capabilities that allow them to achieve the best possible results in market negotiations. This paper studies the influence of context awareness in the decision making process of agents acting in electricity markets. A context analysis mechanism is proposed, considering important characteristics of each negotiation period, so that negotiating agents can adapt their acting strategies to different contexts. The main conclusion is that context-dependant responses improve the decision making process. Suiting actions to different contexts allows adapting the behaviour of negotiating entities to different circumstances, resulting in profitable outcomes.

Multi-agent simulation of competitive electricity markets: Autonomous systems cooperation for European market modeling

The electricity market restructuring, and its worldwide evolution into regional and even continental scales, along with the increasing necessity for an adequate integration of renewable energy sources, is resulting in a rising complexity in power systems operation. Several power system simulators have been developed in recent years with the purpose of helping operators, regulators, and involved players to understand and deal with this complex and constantly changing environment. The main contribution of this paper is given by the integration of several electricity market and power system models, respecting to the reality of different countries. This integration is done through the development of an upper ontology which integrates the essential concepts necessary to interpret all the available information. The continuous development of Multi-Agent System for Competitive Electricity Markets platform provides the means for the exemplification of the usefulness of this ontology. A case study using the proposed multi-agent platform is presented, considering a scenario based on real data that simulates the European Electricity Market environment, and comparing its performance using different market mechanisms. The main goal is to demonstrate the advantages that the integration of various market models and simulation platforms have for the study of the electricity markets’ evolution.

Six thinking hats: A novel metalearner for intelligent decision support in electricity markets

The energy sector has suffered a significant restructuring that has increased the complexity in electricity market players' interactions. The complexity that these changes brought requires the creation of decision support tools to facilitate the study and understanding of these markets. The Multiagent Simulator of Competitive Electricity Markets (MASCEM) arose in this context, providing a simulation framework for deregulated electricity markets. The Adaptive Learning strategic Bidding System (ALBidS) is a multiagent system created to provide decision support to market negotiating players. Fully integrated with MASCEM, ALBidS considers several different strategic methodologies based on highly distinct approaches. Six Thinking Hats (STH) is a powerful technique used to look at decisions from different perspectives, forcing the thinker to move outside its usual way of thinking. This paper aims to complement the ALBidS strategies by combining them and taking advantage of their different perspectives through the use of the STH group decision technique. The combination of ALBidS' strategies is performed through the application of a genetic algorithm, resulting in an evolutionary learning approach.

Ontologies for the interoperability of multiagent electricity markets simulation platforms

Electricity markets worldwide are complex and dynamic environments with very particular characteristics. These are the result of electricity markets’ restructuring and evolution into regional and continental scales, along with the constant changes brought by the increasing necessity for an adequate integration of renewable energy sources. The rising complexity and unpredictability in electricity markets has increased the need for the intervenient entities in foreseeing market behaviour. Market players and regulators are very interested in predicting the market’s behaviour. Market players need to understand the market behaviour and operation in order to maximize their profits, while market regulators need to test new rules and detect market inefficiencies before they are implemented. The growth of usage of simulation tools was driven by the need for understanding those mechanisms and how the involved players' interactions affect the markets' outcomes. Multi-agent based software is particularly well fitted to analyse dynamic and adaptive systems with complex interactions among its constituents, such as electricity markets. Several modelling tools directed to the study of restructured wholesale electricity markets have emerged. Still, they have a common limitation: the lack of interoperability between the various systems to allow the exchange of information and knowledge, to test different market models and to allow market players from different systems to interact in common market environments. This dissertation proposes the development and implementation of ontologies for semantic interoperability between multi-agent simulation platforms in the scope of electricity markets. The added value provided to these platforms is given by enabling them sharing their knowledge and market models with other agent societies, which provides the means for an actual improvement in current electricity markets studies and development. The proposed ontologies are implemented in MASCEM (Multi-Agent Simulator of Competitive Electricity Markets) and tested through the interaction between MASCEM agents and agents from other multi-agent based simulators. The implementation of the proposed ontologies has also required a complete restructuring of MASCEM’s architecture and multi-agent model, which is also presented in this dissertation. The results achieved in the case studies allow identifying the advantages of the novel architecture of MASCEM, and most importantly, the added value of using the proposed ontologies. They facilitate the integration of independent multi-agent simulators, by providing a way for communications to be understood by heterogeneous agents from the various systems.

Analisis De Mercados De Energía Eléctrica Competitivos Con Participación De Energía Eólica. Competitive Electricity Markets Analisis with Wind Power Generation.

La utilización de energía eólica es un hecho cada vez más común en nuestro mundo como respuesta a mitigar el creciente aumento de demanda de energía, los aumentos constantes de precio, la escasez de combustibles fósiles y los impactos del cambio climático, los que son cada día más evidentes.Consecuentemente, el interés por la participación de esta nueva forma de generación de energía en sistema eléctrico de potencia ha aumentado considerablemente en los últimos años. La incorporación de generación de origen eólico en el sistema eléctrico de potencia requiere de un análisis detallado del sistema eléctrico en su conjunto, considerando la interacción entre parques y unidades de generación eólica, plantas de generación convencional y el sistema eléctrico de potencia. La integración de generación de origen renovable en el sistema eléctrico de potencia convencional presenta nuevos desafíos los que pueden ser atribuidos a características propias de este tipo de generación, por ejemplo la fluctuación de energía debido a la naturaleza variable del viento, la naturaleza distribuida de la generación eólica y las características constructivas y método de conexión de los distintos modelos de turbinas eólicas al sistema.La finalidad de este proyecto de investigación consiste en investigar el impacto sobre un mercado de sistema eléctrico competitivo causado por el agregado de generación de origen eolico. Como punto de partida se pretende realizar modelos de plantas de generacion eolica para luego incorporarlos a los modelos de sistemas eléctricos y realizar estudios de de despacho económico, flujo de cargas, análisis transitorio y estudios dinámicos del sistema.

Asymmetric demand information in uniform and discriminatory call auctions: an experimental analysis motivated by electricity markets

We study the outcomes of experimental multi-unit uniform and discriminatory auctions with demand uncertainty. Our study is motivated by the ongoing debate about market design in the electricity industry. Our main aim is to compare the effect of asymmetric demand-information between sellers on the performance of the two auction institutions. In our baseline conditions all sellers have the same information, whereas in our treatment conditions some sellers have better information than others. In both information conditions we find that average transaction prices and price volatility are not significantly different under the two auction institutions. However, when there is asymmetric information among sellers the discriminatory auction is significantly less efficient. These results are not in line with the typical arguments made in favor of discriminatory pricing in electricity industries; namely, lower consumer prices and less price volatility. Moreover, our results provide some indication that discriminatory auctions reduce technical efficiency relative to uniform auctions.

Stochastic optimal bid to electricity markets with environmental risk constraints

There are many factors that influence the day-ahead market bidding strategies of a generation company (GenCo) in the current energy market framework. Environmental policy issues have become more and more important for fossil-fuelled power plants and they have to be considered in their management, giving rise to emission limitations. This work allows to investigate the influence of both the allowances and emission reduction plan, and the incorporation of the derivatives medium-term commitments in the optimal generation bidding strategy to the day-ahead electricity market. Two different technologies have been considered: the coal thermal units, high-emission technology, and the combined cycle gas turbine units, low-emission technology. The Iberian Electricity Market and the Spanish National Emissions and Allocation Plans are the framework to deal with the environmental issues in the day-ahead market bidding strategies. To address emission limitations, some of the standard risk management methodologies developed for financial markets, such as Value-at-Risk (VaR) and Conditional Value-at-Risk (CVaR), have been extended. This study offers to electricity generation utilities a mathematical model to determinate the individual optimal generation bid to the wholesale electricity market, for each one of their generation units that maximizes the long-run profits of the utility abiding by the Iberian Electricity Market rules, the environmental restrictions set by the EU Emission Trading Scheme, as well as the restrictions set by the Spanish National Emissions Reduction Plan. The economic implications for a GenCo of including the environmental restrictions of these National Plans are analyzed and the most remarkable results will be presented.

Generation and interconnection capacity expansion in cross-border electricity markets : the need for policy coordination

Electricity is a strategic service in modern societies. Thus, it is extremely important for governments to be able to guarantee an affordable and reliable supply, which depends to a great extent on an adequate expansion of the generation and transmission capacities. Cross- border integration of electricity markets creates new challenges for the regulators, since the evolution of the market is now influenced by the characteristics and policies of neighbouring countries. There is still no agreement on why and how regions should integrate their electricity markets. The aim of this thesis is to improve the understanding of integrated electricity markets and how their behaviour depends on the prevailing characteristics of the national markets and the policies implemented in each country. We developed a simulation model to analyse under what circumstances integration is desirable. This model is used to study three cases of interconnection between two countries. Several policies regarding interconnection expansion and operation, combined with different generation capacity adequacy mechanisms, are evaluated. The thesis is composed of three papers. The first paper presents a detailed description of the model and an analysis of the case of Colombia and Ecuador. It shows that market coupling can bring important benefits, but the relative size of the countries can lead to import dependency issues in the smaller country. The second paper compares the case of Colombia and Ecuador with the case of Great Britain and France. These countries are significantly different in terms of electricity sources, hydro- storage capacity, complementarity and demand growth. We show that complementarity is essential in order to obtain benefits from integration, while higher demand growth and hydro- storage capacity can lead to counterintuitive outcomes, thus complicating policy design. In the third paper, an extended version of the model presented in the first paper is used to analyse the case of Finland and its interconnection with Russia. Different trading arrangements are considered. We conclude that unless interconnection capacity is expanded, the current trading arrangement, where a single trader owns the transmission rights and limits the flow during peak hours, is beneficial for Finland. In case of interconnection expansion, market coupling would be preferable. We also show that the costs of maintaining a strategic reserve in Finland are justified in order to limit import dependency, while still reaping the benefits of interconnection. In general, we conclude that electricity market integration can bring benefits if the right policies are implemented. However, a large interconnection capacity is only desirable if the countries exhibit significant complementarity and trust each other. The outcomes of policies aimed at guaranteeing security of supply at a national level can be quite counterintuitive due to the interactions between neighbouring countries and their effects on interconnection and generation investments. Thus, it is important for regulators to understand these interactions and coordinate their decisions in order to take advantage of the interconnection without putting security of supply at risk. But it must be taken into account that even when integration brings benefits to the region, some market participants lose and might try to hinder the integration process. -- Dans les sociétés modernes, l'électricité est un service stratégique. Il est donc extrêmement important pour les gouvernements de pouvoir garantir la sécurité d'approvisionnement à des prix abordables. Ceci dépend en grande mesure d'une expansion adéquate des capacités de génération et de transmission. L'intégration des marchés électriques pose des nouveaux défis pour les régulateurs, puisque l'évolution du marché est maintenant influencée par les caractéristiques et les politiques des pays voisins. Il n'est pas encore claire pourquoi ni comment les marches électriques devraient s'intégrer. L'objectif de cette thèse est d'améliorer la compréhension des marchés intégrés d'électricité et de leur comportement en fonction des caractéristiques et politiques de chaque pays. Un modèle de simulation est proposé pour étudier les conditions dans lesquelles l'intégration est désirable. Ce modèle est utilisé pour étudier trois cas d'interconnexion entre deux pays. Plusieurs politiques concernant l'expansion et l'opération de l'interconnexion, combinées avec différents mécanismes de rémunération de la capacité, sont évalués. Cette thèse est compose de trois articles. Le premier présente une description détaillée du modèle et une analyse du cas de la Colombie et de l'Equateur. Il montre que le couplage de marchés peut amener des bénéfices importants ; cependant, la différence de taille entre pays peut créer des soucis de dépendance aux importations pour le pays le plus petit. Le second papier compare le cas de la Colombie et l'Equateur avec le cas de la Grande Bretagne et de la France. Ces pays sont très différents en termes de ressources, taille des réservoirs d'accumulation pour l'hydro, complémentarité et croissance de la demande. Nos résultats montrent que la complémentarité joue un rôle essentiel dans l'obtention des bénéfices potentiels de l'intégration, alors qu'un taux élevé de croissance de la demande, ainsi qu'une grande capacité de stockage, mènent à des résultats contre-intuitifs, ce qui complique les décisions des régulateurs. Dans le troisième article, une extension du modèle présenté dans le premier article est utilisée pour analyser le cas de la Finlande et de la Russie. Différentes règles pour les échanges internationaux d'électricité sont considérées. Nos résultats indiquent qu'à un faible niveau d'interconnexion, la situation actuelle, où un marchand unique possède les droits de transmission et limite le flux pendant les heures de pointe, est bénéfique pour la Finlande. Cependant, en cas d'expansion de la capacité d'interconnexion, «market coupling» est préférable. préférable. Dans tous les cas, la Finlande a intérêt à garder une réserve stratégique, car même si cette politique entraine des coûts, elle lui permet de profiter des avantages de l'intégration tout en limitant ca dépendance envers les importations. En général, nous concluons que si les politiques adéquates sont implémentées, l'intégration des marchés électriques peut amener des bénéfices. Cependant, une grande capacité d'interconnexion n'est désirable que si les pays ont une complémentarité importante et il existe une confiance mutuelle. Les résultats des politiques qui cherchent à préserver la sécurité d'approvisionnement au niveau national peuvent être très contre-intuitifs, étant données les interactions entre les pays voisins et leurs effets sur les investissements en génération et en interconnexion. Il est donc très important pour les régulateurs de comprendre ces interactions et de coordonner décisions à fin de pouvoir profiter de l'interconnexion sans mettre en danger la sécurité d'approvisionnement. Mais il faut être conscients que même quand l'intégration amène de bénéfices pour la région, certains participants au marché sont perdants et pourraient essayer de bloquer le processus d'intégration.