Intelligent decision making in electricity markets: simulated annealing Q-Learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM is integrated with ALBidS, a system that provides several dynamic strategies for agents’ behavior. This paper presents a method that aims at enhancing ALBidS competence in endowing market players with adequate strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible actions. These actions are defined accordingly to the most probable points of bidding success. With the purpose of accelerating the convergence process, a simulated annealing based algorithm is included.

MASGriP – a Multi-Agent Smart Grid Simulation Platform

The increasing number of players that operate in power systems leads to a more complex management. In this paper a new multi-agent platform is proposed, which simulates the real operation of power system players. MASGriP – A Multi-Agent Smart Grid Simulation Platform is presented. Several consumer and producer agents are implemented and simulated, considering real characteristics and different goals and actuation strategies. Aggregator entities, such as Virtual Power Players and Curtailment Service Providers are also included. The integration of MASGriP agents in MASCEM (Multi-Agent System for Competitive Electricity Markets) simulator allows the simulation of technical and economical activities of several players. An energy resources management architecture used in microgrids is also explained.

Balancing market integration in MASCEM electricity market simulator

With the restructuring of the energy sector in industrialized countries there is an increased complexity in market players’ interactions along with emerging problems and new issues to be addressed. Decision support tools that facilitate the study and understanding of these markets are extremely useful to provide players with competitive advantage. In this context arises MASCEM, a multi-agent simulator for competitive electricity markets. It is essential to reinforce MASCEM with the ability to recreate electricity markets reality in the fullest possible extent, making it able to simulate as many types of markets models and players as possible. This paper presents the development of the Balancing Market in MASCEM. A key module to the study of competitive electricity markets, as it has well defined and distinct characteristics previously implemented.

Study of distribution network demand response events in the Portuguese system

In competitive electricity markets with deep concerns for the efficiency level, demand response programs gain considerable significance. As demand response levels have decreased after the introduction of competition in the power industry, new approaches are required to take full advantage of demand response opportunities. Grid operators and utilities are taking new initiatives, recognizing the value of demand response for grid reliability and for the enhancement of organized spot markets’ efficiency. This paper proposes a methodology for the selection of the consumers that participate in an event, which is the responsibility of the Portuguese transmission network operator. The proposed method is intended to be applied in the interruptibility service implemented in Portugal, in convergence with Spain, in the context of the Iberian electricity market. This method is based on the calculation of locational marginal prices (LMP) which are used to support the decision concerning the consumers to be schedule for participation. The proposed method has been computationally implemented and its application is illustrated in this paper using a 937 bus distribution network with more than 20,000 consumers.

Distribution network short term scheduling in Smart Grid context

The large increase of Distributed Generation (DG) in Power Systems (PS) and specially in distribution networks makes the management of distribution generation resources an increasingly important issue. Beyond DG, other resources such as storage systems and demand response must be managed in order to obtain more efficient and “green” operation of PS. More players, such as aggregators or Virtual Power Players (VPP), that operate these kinds of resources will be appearing. This paper proposes a new methodology to solve the distribution network short term scheduling problem in the Smart Grid context. This methodology is based on a Genetic Algorithms (GA) approach for energy resource scheduling optimization and on PSCAD software to obtain realistic results for power system simulation. The paper includes a case study with 99 distributed generators, 208 loads and 27 storage units. The GA results for the determination of the economic dispatch considering the generation forecast, storage management and load curtailment in each period (one hour) are compared with the ones obtained with a Mixed Integer Non-Linear Programming (MINLP) approach.

Using data mining techniques to support DR programs definition in smart grids

In recent years, Power Systems (PS) have experimented many changes in their operation. The introduction of new players managing Distributed Generation (DG) units, and the existence of new Demand Response (DR) programs make the control of the system a more complex problem and allow a more flexible management. An intelligent resource management in the context of smart grids is of huge important so that smart grids functions are assured. This paper proposes a new methodology to support system operators and/or Virtual Power Players (VPPs) to determine effective and efficient DR programs that can be put into practice. This method is based on the use of data mining techniques applied to a database which is obtained for a large set of operation scenarios. The paper includes a case study based on 27,000 scenarios considering a diversity of distributed resources in a 32 bus distribution network.

VPP´s multi-level negotiation in smart grids and competitive electricity markets

The increase of distributed generation (DG) has brought about new challenges in electrical networks electricity markets and in DG units operation and management. Several approaches are being developed to manage the emerging potential of DG, such as Virtual Power Players (VPPs), which aggregate DG plants; and Smart Grids, an approach that views generation and associated loads as a subsystem. This paper presents a multi-level negotiation mechanism for Smart Grids optimal operation and negotiation in the electricity markets, considering the advantages of VPPs’ management. The proposed methodology is implemented and tested in MASCEM – a multiagent electricity market simulator, developed to allow deep studies of the interactions between the players that take part in the electricity market negotiations.

Simulated annealing metaheuristic to solve the optimal power flow

The optimal power flow problem has been widely studied in order to improve power systems operation and planning. For real power systems, the problem is formulated as a non-linear and as a large combinatorial problem. The first approaches used to solve this problem were based on mathematical methods which required huge computational efforts. Lately, artificial intelligence techniques, such as metaheuristics based on biological processes, were adopted. Metaheuristics require lower computational resources, which is a clear advantage for addressing the problem in large power systems. This paper proposes a methodology to solve optimal power flow on economic dispatch context using a Simulated Annealing algorithm inspired on the cooling temperature process seen in metallurgy. The main contribution of the proposed method is the specific neighborhood generation according to the optimal power flow problem characteristics. The proposed methodology has been tested with IEEE 6 bus and 30 bus networks. The obtained results are compared with other wellknown methodologies presented in the literature, showing the effectiveness of the proposed method.

Ant colony search algorithm for the optimal power flow problem

To maintain a power system within operation limits, a level ahead planning it is necessary to apply competitive techniques to solve the optimal power flow (OPF). OPF is a non-linear and a large combinatorial problem. The Ant Colony Search (ACS) optimization algorithm is inspired by the organized natural movement of real ants and has been successfully applied to different large combinatorial optimization problems. This paper presents an implementation of Ant Colony optimization to solve the OPF in an economic dispatch context. The proposed methodology has been developed to be used for maintenance and repairing planning with 48 to 24 hours antecipation. The main advantage of this method is its low execution time that allows the use of OPF when a large set of scenarios has to be analyzed. The paper includes a case study using the IEEE 30 bus network. The results are compared with other well-known methodologies presented in the literature.

Power systems reliability calculation based on fuzzy data mining

This paper presents a methodology supported on the data base knowledge discovery process (KDD), in order to find out the failure probability of electrical equipments’, which belong to a real electrical high voltage network. Data Mining (DM) techniques are used to discover a set of outcome failure probability and, therefore, to extract knowledge concerning to the unavailability of the electrical equipments such us power transformers and high-voltages power lines. The framework includes several steps, following the analysis of the real data base, the pre-processing data, the application of DM algorithms, and finally, the interpretation of the discovered knowledge. To validate the proposed methodology, a case study which includes real databases is used. This data have a heavy uncertainty due to climate conditions for this reason it was used fuzzy logic to determine the set of the electrical components failure probabilities in order to reestablish the service. The results reflect an interesting potential of this approach and encourage further research on the topic.

Multi-agent based electricity market simulator with VPP: Conceptual and implementation issues

This paper presents a new architecture for the MASCEM, a multi-agent electricity market simulator. This is implemented in a Prolog which is integrated in the JAVA program by using the LPA Win-Prolog Intelligence Server (IS) provides a DLL interface between Win-Prolog and other applications. This paper mainly focus on the MASCEM ability to provide the means to model and simulate Virtual Power Producers (VPP). VPPs are represented as a coalition of agents, with specific characteristics and goals. VPPs can reinforce the importance of these generation technologies making them valuable in electricity markets.

Zonal prices analysis supported by a data mining based methodology

A methodology based on data mining techniques to support the analysis of zonal prices in real transmission networks is proposed in this paper. The mentioned methodology uses clustering algorithms to group the buses in typical classes that include a set of buses with similar LMP values. Two different clustering algorithms have been used to determine the LMP clusters: the two-step and K-means algorithms. In order to evaluate the quality of the partition as well as the best performance algorithm adequacy measurements indices are used. The paper includes a case study using a Locational Marginal Prices (LMP) data base from the California ISO (CAISO) in order to identify zonal prices.

Ancillary services market clearing simulation: a comparison between deterministic and heuristic methods

Electricity market players operating in a liberalized environment requires access to an adequate decision support tool, allowing them to consider all the business opportunities and take strategic decisions. Ancillary services represent a good negotiation opportunity that must be considered by market players. For this, decision support tool must include ancillary market simulation. This paper proposes two different methods (Linear Programming and Genetic Algorithm approaches) for ancillary services dispatch. The methodologies are implemented in MASCEM, a multi-agent based electricity market simulator. A test case based on California Independent System Operator (CAISO) data concerning the dispatch of Regulation Down, Regulation Up, Spinning Reserve and Non-Spinning Reserve services is included in this paper.