Intelligent decision making in electricity markets: simulated annealing Q-Learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM is integrated with ALBidS, a system that provides several dynamic strategies for agents’ behavior. This paper presents a method that aims at enhancing ALBidS competence in endowing market players with adequate strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible actions. These actions are defined accordingly to the most probable points of bidding success. With the purpose of accelerating the convergence process, a simulated annealing based algorithm is included.

MASGriP – a Multi-Agent Smart Grid Simulation Platform

The increasing number of players that operate in power systems leads to a more complex management. In this paper a new multi-agent platform is proposed, which simulates the real operation of power system players. MASGriP – A Multi-Agent Smart Grid Simulation Platform is presented. Several consumer and producer agents are implemented and simulated, considering real characteristics and different goals and actuation strategies. Aggregator entities, such as Virtual Power Players and Curtailment Service Providers are also included. The integration of MASGriP agents in MASCEM (Multi-Agent System for Competitive Electricity Markets) simulator allows the simulation of technical and economical activities of several players. An energy resources management architecture used in microgrids is also explained.

Balancing market integration in MASCEM electricity market simulator

With the restructuring of the energy sector in industrialized countries there is an increased complexity in market players’ interactions along with emerging problems and new issues to be addressed. Decision support tools that facilitate the study and understanding of these markets are extremely useful to provide players with competitive advantage. In this context arises MASCEM, a multi-agent simulator for competitive electricity markets. It is essential to reinforce MASCEM with the ability to recreate electricity markets reality in the fullest possible extent, making it able to simulate as many types of markets models and players as possible. This paper presents the development of the Balancing Market in MASCEM. A key module to the study of competitive electricity markets, as it has well defined and distinct characteristics previously implemented.

Dynamic artificial neural network for electricity market prices forecast

This paper presents an artificial neural network applied to the forecasting of electricity market prices, with the special feature of being dynamic. The dynamism is verified at two different levels. The first level is characterized as a re-training of the network in every iteration, so that the artificial neural network can able to consider the most recent data at all times, and constantly adapt itself to the most recent happenings. The second level considers the adaptation of the neural network’s execution time depending on the circumstances of its use. The execution time adaptation is performed through the automatic adjustment of the amount of data considered for training the network. This is an advantageous and indispensable feature for this neural network’s integration in ALBidS (Adaptive Learning strategic Bidding System), a multi-agent system that has the purpose of providing decision support to the market negotiating players of MASCEM (Multi-Agent Simulator of Competitive Electricity Markets).

A multi-agent based approach for intelligent smart grid management

The spread and globalization of distributed generation (DG) in recent years has should highly influence the changes that occur in Electricity Markets (EMs). DG has brought a large number of new players to take action in the EMs, therefore increasing the complexity of these markets. Simulation based on multi-agent systems appears as a good way of analyzing players’ behavior and interactions, especially in a coalition perspective, and the effects these players have on the markets. MASCEM – Multi-Agent System for Competitive Electricity Markets was created to permit the study of the market operation with several different players and market mechanisms. MASGriP – Multi-Agent Smart Grid Platform is being developed to facilitate the simulation of micro grid (MG) and smart grid (SG) concepts with multiple different scenarios. This paper presents an intelligent management method for MG and SG. The simulation of different methods of control provides an advantage in comparing different possible approaches to respond to market events. Players utilize electric vehicles’ batteries and participate in Demand Response (DR) contracts, taking advantage on the best opportunities brought by the use of all resources, to improve their actions in response to MG and/or SG requests.

A simulation tool for demand response programs implementation

The design and development of simulation models and tools for Demand Response (DR) programs are becoming more and more important for adequately taking the maximum advantages of DR programs use. Moreover, a more active consumers’ participation in DR programs can help improving the system reliability and decrease or defer the required investments. DemSi, a DR simulator, designed and implemented by the authors of this paper, allows studying DR actions and schemes in distribution networks. It undertakes the technical validation of the solution using realistic network simulation based on PSCAD. DemSi considers the players involved in DR actions, and the results can be analyzed from each specific player point of view.

Multi-agent simulation of continental, regional, and micro electricity markets

Renewable based power generation has significantly increased over the last years. However, this process has evolved separately from electricity markets, leading to an inadequacy of the present market models to cope with huge quantities of renewable energy resources, and to take full advantage of the presently existing and the increasing envisaged renewable based and distributed energy resources. This paper proposes the modelling of electricity markets at several levels (continental, regional and micro), taking into account the specific characteristics of the players and resources involved in each level and ensuring that the proposed models accommodate adequate business models able to support the contribution of all the resources in the system, from the largest to the smaller ones. The proposed market models are integrated in MASCEM (Multi- Agent Simulator of Competitive Electricity Markets), using the multi agent approach advantages for overcoming the current inadequacy and significant limitations of the presently existing electricity market simulators to deal with the complex electricity market models that must be adopted.

Energy efficiency analysis considering the use of an intelligent systems management in a smart home

In this abstract is presented an energy management system included in a SCADA system existent in a intelligent home. The system control the home energy resources according to the players definitions (electricity consumption and comfort levels), the electricity prices variation in real time mode and the DR events proposed by the aggregators.

Using demand response to deal with unexpected low wind power generation in the context of smart grid

Demand response is assumed an essential resource to fully achieve the smart grids operating benefits, namely in the context of competitive markets. Some advantages of Demand Response (DR) programs and of smart grids can only be achieved through the implementation of Real Time Pricing (RTP). The integration of the expected increasing amounts of distributed energy resources, as well as new players, requires new approaches for the changing operation of power systems. The methodology proposed aims the minimization of the operation costs in a smart grid operated by a virtual power player. It is especially useful when actual and day ahead wind forecast differ significantly. When facing lower wind power generation than expected, RTP is used in order to minimize the impacts of such wind availability change. The proposed model application is here illustrated using the scenario of a special wind availability reduction day in the Portuguese power system (8th February 2012).

A regulatory framework for short-term stochastic maintenance outage scheduling in smart grids

The current regulatory framework for maintenance outage scheduling in distribution systems needs revision to face the challenges of future smart grids. In the smart grid context, generation units and the system operator perform new roles with different objectives, and an efficient coordination between them becomes necessary. In this paper, the distribution system operator (DSO) of a microgrid receives the proposals for shortterm (ST) planned outages from the generation and transmission side, and has to decide the final outage plans, which is mandatory for the members to follow. The framework is based on a coordination procedure between the DSO and other market players. This paper undertakes the challenge of optimization problem in a smart grid where the operator faces with uncertainty. The results show the effectiveness and applicability of the proposed regulatory framework in the modified IEEE 34- bus test system.

Combined heat and power and consumption optimization in a SCADA-based system

The operation of power systems in a Smart Grid (SG) context brings new opportunities to consumers as active players, in order to fully reach the SG advantages. In this context, concepts as smart homes or smart buildings are promising approaches to perform the optimization of the consumption, while reducing the electricity costs. This paper proposes an intelligent methodology to support the consumption optimization of an industrial consumer, which has a Combined Heat and Power (CHP) facility. A SCADA (Supervisory Control and Data Acquisition) system developed by the authors is used to support the implementation of the proposed methodology. An optimization algorithm implemented in the system in order to perform the determination of the optimal consumption and CHP levels in each instant, according to the Demand Response (DR) opportunities. The paper includes a case study with several scenarios of consumption and heat demand in the context of a DR event which specifies a maximum demand level for the consumer.

Electricity Markets Simulation: MASCEM Contributions to the Challenging Reality

Electricity Markets are not only a new reality but an evolving one as the involved players and rules change at a relatively high rate. Multi-agent simulation combined with Artificial Intelligence techniques may result in sophisticated tools very helpful under this context. Some simulation tools have already been developed, some of them very interesting. However, at the present state it is important to go a step forward in Electricity Markets simulators as this is crucial for facing changes in Power Systems. This paper explains the context and needs of electricity market simulation, describing the most important characteristics of available simulators. We present our work concerning MASCEM simulator, presenting its features as well as the improvements being made to accomplish the change and challenging reality of Electricity Markets.

Multi-agent electricity market simulation with dynamic strategies & virtual power producers

Distributed energy resources will provide a significant amount of the electricity generation and will be a normal profitable business. In the new decentralized grid, customers will be among the many decentralized players and may even help to co-produce the required energy services such as demand-side management and load shedding. So, they will gain the opportunity to be more active market players. The aggregation of DG plants gives place to a new concept: the Virtual Power Producer (VPP). VPPs can reinforce the importance of these generation technologies making them valuable in electricity markets. In this paper we propose the improvement of MASCEM, a multi-agent simulation tool to study negotiations in electricity spot markets based on different market mechanisms and behavior strategies, in order to take account of decentralized players such as VPP.

Particle swarm optimization applied to integrated demand response resources scheduling

The concept of demand response has a growing importance in the context of the future power systems. Demand response can be seen as a resource like distributed generation, storage, electric vehicles, etc. All these resources require the existence of an infrastructure able to give players the means to operate and use them in an efficient way. This infrastructure implements in practice the smart grid concept, and should accommodate a large number of diverse types of players in the context of a competitive business environment. In this paper, demand response is optimally scheduled jointly with other resources such as distributed generation units and the energy provided by the electricity market, minimizing the operation costs from the point of view of a virtual power player, who manages these resources and supplies the aggregated consumers. The optimal schedule is obtained using two approaches based on particle swarm optimization (with and without mutation) which are compared with a deterministic approach that is used as a reference methodology. A case study with two scenarios implemented in DemSi, a demand Response simulator developed by the authors, evidences the advantages of the use of the proposed particle swarm approaches.

Multiagent system for adaptive strategy formulation in electricity markets

Electricity markets are complex environments with very particular characteristics. MASCEM is a market simulator developed to allow deep studies of the interactions between the players that take part in the electricity market negotiations. This paper presents a new proposal for the definition of MASCEM players’ strategies to negotiate in the market. The proposed methodology is multiagent based, using reinforcement learning algorithms to provide players with the capabilities to perceive the changes in the environment, while adapting their bids formulation according to their needs, using a set of different techniques that are at their disposal.