Intelligent decision making in electricity markets: simulated annealing Q-Learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM is integrated with ALBidS, a system that provides several dynamic strategies for agents’ behavior. This paper presents a method that aims at enhancing ALBidS competence in endowing market players with adequate strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible actions. These actions are defined accordingly to the most probable points of bidding success. With the purpose of accelerating the convergence process, a simulated annealing based algorithm is included.

Balancing market integration in MASCEM electricity market simulator

With the restructuring of the energy sector in industrialized countries there is an increased complexity in market players’ interactions along with emerging problems and new issues to be addressed. Decision support tools that facilitate the study and understanding of these markets are extremely useful to provide players with competitive advantage. In this context arises MASCEM, a multi-agent simulator for competitive electricity markets. It is essential to reinforce MASCEM with the ability to recreate electricity markets reality in the fullest possible extent, making it able to simulate as many types of markets models and players as possible. This paper presents the development of the Balancing Market in MASCEM. A key module to the study of competitive electricity markets, as it has well defined and distinct characteristics previously implemented.

Dynamic artificial neural network for electricity market prices forecast

This paper presents an artificial neural network applied to the forecasting of electricity market prices, with the special feature of being dynamic. The dynamism is verified at two different levels. The first level is characterized as a re-training of the network in every iteration, so that the artificial neural network can able to consider the most recent data at all times, and constantly adapt itself to the most recent happenings. The second level considers the adaptation of the neural network’s execution time depending on the circumstances of its use. The execution time adaptation is performed through the automatic adjustment of the amount of data considered for training the network. This is an advantageous and indispensable feature for this neural network’s integration in ALBidS (Adaptive Learning strategic Bidding System), a multi-agent system that has the purpose of providing decision support to the market negotiating players of MASCEM (Multi-Agent Simulator of Competitive Electricity Markets).

Remuneration and tariffs in the context of virtual power players

Power systems have been through deep changes in recent years, namely with the operation of competitive electricity markets in the scope and the increasingly intensive use of renewable energy sources and distributed generation. This requires new business models able to cope with the new opportunities that have emerged. Virtual Power Players (VPPs) are a new player type which allows aggregating a diversity of players (Distributed Generation (DG), Storage Agents (SA), Electrical Vehicles, (V2G) and consumers), to facilitate their participation in the electricity markets and to provide a set of new services promoting generation and consumption efficiency, while improving players` benefits. A major task of VPPs is the remuneration of generation and services (maintenance, market operation costs and energy reserves), as well as charging energy consumption. This paper proposes a model to implement fair and strategic remuneration and tariff methodologies, able to allow efficient VPP operation and VPP goals accomplishment in the scope of electricity markets.

Adaptive tool for automatic data collection of real electricity markets

The study of electricity markets operation has been gaining an increasing importance in last years, as result of the new challenges that the electricity markets restructuring produced. This restructuring increased the competitiveness of the market, but with it its complexity. The growing complexity and unpredictability of the market’s evolution consequently increases the decision making difficulty. Therefore, the intervenient entities are forced to rethink their behaviour and market strategies. Currently, lots of information concerning electricity markets is available. These data, concerning innumerous regards of electricity markets operation, is accessible free of charge, and it is essential for understanding and suitably modelling electricity markets. This paper proposes a tool which is able to handle, store and dynamically update data. The development of the proposed tool is expected to be of great importance to improve the comprehension of electricity markets and the interactions among the involved entities.

Self-optimization module for scheduling using case-based reasoning

Metaheuristics performance is highly dependent of the respective parameters which need to be tuned. Parameter tuning may allow a larger flexibility and robustness but requires a careful initialization. The process of defining which parameters setting should be used is not obvious. The values for parameters depend mainly on the problem, the instance to be solved, the search time available to spend in solving the problem, and the required quality of solution. This paper presents a learning module proposal for an autonomous parameterization of Metaheuristics, integrated on a Multi-Agent System for the resolution of Dynamic Scheduling problems. The proposed learning module is inspired on Autonomic Computing Self-Optimization concept, defining that systems must continuously and proactively improve their performance. For the learning implementation it is used Case-based Reasoning, which uses previous similar data to solve new cases. In the use of Case-based Reasoning it is assumed that similar cases have similar solutions. After a literature review on topics used, both AutoDynAgents system and Self-Optimization module are described. Finally, a computational study is presented where the proposed module is evaluated, obtained results are compared with previous ones, some conclusions are reached, and some future work is referred. It is expected that this proposal can be a great contribution for the self-parameterization of Metaheuristics and for the resolution of scheduling problems on dynamic environments.

Cooperative scheduling system with emergent swarm based behavior

This paper presents a Swarm based Cooperation Mechanism for scheduling optimization. We intend to conceptualize real manufacturing systems as interacting autonomous entities in order to support decision making in agile manufacturing environments. Agents coordinate their actions automatically without human supervision considering a common objective – global scheduling solution taking advantages from collective behavior of species through implicit and explicit cooperation. The performance of the cooperation mechanism will be evaluated consider implicit cooperation at first stage through ACS, PSO and ABC algorithms and explicit through cooperation mechanism application.

Intelligent Bio-Inspired system for manufacturing scheduling under uncertainties

A novel approach to scheduling resolution by combining Autonomic Computing (AC), Multi-Agent Systems (MAS), Case-based Reasoning (CBR), and Bio-Inspired Optimization Techniques (BIT) will be described. AC has emerged as a paradigm aiming at incorporating applications with a management structure similar to the central nervous system. The main intentions are to improve resource utilization and service quality. In this paper we envisage the use of MAS paradigm for supporting dynamic and distributed scheduling in Manufacturing Systems with AC properties, in order to reduce the complexity of managing manufacturing systems and human interference. The proposed CBR based Intelligent Scheduling System was evaluated under different dynamic manufacturing scenarios.

Meta-heuristics self-configuration for scheduling

Scheduling resolution requires the intervention of highly skilled human problemsolvers. This is a very hard and challenging domain because current systems are becoming more and more complex, distributed, interconnected and subject to rapidly changing. A natural Autonomic Computing evolution in relation to Current Computing is to provide systems with Self-Managing ability with a minimum human interference. This paper addresses the resolution of complex scheduling problems using cooperative negotiation. A Multi-Agent Autonomic and Meta-heuristics based framework with self-configuring capabilities is proposed.

Self-optimization for dynamic scheduling in manufacturing systems

Scheduling is a critical function that is present throughout many industries and applications. A great need exists for developing scheduling approaches that can be applied to a number of different scheduling problems with significant impact on performance of business organizations. A challenge is emerging in the design of scheduling support systems for manufacturing environments where dynamic adaptation and optimization become increasingly important. In this paper, we describe a Self-Optimizing Mechanism for Scheduling System through Nature Inspired Optimization Techniques (NIT).

A hybrid intelligent system for distributed dynamic scheduling

This chapter addresses the resolution of dynamic scheduling by means of meta-heuristic and multi-agent systems. Scheduling is an important aspect of automation in manufacturing systems. Several contributions have been proposed, but the problem is far from being solved satisfactorily, especially if scheduling concerns real world applications. The proposed multi-agent scheduling system assumes the existence of several resource agents (which are decision-making entities based on meta-heuristics) distributed inside the manufacturing system that interact with other agents in order to obtain optimal or near-optimal global performances.

Intelligent supervisory control system for home devices using a cyber physical approach

Although we have many electric devices at home, there are just few systems to evaluate, monitor and control them. Sometimes users go out and leave their electric devices turned on what can cause energy wasting and dangerous situations. Therefore most of the users may want to know the using states of their electrical appliances through their mobile devices in a pervasive way. In this paper, we propose an Intelligent Supervisory Control System to evaluate, monitor and control the use of electric devices in home, from outside. Because of the transferring data to evaluate, monitor and control user's location and state of home (ex. nobody at home) may be opened to attacks leading to dangerous situations. In our model we include a location privacy module and encryption module to provide security to user location and data. Intelligent Supervising Control System gives to the user the ability to manage electricity loads by means of a multi-agent system involving evaluation, monitoring, control and energy resource agents.

Electricity Markets Simulation: MASCEM Contributions to the Challenging Reality

Electricity Markets are not only a new reality but an evolving one as the involved players and rules change at a relatively high rate. Multi-agent simulation combined with Artificial Intelligence techniques may result in sophisticated tools very helpful under this context. Some simulation tools have already been developed, some of them very interesting. However, at the present state it is important to go a step forward in Electricity Markets simulators as this is crucial for facing changes in Power Systems. This paper explains the context and needs of electricity market simulation, describing the most important characteristics of available simulators. We present our work concerning MASCEM simulator, presenting its features as well as the improvements being made to accomplish the change and challenging reality of Electricity Markets.

Metalearning in ALBidS: A Strategic Bidding System for electricity markets

Metalearning is a subfield of machine learning with special pro-pensity for dynamic and complex environments, from which it is difficult to extract predictable knowledge. The field of study of this work is the electricity market, which due to the restructuring that recently took place, became an especially complex and unpredictable environment, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. This paper presents the development of a metalearner, applied to the decision support of electricity markets’ negotia-tion entities. The proposed metalearner takes advantage on several learning algorithms implemented in ALBidS, an adaptive learning system that pro-vides decision support to electricity markets’ participating players. Using the outputs of each different strategy as inputs, the metalearner creates its own output, considering each strategy with a different weight, depending on its individual quality of performance. The results of the proposed meth-od are studied and analyzed using MASCEM - a multi-agent electricity market simulator that models market players and simulates their operation in the market. This simulator provides the chance to test the metalearner in scenarios based on real electricity market´s data.

Adaptive learning in multiagent systems: a forecasting methodology based on error analysis

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simu-lator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM pro-vides several dynamic strategies for agents’ behaviour. This paper presents a method that aims to provide market players strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses an auxiliary forecasting tool, e.g. an Artificial Neural Net-work, to predict the electricity market prices, and analyses its forecasting error patterns. Through the recognition of such patterns occurrence, the method predicts the expected error for the next forecast, and uses it to adapt the actual forecast. The goal is to approximate the forecast to the real value, reducing the forecasting error.