Híper-heurísticas com aprendizagem

A otimização nos sistemas de suporte à decisão atuais assume um carácter fortemente interdisciplinar relacionando-se com a necessidade de integração de diferentes técnicas e paradigmas na resolução de problemas reais complexos, sendo que a computação de soluções ótimas em muitos destes problemas é intratável. Os métodos de pesquisa heurística são conhecidos por permitir obter bons resultados num intervalo temporal aceitável. Muitas vezes, necessitam que a parametrização seja ajustada de forma a permitir obter bons resultados. Neste sentido, as estratégias de aprendizagem podem incrementar o desempenho de um sistema, dotando-o com a capacidade de aprendizagem, por exemplo, qual a técnica de otimização mais adequada para a resolução de uma classe particular de problemas, ou qual a parametrização mais adequada de um dado algoritmo num determinado cenário. Alguns dos métodos de otimização mais usados para a resolução de problemas do mundo real resultaram da adaptação de ideias de várias áreas de investigação, principalmente com inspiração na natureza - Meta-heurísticas. O processo de seleção de uma Meta-heurística para a resolução de um dado problema é em si um problema de otimização. As Híper-heurísticas surgem neste contexto como metodologias eficientes para selecionar ou gerar heurísticas (ou Meta-heurísticas) na resolução de problemas de otimização NP-difícil. Nesta dissertação pretende-se dar uma contribuição para o problema de seleção de Metaheurísticas respetiva parametrização. Neste sentido é descrita a especificação de uma Híperheurística para a seleção de técnicas baseadas na natureza, na resolução do problema de escalonamento de tarefas em sistemas de fabrico, com base em experiência anterior. O módulo de Híper-heurística desenvolvido utiliza um algoritmo de aprendizagem por reforço (QLearning), que permite dotar o sistema da capacidade de seleção automática da Metaheurística a usar no processo de otimização, assim como a respetiva parametrização. Finalmente, procede-se à realização de testes computacionais para avaliar a influência da Híper- Heurística no desempenho do sistema de escalonamento AutoDynAgents. Como conclusão genérica, é possível afirmar que, dos resultados obtidos é possível concluir existir vantagem significativa no desempenho do sistema quando introduzida a Híper-heurística baseada em QLearning.

Gestão Inteligente de uma Habitação considerando o Controlo de Cargas, de Microgeração e Veículos Elétricos em Função do seu Perfil de Utilização

Os consumidores finais são vistos, no novo paradigma da operação das redes elétricas, como intervenientes ativos com capacidade para gerir os seus recursos energéticos, nomeadamente as cargas, as unidades de produção, os veículos elétricos e a participação em eventos de Demand Response. Tem sido evidente um aumento do consumo de energia, sendo que o setor residencial representa uma importante parte do consumo global dos países desenvolvidos. Para que a participação ativa dos consumidores seja possível, várias abordagens têm vindo a ser propostas, com ênfase nas Smart Grids e nas Microgrids. Diversos sistemas têm sido propostos e desenvolvidos com o intuito de tornar a operação dos sistemas elétricos mais flexível. Neste contexto, os sistemas de gestão de instalações domésticas apresentam-se como um elemento fulcral para a participação ativa dos consumidores na gestão energética, permitindo aos operadores de sistema coordenarem a produção mas também a procura. No entanto, é importante identificar as vantagens da implementação e uso de sistemas de gestão de energia elétrica para os consumidores finais. Nesta dissertação são propostas metodologias de apoio ao consumidor doméstico na gestão dos recursos energéticos existentes e a implementação das mesmas na plataforma de simulação de um sistema de gestão de energia desenvolvido para consumidores domésticos, o SCADA House Intelligent Management (SHIM). Para tal, foi desenvolvida uma interface que permite a simulação em laboratório do sistema de gestão desenvolvido. Adicionalmente, o SHIM foi incluído no simulador Multi-Agent Smart Grid Simulation Plataform (MASGriP) permitindo a simulação de cenários considerando diferentes agentes. Ao nível das metodologias desenvolvidas são propostos diferentes algoritmos de gestão dos recursos energéticos existentes numa habitação, considerando utilizadores com diferentes tipos de recursos (cargas; cargas e veículos elétricos; cargas, veículos elétricos e microgeração). Adicionalmente é proposto um método de gestão dinâmica das cargas para eventos de Demand Response de longa duração, considerando as características técnicas dos equipamentos. Nesta dissertação são apresentados cinco casos de estudos em que cada um deles tem diferentes cenários de simulação. Estes casos de estudos são importantes para verificar a viabilidade da implementação das metodologias propostas para o SHIM. Adicionalmente são apresentados na dissertação perfis reais dos vários recursos energéticos e de consumidores domésticos que são, posteriormente, utilizados para o desenvolvimento dos casos de estudo e aplicação das metodologias.

Intelligent Remuneration and Tariffs for Virtual Power Players

Power systems have been through deep changes in recent years, namely due to the operation of competitive electricity markets in the scope the increasingly intensive use of renewable energy sources and distributed generation. This requires new business models able to cope with the new opportunities that have emerged. Virtual Power Players (VPPs) are a new type of player that allows aggregating a diversity of players (Distributed Generation (DG), Storage Agents (SA), Electrical Vehicles (V2G) and consumers) to facilitate their participation in the electricity markets and to provide a set of new services promoting generation and consumption efficiency, while improving players’ benefits. A major task of VPPs is the remuneration of generation and services (maintenance, market operation costs and energy reserves), as well as charging energy consumption. This paper proposes a model to implement fair and strategic remuneration and tariff methodologies, able to allow efficient VPP operation and VPP goals accomplishment in the scope of electricity markets.

Smart Grid Market Using Joint Energy and Ancillary Services Bids

The power systems operation in the smart grid context increases significantly the complexity of their management. New approaches for ancillary services procurement are essential to ensure the operation of electric power systems with appropriate levels of stability, safety, quality, equity and competitiveness. These approaches should include market mechanisms which allow the participation of small and medium distributed energy resources players in a competitive market environment. In this paper, an energy and ancillary services joint market model used by an aggregator is proposed, considering bids of several types of distributed energy resources. In order to improve economic efficiency in the market, ancillary services cascading market mechanism is also considered in the model. The proposed model is included in MASCEM – a multi-agent system electricity market simulator. A case study considering a distribution network with high penetration of distributed energy resources is presented.

Virtual Power Players Internal Negotiation and Management in MASCEM

Electricity Markets are not only a new reality but an evolving one as the involved players and rules change at a relatively high rate. Multi-agent simulation combined with Artificial Intelligence techniques may result in very helpful sophisticated tools. This paper presents a new methodology for the management of coalitions in electricity markets. This approach is tested using the multi-agent market simulator MASCEM (Multi-Agent Simulator of Competitive Electricity Markets), taking advantage of its ability to provide the means to model and simulate Virtual Power Players (VPP). VPPs are represented as coalitions of agents, with the capability of negotiating both in the market and internally, with their members in order to combine and manage their individual specific characteristics and goals, with the strategy and objectives of the VPP itself. A case study using real data from the Iberian Electricity Market is performed to validate and illustrate the proposed approach.

Contextual Intelligent Load Management Considering Real-Time Pricing in a Smart Grid Environment

The use of demand response programs enables the adequate use of resources of small and medium players, bringing high benefits to the smart grid, and increasing its efficiency. One of the difficulties to proceed with this paradigm is the lack of intelligence in the management of small and medium size players. In order to make demand response programs a feasible solution, it is essential that small and medium players have an efficient energy management and a fair optimization mechanism to decrease the consumption without heavy loss of comfort, making it acceptable for the users. This paper addresses the application of real-time pricing in a house that uses an intelligent optimization module involving artificial neural networks.

Integration in MASCEM of the Joint Dispatch of Energy and Reserves Provided by Generation and Demand Resources

The provision of reserves in power systems is of great importance in what concerns keeping an adequate and acceptable level of security and reliability. This need for reserves and the way they are defined and dispatched gain increasing importance in the present and future context of smart grids and electricity markets due to their inherent competitive environment. This paper concerns a methodology proposed by the authors, which aims to jointly and optimally dispatch both generation and demand response resources to provide the amounts of reserve required for the system operation. Virtual Power Players are especially important for the aggregation of small size demand response and generation resources. The proposed methodology has been implemented in MASCEM, a multi agent system also developed at the authors’ research center for the simulation of electricity markets.

Reinforcement Learning Based on the Bayesian Theorem for Electricity Markets Decision Support

This paper presents the applicability of a reinforcement learning algorithm based on the application of the Bayesian theorem of probability. The proposed reinforcement learning algorithm is an advantageous and indispensable tool for ALBidS (Adaptive Learning strategic Bidding System), a multi-agent system that has the purpose of providing decision support to electricity market negotiating players. ALBidS uses a set of different strategies for providing decision support to market players. These strategies are used accordingly to their probability of success for each different context. The approach proposed in this paper uses a Bayesian network for deciding the most probably successful action at each time, depending on past events. The performance of the proposed methodology is tested using electricity market simulations in MASCEM (Multi-Agent Simulator of Competitive Electricity Markets). MASCEM provides the means for simulating a real electricity market environment, based on real data from real electricity market operators.

Strategic Bidding for Electricity Markets Negotiation Using Support Vector Machines

Energy systems worldwide are complex and challenging environments. Multi-agent based simulation platforms are increasing at a high rate, as they show to be a good option to study many issues related to these systems, as well as the involved players at act in this domain. In this scope the authors’ research group has developed a multi-agent system: MASCEM (Multi- Agent System for Competitive Electricity Markets), which simulates the electricity markets environment. MASCEM is integrated with ALBidS (Adaptive Learning Strategic Bidding System) that works as a decision support system for market players. The ALBidS system allows MASCEM market negotiating players to take the best possible advantages from the market context. This paper presents the application of a Support Vector Machines (SVM) based approach to provide decision support to electricity market players. This strategy is tested and validated by being included in ALBidS and then compared with the application of an Artificial Neural Network, originating promising results. The proposed approach is tested and validated using real electricity markets data from MIBEL - Iberian market operator.

Elspot: Nord Pool Spot Integration in MASCEM Electricity Market Simulator

The energy sector in industrialized countries has been restructured in the last years, with the purpose of decreasing electricity prices through the increase in competition, and facilitating the integration of distributed energy resources. However, the restructuring process increased the complexity in market players' interactions and generated emerging problems and new issues to be addressed. In order to provide players with competitive advantage in the market, decision support tools that facilitate the study and understanding of these markets become extremely useful. In this context arises MASCEM (Multi-Agent Simulator of Competitive Electricity Markets), a multi-agent based simulator that models real electricity markets. To reinforce MASCEM with the capability of recreating the electricity markets reality in the fullest possible extent, it is crucial to make it able to simulate as many market models and player types as possible. This paper presents a new negotiation model implemented in MASCEM based on the negotiation model used in day-ahead market (Elspot) of Nord Pool. This is a key module to study competitive electricity markets, as it presents well defined and distinct characteristics from the already implemented markets, and it is a reference electricity market in Europe (the one with the larger amount of traded power).

Particle Swarm Optimization of Electricity Market Negotiating Players Portfolio

Energy systems worldwide are complex and challenging environments. Multi-agent based simulation platforms are increasing at a high rate, as they show to be a good option to study many issues related to these systems, as well as the involved players at act in this domain. In this scope the authors’ research group has developed a multi-agent system: MASCEM (Multi- Agent System for Competitive Electricity Markets), which performs realistic simulations of the electricity markets. MASCEM is integrated with ALBidS (Adaptive Learning Strategic Bidding System) that works as a decision support system for market players. The ALBidS system allows MASCEM market negotiating players to take the best possible advantages from each market context. However, it is still necessary to adequately optimize the players’ portfolio investment. For this purpose, this paper proposes a market portfolio optimization method, based on particle swarm optimization, which provides the best investment profile for a market player, considering different market opportunities (bilateral negotiation, market sessions, and operation in different markets) and the negotiation context such as the peak and off-peak periods of the day, the type of day (business day, weekend, holiday, etc.) and most important, the renewable based distributed generation forecast. The proposed approach is tested and validated using real electricity markets data from the Iberian operator – MIBEL.

Metalearner based on Dynamic Neural Network for Strategic Bidding in electricity Markets

The restructuring of electricity markets, conducted to increase the competition in this sector, and decrease the electricity prices, brought with it an enormous increase in the complexity of the considered mechanisms. The electricity market became a complex and unpredictable environment, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. Software tools became, therefore, essential to provide simulation and decision support capabilities, in order to potentiate the involved players’ actions. This paper presents the development of a metalearner, applied to the decision support of electricity markets’ negotiation entities. The proposed metalearner executes a dynamic artificial neural network to create its own output, taking advantage on several learning algorithms implemented in ALBidS, an adaptive learning system that provides decision support to electricity markets’ players. The proposed metalearner considers different weights for each strategy, depending on its individual quality of performance. The results of the proposed method are studied and analyzed in scenarios based on real electricity markets’ data, using MASCEM - a multi-agent electricity market simulator that simulates market players’ operation in the market.

Electricity Markets Portfolio Optimization using a Particle Swarm Approach

Energy systems worldwide are complex and challenging environments. Multi-agent based simulation platforms are increasing at a high rate, as they show to be a good option to study many issues related to these systems, as well as the involved players at act in this domain. In this scope the authors’ research group has developed a multi-agent system: MASCEM (Multi-Agent System for Competitive Electricity Markets), which simulates the electricity markets. MASCEM is integrated with ALBidS (Adaptive Learning Strategic Bidding System) that works as a decision support system for market players. The ALBidS system allows MASCEM market negotiating players to take the best possible advantages from the market context. However, it is still necessary to adequately optimize the player’s portfolio investment. For this purpose, this paper proposes a market portfolio optimization method, based on particle swarm optimization, which provides the best investment profile for a market player, considering the different markets the player is acting on in each moment, and depending on different contexts of negotiation, such as the peak and offpeak periods of the day, and the type of day (business day, weekend, holiday, etc.). The proposed approach is tested and validated using real electricity markets data from the Iberian operator – OMIE.

Domestic Consumption Simulation and Management Using a Continuous Consumption Management and Optimization Algorithm

The recent changes concerning the consumers’ active participation in the efficient management of load devices for one’s own interest and for the interest of the network operator, namely in the context of demand response, leads to the need for improved algorithms and tools. A continuous consumption optimization algorithm has been improved in order to better manage the shifted demand. It has been done in a simulation and user-interaction tool capable of being integrated in a multi-agent smart grid simulator already developed, and also capable of integrating several optimization algorithms to manage real and simulated loads. The case study of this paper enhances the advantages of the proposed algorithm and the benefits of using the developed simulation and user interaction tool.

Towards a unified European electricity market: The contribution of data-mining to support realistic simulation studies

Worldwide electricity markets have been evolving into regional and even continental scales. The aim at an efficient use of renewable based generation in places where it exceeds the local needs is one of the main reasons. A reference case of this evolution is the European Electricity Market, where countries are connected, and several regional markets were created, each one grouping several countries, and supporting transactions of huge amounts of electrical energy. The continuous transformations electricity markets have been experiencing over the years create the need to use simulation platforms to support operators, regulators, and involved players for understanding and dealing with this complex environment. This paper focuses on demonstrating the advantage that real electricity markets data has for the creation of realistic simulation scenarios, which allow the study of the impacts and implications that electricity markets transformations will bring to the participant countries. A case study using MASCEM (Multi-Agent System for Competitive Electricity Markets) is presented, with a scenario based on real data, simulating the European Electricity Market environment, and comparing its performance when using several different market mechanisms.