Adaptive learning in agents behaviour: a framework for electricity markets simulation

Electricity markets are complex environments with very particular characteristics. A critical issue regarding these specific characteristics concerns the constant changes they are subject to. This is a result of the electricity markets’ restructuring, which was performed so that the competitiveness could be increased, but it also had exponential implications in the increase of the complexity and unpredictability in those markets scope. The constant growth in markets unpredictability resulted in an amplified need for market intervenient entities in foreseeing market behaviour. The need for understanding the market mechanisms and how the involved players’ interaction affects the outcomes of the markets, contributed to the growth of usage of simulation tools. Multi-agent based software is particularly well fitted to analyze dynamic and adaptive systems with complex interactions among its constituents, such as electricity markets. This dissertation presents ALBidS – Adaptive Learning strategic Bidding System, a multiagent system created to provide decision support to market negotiating players. This system is integrated with the MASCEM electricity market simulator, so that its advantage in supporting a market player can be tested using cases based on real markets’ data. ALBidS considers several different methodologies based on very distinct approaches, to provide alternative suggestions of which are the best actions for the supported player to perform. The approach chosen as the players’ actual action is selected by the employment of reinforcement learning algorithms, which for each different situation, simulation circumstances and context, decides which proposed action is the one with higher possibility of achieving the most success. Some of the considered approaches are supported by a mechanism that creates profiles of competitor players. These profiles are built accordingly to their observed past actions and reactions when faced with specific situations, such as success and failure. The system’s context awareness and simulation circumstances analysis, both in terms of results performance and execution time adaptation, are complementary mechanisms, which endow ALBidS with further adaptation and learning capabilities.

Personalised placement in networked video

Personalised video can be achieved by inserting objects into a video play-out according to the viewer's profile. Content which has been authored and produced for general broadcast can take on additional commercial service features when personalised either for individual viewers or for groups of viewers participating in entertainment, training, gaming or informational activities. Although several scenarios and use-cases can be envisaged, we are focussed on the application of personalised product placement. Targeted advertising and product placement are currently garnering intense interest in the commercial networked media industries. Personalisation of product placement is a relevant and timely service for next generation online marketing and advertising and for many other revenue generating interactive services. This paper discusses the acquisition and insertion of media objects into a TV video play-out stream where the objects are determined by the profile of the viewer. The technology is based on MPEG-4 standards using object based video and MPEG-7 for metadata. No proprietary technology or protocol is proposed. To trade the objects into the video play-out, a Software-as-a-Service brokerage platform based on intelligent agent technology is adopted. Agencies, libraries and service providers are represented in a commercial negotiation to facilitate the contractual selection and usage of objects to be inserted into the video play-out.

Metalearner based on Dynamic Neural Network for Strategic Bidding in electricity Markets

The restructuring of electricity markets, conducted to increase the competition in this sector, and decrease the electricity prices, brought with it an enormous increase in the complexity of the considered mechanisms. The electricity market became a complex and unpredictable environment, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. Software tools became, therefore, essential to provide simulation and decision support capabilities, in order to potentiate the involved players’ actions. This paper presents the development of a metalearner, applied to the decision support of electricity markets’ negotiation entities. The proposed metalearner executes a dynamic artificial neural network to create its own output, taking advantage on several learning algorithms implemented in ALBidS, an adaptive learning system that provides decision support to electricity markets’ players. The proposed metalearner considers different weights for each strategy, depending on its individual quality of performance. The results of the proposed method are studied and analyzed in scenarios based on real electricity markets’ data, using MASCEM - a multi-agent electricity market simulator that simulates market players’ operation in the market.

Domestic Consumption Simulation and Management Using a Continuous Consumption Management and Optimization Algorithm

The recent changes concerning the consumers’ active participation in the efficient management of load devices for one’s own interest and for the interest of the network operator, namely in the context of demand response, leads to the need for improved algorithms and tools. A continuous consumption optimization algorithm has been improved in order to better manage the shifted demand. It has been done in a simulation and user-interaction tool capable of being integrated in a multi-agent smart grid simulator already developed, and also capable of integrating several optimization algorithms to manage real and simulated loads. The case study of this paper enhances the advantages of the proposed algorithm and the benefits of using the developed simulation and user interaction tool.

Contextual and Environmental Awareness Laboratory for Energy Consumption Management

The recent changes on power systems paradigm requires the active participation of small and medium players in energy management. With an electricity price fluctuation these players must manage the consumption. Lowering costs and ensuring adequate user comfort levels. Demand response can improve the power system management and bring benefits for the small and medium players. The work presented in this paper, which is developed aiming the smart grid context, can also be used in the current power system paradigm. The proposed system is the combination of several fields of research, namely multi-agent systems and artificial neural networks. This system is physically implemented in our laboratories and it is used daily by researchers. The physical implementation gives the system an improvement in the proof of concept, distancing itself from the conventional systems. This paper presents a case study illustrating the simulation of real-time pricing in a laboratory.

Desenvolvimento de uma framework de argumentação Para apoio à tomada de decisão ubíqua

Os Sistemas de Apoio à Tomada de Decisão em Grupo (SADG) surgiram com o objetivo de apoiar um conjunto de decisores no processo de tomada de decisão. Uma das abordagens mais comuns na literatura para a implementação dos SADG é a utilização de Sistemas Multi-Agente (SMA). Os SMA permitem refletir com maior transparência o contexto real, tanto na representação que cada agente faz do decisor que representa como no formato de comunicação utilizado. Com o crescimento das organizações, atualmente vive-se uma viragem no conceito de tomada de decisão. Cada vez mais, devido a questões como: o estilo de vida, os mercados globais e o tipo de tecnologias disponíveis, faz sentido falar de decisão ubíqua. Isto significa que o decisor deverá poder utilizar o sistema a partir de qualquer local, a qualquer altura e através dos mais variados tipos de dispositivos eletrónicos tais como tablets, smartphones, etc. Neste trabalho é proposto um novo modelo de argumentação, adaptado ao contexto da tomada de decisão ubíqua para ser utilizado por um SMA na resolução de problemas multi-critério. É assumido que cada agente poderá utilizar um estilo de comportamento que afeta o modo como esse agente interage com outros agentes em situações de conflito. Sendo assim, pretende-se estudar o impacto da utilização de estilos de comportamento ao longo do processo da tomada de decisão e perceber se os agentes modelados com estilos de comportamento conseguem atingir o consenso mais facilmente quando comparados com agentes que não apresentam nenhum estilo de comportamento. Pretende-se ainda estudar se o número de argumentos trocados entre os agentes é proporcional ao nível de consenso final após o processo de tomada de decisão. De forma a poder estudar as hipóteses de investigação desenvolveu-se um protótipo de um SADG, utilizando um SMA. Desenvolveu-se ainda uma framework de argumentação que foi adaptada ao protótipo desenvolvido. Os resultados obtidos permitiram validar as hipóteses definidas neste trabalho tendo-se concluído que os agentes modelados com estilos de comportamento conseguem na maioria das vezes atingir um consenso mais facilmente comparado com agentes que não apresentam nenhum estilo de comportamento e que o número de argumentos trocados entre os agentes durante o processo de tomada de decisão não é proporcional ao nível de consenso final.

Balancing market integration in MASCEM electricity market simulator

With the restructuring of the energy sector in industrialized countries there is an increased complexity in market players’ interactions along with emerging problems and new issues to be addressed. Decision support tools that facilitate the study and understanding of these markets are extremely useful to provide players with competitive advantage. In this context arises MASCEM, a multi-agent simulator for competitive electricity markets. It is essential to reinforce MASCEM with the ability to recreate electricity markets reality in the fullest possible extent, making it able to simulate as many types of markets models and players as possible. This paper presents the development of the Balancing Market in MASCEM. A key module to the study of competitive electricity markets, as it has well defined and distinct characteristics previously implemented.

Remuneration and tariffs in the context of virtual power players

Power systems have been through deep changes in recent years, namely with the operation of competitive electricity markets in the scope and the increasingly intensive use of renewable energy sources and distributed generation. This requires new business models able to cope with the new opportunities that have emerged. Virtual Power Players (VPPs) are a new player type which allows aggregating a diversity of players (Distributed Generation (DG), Storage Agents (SA), Electrical Vehicles, (V2G) and consumers), to facilitate their participation in the electricity markets and to provide a set of new services promoting generation and consumption efficiency, while improving players` benefits. A major task of VPPs is the remuneration of generation and services (maintenance, market operation costs and energy reserves), as well as charging energy consumption. This paper proposes a model to implement fair and strategic remuneration and tariff methodologies, able to allow efficient VPP operation and VPP goals accomplishment in the scope of electricity markets.

Adaptive tool for automatic data collection of real electricity markets

The study of electricity markets operation has been gaining an increasing importance in last years, as result of the new challenges that the electricity markets restructuring produced. This restructuring increased the competitiveness of the market, but with it its complexity. The growing complexity and unpredictability of the market’s evolution consequently increases the decision making difficulty. Therefore, the intervenient entities are forced to rethink their behaviour and market strategies. Currently, lots of information concerning electricity markets is available. These data, concerning innumerous regards of electricity markets operation, is accessible free of charge, and it is essential for understanding and suitably modelling electricity markets. This paper proposes a tool which is able to handle, store and dynamically update data. The development of the proposed tool is expected to be of great importance to improve the comprehension of electricity markets and the interactions among the involved entities.

Electricity Markets Simulation: MASCEM Contributions to the Challenging Reality

Electricity Markets are not only a new reality but an evolving one as the involved players and rules change at a relatively high rate. Multi-agent simulation combined with Artificial Intelligence techniques may result in sophisticated tools very helpful under this context. Some simulation tools have already been developed, some of them very interesting. However, at the present state it is important to go a step forward in Electricity Markets simulators as this is crucial for facing changes in Power Systems. This paper explains the context and needs of electricity market simulation, describing the most important characteristics of available simulators. We present our work concerning MASCEM simulator, presenting its features as well as the improvements being made to accomplish the change and challenging reality of Electricity Markets.

Virtual power producers market strategies

This paper presents MASCEM - a multi-agent based electricity market simulator. MASCEM uses game theory, machine learning techniques, scenario analysis and optimisation techniques to model market agents and to provide them with decision-support. This paper mainly focus on the MASCEM ability to provide the means to model and simulate Virtual Power Producers (VPP). VPPs are represented as a coalition of agents, with specific characteristics and goals. The paper detail some of the most important aspects considered in VPP formation and in the aggregation of new producers and includes a case study.

Cost dependent strategy for electricity markets bidding based on adaptive reinforcement learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM provides several dynamic strategies for agents’ behavior. This paper presents a method that aims to provide market players with strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible bids. These bids are defined accordingly to the cost function that each producer presents.

Virtual power producers simulation - negotiating renewable distributed generation in competitive electricity markets

Sustainable development concerns made renewable energy sources to be increasingly used for electricity distributed generation. However, this is mainly due to incentives or mandatory targets determined by energy policies as in European Union. Assuring a sustainable future requires distributed generation to be able to participate in competitive electricity markets. To get more negotiation power in the market and to get advantages of scale economy, distributed generators can be aggregated giving place to a new concept: the Virtual Power Producer (VPP). VPPs are multi-technology and multisite heterogeneous entities that should adopt organization and management methodologies so that they can make distributed generation a really profitable activity, able to participate in the market. This paper presents ViProd, a simulation tool that allows simulating VPPs operation, in the context of MASCEM, a multi-agent based eletricity market simulator.

The role of demand response in future power systems

Power systems are planed and operated according to the optimization of the available resources. Traditionally these tasks were mostly undertaken in a centralized way which is no longer adequate in a competitive environment. Demand response can play a very relevant role in this context but adequate tools to negotiate this kind of resources are required. This paper presents an approach to deal with these issues, by using a multi-agent simulator able to model demand side players and simulate their strategic behavior. The paper includes an illustrative case study that considers an incident situation. The distribution company is able to reduce load curtailment due to load flexibility contracts previously established with demand side players.

Seller, buyer and VPP agents behavior on a simulated electricity market

Competitive electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is an electricity market simulator able to model market players and simulate their operation in the market. As market players are complex entities, having their characteristics and objectives, making their decisions and interacting with other players, a multi-agent architecture is used and proved to be adequate. MASCEM players have learning capabilities and different risk preferences. They are able to refine their strategies according to their past experience (both real and simulated) and considering other agents’ behavior. Agents’ behavior is also subject to its risk preferences.