Adaptive Learning in Games: Defining Profiles of Competitor Players

Artificial Intelligence has been applied to dynamic games for many years. The ultimate goal is creating responses in virtual entities that display human-like reasoning in the definition of their behaviors. However, virtual entities that can be mistaken for real persons are yet very far from being fully achieved. This paper presents an adaptive learning based methodology for the definition of players’ profiles, with the purpose of supporting decisions of virtual entities. The proposed methodology is based on reinforcement learning algorithms, which are responsible for choosing, along the time, with the gathering of experience, the most appropriate from a set of different learning approaches. These learning approaches have very distinct natures, from mathematical to artificial intelligence and data analysis methodologies, so that the methodology is prepared for very distinct situations. This way it is equipped with a variety of tools that individually can be useful for each encountered situation. The proposed methodology is tested firstly on two simpler computer versus human player games: the rock-paper-scissors game, and a penalty-shootout simulation. Finally, the methodology is applied to the definition of action profiles of electricity market players; players that compete in a dynamic game-wise environment, in which the main goal is the achievement of the highest possible profits in the market.

Particle Swarm Optimization of Electricity Market Negotiating Players Portfolio

Energy systems worldwide are complex and challenging environments. Multi-agent based simulation platforms are increasing at a high rate, as they show to be a good option to study many issues related to these systems, as well as the involved players at act in this domain. In this scope the authors’ research group has developed a multi-agent system: MASCEM (Multi- Agent System for Competitive Electricity Markets), which performs realistic simulations of the electricity markets. MASCEM is integrated with ALBidS (Adaptive Learning Strategic Bidding System) that works as a decision support system for market players. The ALBidS system allows MASCEM market negotiating players to take the best possible advantages from each market context. However, it is still necessary to adequately optimize the players’ portfolio investment. For this purpose, this paper proposes a market portfolio optimization method, based on particle swarm optimization, which provides the best investment profile for a market player, considering different market opportunities (bilateral negotiation, market sessions, and operation in different markets) and the negotiation context such as the peak and off-peak periods of the day, the type of day (business day, weekend, holiday, etc.) and most important, the renewable based distributed generation forecast. The proposed approach is tested and validated using real electricity markets data from the Iberian operator – MIBEL.

Dynamic Approach and Testbed for Small and Medium Players Simulation in Smart Grid Environments

The Smart Grid environment allows the integration of resources of small and medium players through the use of Demand Response programs. Despite the clear advantages for the grid, the integration of consumers must be carefully done. This paper proposes a system which simulates small and medium players. The system is essential to produce tests and studies about the active participation of small and medium players in the Smart Grid environment. When comparing to similar systems, the advantages comprise the capability to deal with three types of loads – virtual, contextual and real. It can have several loads optimization modules and it can run in real time. The use of modules and the dynamic configuration of the player results in a system which can represent different players in an easy and independent way. This paper describes the system and all its capabilities.

Resource Scheduling in Residential Microgrids Considering Energy Selling to External Players

The operation of distribution networks has been facing changes with the implementation of smart grids and microgrids, and the increasing use of distributed generation. The specific case of distribution networks that accommodate residential buildings, small commerce, and distributed generation as the case of storage and PV generation lead to the concept of microgrids, in the cases that the network is able to operate in islanding mode. The microgrid operator in this context is able to manage the consumption and generation resources, also including demand response programs, obtaining profits from selling electricity to the main network. The present paper proposes a methodology for the energy resource scheduling considering power flow issues and the energy buying and selling from/to the main network in each bus of the microgrid. The case study uses a real distribution network with 25 bus, residential and commercial consumers, PV generation, and storage.

Design of the ISePorto robocup Middle-Size League Robotic Soccer Team: Control, Localisation and Coordination

Proceedings of the 10th Mediterranean Conference on Control and Automation - MED2002 Lisbon, Portugal, July 9-12, 2002

Control and Localisation for the ISePorto Robotic Soccer Team

International Conference on Advanced Robotics, Coimbra, Portugal, Julho 2003

ISePorto Robotic Soccer Team: A New Player Generation

Proceedings of the Scientific Meeting of the Portuguese Robotics Open 2004

Coalition of distributed generation units to Virtual Power Players - a game theory approach

Smart Grids (SGs) have emerged as the new paradigm for power system operation and management, being designed to include large amounts of distributed energy resources. This new paradigm requires new Energy Resource Management (ERM) methodologies considering different operation strategies and the existence of new management players such as several types of aggregators. This paper proposes a methodology to facilitate the coalition between distributed generation units originating Virtual Power Players (VPP) considering a game theory approach. The proposed approach consists in the analysis of the classifications that were attributed by each VPP to the distributed generation units, as well as in the analysis of the previous established contracts by each player. The proposed classification model is based in fourteen parameters including technical, economical and behavioural ones. Depending of the VPP strategies, size and goals, each parameter has different importance. VPP can also manage other type of energy resources, like storage units, electric vehicles, demand response programs or even parts of the MV and LV distribution network. A case study with twelve VPPs with different characteristics and one hundred and fifty real distributed generation units is included in the paper.

Gestão de Recursos Energéticos para Apoio à Operação dos Virtual Power Players

A liberalização dos mercados de energia elétrica e a crescente integração dos recursos energéticos distribuídos nas redes de distribuição, nomeadamente as unidades de produção distribuída, os sistemas de controlo de cargas através dos programas de demand response, os sistemas de armazenamento e os veículos elétricos, representaram uma evolução no paradigma de operação e gestão dos sistemas elétricos. Este novo paradigma de operação impõe o desenvolvimento de novas metodologias de gestão e controlo que permitam a integração de todas as novas tecnologias de forma eficiente e sustentável. O principal contributo deste trabalho reside no desenvolvimento de metodologias para a gestão de recursos energéticos no contexto de redes inteligentes, que contemplam três horizontes temporais distintos (24 horas, 1 hora e 5 minutos). As metodologias consideram os escalonamentos anteriores assim como as previsões atualizadas de forma a melhorar o desempenho total do sistema e consequentemente aumentar a rentabilidade dos agentes agregadores. As metodologias propostas foram integradas numa ferramenta de simulação, que servirá de apoio à decisão de uma entidade agregadora designada por virtual power player. Ao nível das metodologias desenvolvidas são propostos três algoritmos de gestão distintos, nomeadamente para a segunda (1 hora) e terceira fase (5 minutos) da ferramenta de gestão, diferenciados pela influência que os períodos antecedentes e seguintes têm no período em escalonamento. Outro aspeto relevante apresentado neste documento é o teste e a validação dos modelos propostos numa plataforma de simulação comercial. Para além das metodologias propostas, a aplicação permitiu validar os modelos dos equipamentos considerados, nomeadamente, ao nível das redes de distribuição e dos recursos energéticos distribuidos. Nesta dissertação são apresentados três casos de estudos, cada um com diferentes cenários referentes a cenários de operação futuros. Estes casos de estudos são importantes para verificar a viabilidade da implementação das metodologias e algoritmos propostos. Adicionalmente são apresentadas comparações das metodologias propostas relativamente aos resultados obtidos, complexidade de gestão em ambiente de simulação para as diferentes fases da ferramenta proposta e os benefícios e inconvenientes no uso da ferramenta proposta.

A fisioterapia no andebol

A articulação tíbio-társica (tornozelo) é considerada uma das articulações mais relevantes e mais susceptível à lesão, sendo que a entorse do tornozelo consiste na lesão mais comum em desportistas, com uma percentagem de ocorrência de 35% no andebol. Este trabalho centra-se, assim, na problemática das entorses do tornozelo nos jogadores de andebol e da sua relação com o treino proprioceptivo como medida preventiva. O objectivo do presente trabalho foi a realização de uma revisão sistemática que estuda o treino proprioceptivo na prevenção da entorse do tornozelo, para isso, tentámos identificar a possível relação entre entorse do tornozelo e treino proprioceptivo. A metodologia utilizada neste trabalho foi uma revisão da literatura, sendo realizada uma pesquisa electrónica na base de dados da B-On, utilizando combinações de palavras-chave. No total resultaram 758 artigos, tendo sido pela sua relevância selecionados 29. Neste trabalho ficou evidente a relação do treino proprioceptivo com a prevenção da entorse do tornozelo. De todas as medidas preventivas o treino proprioceptivo é o que apresenta melhores resultados, pois, este tipo de treino está direccionado para uma estruturação da zona lesionada, fortalecendo os músculos e ligamentos, restaurando a componente proprioceptiva das estruturas danificadas e permitindo uma resposta mais rápida do Sistema Nervoso Central, aos vários estímulos a que está sujeito o atleta. Foi possível descrever em que consiste o treino proprioceptivo e os seus constituintes, adaptando-o ao andebol. Com esta revisão da literatura não foi possível obter um consenso sobre o efeito do treino proprioceptivo na prevenção de entorses do tornozelo.

Virtual power producers integration into MASCEM

All over the world Distributed Generation is seen as a valuable help to get cleaner and more efficient electricity. Under this context distributed generators, owned by different decentralized players can provide a significant amount of the electricity generation. To get negotiation power and advantages of scale economy, these players can be aggregated giving place to a new concept: the Virtual Power Producer. Virtual Power Producers are multi-technology and multi-site heterogeneous entities. Virtual Power Producers should adopt organization and management methodologies so that they can make Distributed Generation a really profitable activity, able to participate in the market. In this paper we address the integration of Virtual Power Producers into an electricity market simulator –MASCEM – as a coalition of distributed producers.

ANN-Based LMP forecasting in a distribution network with large penetration of DG

In recent years, power systems have experienced many changes in their paradigm. The introduction of new players in the management of distributed generation leads to the decentralization of control and decision-making, so that each player is able to play in the market environment. In the new context, it will be very relevant that aggregator players allow midsize, small and micro players to act in a competitive environment. In order to achieve their objectives, virtual power players and single players are required to optimize their energy resource management process. To achieve this, it is essential to have financial resources capable of providing access to appropriate decision support tools. As small players have difficulties in having access to such tools, it is necessary that these players can benefit from alternative methodologies to support their decisions. This paper presents a methodology, based on Artificial Neural Networks (ANN), and intended to support smaller players. In this case the present methodology uses a training set that is created using energy resource scheduling solutions obtained using a mixed-integer linear programming (MIP) approach as the reference optimization methodology. The trained network is used to obtain locational marginal prices in a distribution network. The main goal of the paper is to verify the accuracy of the ANN based approach. Moreover, the use of a single ANN is compared with the use of two or more ANN to forecast the locational marginal price.

Multi-criteria short-term maintenance outage scheduling in smart distribution systems

Effective legislation and standards for the coordination procedures between consumers, producers and the system operator supports the advances in the technologies that lead to smart distribution systems. In short-term (ST) maintenance scheduling procedure, the energy producers in a distribution system access to the long-term (LT) outage plan that is released by the distribution system operator (DSO). The impact of this additional information on the decision-making procedure of producers in ST maintenance scheduling is studied in this paper. The final ST maintenance plan requires the approval of the DSO that has the responsibility to secure the network reliability and quality, and other players have to follow the finalized schedule. Maintenance scheduling in the producers’ layer and the coordination procedure between them and the DSO is modelled in this paper. The proposed method is applied to a 33-bus distribution system.

Intelligent decision making in electricity markets: simulated annealing Q-Learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM is integrated with ALBidS, a system that provides several dynamic strategies for agents’ behavior. This paper presents a method that aims at enhancing ALBidS competence in endowing market players with adequate strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible actions. These actions are defined accordingly to the most probable points of bidding success. With the purpose of accelerating the convergence process, a simulated annealing based algorithm is included.

MASGriP – a Multi-Agent Smart Grid Simulation Platform

The increasing number of players that operate in power systems leads to a more complex management. In this paper a new multi-agent platform is proposed, which simulates the real operation of power system players. MASGriP – A Multi-Agent Smart Grid Simulation Platform is presented. Several consumer and producer agents are implemented and simulated, considering real characteristics and different goals and actuation strategies. Aggregator entities, such as Virtual Power Players and Curtailment Service Providers are also included. The integration of MASGriP agents in MASCEM (Multi-Agent System for Competitive Electricity Markets) simulator allows the simulation of technical and economical activities of several players. An energy resources management architecture used in microgrids is also explained.