Adaptive learning in agents behaviour: A framework for electricity markets simulation

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM (Multi-Agent System for Competitive Electricity Markets) is a multi-agent electricity market simulator that models market players and simulates their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. This paper presents a methodology to provide decision support to electricity market negotiating players. This model allows integrating different strategic approaches for electricity market negotiations, and choosing the most appropriate one at each time, for each different negotiation context. This methodology is integrated in ALBidS (Adaptive Learning strategic Bidding System) – a multiagent system that provides decision support to MASCEM's negotiating agents so that they can properly achieve their goals. ALBidS uses artificial intelligence methodologies and data analysis algorithms to provide effective adaptive learning capabilities to such negotiating entities. The main contribution is provided by a methodology that combines several distinct strategies to build actions proposals, so that the best can be chosen at each time, depending on the context and simulation circumstances. The choosing process includes reinforcement learning algorithms, a mechanism for negotiating contexts analysis, a mechanism for the management of the efficiency/effectiveness balance of the system, and a mechanism for competitor players' profiles definition.

Adaptive Learning in Games: Defining Profiles of Competitor Players

Artificial Intelligence has been applied to dynamic games for many years. The ultimate goal is creating responses in virtual entities that display human-like reasoning in the definition of their behaviors. However, virtual entities that can be mistaken for real persons are yet very far from being fully achieved. This paper presents an adaptive learning based methodology for the definition of players’ profiles, with the purpose of supporting decisions of virtual entities. The proposed methodology is based on reinforcement learning algorithms, which are responsible for choosing, along the time, with the gathering of experience, the most appropriate from a set of different learning approaches. These learning approaches have very distinct natures, from mathematical to artificial intelligence and data analysis methodologies, so that the methodology is prepared for very distinct situations. This way it is equipped with a variety of tools that individually can be useful for each encountered situation. The proposed methodology is tested firstly on two simpler computer versus human player games: the rock-paper-scissors game, and a penalty-shootout simulation. Finally, the methodology is applied to the definition of action profiles of electricity market players; players that compete in a dynamic game-wise environment, in which the main goal is the achievement of the highest possible profits in the market.

Adaptive offloading for infotainment systems

Infotainment applications in vehicles are currently supported both by the in-vehicle platform, as well as by user’s smart devices, such as smartphones and tablets. More and more the user expects that there is a continuous service of applications inside or outside of the vehicle, provided in any of these devices (a simple but common example is hands-free mobile phone calls provided by the vehicle platform). With the increasing complexity of ‘apps’, it is necessary to support increasing levels of Quality of Service (QoS), with varying resource requirements. Users may want to start listening to music in the smartphone, or video in the tablet, being this application transparently ‘moved’ into the vehicle when it is started. This paper presents an adaptable offloading mechanism, following a service-oriented architecture pattern, which takes into account the QoS requirements of the applications being executed when making decisions.

A Random Access Protocol incorporating Multi-Packet Reception, Retransmission Diversity and Successive Interference Cancellation

8th International Workshop on Multiple Access Communications (MACOM2015), Helsinki, Finland.

Improved Numerical Simulation for a Novel Adaptive Control Using Fractional Order Derivatives

A novel control technique is investigated in the adaptive control of a typical paradigm, an approximately and partially modeled cart plus double pendulum system. In contrast to the traditional approaches that try to build up ”complete” and ”permanent” system models it develops ”temporal” and ”partial” ones that are valid only in the actual dynamic environment of the system, that is only within some ”spatio-temporal vicinity” of the actual observations. This technique was investigated for various physical systems via ”preliminary” simulations integrating by the simplest 1st order finite element approach for the time domain. In 2004 INRIA issued its SCILAB 3.0 and its improved numerical simulation tool ”Scicos” making it possible to generate ”professional”, ”convenient”, and accurate simulations. The basic principles of the adaptive control, the typical tools available in Scicos, and others developed by the authors, as well as the improved simulation results and conclusions are presented in the contribution.

Decision Support for Energy Contracts Negotiation with Game Theory and Adaptive Learning

This paper presents a decision support methodology for electricity market players’ bilateral contract negotiations. The proposed model is based on the application of game theory, using artificial intelligence to enhance decision support method’s adaptive features. This model is integrated in AiD-EM (Adaptive Decision Support for Electricity Markets Negotiations), a multi-agent system that provides electricity market players with strategic behavior capabilities to improve their outcomes from energy contracts’ negotiations. Although a diversity of tools that enable the study and simulation of electricity markets has emerged during the past few years, these are mostly directed to the analysis of market models and power systems’ technical constraints, making them suitable tools to support decisions of market operators and regulators. However, the equally important support of market negotiating players’ decisions is being highly neglected. The proposed model contributes to overcome the existing gap concerning effective and realistic decision support for electricity market negotiating entities. The proposed method is validated by realistic electricity market simulations using real data from the Iberian market operator—MIBEL. Results show that the proposed adaptive decision support features enable electricity market players to improve their outcomes from bilateral contracts’ negotiations.

Waste collection routing-limited multiple landfills and heterogeneous fleet

This article deals with a real-life waste collection routing problem. To efficiently plan waste collection, large municipalities may be partitioned into convenient sectors and only then can routing problems be solved in each sector. Three diverse situations are described, resulting in three different new models. In the first situation, there is a single point of waste disposal from where the vehicles depart and to where they return. The vehicle fleet comprises three types of collection vehicles. In the second, the garage does not match any of the points of disposal. The vehicle is unique and the points of disposal (landfills or transfer stations) may have limitations in terms of the number of visits per day. In the third situation, disposal points are multiple (they do not coincide with the garage), they are limited in the number of visits, and the fleet is composed of two types of vehicles. Computational results based not only on instances adapted from the literature but also on real cases are presented and analyzed. In particular, the results also show the effectiveness of combining sectorization and routing to solve waste collection problems.

Quality-of-service aware routing for static and mobile IPv6-based low-power and lossy sensor networks using RPL

The Internet of Things (IoT) has emerged as a paradigm over the last few years as a result of the tight integration of the computing and the physical world. The requirement of remote sensing makes low-power wireless sensor networks one of the key enabling technologies of IoT. These networks encompass several challenges, especially in communication and networking, due to their inherent constraints of low-power features, deployment in harsh and lossy environments, and limited computing and storage resources. The IPv6 Routing Protocol for Low Power and Lossy Networks (RPL) [1] was proposed by the IETF ROLL (Routing Over Low-power Lossy links) working group and is currently adopted as an IETF standard in the RFC 6550 since March 2012. Although RPL greatly satisfied the requirements of low-power and lossy sensor networks, several issues remain open for improvement and specification, in particular with respect to Quality of Service (QoS) guarantees and support for mobility. In this paper, we focus mainly on the RPL routing protocol. We propose some enhancements to the standard specification in order to provide QoS guarantees for static as well as mobile LLNs. For this purpose, we propose OF-FL (Objective Function based on Fuzzy Logic), a new objective function that overcomes the limitations of the standardized objective functions that were designed for RPL by considering important link and node metrics, namely end-to-end delay, number of hops, ETX (Expected transmission count) and LQL (Link Quality Level). In addition, we present the design of Co-RPL, an extension to RPL based on the corona mechanism that supports mobility in order to overcome the problem of slow reactivity to frequent topology changes and thus providing a better quality of service mainly in dynamic networks application. Performance evaluation results show that both OF-FL and Co-RPL allow a great improvement when compared to the standard specification, mainly in terms of packet loss ratio and average network latency. 2015 Elsevier B.V. Al

Adaptive Penalty and Barrier function based on Fuzzy Logic

Optimization methods have been used in many areas of knowledge, such as Engineering, Statistics, Chemistry, among others, to solve optimization problems. In many cases it is not possible to use derivative methods, due to the characteristics of the problem to be solved and/or its constraints, for example if the involved functions are non-smooth and/or their derivatives are not know. To solve this type of problems a Java based API has been implemented, which includes only derivative-free optimization methods, and that can be used to solve both constrained and unconstrained problems. For solving constrained problems, the classic Penalty and Barrier functions were included in the API. In this paper a new approach to Penalty and Barrier functions, based on Fuzzy Logic, is proposed. Two penalty functions, that impose a progressive penalization to solutions that violate the constraints, are discussed. The implemented functions impose a low penalization when the violation of the constraints is low and a heavy penalty when the violation is high. Numerical results, obtained using twenty-eight test problems, comparing the proposed Fuzzy Logic based functions to six of the classic Penalty and Barrier functions are presented. Considering the achieved results, it can be concluded that the proposed penalty functions besides being very robust also have a very good performance.

Adaptive modeling and high quality spectral estimation for speech enhancement

In this work an adaptive modeling and spectral estimation scheme based on a dual Discrete Kalman Filtering (DKF) is proposed for speech enhancement. Both speech and noise signals are modeled by an autoregressive structure which provides an underlying time frame dependency and improves time-frequency resolution. The model parameters are arranged to obtain a combined state-space model and are also used to calculate instantaneous power spectral density estimates. The speech enhancement is performed by a dual discrete Kalman filter that simultaneously gives estimates for the models and the signals. This approach is particularly useful as a pre-processing module for parametric based speech recognition systems that rely on spectral time dependent models. The system performance has been evaluated by a set of human listeners and by spectral distances. In both cases the use of this pre-processing module has led to improved results.

Adaptive filtering for high quality HMM based speech synthesis

In this work an adaptive filtering scheme based on a dual Discrete Kalman Filtering (DKF) is proposed for Hidden Markov Model (HMM) based speech synthesis quality enhancement. The objective is to improve signal smoothness across HMMs and their related states and to reduce artifacts due to acoustic model's limitations. Both speech and artifacts are modelled by an autoregressive structure which provides an underlying time frame dependency and improves time-frequency resolution. Themodel parameters are arranged to obtain a combined state-space model and are also used to calculate instantaneous power spectral density estimates. The quality enhancement is performed by a dual discrete Kalman filter that simultaneously gives estimates for the models and the signals. The system's performance has been evaluated using mean opinion score tests and the proposed technique has led to improved results.

Scalable and interference aware wi-fi mesh networks using cots devices

A crescente tendencia no acesso móvel tem sido potenciada pela tecnologia IEEE 802.11. Contudo, estas redes têm alcance rádio limitado. Para a extensão da sua cobertura é possível recorrer a redes emalhadas sem fios baseadas na tecnologia IEEE 802.11, com vantagem do ponto de vista do custo e da flexibilidade de instalação, face a soluções cabladas. Redes emalhadas sem fios constituídas por nós com apenas uma interface têm escalabilidade reduzida. A principal razão dessa limitação deve-se ao uso do mecanismo de acesso ao meio partilhado Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) em topologias multi-hop. Especificamente, o CSMA/CA não evita o problema do nó escondido levando ao aumento do número de colisões e correspondente degradação de desempenho com impacto direto no throughput e na latência. Com a redução da tecnologia rádio torna-se viável a utilização de múltiplos rádios por nó, sem com isso aumentar significativamente o custo da solução final de comunicações. A utilização de mais do que um rádio por nó de comuniações permite superar os problemas de desempenho inerentes ás redes formadas por nós com apenas um rádio. O objetivo desta tese, passa por desenvolver uma nova solução para redes emalhadas multi-cana, duar-radio, utilizando para isso novos mecanismos que complementam os mecanismos definidos no IEEE 802.11 para o estabelecimento de um Basic Service Set (BSS). A solução é baseada na solução WiFIX, um protocolo de routing para redes emalhadas de interface única e reutiliza os mecanismos já implementados nas redes IEEE 802.11 para difundir métricas que permitam à rede escalar de forma eficaz minimizando o impacto na performance. A rede multi-hop é formada por nós equipados com duas interfaces, organizados numa topologia hierárquica sobre múltiplas relações Access Point (AP) – Station (STA). Os resultados experimentais obtidos mostram a eficácia e o bom desempenho da solução proposta face à solução WiFIX original.

Adaptive Game Design Study of a fully adaptive framework

Adventure! The Paladin Order foi um projecto ambicioso que começou por ser desenvolvido como um video jogo completo. Tinha como objéctivo implementar uma ferramenta diferente que permitisse tornar o jogo completamente adaptativo às decisões do jogador tanto nas interacções e no diálogo com outras personagens, assim como no combate contra os variados inímigos do jogo. Devido à inexperiência do autor uma grande parte do tempo foi passado a estudar e a pesquisar várias possíveis soluções que permitissem criar um ambiente que fosse adaptativo de uma forma simples e interessante, não só para os programadores mas também para qualquer pessoa que fosse responsável por editar o diálogo e a história do jogo. Os resultados foram bastante interessantes, revelando um sistema que depende simultaneamente dos ficheiros de onde é retirado o diálogo, e de um sistema de personalidades que permite definir qual será o comportamento de qualquer objecto do jogo ou, pelo menos, como as outras personagens irão reagir. O produto final é uma ferramenta de bases sólidas que permite uma implementação relativamente simples de um sistema abrangente e adaptativo, com poucas falhas e apenas algumas questões de simplicidade de código.

Providing Adaptation for an Adaptive Hypermedia System

O ensino à distância cresceu consideravelmente nos últimos anos e a tendência é para que continue a crescer em anos vindouros. No entanto, enquanto que a maioria das plataformas de ensino à distância utilizam a mesma abordagem de ensino para todos os utilizadores, os estudantes que as usam são na realidade pessoas de diferentes culturas, locais, idades e géneros, e que possuem diferentes níveis de educação. Ao contrário do ensino à distância tradicional, os sistemas de hipermédia adaptativa educacional adaptam interface, apresentação de conteúdos e navegação, entre outros, às características, necessidades e interesses específicos de diferentes utilizadores. Apesar da investigação na área de sistemas de hipermédia adaptativa já estar bastante desenvolvida, é necessário efetuar mais desenvolvimento e experimentação de modo a determinar quais são os aspetos mais eficazes destes sistemas e avaliar o seu sucesso. A Plataforma de Aprendizagem Colaborativa da Matemática (PCMAT) é um sistema de hipermédia adaptativa educacional com uma abordagem construtivista, que foi desenvolvido com o objetivo de contribuir para a investigação na área de sistemas de hipermédia adaptativa. A plataforma avalia o conhecimento do utilizador e apresenta conteúdos e atividades adaptadas às características e estilo de aprendizagem dominante de estudantes de matemática do segundo ciclo. O desenvolvimento do PCMAT tem também o propósito de auxiliar os alunos Portugueses com a aprendizagem da matemática. De acordo com o estudo PISA 2012 da OCDE [OECD, 2014], o desempenho dos alunos Portugueses na área da matemática melhorou em relação à edição anterior do estudo, mas os resultados obtidos permanecem abaixo da média da OCDE. Por este motivo, uma das finalidades deste projeto é desenvolver um sistema de hipermédia adaptativa que, ao adequar o ensino da matemática às necessidades específicas de cada aluno, os assista com a aquisição de conhecimento. A adaptação é efetuada pelo sistema usando a informação constante no modelo do utilizador para definir um grafo de conceitos do domínio específico. Este grafo é adaptado do modelo do domínio e utilizado para dar resposta às necessidades particulares de cada aluno. Embora a trajetória inicial seja definida pelo professor, o percurso percorrido no grafo por cada aluno é determinado pela sua interação com o sistema, usando para o efeito a representação do conhecimento do aluno e outras características disponíveis no modelo do utilizador, assim como avaliação progressiva. A adaptação é conseguida através de alterações na apresentação de conteúdos e na estrutura e anotações das hiperligações. A apresentação de conteúdos é alterada mostrando ou ocultando cada um dos vários fragmentos que compõe as páginas dum curso. Estes fragmentos são compostos por diferentes objetos de aprendizagem, tais como exercícios, figuras, diagramas, etc. As mudanças efetuadas na estrutura e anotações das hiperligações têm o objetivo de guiar o estudante, apontando-o na direção do conhecimento mais relevante e mantendo-o afastado de informação inadequada. A escolha de objectos de aprendizagem adequados às características particulares de cada aluno é um aspecto essencial do modelo de adaptação do PCMAT. A plataforma inclui para esse propósito um módulo responsável pela recomendação de objectos de aprendizagem, e um módulo para a pesquisa e recuperação dos mesmos. O módulo de recomendação utiliza lógica Fuzzy para converter determinados atributos do aluno num conjunto de parâmetros que caracterizam o objecto de aprendizagem que idealmente deveria ser apresentado ao aluno. Uma vez que o objecto “ideal” poderá não existir no repositório de objectos de aprendizagem do sistema, esses parâmetros são utilizados pelo módulo de pesquisa e recuperação para procurar e devolver ao módulo de recomendação uma lista com os objectos que mais se assemelham ao objecto “ideal”. A pesquisa é feita numa árvore k-d usando o algoritmo k-vizinhos mais próximos. O modelo de recomendação utiliza a lista devolvida pelo módulo de pesquisa e recuperação para seleccionar o objecto de aprendizagem mais apropriado para o aluno e processa-o para inclusão numa das páginas Web do curso. O presente documento descreve o trabalho desenvolvido no âmbito do projeto PCMAT (PTDS/CED/108339/2008), dando relevância à adaptação de conteúdos.