Learning Mechanisms for Information Filtering Agents

Postprint

DATA DRIVEN INTELLIGENT AGENT NETWORKS FOR ADAPTIVE MONITORING AND CONTROL

To analyze the characteristics and predict the dynamic behaviors of complex systems over time, comprehensive research to enable the development of systems that can intelligently adapt to the evolving conditions and infer new knowledge with algorithms that are not predesigned is crucially needed. This dissertation research studies the integration of the techniques and methodologies resulted from the fields of pattern recognition, intelligent agents, artificial immune systems, and distributed computing platforms, to create technologies that can more accurately describe and control the dynamics of real-world complex systems. The need for such technologies is emerging in manufacturing, transportation, hazard mitigation, weather and climate prediction, homeland security, and emergency response. Motivated by the ability of mobile agents to dynamically incorporate additional computational and control algorithms into executing applications, mobile agent technology is employed in this research for the adaptive sensing and monitoring in a wireless sensor network. Mobile agents are software components that can travel from one computing platform to another in a network and carry programs and data states that are needed for performing the assigned tasks. To support the generation, migration, communication, and management of mobile monitoring agents, an embeddable mobile agent system (Mobile-C) is integrated with sensor nodes. Mobile monitoring agents visit distributed sensor nodes, read real-time sensor data, and perform anomaly detection using the equipped pattern recognition algorithms. The optimal control of agents is achieved by mimicking the adaptive immune response and the application of multi-objective optimization algorithms. The mobile agent approach provides potential to reduce the communication load and energy consumption in monitoring networks. The major research work of this dissertation project includes: (1) studying effective feature extraction methods for time series measurement data; (2) investigating the impact of the feature extraction methods and dissimilarity measures on the performance of pattern recognition; (3) researching the effects of environmental factors on the performance of pattern recognition; (4) integrating an embeddable mobile agent system with wireless sensor nodes; (5) optimizing agent generation and distribution using artificial immune system concept and multi-objective algorithms; (6) applying mobile agent technology and pattern recognition algorithms for adaptive structural health monitoring and driving cycle pattern recognition; (7) developing a web-based monitoring network to enable the visualization and analysis of real-time sensor data remotely. Techniques and algorithms developed in this dissertation project will contribute to research advances in networked distributed systems operating under changing environments.

Análise funcional comparativa de algoritmos de aprendizagem por reforço

De entre todos os paradigmas de aprendizagem actualmente identificados, a Aprendizagem por Reforço revela-se de especial interesse e aplicabilidade nos inúmeros processos que nos rodeiam: desde a solitária sonda que explora o planeta mais remoto, passando pelo programa especialista que aprende a apoiar a decisão médica pela experiencia adquirida, até ao cão de brincar que faz as delícias da criança interagindo com ela e adaptando-se aos seus gostos, e todo um novo mundo que nos rodeia e apela crescentemente a que façamos mais e melhor nesta área. Desde o aparecimento do conceito de aprendizagem por reforço, diferentes métodos tem sido propostos para a sua concretização, cada um deles abordando aspectos específicos. Duas vertentes distintas, mas complementares entre si, apresentam-se como características chave do processo de aprendizagem por reforço: a obtenção de experiência através da exploração do espaço de estados e o aproveitamento do conhecimento obtido através dessa mesma experiência. Esta dissertação propõe-se seleccionar alguns dos métodos propostos mais promissores de ambas as vertentes de exploração e aproveitamento, efectuar uma implementação de cada um destes sobre uma plataforma modular que permita a simulação do uso de agentes inteligentes e, através da sua aplicação na resolução de diferentes configurações de ambientes padrão, gerar estatísticas funcionais que permitam inferir conclusões que retractem entre outros aspectos a sua eficiência e eficácia comparativas em condições específicas.

Manutenção proactiva de sistemas AVAC com recurso aos sistemas inteligentes multiagente

Nos tempos actuais os equipamentos para Aquecimento Ventilação e Ar Condicionado (AVAC) ocupam um lugar de grande importância na concepção, desenvolvimento e manutenção de qualquer edifício por mais pequeno que este seja. Assim, surge a necessidade premente de racionalizar os consumos energéticos optimizando-os. A alta fiabilidade desejada nestes sistemas obriga-nos cada vez mais a descobrir formas de tornar a sua manutenção mais eficiente, pelo que é necessário prevenir de uma forma proactiva todas as falhas que possam prejudicar o bom desempenho destas instalações. Como tal, torna-se necessário detectar estas falhas/anomalias, sendo imprescíndivel que nos antecipemos a estes eventos prevendo o seu acontecimento num horizonte temporal pré-definido, permitindo actuar o mais cedo possível. É neste domínio que a presente dissertação tenta encontrar soluções para que a manutenção destes equipamentos aconteça de uma forma proactiva e o mais eficazmente possível. A ideia estruturante é a de tentar intervir ainda numa fase incipiente do problema, alterando o comportamento dos equipamentos monitorizados, de uma forma automática, com recursos a agentes inteligentes de diagnóstico de falhas. No caso em estudo tenta-se adaptar de forma automática o funcionamento de uma Unidade de Tratamento de Ar (UTA) aos desvios/anomalias detectadas, promovendo a paragem integral do sistema apenas como último recurso. A arquitectura aplicada baseia-se na utilização de técnicas de inteligência artificial, nomeadamente dos sistemas multiagente. O algoritmo utilizado e testado foi construído em Labview®, utilizando um kit de ferramentas de controlo inteligente para Labview®. O sistema proposto é validado através de um simulador com o qual se conseguem reproduzir as condições reais de funcionamento de uma UTA.

MASCEM improvement for agent based coalitions

This paper presents MASCEM - Multi-Agent Simulator for Electricity Markets improvement towards an enlarged model for Seller Agents coalitions. The simulator has been improved, both regarding its user interface and internal structure. The OOA, used as development platform, version was updated and the multi-agent model was adjusted for implementing and testing several negotiations regarding Seller agents’ coalitions. Seller coalitions are a very important subject regarding the increased relevance of Distributed Generation under liberalised electricity markets.

Agent-based simulation of electronic marketplaces with decision support

This paper presents a Multi-Agent Market simulator designed for analyzing agent market strategies based on a complete understanding of buyer and seller behaviors, preference models and pricing algorithms, considering user risk preferences and game theory for scenario analysis. The system includes agents that are capable of improving their performance with their own experience, by adapting to the market conditions, and capable of considering other agents reactions.

A multi-agent system for the support of producer coalition formation in electricity markets

All over the world Distributed Generation is seen as a valuable help to get cleaner and more efficient electricity. To get negotiation power and advantages of scale economy, distributed producers can be aggregated giving place to a new concept: the Virtual Power Producer. Virtual Power Producers are multitechnology and multi-site heterogeneous entities. Virtual Power Producers should adopt organization and management methodologies so that they can make Distributed Generation a really profitable activity, able to participate in the market. In this paper we address the development of a multi-agent market simulator – MASCEM – able to study alternative coalitions of distributed producers in order to identify promising Virtual Power Producers in an electricity market.

Metalearning in ALBidS: A Strategic Bidding System for electricity markets

Metalearning is a subfield of machine learning with special pro-pensity for dynamic and complex environments, from which it is difficult to extract predictable knowledge. The field of study of this work is the electricity market, which due to the restructuring that recently took place, became an especially complex and unpredictable environment, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. This paper presents the development of a metalearner, applied to the decision support of electricity markets’ negotia-tion entities. The proposed metalearner takes advantage on several learning algorithms implemented in ALBidS, an adaptive learning system that pro-vides decision support to electricity markets’ participating players. Using the outputs of each different strategy as inputs, the metalearner creates its own output, considering each strategy with a different weight, depending on its individual quality of performance. The results of the proposed meth-od are studied and analyzed using MASCEM - a multi-agent electricity market simulator that models market players and simulates their operation in the market. This simulator provides the chance to test the metalearner in scenarios based on real electricity market´s data.

Virtual power producers market strategies

This paper presents MASCEM - a multi-agent based electricity market simulator. MASCEM uses game theory, machine learning techniques, scenario analysis and optimisation techniques to model market agents and to provide them with decision-support. This paper mainly focus on the MASCEM ability to provide the means to model and simulate Virtual Power Producers (VPP). VPPs are represented as a coalition of agents, with specific characteristics and goals. The paper detail some of the most important aspects considered in VPP formation and in the aggregation of new producers and includes a case study.

Multi-agent electricity market simulation with dynamic strategies & virtual power producers

Distributed energy resources will provide a significant amount of the electricity generation and will be a normal profitable business. In the new decentralized grid, customers will be among the many decentralized players and may even help to co-produce the required energy services such as demand-side management and load shedding. So, they will gain the opportunity to be more active market players. The aggregation of DG plants gives place to a new concept: the Virtual Power Producer (VPP). VPPs can reinforce the importance of these generation technologies making them valuable in electricity markets. In this paper we propose the improvement of MASCEM, a multi-agent simulation tool to study negotiations in electricity spot markets based on different market mechanisms and behavior strategies, in order to take account of decentralized players such as VPP.

Strategic bidding methodology for electricity markets using adaptive learning

The very particular characteristics of electricity markets, require deep studies of the interactions between the involved players. MASCEM is a market simulator developed to allow studying electricity market negotiations. This paper presents a new proposal for the definition of MASCEM players’ strategies to negotiate in the market. The proposed methodology is implemented as a multiagent system, using reinforcement learning algorithms to provide players with the capabilities to perceive the changes in the environment, while adapting their bids formulation according to their needs, using a set of different techniques that are at their disposal. This paper also presents a methodology to define players’ models based on the historic of their past actions, interpreting how their choices are affected by past experience, and competition.

Multi-agent based electricity market simulator with VPP: Conceptual and implementation issues

This paper presents a new architecture for the MASCEM, a multi-agent electricity market simulator. This is implemented in a Prolog which is integrated in the JAVA program by using the LPA Win-Prolog Intelligence Server (IS) provides a DLL interface between Win-Prolog and other applications. This paper mainly focus on the MASCEM ability to provide the means to model and simulate Virtual Power Producers (VPP). VPPs are represented as a coalition of agents, with specific characteristics and goals. VPPs can reinforce the importance of these generation technologies making them valuable in electricity markets.

MASCEM - an electricity market simulator providing coalition support for virtual power players

This paper presents MASCEM - a multi-agent based electricity market simulator. MASCEM uses game theory, machine learning techniques, scenario analysis and optimization techniques to model market agents and to provide them with decision-support. This paper mainly focus on the MASCEM ability to provide the means to model and simulate Virtual Power Players (VPP). VPPs are represented as a coalition of agents, with specific characteristics and goals. The paper details some of the most important aspects considered in VPP formation and in the aggregation of new producers and includes a case study based on real data.

Ambient intelligence in emotion based ubiquitous decision making

As the time goes on, it is a question of common sense to involve in the process of decision making people scattered around the globe. Groups are created in a formal or informal way, exchange ideas or engage in a process of argumentation and counterargumentation, negotiate, cooperate, collaborate or even discuss techniques and/or methodologies for problem solving. In this work it is proposed an agent-based architecture to support a ubiquitous group decision support system, i.e. based on the concept of agent, which is able to exhibit intelligent, and emotional-aware behaviour, and support argumentation, through interaction with individual persons or groups. It is enforced the paradigm of Mixed Initiative Systems, so the initiative is to be pushed by human users and/or intelligent agents.

Context-aware emotion-based model for group decision making

Involving groups in important management processes such as decision making has several advantages. By discussing and combining ideas, counter ideas, critical opinions, identified constraints, and alternatives, a group of individuals can test potentially better solutions, sometimes in the form of new products, services, and plans. In the past few decades, operations research, AI, and computer science have had tremendous success creating software systems that can achieve optimal solutions, even for complex problems. The only drawback is that people don’t always agree with these solutions. Sometimes this dissatisfaction is due to an incorrect parameterization of the problem. Nevertheless, the reasons people don’t like a solution might not be quantifiable, because those reasons are often based on aspects such as emotion, mood, and personality. At the same time, monolithic individual decisionsupport systems centered on optimizing solutions are being replaced by collaborative systems and group decision-support systems (GDSSs) that focus more on establishing connections between people in organizations. These systems follow a kind of social paradigm. Combining both optimization- and socialcentered approaches is a topic of current research. However, even if such a hybrid approach can be developed, it will still miss an essential point: the emotional nature of group participants in decision-making tasks. We’ve developed a context-aware emotion based model to design intelligent agents for group decision-making processes. To evaluate this model, we’ve incorporated it in an agent-based simulator called ABS4GD (Agent-Based Simulation for Group Decision), which we developed. This multiagent simulator considers emotion- and argument based factors while supporting group decision-making processes. Experiments show that agents endowed with emotional awareness achieve agreements more quickly than those without such awareness. Hence, participant agents that integrate emotional factors in their judgments can be more successful because, in exchanging arguments with other agents, they consider the emotional nature of group decision making.