Aprendizado por Reforço com Valores deInfluência em Sistemas Multi-Agente

We propose a new paradigm for collective learning in multi-agent systems (MAS) as a solution to the problem in which several agents acting over the same environment must learn how to perform tasks, simultaneously, based on feedbacks given by each one of the other agents. We introduce the proposed paradigm in the form of a reinforcement learning algorithm, nominating it as reinforcement learning with influence values. While learning by rewards, each agent evaluates the relation between the current state and/or action executed at this state (actual believe) together with the reward obtained after all agents that are interacting perform their actions. The reward is a result of the interference of others. The agent considers the opinions of all its colleagues in order to attempt to change the values of its states and/or actions. The idea is that the system, as a whole, must reach an equilibrium, where all agents get satisfied with the obtained results. This means that the values of the state/actions pairs match the reward obtained by each agent. This dynamical way of setting the values for states and/or actions makes this new reinforcement learning paradigm the first to include, naturally, the fact that the presence of other agents in the environment turns it a dynamical model. As a direct result, we implicitly include the internal state, the actions and the rewards obtained by all the other agents in the internal state of each agent. This makes our proposal the first complete solution to the conceptual problem that rises when applying reinforcement learning in multi-agent systems, which is caused by the difference existent between the environment and agent models. With basis on the proposed model, we create the IVQ-learning algorithm that is exhaustive tested in repetitive games with two, three and four agents and in stochastic games that need cooperation and in games that need collaboration. This algorithm shows to be a good option for obtaining solutions that guarantee convergence to the Nash optimum equilibrium in cooperative problems. Experiments performed clear shows that the proposed paradigm is theoretical and experimentally superior to the traditional approaches. Yet, with the creation of this new paradigm the set of reinforcement learning applications in MAS grows up. That is, besides the possibility of applying the algorithm in traditional learning problems in MAS, as for example coordination of tasks in multi-robot systems, it is possible to apply reinforcement learning in problems that are essentially collaborative

Valoración de efectos ecotoxicológicos de oxitetraciclina en organismos terrestres y acuáticos mediante el empleo de sistemas multi-especie en suelo (MS3)

La importancia del suelo radica en las numerosas funciones que desempeña, tanto ambientales como socio-económicas y culturales. El suelo es el fundamento del sistema alimentario, la base de la agricultura y el medio en el que crecen casi todas las plantas destinadas a la producción de alimento, es un recurso prácticamente no renovable y es un medio vivo con gran biodiversidad cuya actividad biológica contribuye a determinar la estructura y fertilidad, y resulta ser fundamental para que este pueda realizar algunas de sus funciones. La incorporación al suelo de agentes contaminantes químicos o abióticos por encima de su capacidad de amortiguación supone su contaminación y en consecuencia la contaminación de las aguas subterráneas y/o superficiales. La presencia en el suelo de elementos tóxicos puede suponer un riesgo para la salud humana y/o los ecosistemas La presencia de medicamentos en el medio ambiente se ha convertido en un tema muy actual de investigación. Las técnicas cromatográficas actuales permiten alcanzar límites de detección analítica, en rangos comprendidos entre ng/l a μg/l, lo que ha permitido cuantificar un gran número de principios activos de uso farmacológico y excipientes en el medio ambiente, obligando a la comunidad científica a considerar este tipo de contaminación como un potencial problema que merece su atención. Hoy en día, se conoce, su amplia difusión a bajas concentraciones principalmente en el medio ambiente acuático. Tales concentraciones se han detectado en los compartimentos acuáticos, tales como los influentes y efluentes de plantas depuradoras de aguas residuales (EDAR), las aguas superficiales (ríos, lagos, arroyos, y estuarios, entre otros), el agua de mar, las aguas subterráneas y el agua potable...

Seguimento de pessoas em sistemas multi-câmara

Trabalho Final de Mestrado para obtenção do grau de Mestre em Engenharia de Redes de Comunicação e Multimédia

Optimização energética de sistemas multi-componente da zona fria do Steam Cracker

Dissertação para obtenção do Grau de Mestre em Engenharia Química e Bioquímica

Segurança no nível físico em sistemas multi-antena com diretividade na informação

Os sistemas de comunicação sem fios são sistemas de difusão por natureza. Devido a essa sua natureza, um dos problemas inerentes à mesma deve-se à segurança e ao secretismo, pois se o canal é partilhado a informação facilmente é obtida por um utilizador não autorizado, ao contrário dos sistemas de comunicação com fios. Tradicionalmente, a introdução de segurança em sistemas de comunicação, resulta na encriptação da informação, resultante de protocolos de encriptação. No entanto, a segurança através da criptografia baseia-se na premissa de que o utilizador não autorizado tem uma capacidade de processamento limitada, pois senão poderia simplesmente tentar todas as combinações possíveis e obter a chave de encriptação. Como a capacidade de processamento tem crescido exponencialmente, este tipo de sistemas tem se tornado cada vez mais complexos para não se tornarem obsoletos. A introdução de segurança na camada física torna-se então uma opção apelativa pois pode servir como um complemento, visto que os sistemas de criptografia funcionam em camadas superiores independentes da camada fisica, apresentando assim uma abordagem multi-camada em termos de segurança. Tipicamente as técnicas de segurança no nível físico podem se agrupar em 2 tipos: técnicas que se baseiam em códigos, ou técnicas que exploram variações temporais e espaciais do canal. As primeiras diminuem a eficiência espectral do sistema, e as segundas apresentam bons resultados em ambientes dinâmicos, mas em ambientes estáticos não são muito promissores. Há também a necessidade de aumentar as taxas de transmissão nos próximos sistemas de comunicação. Devido a estes requisitos, uma das tecnologias propostas para a nova geração de comunicações, é uma tecnologia baseada numa arquitectura Multiple-Input-Multiple-Output(MIMO). Esta tecnologia é promissora e consegue atingir taxas de transferências que correspondem aos requisitos propostos. Apresenta-se assim uma nova técnica de segurança no nível físico, que explora as caracteristicas físicas do sistema, como um complemento a outras medidas de segurança em camadas mais altas. Esta técnica não provoca diminuição da eficiência espectral e é independente do canal, o que tenta solucionar os problemas das restantes técnicas já existentes.

Aplicação de sistemas multi-classificadores no diagnóstico de falhas em motores de indução trifásicos

Equipment maintenance is the major cost factor in industrial plants, it is very important the development of fault predict techniques. Three-phase induction motors are key electrical equipments used in industrial applications mainly because presents low cost and large robustness, however, it isn t protected from other fault types such as shorted winding and broken bars. Several acquisition ways, processing and signal analysis are applied to improve its diagnosis. More efficient techniques use current sensors and its signature analysis. In this dissertation, starting of these sensors, it is to make signal analysis through Park s vector that provides a good visualization capability. Faults data acquisition is an arduous task; in this way, it is developed a methodology for data base construction. Park s transformer is applied into stationary reference for machine modeling of the machine s differential equations solution. Faults detection needs a detailed analysis of variables and its influences that becomes the diagnosis more complex. The tasks of pattern recognition allow that systems are automatically generated, based in patterns and data concepts, in the majority cases undetectable for specialists, helping decision tasks. Classifiers algorithms with diverse learning paradigms: k-Neighborhood, Neural Networks, Decision Trees and Naïves Bayes are used to patterns recognition of machines faults. Multi-classifier systems are used to improve classification errors. It inspected the algorithms homogeneous: Bagging and Boosting and heterogeneous: Vote, Stacking and Stacking C. Results present the effectiveness of constructed model to faults modeling, such as the possibility of using multi-classifiers algorithm on faults classification

Utilizando Pesos estáticos e dinâmicos em sistemas multi-classificadores com diferentes níveis de diversidade

Although some individual techniques of supervised Machine Learning (ML), also known as classifiers, or algorithms of classification, to supply solutions that, most of the time, are considered efficient, have experimental results gotten with the use of large sets of pattern and/or that they have a expressive amount of irrelevant data or incomplete characteristic, that show a decrease in the efficiency of the precision of these techniques. In other words, such techniques can t do an recognition of patterns of an efficient form in complex problems. With the intention to get better performance and efficiency of these ML techniques, were thought about the idea to using some types of LM algorithms work jointly, thus origin to the term Multi-Classifier System (MCS). The MCS s presents, as component, different of LM algorithms, called of base classifiers, and realized a combination of results gotten for these algorithms to reach the final result. So that the MCS has a better performance that the base classifiers, the results gotten for each base classifier must present an certain diversity, in other words, a difference between the results gotten for each classifier that compose the system. It can be said that it does not make signification to have MCS s whose base classifiers have identical answers to the sames patterns. Although the MCS s present better results that the individually systems, has always the search to improve the results gotten for this type of system. Aim at this improvement and a better consistency in the results, as well as a larger diversity of the classifiers of a MCS, comes being recently searched methodologies that present as characteristic the use of weights, or confidence values. These weights can describe the importance that certain classifier supplied when associating with each pattern to a determined class. These weights still are used, in associate with the exits of the classifiers, during the process of recognition (use) of the MCS s. Exist different ways of calculating these weights and can be divided in two categories: the static weights and the dynamic weights. The first category of weights is characterizes for not having the modification of its values during the classification process, different it occurs with the second category, where the values suffers modifications during the classification process. In this work an analysis will be made to verify if the use of the weights, statics as much as dynamics, they can increase the perfomance of the MCS s in comparison with the individually systems. Moreover, will be made an analysis in the diversity gotten for the MCS s, for this mode verify if it has some relation between the use of the weights in the MCS s with different levels of diversity

Sistema de agentes poligínicos para esteganálise de imagens digitais

In this work, we propose a multi agent system for digital image steganalysis, based on the poliginic bees model. Such approach aims to solve the problem of automatic steganalysis for digital media, with a case study on digital images. The system architecture was designed not only to detect if a file is suspicious of covering a hidden message, as well to extract the hidden message or information regarding it. Several experiments were performed whose results confirm a substantial enhancement (from 67% to 82% success rate) by using the multi-agent approach, fact not observed in traditional systems. An ongoing application using the technique is the detection of anomalies in digital data produced by sensors that capture brain emissions in little animals. The detection of such anomalies can be used to prove theories and evidences of imagery completion during sleep provided by the brain in visual cortex areas

Contribuciones al proceso de súper-resolución mediante técnicas de filtros selectivos, topología de macro-bloques adaptable y sistemas multi-cámara

Programa de doctorado: Ingeniería de Telecomunicación Avanzada

Agente de coligação com definição gráfica de processos habilitados por executor de regras: execução e composição dinâmica de skills num sistema de manufactura multiagente

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para obtenção do grau de Mestre em Engenharia Electrotécnica e de Computadores

Plataforma per a la simulació de xarxes de sensors

Aquest projecte descriu una plataforma de simulació per a xarxes de sensors des de la perspectiva dels sistemes multi-agents. La plataforma s'ha dissenyat per facilitar la simulació de diferents aplicacions concretes de xarxes de sensors. A més, s'ha entregat com a artefacte del projecte IEA (Institucions Electròniques Autònomes, TIN2006-15662-C02-0) de l'IIIACSIC. Dins l'entorn de l'IEA, aquesta és l'eina que aporta les capacitats de simulació per donar suport al disseny d'algorismes adaptatius per a xarxes de sensors.

Infra-estrutura para avaliação e testes de protocolos sociais de comunicação em ambientes tridimencionais compartilhados

The advent of the Internet stimulated the appearance of several services. An example is the communication ones present in the users day-by-day. Services as chat and e-mail reach an increasing number of users. This fact is turning the Net a powerful communication medium. The following work explores the use of communication conventional services into the Net infrastructure. We introduce the concept of communication social protocols applied to a shared virtual environment. We argue that communication tools have to be adapted to the Internet potentialities. To do that, we approach some theories of the Communication area and its applicability in a virtual environment context. We define multi-agent architecture to support the offer of these services, as well as, a software and hardware platform to support the accomplishment of experiments using Mixed Reality. Finally, we present the obtained results, experiments and products