Aprendizado por Reforço com Valores deInfluência em Sistemas Multi-Agente

We propose a new paradigm for collective learning in multi-agent systems (MAS) as a solution to the problem in which several agents acting over the same environment must learn how to perform tasks, simultaneously, based on feedbacks given by each one of the other agents. We introduce the proposed paradigm in the form of a reinforcement learning algorithm, nominating it as reinforcement learning with influence values. While learning by rewards, each agent evaluates the relation between the current state and/or action executed at this state (actual believe) together with the reward obtained after all agents that are interacting perform their actions. The reward is a result of the interference of others. The agent considers the opinions of all its colleagues in order to attempt to change the values of its states and/or actions. The idea is that the system, as a whole, must reach an equilibrium, where all agents get satisfied with the obtained results. This means that the values of the state/actions pairs match the reward obtained by each agent. This dynamical way of setting the values for states and/or actions makes this new reinforcement learning paradigm the first to include, naturally, the fact that the presence of other agents in the environment turns it a dynamical model. As a direct result, we implicitly include the internal state, the actions and the rewards obtained by all the other agents in the internal state of each agent. This makes our proposal the first complete solution to the conceptual problem that rises when applying reinforcement learning in multi-agent systems, which is caused by the difference existent between the environment and agent models. With basis on the proposed model, we create the IVQ-learning algorithm that is exhaustive tested in repetitive games with two, three and four agents and in stochastic games that need cooperation and in games that need collaboration. This algorithm shows to be a good option for obtaining solutions that guarantee convergence to the Nash optimum equilibrium in cooperative problems. Experiments performed clear shows that the proposed paradigm is theoretical and experimentally superior to the traditional approaches. Yet, with the creation of this new paradigm the set of reinforcement learning applications in MAS grows up. That is, besides the possibility of applying the algorithm in traditional learning problems in MAS, as for example coordination of tasks in multi-robot systems, it is possible to apply reinforcement learning in problems that are essentially collaborative

Estratégias baseadas em aprendizado para coordenação de uma frota de robôs em tarefas cooperativas

In multi-robot systems, both control architecture and work strategy represent a challenge for researchers. It is important to have a robust architecture that can be easily adapted to requirement changes. It is also important that work strategy allows robots to complete tasks efficiently, considering that robots interact directly in environments with humans. In this context, this work explores two approaches for robot soccer team coordination for cooperative tasks development. Both approaches are based on a combination of imitation learning and reinforcement learning. Thus, in the first approach was developed a control architecture, a fuzzy inference engine for recognizing situations in robot soccer games, a software for narration of robot soccer games based on the inference engine and the implementation of learning by imitation from observation and analysis of others robotic teams. Moreover, state abstraction was efficiently implemented in reinforcement learning applied to the robot soccer standard problem. Finally, reinforcement learning was implemented in a form where actions are explored only in some states (for example, states where an specialist robot system used them) differently to the traditional form, where actions have to be tested in all states. In the second approach reinforcement learning was implemented with function approximation, for which an algorithm called RBF-Sarsa($lambda$) was created. In both approaches batch reinforcement learning algorithms were implemented and imitation learning was used as a seed for reinforcement learning. Moreover, learning from robotic teams controlled by humans was explored. The proposal in this work had revealed efficient in the robot soccer standard problem and, when implemented in other robotics systems, they will allow that these robotics systems can efficiently and effectively develop assigned tasks. These approaches will give high adaptation capabilities to requirements and environment changes.

Uma proposta de ensino para o nível fundamental a partir das concepções sobre insetos

O presente trabalho consiste em uma investigação acerca das concepções de estudantes sobre o que é um inseto. Partiu-se do pressuposto de que os alunos expressam concepções errôneas no momento de identificar os representantes do táxon Insecta e que isso é consequência das experiências do cotidiano e de erros conceituais oriundos da formação escolar incorreta. Os objetivos deste trabalho foram investigar sobre concepções alternativas acerca dos insetos com alunos do ensino fundamental II e propor situações para promoção de uma aprendizagem mais ativa. Os dados para analise foram coletados em três turmas de escolas diferentes: uma escola em Nova Parnamirim, da rede privada, e duas da rede pública municipal de ensino: uma do município de Natal/RN e a outra do município de Parnamirim/RN. Utilizou-se como instrumento um questionário aplicado em duas fases. Na primeira etapa foram feitas três questões abertas e duas fechadas. Na segunda, aplicaram-se mais três questões. Nesta oportunidade foram apresentados aos alunos uma lista com imagens de insetos e animais considerados não insetos. Os resultados dos questionários mostraram que os alunos apresentaram na sua maioria, 62%, sentimentos negativos sobre os insetos, expressos por palavras depreciativas, entre elas: nojentos, feios, asquerosos, perigosos, sendo estes mais da metade dos alunos da amostra. Sentimentos positivos representaram 20%: fazem bem ao homem, Deus criou, são bons, são bonitos, e 18% apresentaram neutralidade nessas categorias. Outro resultado apresentado foi que 82% generalizaram os animais artrópodes como sendo insetos. Como forma de resolver problemas relacionados as concepções uma unidade didática foi aplicada em uma das escolas envolvidas na pesquisa, validando assim um instrumento pedagógico que incluiu atividade lúdica constituída de um jogo de tabuleiro proporcionador de uma vivência de aprendizagem focada em conceitos científicos sobre os insetos fáceis de serem apreendidos

Geometria e arirmética numa visão multicultural: uma experiência pedagógica

This paper aims to build a notebook of activities that can help the teacher of elementary school mathematics. Topics covered are arithmetic and geometry and the activities proposed here were developed aiming print them a multicultural character. We take as a base line developed by Claudia Zaslavsky multiculturalism and reflected in his books "Games and activities worldwide" and "More games and activities worldwide." We structure our work around four themes: the symbol of the Olympic Games, the pyramids of Egypt, the Russian abacus abacus and Chinese. The first two themes allow you to explore basic concepts of geometry while the latter two themes allow us to explore numerical notation and arithmetic operations

O ser elástico, mola, agulha, trepidação: expressões do homoerotismo em Fernando Pessoa

This Masters thesis investigates and analyzes the homoerotic expressions in Fernando Pessoa s poetry and in his heteronyms : Álvaro de Campos, Alberto Caeiro and Ricardo Reis. Through the elaboration of deceptive strategies as a way to escape from the insults and the constraints, the poet by means of creative runaway lines builds up a homoerotic poetry that differ from the established patterns and representational models of the literature. We can find, bearing in mind a problematic sexuality which overflows in the various I s, of the sinuous identities and the movable boundaries, and intense mask games. They become real in poems, especially by means of sensations, yielding the possibility of a queer discourse and generating a homoerotic poetry. Pessoa s homoerotism is, thus, presented amid friendly relationships, smooth spaces, of heterotrophy of the city, or others space and the closet s aesthetics. Based on this, it is noted the articulations of a performance gender as body scripture and inscription, thus establishing a physical-writing which reveals a transgressive sexuality, with multiple entrances and exits, affirmations and denials that originate paradoxes. In this matter, we develop links between Foucault conceptions (2010a, 2010b, 2011) towards the history of sexuality, friendship and the power-knowledge relations with traces of significance and subjectivity, in processes of faciality and BwO (Body-without-Organs) in Deleuze and Guattari (1996). We still consider the ontological questionings towards the Gay dimension in Eribon (2008) and we raise aspects concerning the construction of the idea of masculinity in Bourdieu (2010). Finally, we base ourselves on Sedgwick s 1993 closet epistemological approach, as well as on Butler s 2012 gender and performance theory, also going through a few Bataille 1987 passages about erotism, that redimension the homoaffective images of the Portuguese poet

Poesia digital: três leituras da Obra de Jason Nelson

The development of digital technologies has opened up a new niche for artistic production. It began to emerge artworks that could only happen in the digital context. In this study, we investigated the syntax of defining digital poetry, recognized internationally as one of the possibilities of this new artistic practice, and how happens the meaning during the reading process. As research corpus, we chose the poem "Birds Still Warm From Flying", by american poet Jason Nelson. And we structure our analysis looking through three perspectives: the digital poem as a game; as the evolution of experimental trends of twentieth-century poetry; and as a representation of Rubik`s Cube, a famous three-dimensional puzzle from the 1970`s. Initially, we made some considerations about the construction of digital poetry as a hypermedia artwork, looking through convergence and hybridization of artistic and media languages. Then, we saw some similarities between Nelson`s poem and electronic games, based on our critical observations about the concept of interactivity. Subsequently, we wrote a historical overview about the poetic experimentation in the twentieth century, bringing examples of sound poetry and visual poetry as evidence that the birth of digital poetry is also the result of the evolution of these experiments. Finally, we use the Charles Sanders Peirce`s semiotics to analyze the signs that give references able to make us recognize the Rubik`s Cube in "Birds Still Warm From Flying"

Realidade virtual na atividade cerebral: um estudo comparativo entre estados emocionais de adultos jovens e idosos

The association of Virtual Reality (VR) to clinical practice has become common in the recent years, showing to be an additional tool on health care, especially for elderly. Its use has been related to higher therapeutic adhesion levels and well being sensation. Such emotional based aspects are often observed by subjective tools of relative validity. This study analyzed the immediate effects of varied VR contexts balance training over emotional behavior, which was observed under peaks of maximum expression of EEG waves. Methodology: 40 individuals, divided in two groups, both gender, 20 young and 20 elderly, were submitted to a 60 minutes intervention, including balance training under VR. The first 25 minutes referred to initial evaluation, general orientation and cognitive assessment by the use of Mini Mental. The next ten minutes were designated to the avatar creation and tutorial video presentation. Through the following 20 minutes, the individuals from both groups were exposed to the exact same sequence of games under virtual contexts, while submitted to electroencephalography by Emotiv EPOC® focusing Adhesion, Frustration and Meditation states. The virtual interface was provided by the Nintendo® game, Wii Fit Plus, with the scenarios Balance Bubble (1), Penguin (2), Soccer (3), Tight Rope (4) and Table Tilt (5). Finally, a questionnaire of personal impressions was applied on the 5 minutes left. Results: data collected showed 64,7% of individuals from both groups presented higher concentration of adhesion peaks on Balance Bubble game. Both groups also presented similar behavior regarding meditation state, with marks close to 40%, each, on the same game, Table Tilt. There was divergence related to the frustration state, being the maximum concentration for the young group on the Soccer game (29,3%), whilst the elderly group referred highest marks to Tight Rope game (35,2%). Conclusion: Findings suggest virtual contexts can be favorable to adhesion and meditation emotional patterns induction, regardless age and for both sexes, whilst frustration seems to be more related to cognitive motor affordance, likely to be influenced by age. This information is relevant and contributes to the orientation for the best choice of games applied in clinical practice, as for other studies regarding this topic

A cooperação em crianças de rede pública de Natal/RN:uma abordagem evolucionista

Cooperation is a well known behavior and influenced by all cultures. Probably selective pressures brought advantages to individuals that cooperate, and then this behavior is current in human societies. Most of it is studied about cooperation and natural selection was understood by the game theory, a mathematical approach that helps to understand the conflict and cooperation. We believe that natural selection and game theory could facilitate understanding these behaviors and two theoretical articles were written regarding this view. It was also found that most of data about cooperation was obtained in (with) adults. Since game theory is effective to understand this phenomenon, and to be used and understood, two games were used with five and eleven year old children: the common pool and public goods games. The results are presented in four empirical articles. We found that children respond to social dilemmas of game theory like the adults do. They adjust their rounds regarding the feedback obtained of their partness; in the beginning they cooperate and reduce the degree of cooperation along (throughout) the following session; in the absence of punition the level of opportunism increased, mainly in larger groups; boys and girls behave differently when donate. This research suggest that cooperation has an evolutionary basis in human and it is since earlier in the behavioral pattern shown by adults.

Fatores que influenciam a cooperação em humanos

Human cooperation is fundamentally affected by reciprocal exchange, but it is also remarkably common on the context of large and symbolically marked in-groups, which promote cooperation through the feeling of belonging to a group. In this thesis, two empirical articles were produced in order to investigate how human cooperation is affected by factors such as reciprocity, in-group behavior, in-group markers and gender. We investigated this subject through the administration of online games consisting of token donations, on which the subjects faced virtual players controlled by the experiment. We found that cooperative behavior is strongly influenced by reciprocity, and it is also affected by the in-group behavior, observed on the context of the social variables place of birth, ethnicity, and religions, once all of them acted as in-group markers. The subjects´ in-group behavior was enhanced when they played with generous in-group opponents, but weakened when their in-group opponents were non-generous. It was also found that cooperation is not affected by gender, but men and women cooperated in different ways under the influence of reciprocity and in-group behavior. Women are much more reciprocal on their cooperative behavior and men are less willing to cooperate with outgroupers, even when they act generously. The overall results contribute to a better understanding of the adaptive value of cooperation, reciprocity and in-group behavior on the solution of important challenges through the human evolutionary history