Aprendizado por Reforço com Valores deInfluência em Sistemas Multi-Agente

We propose a new paradigm for collective learning in multi-agent systems (MAS) as a solution to the problem in which several agents acting over the same environment must learn how to perform tasks, simultaneously, based on feedbacks given by each one of the other agents. We introduce the proposed paradigm in the form of a reinforcement learning algorithm, nominating it as reinforcement learning with influence values. While learning by rewards, each agent evaluates the relation between the current state and/or action executed at this state (actual believe) together with the reward obtained after all agents that are interacting perform their actions. The reward is a result of the interference of others. The agent considers the opinions of all its colleagues in order to attempt to change the values of its states and/or actions. The idea is that the system, as a whole, must reach an equilibrium, where all agents get satisfied with the obtained results. This means that the values of the state/actions pairs match the reward obtained by each agent. This dynamical way of setting the values for states and/or actions makes this new reinforcement learning paradigm the first to include, naturally, the fact that the presence of other agents in the environment turns it a dynamical model. As a direct result, we implicitly include the internal state, the actions and the rewards obtained by all the other agents in the internal state of each agent. This makes our proposal the first complete solution to the conceptual problem that rises when applying reinforcement learning in multi-agent systems, which is caused by the difference existent between the environment and agent models. With basis on the proposed model, we create the IVQ-learning algorithm that is exhaustive tested in repetitive games with two, three and four agents and in stochastic games that need cooperation and in games that need collaboration. This algorithm shows to be a good option for obtaining solutions that guarantee convergence to the Nash optimum equilibrium in cooperative problems. Experiments performed clear shows that the proposed paradigm is theoretical and experimentally superior to the traditional approaches. Yet, with the creation of this new paradigm the set of reinforcement learning applications in MAS grows up. That is, besides the possibility of applying the algorithm in traditional learning problems in MAS, as for example coordination of tasks in multi-robot systems, it is possible to apply reinforcement learning in problems that are essentially collaborative

Participação da neurotransmissão dopaminergica no efeito hiperlocomotor do neuropeptideo S

Neuropeptide S (NPS) is an endogenous 20-aminoacid peptide which binds a G protein-coupled receptor named NPSR. This peptidergic system is involved in the modulation of several biological functions, such as locomotion, anxiety, nociception, food intake and motivational behaviors. Studies have shown the participation of NPSR receptors in mediating the hyperlocomotor effects of NPS. A growing body of evidence suggests the participation of adenosinergic, dopaminergic and CRF systems on the hyperlocomotor effects of NPS. Considering that little is known about the role of dopaminergic system in mediating NPS-induced hyperlocomotion, the present study aims to investigate the locomotor actions of intracerebroventricular (icv) NPS in mice pretreated with α-metil-p-tirosine (AMPT, inhibitor of dopamine synthesis), reserpine (inhibitor of dopamine vesicle storage) or sulpiride (D2 receptor antagonist) in the open field test. A distinct group of animals received the same pretreatments described above (AMPT, reserpine or sulpiride) and the hyperlocomotor effects of methylphenidate (dopamine reuptake inhibitor) were investigated in the open field. NPS and methylphenidate increased the mouse locomotor activity. AMPT per se did not change the locomotion of the animals, but it partially reduced the hyperlocomotion of methylphenidate. The pretreatment with AMPT did not affect the psychostimulant effects of NPS. Both reserpine and sulpiride inhibited the stimulatory actions of NPS and methylphenidate. These findings show that the hyperlocomotor effects of methylphenidate, but not NPS, were affected by the pretreatment with AMPT. Furthermore, methylphenidate- and NPS-induced hyperlocomotion was impaired by reserpine and sulpiride pretreatments. Together, data suggests that NPS can increase locomotion even when the synthesis of catecholamines was impaired. Additionally, the hyperlocomotor effects of NPS and methylphenidate depend on monoamines vesicular storaged, mainly dopamine, and on the activation of D2 receptors. The psychostimulant effects of NPS via activation of dopaminergic system display clinical significance on the treatment of diseases which involves dopaminergic pathways, such as Parkinson s disease and drug addiction

Increased hippocampal excitability and impaired spatial memory function in mice lacking VGLUT2 selectively in neurons defined by tyrosine hydroxylase promoter activity

Three populations of neurons expressing the vesicular glutamate transporter 2 (Vglut2) were recently described in the A10 area of the mouse midbrain, of which two populations were shown to express the gene encoding, the rate-limiting enzyme for catecholamine synthesis, tyrosine hydroxylase (TH).One of these populations (‘‘TH– Vglut2 Class1’’) also expressed the dopamine transporter (DAT) gene while one did not ("TH–Vglut2 Class2"), and the remaining population did not express TH at all ("TH-Vglut2-only"). TH is known to be expressed by a promoter which shows two phases of activation, a transient one early during embryonal development, and a later one which gives rise to stable endogenous expression of the TH gene. The transient phase is, however, not specific to catecholaminergic neurons, a feature taken to advantage here as it enabled Vglut2 gene targeting within all three A10 populations expressing this gene, thus creating a new conditional knockout. These knockout mice showed impairment in spatial memory function. Electrophysiological analyses revealed a profound alteration of oscillatory activity in the CA3 region of the hippocampus. In addition to identifying a novel role for Vglut2 in hippocampus function, this study points to the need for improved genetic tools for targeting of the diversity of subpopulations of the A10 area

Increased hippocampal excitability and impaired spatial memory function in mice lacking VGLUT2 selectively in neurons defined by tyrosine hydroxylase promoter activity

Three populations of neurons expressing the vesicular glutamate transporter 2 (Vglut2) were recently described in the A10 area of the mouse midbrain, of which two populations were shown to express the gene encoding, the rate-limiting enzyme for catecholamine synthesis, tyrosine hydroxylase (TH).One of these populations (‘‘TH– Vglut2 Class1’’) also expressed the dopamine transporter (DAT) gene while one did not ("TH–Vglut2 Class2"), and the remaining population did not express TH at all ("TH-Vglut2-only"). TH is known to be expressed by a promoter which shows two phases of activation, a transient one early during embryonal development, and a later one which gives rise to stable endogenous expression of the TH gene. The transient phase is, however, not specific to catecholaminergic neurons, a feature taken to advantage here as it enabled Vglut2 gene targeting within all three A10 populations expressing this gene, thus creating a new conditional knockout. These knockout mice showed impairment in spatial memory function. Electrophysiological analyses revealed a profound alteration of oscillatory activity in the CA3 region of the hippocampus. In addition to identifying a novel role for Vglut2 in hippocampus function, this study points to the need for improved genetic tools for targeting of the diversity of subpopulations of the A10 area