Dynamic equilibrium through reinforcement learning

Reinforcement Learning is an area of Machine Learning that deals with how an agent should take actions in an environment such as to maximize the notion of accumulated reward. This type of learning is inspired by the way humans learn and has led to the creation of various algorithms for reinforcement learning. These algorithms focus on the way in which an agent’s behaviour can be improved, assuming independence as to their surroundings. The current work studies the application of reinforcement learning methods to solve the inverted pendulum problem. The importance of the variability of the environment (factors that are external to the agent) on the execution of reinforcement learning agents is studied by using a model that seeks to obtain equilibrium (stability) through dynamism – a Cart-Pole system or inverted pendulum. We sought to improve the behaviour of the autonomous agents by changing the information passed to them, while maintaining the agent’s internal parameters constant (learning rate, discount factors, decay rate, etc.), instead of the classical approach of tuning the agent’s internal parameters. The influence of changes on the state set and the action set on an agent’s capability to solve the Cart-pole problem was studied. We have studied typical behaviour of reinforcement learning agents applied to the classic BOXES model and a new form of characterizing the environment was proposed using the notion of convergence towards a reference value. We demonstrate the gain in performance of this new method applied to a Q-Learning agent.

Sobrevivência: jogo interactivo para estudo de arquitecturas de agentes autónomos

Trabalho de projeto para obtenção do grau de Mestre em Engenharia Informática e de Computadores

(Virtual) Agents for running electricity markets

This paper describes a multi-agent based simulation (MABS) framework to construct an artificial electric power market populated with learning agents. The artificial market, named TEMMAS (The Electricity Market Multi-Agent Simulator), explores the integration of two design constructs: (i) the specification of the environmental physical market properties and (ii) the specification of the decision-making (deliberative) and reactive agents. TEMMAS is materialized in an experimental setup involving distinct power generator companies that operate in the market and search for the trading strategies that best exploit their generating units' resources. The experimental results show a coherent market behavior that emerges from the overall simulated environment.

A new control strategy with saturation effect compensation for an autonomous induction generator drivenby wide speed range turbines

This paper presents a variable speed autonomous squirrel cage generator excited by a current-controlled voltage source inverter to be used in stand-alone micro-hydro power plants. The paper proposes a system control strategy aiming to properly excite the machine as well as to achieve the load voltage control. A feed-forward control sets the appropriate generator flux by taking into account the actual speed and the desired load voltage. A load voltage control loop is used to adjust the generated active power in order to sustain the load voltage at a reference value. The control system is based on a rotor flux oriented vector control technique which takes into account the machine saturation effect. The proposed control strategy and the adopted system models were validated both by numerical simulation and by experimental results obtained from a laboratory prototype. Results covering the prototype start-up, as well as its steady-state and dynamical behavior are presented. (C) 2011 Elsevier Ltd. All rights reserved.

A Service Integration Platform for Collaborative Networks

Integrated manufacturing constitutes a complex system made of heterogeneous information and control subsystems. Those subsystems are not designed to the cooperation. Typically each subsystem automates specific processes, and establishes closed application domains, therefore it is very difficult to integrate it with other subsystems in order to respond to the needed process dynamics. Furthermore, to cope with ever growing marketcompetition and demands, it is necessary for manufacturing/enterprise systems to increase their responsiveness based on up-to-date knowledge and in-time data gathered from the diverse information and control systems. These have created new challenges for manufacturing sector, and even bigger challenges for collaborative manufacturing. The growing complexity of the information and communication technologies when coping with innovative business services based on collaborative contributions from multiple stakeholders, requires novel and multidisciplinary approaches. Service orientation is a strategic approach to deal with such complexity, and various stakeholders' information systems. Services or more precisely the autonomous computational agents implementing the services, provide an architectural pattern able to cope with the needs of integrated and distributed collaborative solutions. This paper proposes a service-oriented framework, aiming to support a virtual organizations breeding environment that is the basis for establishing short or long term goal-oriented virtual organizations. The notion of integrated business services, where customers receive some value developed through the contribution from a network of companies is a key element.

Commercial agents portfolio optimization in electricity markets

As it is well known, competitive electricity markets require new computing tools for power companies that operate in retail markets in order to enhance the management of its energy resources. During the last years there has been an increase of the renewable penetration into the micro-generation which begins to co-exist with the other existing power generation, giving rise to a new type of consumers. This paper develops a methodology to be applied to the management of the all the aggregators. The aggregator establishes bilateral contracts with its clients where the energy purchased and selling conditions are negotiated not only in terms of prices but also for other conditions that allow more flexibility in the way generation and consumption is addressed. The aggregator agent needs a tool to support the decision making in order to compose and select its customers' portfolio in an optimal way, for a given level of profitability and risk.

Symbolic dynamics and renormalization of non-autonomous k periodic dynamical systems

The purpose of this paper was to introduce the symbolic formalism based on kneading theory, which allows us to study the renormalization of non-autonomous periodic dynamical systems.

Nanofiltration of Cork Wastewaters and Their Possible Use in Leather Industry as Tanning Agents

Cork processing wastewater is a very complex mixture of vegetal extracts and has, among other natural compounds, a very high content of phenolic/tannic colloidal matter that is responsible for severe environmental problems. In the present work, the concentration of this wastewater by nanofiltration was investigated with the aim of producing a cork tannin concentrate to be utilized in tanning. Permeation results showed that the permeate fluxes are controlled by both osmotic pressure and fouling/gel layer phenomena, leading to a rapid decrease of permeate fluxes with the concentration factor. The rejection coefficients to organic matter were higher than 95%, indicating that nanofiltration has a very good ability to concentrate the tannins and produce a permeate stream depleted from organic matter. The cork tannin concentrate obtained by nanofiltration and evaporation had total solids concentration of 34.8 g/l. The skins tanned by this concentrate were effectively converted to leather with a shrinking temperature of 7 degrees C.

Molecular imaging agents for detection of β-amyloid plaques in Alzheimer’s disease

The formation of amyloid structures is a neuropathological feature that characterizes several neurodegenerative disorders, such as Alzheimer´s and Parkinson´s disease. Up to now, the definitive diagnosis of these diseases can only be accomplished by immunostaining of post mortem brain tissues with dyes such Thioflavin T and congo red. Aiming at early in vivo diagnosis of Alzheimer´s disease (AD), several amyloid-avid radioprobes have been developed for b-amyloid imaging by positron emission tomography (PET) and single-photon emission computed tomography (SPECT). The aim of this paper is to present a perspective of the available amyloid imaging agents, special those that have been selected for clinical trials and are at the different stages of the US Food and Drugs Administration (FDA) approval.

Implementação das medidas de autoproteção em edifícios escolares

Mestrado em Segurança e Higiene no Trabalho

Road lane detection for autonomous robot guidance

Computational Vision stands as the most comprehensive way of knowing the surrounding environment. Accordingly to that, this study aims to present a method to obtain from a common webcam, environment information to guide a mobile differential robot through a path similar to a roadway.

Genotoxic assessment in different exposure groups working with antineoplastic agents

Antioneoplastic drugs are widely used in treatment of cancer, and several studies suggest acute and long-term effects associated to antineoplastic drug exposures, namely associating workplace exposure with health effects. Cytokinesis blocked micronucleus (CBMN) assay is one promising short-term genotoxicity assays for human risk assessment and their combination is recommended to monitor populations chronically exposed to genotoxic agents. The aim of this investigation is the genotoxicity assessment in different professionals that handle cytostatics drugs. This research is case-control blinded study constituted by 46 non-exposed subjects and 44 workers that handle antineoplastic drugs, such as pharmacists, pharmacy technicians, and nurses. It was found statistically significant increases in the genotoxicity biomarkers in exposed comparising with controls (p<0.05). The findings address the need for regular biomonitoring of personnel occupationally exposed to these drugs, confirming to an enhanced health risk assessment.

Road lane detection for autonomous robot guidance

Computational Vision stands as the most comprehensive way of knowing the surrounding environment. Accordingly to that, this study aims to present a method to obtain from a common webcam, environment information to guide a mobile differential robot through a path similar to a roadway.

Equilibrium distributions of discrete non-autonomous graphs

We introduce the notions of equilibrium distribution and time of convergence in discrete non-autonomous graphs. Under some conditions we give an estimate to the convergence time to the equilibrium distribution using the second largest eigenvalue of some matrices associated with the system.

Generalized synchronization in a system of several non-autonomous oscillators coupled by a medium

An abstract theory on general synchronization of a system of several oscillators coupled by a medium is given. By generalized synchronization we mean the existence of an invariant manifold that allows a reduction in dimension. The case of a concrete system modeling the dynamics of a chemical solution on two containers connected to a third container is studied from the basics to arbitrary perturbations. Conditions under which synchronization occurs are given. Our theoretical results are complemented with a numerical study.