Dynamic equilibrium through reinforcement learning

Reinforcement Learning is an area of Machine Learning that deals with how an agent should take actions in an environment such as to maximize the notion of accumulated reward. This type of learning is inspired by the way humans learn and has led to the creation of various algorithms for reinforcement learning. These algorithms focus on the way in which an agent’s behaviour can be improved, assuming independence as to their surroundings. The current work studies the application of reinforcement learning methods to solve the inverted pendulum problem. The importance of the variability of the environment (factors that are external to the agent) on the execution of reinforcement learning agents is studied by using a model that seeks to obtain equilibrium (stability) through dynamism – a Cart-Pole system or inverted pendulum. We sought to improve the behaviour of the autonomous agents by changing the information passed to them, while maintaining the agent’s internal parameters constant (learning rate, discount factors, decay rate, etc.), instead of the classical approach of tuning the agent’s internal parameters. The influence of changes on the state set and the action set on an agent’s capability to solve the Cart-pole problem was studied. We have studied typical behaviour of reinforcement learning agents applied to the classic BOXES model and a new form of characterizing the environment was proposed using the notion of convergence towards a reference value. We demonstrate the gain in performance of this new method applied to a Q-Learning agent.

On the Problem of Balancing the DC Capacitor Voltage Divider in Back-to-Back Multilevel Converters

This paper presents a new generalized solution for DC bus capacitors voltage balancing in back-to-back m level diode-clamped multilevel converters connecting AC networks. The solution is based on the DC bus average power flow and exploits the switching configuration redundancies. The proposed balancing solution is particularized for the back-to-back multilevel structure with m=5 levels. This back-to-back converter is studied working with bidirectional power flow, connecting an induction machine to the power grid.

Equilibrium self-assembly of colloids with distinct interaction sites: Thermodynamics, percolation, and cluster distribution functions

We calculate the equilibrium thermodynamic properties, percolation threshold, and cluster distribution functions for a model of associating colloids, which consists of hard spherical particles having on their surfaces three short-ranged attractive sites (sticky spots) of two different types, A and B. The thermodynamic properties are calculated using Wertheim's perturbation theory of associating fluids. This also allows us to find the onset of self-assembly, which can be quantified by the maxima of the specific heat at constant volume. The percolation threshold is derived, under the no-loop assumption, for the correlated bond model: In all cases it is two percolated phases that become identical at a critical point, when one exists. Finally, the cluster size distributions are calculated by mapping the model onto an effective model, characterized by a-state-dependent-functionality (f) over bar and unique bonding probability (p) over bar. The mapping is based on the asymptotic limit of the cluster distributions functions of the generic model and the effective parameters are defined through the requirement that the equilibrium cluster distributions of the true and effective models have the same number-averaged and weight-averaged sizes at all densities and temperatures. We also study the model numerically in the case where BB interactions are missing. In this limit, AB bonds either provide branching between A-chains (Y-junctions) if epsilon(AB)/epsilon(AA) is small, or drive the formation of a hyperbranched polymer if epsilon(AB)/epsilon(AA) is large. We find that the theoretical predictions describe quite accurately the numerical data, especially in the region where Y-junctions are present. There is fairly good agreement between theoretical and numerical results both for the thermodynamic (number of bonds and phase coexistence) and the connectivity properties of the model (cluster size distributions and percolation locus).

Percolation of colloids with distinct interaction sites

We generalize the Flory-Stockmayer theory of percolation to a model of associating (patchy) colloids, which consists of hard spherical particles, having on their surfaces f short-ranged-attractive sites of m different types. These sites can form bonds between particles and thus promote self-assembly. It is shown that the percolation threshold is given in terms of the eigenvalues of a m x m matrix, which describes the recursive relations for the number of bonded particles on the ith level of a cluster with no loops; percolation occurs when the largest of these eigenvalues equals unity. Expressions for the probability that a particle is not bonded to the giant cluster, for the average cluster size and the average size of a cluster to which a randomly chosen particle belongs, are also derived. Explicit results for these quantities are computed for the case f = 3 and m = 2. We show how these structural properties are related to the thermodynamics of the associating system by regarding bond formation as a (equilibrium) chemical reaction. This solution of the percolation problem, combined with Wertheim's thermodynamic first-order perturbation theory, allows the investigation of the interplay between phase behavior and cluster formation for general models of patchy colloids.

Formaldehyde in indoor air: a public health problem?

Formaldehyde was the first air pollutant, which already in the 1970s emerged as a specifically non-industrial indoor air quality problem. Yet formaldehyde remained an indoor air quality issue and the formaldehyde level in residential indoor air is among the highest of any indoor air contaminant. Formaldehyde concentrations in 4 different indoor settings (schools, office buildings, new dwellings and occupied dwellings) in Portugal were measured using Photo Ionization Detection (PID) equipment (11,7 eV lamps). All the settings presented results higher than the reference value proposed by Portuguese legislation. Furthermore, occupied dwellings showed 3 units with results above the reference. We could conclude that formaldehyde presence is a reality in monitored indoor settings. Concentration levels are higher than the Portuguese reference value for indoor settings and these can indicate health problems for occupants.

Simulação numérica do comportamento dinâmico de uma laje

Nesta dissertação pretende-se simular o comportamento dinâmico de uma laje de betão armado aplicando o Método de Elementos Finitos através da sua implementação no programa FreeFEM++. Este programa permite-nos a análise do modelo matemático tridimensional da Teoria da Elasticidade Linear, englobando a Equação de Equilíbrio, Equação de Compatibilidade e Relações Constitutivas. Tratando-se de um problema dinâmico é necessário recorrer a métodos numéricos de Integração Directa de modo a obter a resposta em termos de deslocamento ao longo do tempo. Para este trabalho escolhemos o Método de Newmark e o Método de Euler para a discretização temporal, um pela sua popularidade e o outro pela sua simplicidade de implementação. Os resultados obtidos pelo FreeFEM++ são validados através da comparação com resultados adquiridos a partir do SAP2000 e de Soluções Teóricas, quando possível.

The problem of estimating the volatility of zero coupon bond interest rate

Financial literature and financial industry use often zero coupon yield curves as input for testing hypotheses, pricing assets or managing risk. They assume this provided data as accurate. We analyse implications of the methodology and of the sample selection criteria used to estimate the zero coupon bond yield term structure on the resulting volatility of spot rates with different maturities. We obtain the volatility term structure using historical volatilities and Egarch volatilities. As input for these volatilities we consider our own spot rates estimation from GovPX bond data and three popular interest rates data sets: from the Federal Reserve Board, from the US Department of the Treasury (H15), and from Bloomberg. We find strong evidence that the resulting zero coupon bond yield volatility estimates as well as the correlation coefficients among spot and forward rates depend significantly on the data set. We observe relevant differences in economic terms when volatilities are used to price derivatives.

Fungal contamination of poultries litter: a public health problem

Exposure to certain fungi can cause human illness. Fungi cause adverse human health effects through three specific mechanisms: generation of a harmful immune response (e.g., allergy or hypersensitivity pneumonitis); direct infection by the fungal organism; by toxic-irritant effects from mold byproducts, such as mycotoxins. In Portugal there is an increasingly industry of large facilities that produce whole chickens for domestic consumption and only few investigations have reported on fungal contamination of the poultry litter. The material used for poultry litter is varied but normally can be constitute by: pine shavings; sawdust of eucalyptus; other types of wood; peanut; coffee; sugar cane; straw; hay; grass; paper processed. Litter is one of the most contributive factors to fungal contamination in poultries. Spreading litter is one of the tasks that normally involve higher exposure of the poultry workers to dust, fungi and their metabolites, such as VOC’s and mycotoxins. After being used and removed from poultries, litter is ploughed into agricultural soils, being this practice potentially dangerous for the soil environment, as well for both humans and animals. The goal of this study was to characterize litter’s fungal contamination and also to report the incidence of keratinophilic and toxigenic fungi.

Positive Solutions of the Dirichlet Problem for the One-dimensional Minkowski-Curvature Equation

We discuss existence and multiplicity of positive solutions of the Dirichlet problem for the quasilinear ordinary differential equation-(u' / root 1 - u'(2))' = f(t, u). Depending on the behaviour of f = f(t, s) near s = 0, we prove the existence of either one, or two, or three, or infinitely many positive solutions. In general, the positivity of f is not required. All results are obtained by reduction to an equivalent non-singular problem to which variational or topological methods apply in a classical fashion.

Fungal contamination of poultry litter: a public health problem

Although numerous studies have been conducted on microbial contaminants associated with various stages related to poultry and meat products processing, only a few reported on fungal contamination of poultry litter. The goals of this study were to (1) characterize litter fungal contamination and (2) report the incidence of keratinophilic and toxigenic fungi presence. Seven fresh and 14 aged litter samples were collected from 7 poultry farms. In addition, 27 air samples of 25 litters were also collected through impaction method, and after laboratory processing and incubation of collected samples, quantitative colony-forming units (CFU/m3) and qualitative results were obtained. Twelve different fungal species were detected in fresh litter and Penicillium was the most frequent genus found (59.9%), followed by Alternaria (17.8%), Cladosporium (7.1%), and Aspergillus (5.7%). With respect to aged litter, 19 different fungal species were detected, with Penicillium sp. the most frequently isolated (42.3%), followed by Scopulariopsis sp. (38.3%), Trichosporon sp. (8.8%), and Aspergillus sp. (5.5%). A significant positive correlation was found between litter fungal contamination (CFU/g) and air fungal contamination (CFU/m3). Litter fungal quantification and species identification have important implications in the evaluation of potential adverse health risks to exposed workers and animals. Spreading of poultry litter in agricultural fields is a potential public health concern, since keratinophilic (Scopulariopsis and Fusarium genus) as well as toxigenic fungi (Aspergillus, Fusarium, and Penicillium genus) were isolated.

Fast growing fungi: a problem to be solved to achieve characterization of occupational exposure to fungi in cork industry

Chrysonilia sitophila is a common mould in cork industry and has been identified as a cause of IgE sensitization and occupational asthma. This fungal species have a fast growth rate that may inhibit others species’ growth causing underestimated data from characterization of occupational fungal exposure. Aiming to ascertain occupational exposure to fungi in cork industry, were analyzed papers from 2000 about the best air sampling method, to obtain quantification and identification of all airborne culturable fungi, besides the ones that have fast-growing rates. Impaction method don’t allows the collection of a representative air volume, because even with some media that restricts the growth of the colonies, in environments with higher fungal load, such as cork industry, the counting of the colonies is very difficult. Otherwise, impinger method permits the collection of a representative air volume, since we can make dilution of the collected volume. Besides culture methods that allows fungal identification trough macro- and micro-morphology, growth features, thermotolerance and ecological data, we can apply molecular biology with the impinger method, to detect the presence of non-viable particles and potential mycotoxin producers’ strains, and also to detect mycotoxins presence with ELISA or HPLC. Selection of the best air sampling method in each setting is crucial to achieve characterization of occupational exposure to fungi. Information about the prevalent fungal species in each setting and also the eventual fungal load it’s needed for a criterious selection.

EWASTEU Programme: proposals to minimise the problem of e‑waste

Hoje em dia muitos dos equipamentos elétricos e eletrónicos que compramos ficam obsoletos num curto espaço de tempo por causa dos rápidos avanços tecnológicos neste campo. Equipamentos como computadores, telemóveis e equipamentos elétricos e eletrónicos de pequeno e grande porte são transformados em lixo eletrónico e muitos deles são despejados no lixo comum. Para alterar este cenário, a União Europeia publicou diretivas neste domínio com o intuito de controlar o crescimento do lixo eletrónico e reduzir o seu impacto. Neste contexto, a Universidade de Yaşar (Turquia) submeteu à União Europeia um projeto (EWASTEU) com o objetivo de fornecer uma visão do que está acontecer com o equipamento transformado em lixo eletrónico e de apresentar algumas propostas para minimizar este problema. Uma das principais questões a ser respondida será a adequação das diretivas europeias.

Equilibrium distributions of discrete non-autonomous graphs

We introduce the notions of equilibrium distribution and time of convergence in discrete non-autonomous graphs. Under some conditions we give an estimate to the convergence time to the equilibrium distribution using the second largest eigenvalue of some matrices associated with the system.

Positive radial solutions of the Dirichlet problem for the Minkowski-Curvature equation in a ball

We study the existence and multiplicity of positive radial solutions of the Dirichlet problem for the Minkowski-curvature equation { -div(del upsilon/root 1-vertical bar del upsilon vertical bar(2)) in B-R, upsilon=0 on partial derivative B-R,B- where B-R is a ball in R-N (N >= 2). According to the behaviour off = f (r, s) near s = 0, we prove the existence of either one, two or three positive solutions. All results are obtained by reduction to an equivalent non-singular one-dimensional problem, to which variational methods can be applied in a standard way.