Dynamic equilibrium through reinforcement learning

Reinforcement Learning is an area of Machine Learning that deals with how an agent should take actions in an environment such as to maximize the notion of accumulated reward. This type of learning is inspired by the way humans learn and has led to the creation of various algorithms for reinforcement learning. These algorithms focus on the way in which an agent’s behaviour can be improved, assuming independence as to their surroundings. The current work studies the application of reinforcement learning methods to solve the inverted pendulum problem. The importance of the variability of the environment (factors that are external to the agent) on the execution of reinforcement learning agents is studied by using a model that seeks to obtain equilibrium (stability) through dynamism – a Cart-Pole system or inverted pendulum. We sought to improve the behaviour of the autonomous agents by changing the information passed to them, while maintaining the agent’s internal parameters constant (learning rate, discount factors, decay rate, etc.), instead of the classical approach of tuning the agent’s internal parameters. The influence of changes on the state set and the action set on an agent’s capability to solve the Cart-pole problem was studied. We have studied typical behaviour of reinforcement learning agents applied to the classic BOXES model and a new form of characterizing the environment was proposed using the notion of convergence towards a reference value. We demonstrate the gain in performance of this new method applied to a Q-Learning agent.

Carbon balance and crop residue management in dynamic equilibrium under a no-till system in Campos Gerais

The adoption of no-tillage systems (NT) and the maintenance of crop residues on the soil surface result in the long-term increase of carbon (C) in the system, promoting C sequestration and reducing C-CO2 emissions to the atmosphere. The purpose of this study was to evaluate the C sequestration rate and the minimum amount of crop residues required to maintain the dynamic C equilibrium (dC/dt = 0) of two soils (Typic Hapludox) with different textural classes. The experiment was arranged in a 2 x 2 x 2 randomized block factorial design. The following factors were analyzed: (a) two soil types: Typic Hapludox (Oxisol) with medium texture (LVTM) and Oxisol with clay texture (LVTA), (b) two sampling layers (0-5 and 5-20 cm), and (c) two sampling periods (P1 - October 2007; P2 - September 2008). Samples were collected from fields under a long-term (20 years) NT system with the following crop rotations: wheat/soybean/black oat + vetch/maize (LVTM) and wheat/maize/black oat + vetch/soybean (LVTA). The annual C sequestration rates were 0.83 and 0.76 Mg ha-1 for LVTM and LVTA, respectively. The estimates of the minimum amount of crop residues required to maintain a dynamic equilibrium (dC/dt = 0) were 7.13 and 6.53 Mg ha-1 year-1 for LVTM and LVTA, respectively. The C conversion rate in both studied soils was lower than that reported in other studies in the region, resulting in a greater amount of crop residues left on the soil surface.

Is the association between late-life morbidity and disability attenuated over time? : Exploring the dynamic equilibrium of morbidity hypothesis

Context: There is evidence suggesting that the prevalence of disability in late life has declined over time while the prevalence of disabling chronic diseases has increased. The dynamic equilibrium of morbidity hypothesis suggests that these seemingly contradictory trends are due to the attenuation of the morbidity-disability link over time. The aim of this study was to empirically test this assumption.Methods: Data were drawn from three repeated cross-sections of SWEOLD, a population-based survey among the Swedish men and women ages 77 and older. Logistic regression models were fitted to assess the trends in the prevalence of Activities of Daily Living (ADL) disability, Instrumental ADL (IADL) disability, and selected groups of chronic conditions. The changes in the associations between chronic conditions and disabilities were examined in both multiplicative and additive models.Results: Between 1992 and 2011, the odds of ADL disability significantly declined among women whereas the odds of IADL disability significantly declined among men. During the same period, the prevalence of most chronic morbidities including multimorbidity went up. Significant attenuations of the morbidity-disability associations were found for cardiovascular diseases, metabolic disorders, poor lung function, psychological distress, and multimorbidity.Conclusion: In agreement with the dynamic equilibrium hypothesis, this study concludes that the associations between chronic conditions and disability among the Swedish older adults have largely waned over time.

Consumption externalities, habit formation, and equilibrium efficiency

We analyze the welfare properties of the competitive equilibrium in a capital accumulation model where individual preferences are subjected to both habit formation and consumption spillovers. We also discuss how consumption externalities and habits interact to generate an inefficient dynamic equilibrium. Finally, we characterize optimal tax policies aimed to restore efficient decentralized paths.

Dynamic linear economies with social interactions

Social interactions arguably provide a rationale for several important phenomena, from smoking and other risky behavior in teens to e.g., peer effects in school performance. We study social interactions in dynamic economies. For these economies, we provide existence (Markov Perfect Equilibrium in pure strategies), ergodicity, and welfare results. Also, we characterize equilibria in terms of agents' policy function, spatial equilibrium correlations and social multiplier effects, depending on the nature of interactions. Most importantly, we study formally the issue of the identification of social interactions, with special emphasis on the restrictions imposed by dynamic equilibrium conditions.

Dynamic behavior, stochasticity and enviromental heterogeneity in Tribolium castaneum

In this study we analyzed the influence of demographic parameters on the population dynamics of Tribolium castaneum, combining empiricism and population theory to analyze the different effects of environmental heterogeneity, by employing Ricker models, designed to study a two-patch system taking into account deterministic and stochastic analysis. Results were expressed by bifurcation diagrams and stochastic simulations. Dynamic equilibrium was widely investigated with results suggesting specific parametric spaces in response to environmental heterogeneity and migration. Population equilibrium patterns, synchrony and persistence in T. castaneum were discussed

Double-level “orthogonal” dynamic combinatorial libraries on transition metal template

Dynamic combinatorial libraries are mixtures of compounds that exist in a dynamic equilibrium and can be driven to compositional self adaptation via selective binding of a specific assembly of certain components to a molecular target. We present here an extension of this initial concept to dynamic libraries that consists of two levels, the first formed by the coordination of terpyridine-based ligands to the transition metal template, and the second, by the imine formation with the aldehyde substituents on the terpyridine moieties. Dialdehyde 7 has been synthesized, converted into a variety of ligands, oxime ethers L11–L33 and acyl hydrazones L44–L77, and subsequently into corresponding cobalt complexes. A typical complex, Co(L22)22+ is shown to engage in rapid exchange with a competing ligand L11 and with another complex, Co(L22)22+ in 30% acetonitrile/water at pH 7.0 and 25°C. The exchange in the corresponding Co(III) complexes is shown to be much slower. Imine exchange in the acyl hydrazone complexes (L44–L77) is strongly controlled by pH and temperature. The two types of exchange, ligand and imine, can thus be used as independent equilibrium processes controlled by different types of external intervention, i.e., via oxidation/reduction of the metal template and/or change in the pH/temperature of the medium. The resulting double-level dynamic libraries are therefore named orthogonal, in similarity with the orthogonal protecting groups in organic synthesis. Sample libraries of this type have been synthesized and showed the complete expected set of components in electrospray ionization MS.

Qualidade de vida e avaliação fisica em intoxicados por mercurio : estudo observacional transversal descritivo

Universidade Estadual de Campinas . Faculdade de Educação Física

Headland-bay beach planform stability of Santa Catarina State and of the Northern Coast of São Paulo State

This paper presents the results of the planform stability classification for the headland-bay beaches of the State of Santa Catarina and of the Northern Coast of São Paulo, based on the application of the Parabolic Bay-Shape Equation (PBSE) to aerial images of the beaches, using the software MEPBAY®. For this purpose, georeferenced mosaics of the QuickBird2® satellite imagery (for the State of Santa Catarina) and vertical aerial photographs (for the northern coast of São Paulo State) were used. Headland-bay beach planform stability can be classified as: (1) in static equilibrium, (2) in dynamic equilibrium, (3) unstable or (4) in a state of natural beach reshaping. Static equilibrium beaches are the most frequent along the coast of the State of Santa Catarina and the Northern Shore of São Paulo, notably along the most rugged sectors of the coast and those with experiencing lower fluvial discharge. By comparison, dynamic equilibrium beaches occur primarily on the less rugged sectors of the coast and along regions with higher fluvial discharge. Beaches in a state of natural beach reshaping have only been found in SC, associated with stabilized estuarine inlets or port breakwaters. However, it is not possible to classify any of these beaches as unstable because only one set of images was used. No clear relation was observed between a beach's planform stability and other classification factors, such as morphodynamics or orientation.

Debt Maturity: Is Long-Term Debt Optimal?

We model and calibrate the arguments in favor and against short-term and long-term debt. These arguments broadly include: maturity premium, sustainability, and service smoothing. We use a dynamic-equilibrium model with tax distortions and government outlays uncertainty, and model maturity as the fraction of debt that needs to be rolled over every period. In the model, the benefits of defaulting are tempered by higher future interest rates. We then calibrate our artificial economy and solve for the optimal debt maturity for Brazil as an example of a developing country and the US as an example of a mature economy. We obtain that the calibrated costs from defaulting on long-term debt more than offset costs associated with short-term debt. Therefore, short-term debt implies higher welfare levels.

Optimal reserve management and sovereign debt

Most models currently used to determine optimal foreign reserve holdings take the level of international debt as given. However, given the sovereign`s willingness-to-pay incentive problems, reserve accumulation may reduce sustainable debt levels. In addition, assuming constant debt levels does not allow addressing one of the puzzles behind using reserves as a means to avoid the negative effects of crisis: why do not sovereign countries reduce their sovereign debt instead? To study the joint decision of holding sovereign debt and reserves, we construct a stochastic dynamic equilibrium model calibrated to a sample of emerging markets. We obtain that the reserve accumulation does not play a quantitatively important role in this model. In fact, we find the optimal policy is not to hold reserves at all. This finding is robust to considering interest rate shocks, sudden stops, contingent reserves and reserve dependent output costs. (c) 2008 Elsevier B.V. All rights reserved.

Nominal versus indexed debt: A quantitative horse race

The main arguments in favor and against nominal and indexed debts are the incentive to default through inflation versus hedging against unforeseen shocks. We model and calibrate these arguments to assess their quantitative importance. We use a dynamic equilibrium model with tax distortion, government outlays uncertainty, and contingent-debt service. Our framework also recognizes that contingent debt can be associated with incentive problems and lack of commitment. Thus, the benefits of unexpected inflation are tempered by higher interest rates. We obtain that costs from inflation more than offset the benefits from reducing tax distortions. We further discuss sustainability of nominal debt in developing (volatile) countries. (C) 2010 Elsevier Ltd. All rights reserved.

Galectin-3 regulates peritoneal B1-cell differentiation into plasma cells

Extracellular galectin-3 participates in the control of B2 lymphocyte migration and adhesion and of their differentiation into plasma cells. Here, we analyzed the role of galectin-3 in B1-cell physiology and the balance between B1a and B1b lymphocytes in the peritoneal cavity. In galectin-3(-/-) mice, the total number of B1a lymphocytes was lower, while B1b lymphocyte number was higher as compared to wild-type mice. The differentiation of B1a cells into plasma cells was associated with their abnormal adhesion and location on the mesentery. The B220 and CD43, constitutively expressed by B1 lymphocytes, were respectively up- and downregulated in galectin-3(-/-) mice. Mononuclear cells were strongly adhered to the mesenteric membranes of both CD43(-/-) and galectin-3(-/-) mice, but in contrast to CD43(-/-) mice, the accumulation of B1 cells in peritoneal membranes in galectin-3(-/-) mice was accompanied by their functional differentiation into plasma cells. We have shown that in the absence of galectin-3, B1-cell differentiation into plasma cells is favored and the dynamic equilibrium of B1-cell populations in the peritoneum is maintained through a compensatory increase in B1b lymphocytes.

Structural Dynamics in (ND4)2[Cu(D2O)6](SO4)2

The extended X-ray absorption fine structure spectroscopy (EXAFS) of (ND4)(2)[CU(D2O)(6)](SO4)(2) at 5, 14,100, 200, and 298 K is reported. This indicates that the Cu-O bond lengths of the Cu(D2O)(6)(2+) ion do not change significantly within this temperature range, which contrasts with EPR results and X-ray and neutron diffraction experiments, which imply that two of the Cu-(D2O) bonds converge in length as the temperature is raised. The EXAFS measurements thus confirm that the bond distances yielded by the diffraction experiments refer to the average positions of ligands involved in a dynamic equilibrium in which the directions of the long and intermediate bonds of the Jahn-Teller distorted Cu(D2O)(6)(2+) ion are interchanged in the crystal lattice. Analysis of the displacement parameters is consistent with this interpretation, as are the wave functions calculated using a model involving Jahn-Teller vibronic coupling and the influence of lattice strain interactions.

Kinetics of baculovirus replication and release using real-time quantitative polymerase chain reaction

The study of viral-based processes is hampered by (a) their complex, transient nature, (b) the instability of products, and (c) the lack of accurate diagnostic assays. Here, we describe the use of real-time quantitative polymerase chain reaction to characterize baculoviral infection. Baculovirus DNA content doubles every 1.7 h from 6 h post-infection until replication is halted at the onset of budding. No dynamic equilibrium exists between replication and release, and the kinetics are independent of the cell density at the time of infection. No more than 16% of the intracellular virus copies bud from the cell. (C) 2002 John Wiley & Sons, Inc. Biotechnol Bioeng 77: 476-480, 2002; DOI 10.1002/bit.10126.