Dynamic equilibrium through reinforcement learning

Reinforcement Learning is an area of Machine Learning that deals with how an agent should take actions in an environment such as to maximize the notion of accumulated reward. This type of learning is inspired by the way humans learn and has led to the creation of various algorithms for reinforcement learning. These algorithms focus on the way in which an agent’s behaviour can be improved, assuming independence as to their surroundings. The current work studies the application of reinforcement learning methods to solve the inverted pendulum problem. The importance of the variability of the environment (factors that are external to the agent) on the execution of reinforcement learning agents is studied by using a model that seeks to obtain equilibrium (stability) through dynamism – a Cart-Pole system or inverted pendulum. We sought to improve the behaviour of the autonomous agents by changing the information passed to them, while maintaining the agent’s internal parameters constant (learning rate, discount factors, decay rate, etc.), instead of the classical approach of tuning the agent’s internal parameters. The influence of changes on the state set and the action set on an agent’s capability to solve the Cart-pole problem was studied. We have studied typical behaviour of reinforcement learning agents applied to the classic BOXES model and a new form of characterizing the environment was proposed using the notion of convergence towards a reference value. We demonstrate the gain in performance of this new method applied to a Q-Learning agent.

Harvesting dynamic aspects from a real rational map : the bulb of period 5

This journal provides immediate open access to its content on the principle that making research freely available to the public supports a greater global exchange of knowledge.

The effect of metal transfer stability (spattering) on fume generation, morphology and composition in short-circuit MAG welding

The main objective of this work was to evaluate the hypothesis that the greater transfer stability leads also to less volume of fumes. Using an Ar + 25%CO2 blend as shielding gas and maintaining constant the average current, wire feed speed and welding speed, bead-on-plate welds were carried out with plain carbon steel solid wire. The welding voltage was scanned to progressively vary the transfer stability. Using two conditions of low stability and one with high stability, fume generation was evaluated by means of the AWS F1.2:2006 standard. The influence of these conditions on fume morphology and composition was also verified. A condition with greater transfer stability does not generate less fume quantity, despite the fact that this condition produces fewer spatters. Other factors such as short-circuit current, arcing time, droplet diameters and arc length are the likely governing factors, but in an interrelated way. Metal transfer stability does not influence either the composition or the size/morphology of fume particulates. (c) 2014 Elsevier B.V. All rights reserved.

Dynamical Stability and Predictability of Football Players: The Study of One Match

The game of football demands new computational approaches to measure individual and collective performance. Understanding the phenomena involved in the game may foster the identification of strengths and weaknesses, not only of each player, but also of the whole team. The development of assertive quantitative methodologies constitutes a key element in sports training. In football, the predictability and stability inherent in the motion of a given player may be seen as one of the most important concepts to fully characterise the variability of the whole team. This paper characterises the predictability and stability levels of players during an official football match. A Fractional Calculus (FC) approach to define a player’s trajectory. By applying FC, one can benefit from newly considered modeling perspectives, such as the fractional coefficient, to estimate a player’s predictability and stability. This paper also formulates the concept of attraction domain, related to the tactical region of each player, inspired by stability theory principles. To compare the variability inherent in the player’s process variables (e.g., distance covered) and to assess his predictability and stability, entropy measures are considered. Experimental results suggest that the most predictable player is the goalkeeper while, conversely, the most unpredictable players are the midfielders. We also conclude that, despite his predictability, the goalkeeper is the most unstable player, while lateral defenders are the most stable during the match.

In vitro digestion and stability assessment of b-lactoglobulin/riboflavin nanostructures

B-Lactoglobulin (b-Lg) is the major protein fraction of bovine whey serum and a primary gelling agent. b-Lg has a high nutritional value, is stable at low pH being highly resistant to proteolytic degradation in the stomach, besides, it has the ability of acting as an encapsulating agent. This study aims at assessing the ability of b-Lg nanostructures to associate a nutraceutical - i.e. riboflavin - and release it in a controlled manner throughout an in vitro gastrointestinal (GI) system. For this reason b-Lg nanostructures loaded with riboflavin were critically characterized in terms of their morphology (i.e. size, polydispersity, -potential and shape) by dynamic light scattering (DLS) and transmission electron microscopy (TEM), and efficiency to associate to riboflavin through spectrofluorimetry. Furthermore, these nanocomplexes were evaluated in an in vitro GI model, simulating the physiological conditions. Stable b-Lg nanostructures were obtained at pH 6, of spherical shape, characterized by particle size of 172±1 nm, low polydispersity (i.e. PDI of 0.06±0.02), -potential of -32±3 mV and association efficiency (AE) of 26±1 %. b-Lg nanostructures showed to be stable upon their passage throughout stomach (i.e. particle size, PDI and potential of 248±10 nm, 0.18±0.03 and 18±3 mV, respectively). Concerning their passage throughout the intestine, such nanostructures were mostly degraded in the duodenum. Regarding riboflavin, a release of about 11 % was observed after their passage through stomach, while 35 %, 38 % and 5 % were the released percentages of the total riboflavin associated observed after passage through duodenum, jejunum and ileum, respectively. Hence,b-Lg nanostructures showed to be suitable carriers for riboflavin until the intestine, where their degradation occurs. b-Lg also showed to be structurally stable, under food simulant conditions (yoghurt simulant, composed of 3 % acetic acid), over 14 days, with a protective effect upon riboflavin activity, releasing it in a 7 day period.

DINAMICA MICROBIANA Y CALIDAD DE LA MATERIA ORGANICA DEL SUELO COMO INDICADORES DE SUSTENTABILIDAD EN EL CHACO ARIDO DE LA PROVINCIA DE CORDOBA Microbial dynamic and organic matter quality as indicators of soil sustainability in Dry Chaco of Córdoba Province

Se propone analizar el efecto del uso productivo en el Chaco Árido de la provincia de Córdoba, mediante la aplicación de indicadores de sustentabilidad relacionados con la calidad de la materia orgánica y la liberación de nutrientes en el suelo, con la finalidad de aportar a un tema de suma interes para la provincia de Córdoba como es la formulación de criterios y pautas de manejo para la implementación de la Ley de Bosques (N° 26331). Se trabajará en la localidad de San Miguel en el departamento Pocho, en un sitio de bosque no disturbado y en tres sistemas productivos: desmonte selectivo con implantación de pasturas; desmonte total con agricultura bajo riego y desmonte total sobrepastoreado. En cada sitio se medirá “in situ” la emisión de CO2 y se tomaran muestras de suelo a las que se les determinará: a) contenido de materia orgánica total (MO), b) contenido de sustancias húmicas (SH), diferenciando ácidos húmicos (AH) y fúlvicos (AF), c) abundancia y actividad de microorganismos nitrificadores y d) propiedades químicas de los AH y AF. Se calcularán los siguientes índices de sustentabilidad a) materia orgánica biodisponible (MOB=MO–SH); b) índice de humificación (IH=SH/MO); c) tipo de humus (TH=AF/AH; d) índice de mineralización de C (IMC=CO2/MO); e) índice de nitrificación (IN=actividad/abundancia); y f) índice de estabilidad de las fracciones humificadas: compuestos aromáticos/ alifáticos. Los datos serán analizados estadísticamente mediante ANOVA y comparación de medias por LSD (P<0.05) y tests multivariados. We proposed analyze the effect of land use in Arid Chaco of Cordoba province, using sustainability indicators related to organic matter quality and nutrient release in soil, with the aim to formulate management criteria for the implementation of the Ley de Bosques (N° 26331) in Córdoba province. The study will be conducted in San Miguel village in Pocho department, in one undisturbed forest site and three productive systems: selective clearing with grass sowing; total clearing with irrigation agriculture and total clearing with overgrazed. In each site "in situ" CO2 emission will be measured and soil samples will be taken, in which the following parameters will be determined: a) total organic matter content (MO), b) humic substances content (SH), in humic acids (AH) and fulvic acids (AF), c) abundance and activity of nitrifier microorganisms and d) chemical properties of AH and AF. The sustainability indexes will be calculated: biodisponible organic matter (MOB=MO–SH); b) humification index (IH=SH/MO); c) humus type (TH=AF/AH; d) C mineralization index (IMC=CO2/MO); e) nitrifying index (IN=activity/abundance); and f) humic fractions stability index: aromatic/aliphatic compounds. The data will be statistically analyzed by ANOVA and the means will be compared by LSD (P<0.05) and multivariate tests.

Stability of Growth Models with Generalised Lag Structures

This paper considers the lag structures of dynamic models in economics, arguing that the standard approach is too simple to capture the complexity of actual lag structures arising, for example, from production and investment decisions. It is argued that recent (1990s) developments in the the theory of functional differential equations provide a means to analyse models with generalised lag structures. The stability and asymptotic stability of two growth models with generalised lag structures are analysed. The paper concludes with some speculative discussion of time-varying parameters.

Intra-individual stability of neuromotor tasks from 6 to 18 years: a longitudinal study.

This study investigates the intra-individual stability of the speed of several motor tasks and the intensity of associated movements in 256 children (131 girls, 125 boys) from the Zurich generational study using the Zurich neuromotor assessment battery (ZNA) over a 12-year period from the age of 6 to 18 years. The stability was assessed by correlograms of standard deviation scores calculated from age- and gender-adjusted normative values and compared with standing height and full scale intelligence quotient (IQ). While motor tasks of hand, finger and foot (HFT) and contralateral associated movements (CAM) exhibited a moderate stability (summary measure as correlation coefficients between two measurements made 4 years apart: .61 and .60), other tasks (dynamic balance, static balance and pegboard) were only weakly stable (.46, .47 and .49). IQ and height were more stable than neuromotor components (.72 and .86). We conclude that the moderately stable HFT and CAM may reflect "motor traits", while the stability of the pegboard and balance tasks is weaker because these skills are more experience related and state-dependent.

Decoding stimulus-related information from single-trial EEG responses based on voltage topographies

Neuroimaging studies typically compare experimental conditions using average brain responses, thereby overlooking the stimulus-related information conveyed by distributed spatio-temporal patterns of single-trial responses. Here, we take advantage of this rich information at a single-trial level to decode stimulus-related signals in two event-related potential (ERP) studies. Our method models the statistical distribution of the voltage topographies with a Gaussian Mixture Model (GMM), which reduces the dataset to a number of representative voltage topographies. The degree of presence of these topographies across trials at specific latencies is then used to classify experimental conditions. We tested the algorithm using a cross-validation procedure in two independent EEG datasets. In the first ERP study, we classified left- versus right-hemifield checkerboard stimuli for upper and lower visual hemifields. In a second ERP study, when functional differences cannot be assumed, we classified initial versus repeated presentations of visual objects. With minimal a priori information, the GMM model provides neurophysiologically interpretable features - vis à vis voltage topographies - as well as dynamic information about brain function. This method can in principle be applied to any ERP dataset testing the functional relevance of specific time periods for stimulus processing, the predictability of subject's behavior and cognitive states, and the discrimination between healthy and clinical populations.

Symmetries and fixed point stability of stochastic differential equations modeling self-organized criticality

A stochastic nonlinear partial differential equation is constructed for two different models exhibiting self-organized criticality: the Bak-Tang-Wiesenfeld (BTW) sandpile model [Phys. Rev. Lett. 59, 381 (1987); Phys. Rev. A 38, 364 (1988)] and the Zhang model [Phys. Rev. Lett. 63, 470 (1989)]. The dynamic renormalization group (DRG) enables one to compute the critical exponents. However, the nontrivial stable fixed point of the DRG transformation is unreachable for the original parameters of the models. We introduce an alternative regularization of the step function involved in the threshold condition, which breaks the symmetry of the BTW model. Although the symmetry properties of the two models are different, it is shown that they both belong to the same universality class. In this case the DRG procedure leads to a symmetric behavior for both models, restoring the broken symmetry, and makes accessible the nontrivial fixed point. This technique could also be applied to other problems with threshold dynamics.

Generation of dynamic structures in nonequilibrium reactive membranes

We present a nonequlibrium approach for the study of a flexible bilayer whose two components induce distinct curvatures. In turn, the two components are interconverted by an externally promoted reaction. Phase separation of the two species in the surface results in the growth of domains characterized by different local composition and curvature modulations. This domain growth is limited by the effective mixing due to the interconversion reaction, leading to a finite characteristic domain size. In addition to these effects, first introduced in our earlier work [ Phys. Rev. E 71 051906 (2005)], the important new feature is the assumption that the reactive process actively affects the local curvature of the bilayer. Specifically, we suggest that a force energetically activated by external sources causes a modification of the shape of the membrane at the reaction site. Our results show the appearance of a rich and robust dynamical phenomenology that includes the generation of traveling and/or oscillatory patterns. Linear stability analysis, amplitude equations, and numerical simulations of the model kinetic equations confirm the occurrence of these spatiotemporal behaviors in nonequilibrium reactive bilayers.

The evolutionary conserved BER1 gene is involved in microtubule stability in yeast

In yeast, microtubules are dynamic filaments necessary for spindle and nucleus positioning, as well as for proper chromosome segregation. We identify a function for the yeast gene BER1 (Benomyl REsistant 1) in microtubule stability. BER1 belongs to an evolutionary conserved gene family whose founding member Sensitivity to Red light Reduced is involved in red-light perception and circadian rhythms in Arabidopsis. Here, we present data showing that the ber1Delta mutant is affected in microtubule stability, particularly in presence of microtubule-depolymerising drugs. The pattern of synthetic lethal interactions obtained with the ber1Delta mutant suggests that Ber1 may function in N-terminal protein acetylation. Our work thus suggests that microtubule stability might be regulated through this post-translational modification on yet-to-be determined proteins

Dynamic impacts of the inhibition of the molecular chaperone hsp90 on the T-cell proteome have implications for anti-cancer therapy.

The molecular chaperone Hsp90-dependent proteome represents a complex protein network of critical biological and medical relevance. Known to associate with proteins with a broad variety of functions termed clients, Hsp90 maintains key essential and oncogenic signalling pathways. Consequently, Hsp90 inhibitors are being tested as anti-cancer drugs. Using an integrated systematic approach to analyse the effects of Hsp90 inhibition in T-cells, we quantified differential changes in the Hsp90-dependent proteome, Hsp90 interactome, and a selection of the transcriptome. Kinetic behaviours in the Hsp90-dependent proteome were assessed using a novel pulse-chase strategy (Fierro-Monti et al., accompanying article), detecting effects on both protein stability and synthesis. Global and specific dynamic impacts, including proteostatic responses, are due to direct inhibition of Hsp90 as well as indirect effects. As a result, a decrease was detected in most proteins that changed their levels, including known Hsp90 clients. Most likely, consequences of the role of Hsp90 in gene expression determined a global reduction in net de novo protein synthesis. This decrease appeared to be greater in magnitude than a concomitantly observed global increase in protein decay rates. Several novel putative Hsp90 clients were validated, and interestingly, protein families with critical functions, particularly the Hsp90 family and cofactors themselves as well as protein kinases, displayed strongly increased decay rates due to Hsp90 inhibitor treatment. Remarkably, an upsurge in survival pathways, involving molecular chaperones and several oncoproteins, and decreased levels of some tumour suppressors, have implications for anti-cancer therapy with Hsp90 inhibitors. The diversity of global effects may represent a paradigm of mechanisms that are operating to shield cells from proteotoxic stress, by promoting pro-survival and anti-proliferative functions. Data are available via ProteomeXchange with identifier PXD000537.

Stability Evaluation of a Paper Machine Wet End

Työssä analysoidaanprosessin vaikutusta paperikoneen stabiiliuteen. Kaksi modernia sanomalehtipaperikonetta analysoitiin ja sen perusteella molemmista prosesseista rakennettiin fysiikan lakeihin perustuvat simulointimallit APROS Paper simulointiohjelmistolla. Työn tavoitteena on selvittää, miten kyseisten koneiden prosessit eroavat toisistaan ja arvioida, miten havaitut erot vaikuttavat prosessien stabiiliuteen. Työssä tarkastellaan periodisten häiriöiden vaimenemista prosessissa. Simuloinnissa herätteenä käytettiin puhdasta valkoista kohinaa, jonka avulla eri taajuistenperiodisten häiriöiden vaimenemista analysoitiin. Prosessien häiriövasteet esitetään taajuuskoordinaatistossa. Suurimmat erot prosessien välillä löytyivät viirakaivosta ja sen sekoitusdynamiikasta. Perinteisen viirakaivon todettiin muistuttavan käyttäytymiseltään sarjaan kytkettyjä ideaalisekoittimia, kun taas pienempitilavuuksisen fluumin todettiin käyttäytyvän lähes kuin putkiviive. Vaikka erotprosessitilavuudessa sekä viirakaivon sekoitusdynamiikassa olivat hyvin selkeät, havaittiin vain marginaalinen ero prosessin välillä periodisten häiriöiden vaimenemisessa, koska erot viiraretentiotasoissa vaikuttivat eniten simulointituloksia. Matalammalla viiraretentiolla operoivan paperikoneen todettiin vaimentavan tehokkaammin prosessihäiriöitä. Samalla retentiotasolla pienempitilavuuksisen prosessin todettiin vaimentavan hitaita prosessihäiriöitä marginaalisesti paremmin. Tutkituista paperikoneista toisella simuloitiin viiraosan vedenpoistomuutoksenvaikutusta viiraretentioon ja paperin koostumukseen. Lisäksi arvioitiin viiraretention säädön toimivuutta. Viiraosan listakengän vedenpoiston todettiin aiheuttavan merkittäviä sakeus- ja retentiohäiriöitä, mikäli sen avulla poistettavan kiintoaineen virtaus tuplaantuisi. Viiraretention säädön todettiin estävän häiriöiden kierron prosessissa, mutta siirtävän ne suoraan rainaan. Retention säädön eikuitenkaan todettu olevan suoranainen häiriön lähde.

Dynamic Simulations of Rotors during Drop on Retainer Bearings

The active magnetic bearings have recently been intensively developed because of noncontact support having several advantages compared to conventional bearings. Due to improved materials, strategies of control, and electrical components, the performance and reliability of the active magnetic bearings are improving. However, additional bearings, retainer bearings, still have a vital role in the applications of the active magnetic bearings. The most crucial moment when the retainer bearings are needed is when the rotor drops from the active magnetic bearings on the retainer bearings due to component or power failure. Without appropriate knowledge of the retainer bearings, there is a chance that an active magnetic bearing supported rotor system will be fatal in a drop-down situation. This study introduces a detailed simulation model of a rotor system in order to describe a rotor drop-down situation on the retainer bearings. The introduced simulation model couples a finite element model with component mode synthesis and detailed bearing models. In this study, electrical components and electromechanical forces are not in the focus. The research looks at the theoretical background of the finite element method with component mode synthesis that can be used in the dynamic analysis of flexible rotors. The retainer bearings are described by using two ball bearing models, which include damping and stiffness properties, oil film, inertia of rolling elements and friction between races and rolling elements. Thefirst bearing model assumes that the cage of the bearing is ideal and that the cage holds the balls in their predefined positions precisely. The second bearing model is an extension of the first model and describes the behavior of the cageless bearing. In the bearing model, each ball is described by using two degrees of freedom. The models introduced in this study are verified with a corresponding actual structure. By using verified bearing models, the effects of the parameters of the rotor system onits dynamics during emergency stops are examined. As shown in this study, the misalignment of the retainer bearings has a significant influence on the behavior of the rotor system in a drop-down situation. In this study, a stability map of the rotor system as a function of rotational speed of the rotor and the misalignment of the retainer bearings is presented. In addition, the effects of parameters of the simulation procedure and the rotor system on the dynamics of system are studied.