Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.

DESIGN OF CONCRETE PLATFORMS OFFSHORE USING THE THREE-LAYER SHELL THEORY

The concrete offshore platforms, which are subjected a several loading combinations and, thus, requires an analysis more generic possible, can be designed using the concepts adopted to shell elements, but the resistance must be verify in particular cross-sections to shear forces. This work about design of shell elements will be make using the three-layer shell theory. The elements are subject to combined loading of membrane and plate, totalizing eight components of internal forces, which are three membrane forces, three moments (two out-of-plane bending moments and one in-plane, or torsion, moment) and two shear forces. The design method adopted, utilizing the iterative process proposed by Lourenco & Figueiras (1993) obtained from equations of equilibrium developed by Gupta (1896) , will be compared to results of experimentally tested shell elements found in the literature using the program DIANA.

Debris flow affecting the Cubatauo Oil Refinery, Brazil

On February 6, 1994, a large debris flow developed because of intense rains in a 800-m-high mountain range called Serra do Cubatao, the local name for the Serra do Mar, located along the coast of the state of Sao Paulo, Brazil. It affected the Presidente Bernardes Refinery, owned by Petrobras, in Cubatao. The damages amounted to about US $40 million because of the muck cleaning, repairs, and 3-week interruption of the operations. This prompted Petrobras to conduct studies, carried out by the authors, to develop protection works, which were done at a cost of approximately US $12 million. The paper describes the studies conducted on debris flow mechanics. A new criteria to define rainfall intensities that trigger debris flows is presented, as well as a correlation of slipped area with soil porosity and rain intensity. Also presented are (a) an actual grain size distribution of a deposited material, determined by laboratory and a large-scale field test, and (b) the size distribution of large boulders along the river bed. Based on theory, empirical experience and back-analysis of the events, the main parameters as the front velocity, the peak discharge and the volume of the transported sediments were determined in a rational basis for the design of the protection works. Finally, the paper describes the set of the protection works built, emphasizing their concept and function. They also included some low-cost innovative works.

Role of price and enforcement in water allocation: Insights from Game Theory

As many countries are moving toward water sector reforms, practical issues of how water management institutions can better effect allocation, regulation, and enforcement of water rights have emerged. The problem of nonavailability of water to tailenders on an irrigation system in developing countries, due to unlicensed upstream diversions is well documented. The reliability of access or equivalently the uncertainty associated with water availability at their diversion point becomes a parameter that is likely to influence the application by users for water licenses, as well as their willingness to pay for licensed use. The ability of a water agency to reduce this uncertainty through effective water rights enforcement is related to the fiscal ability of the agency to monitor and enforce licensed use. In this paper, this interplay across the users and the agency is explored, considering the hydraulic structure or sequence of water use and parameters that define the users and the agency`s economics. The potential for free rider behavior by the users, as well as their proposals for licensed use are derived conditional on this setting. The analyses presented are developed in the framework of the theory of ""Law and Economics,`` with user interactions modeled as a game theoretic enterprise. The state of Ceara, Brazil, is used loosely as an example setting, with parameter values for the experiments indexed to be approximately those relevant for current decisions. The potential for using the ideas in participatory decision making is discussed. This paper is an initial attempt to develop a conceptual framework for analyzing such situations but with a focus on the reservoir-canal system water rights enforcement.

Modelling grain boundary migration during geometric dynamic recrystallization

High-angle grain boundary migration is predicted during geometric dynamic recrystallization (GDRX) by two types of mathematical models. Both models consider the driving pressure due to curvature and a sinusoidal driving pressure owing to subgrain walls connected to the grain boundary. One model is based on the finite difference solution of a kinetic equation, and the other, on a numerical technique in which the boundary is subdivided into linear segments. The models show that an initially flat boundary becomes serrated, with the peak and valley migrating into both adjacent grains, as observed during GDRX. When the sinusoidal driving pressure amplitude is smaller than 2 pi, the boundary stops migrating, reaching an equilibrium shape. Otherwise, when the amplitude is larger than 2 pi, equilibrium is never reached and the boundary migrates indefinitely, which would cause the protrusions of two serrated parallel boundaries to impinge on each other, creating smaller equiaxed grains.

Robust model predictive controller with output feedback and target tracking

A model predictive controller (MPC) is proposed, which is robustly stable for some classes of model uncertainty and to unknown disturbances. It is considered as the case of open-loop stable systems, where only the inputs and controlled outputs are measured. It is assumed that the controller will work in a scenario where target tracking is also required. Here, it is extended to the nominal infinite horizon MPC with output feedback. The method considers an extended cost function that can be made globally convergent for any finite input horizon considered for the uncertain system. The method is based on the explicit inclusion of cost contracting constraints in the control problem. The controller considers the output feedback case through a non-minimal state-space model that is built using past output measurements and past input increments. The application of the robust output feedback MPC is illustrated through the simulation of a low-order multivariable system.

On the solubility of proteins as a function of pH: Mathematical development and application

Thermodynamic relations between the solubility of a protein and the solution pH are presented in this work. The hypotheses behind the development are that the protein chemical potential in liquid phase can be described by Henry`s law and that the solid-liquid equilibrium is established only between neutral molecules. The mathematical development results in an analytical expression of the solubility curve, as a function of the ionization equilibrium constants, the pH and the solubility at the isoelectric point. It is shown that the same equation can be obtained either by directly calculating the fraction of neutral protein molecules or by integrating the curve of the protein average charge. The methodology was successfully applied to the description of the solubility of porcine insulin as a function of pH at three different temperatures and of bovine beta-lactoglobulin at four different ionic strengths. (C) 2011 Elsevier B.V. All rights reserved.

Robust model predictive control with zone control

Model predictive control (MPC) is usually implemented as a control strategy where the system outputs are controlled within specified zones, instead of fixed set points. One strategy to implement the zone control is by means of the selection of different weights for the output error in the control cost function. A disadvantage of this approach is that closed-loop stability cannot be guaranteed, as a different linear controller may be activated at each time step. A way to implement a stable zone control is by means of the use of an infinite horizon cost in which the set point is an additional variable of the control problem. In this case, the set point is restricted to remain inside the output zone and an appropriate output slack variable is included in the optimisation problem to assure the recursive feasibility of the control optimisation problem. Following this approach, a robust MPC is developed for the case of multi-model uncertainty of open-loop stable systems. The controller is devoted to maintain the outputs within their corresponding feasible zone, while reaching the desired optimal input target. Simulation of a process of the oil re. ning industry illustrates the performance of the proposed strategy.

On state representations of nonlinear implicit systems

This work considers a semi-implicit system A, that is, a pair (S, y), where S is an explicit system described by a state representation (x)over dot(t) = f(t, x(t), u(t)), where x(t) is an element of R(n) and u(t) is an element of R(m), which is subject to a set of algebraic constraints y(t) = h(t, x(t), u(t)) = 0, where y(t) is an element of R(l). An input candidate is a set of functions v = (v(1),.... v(s)), which may depend on time t, on x, and on u and its derivatives up to a Finite order. The problem of finding a (local) proper state representation (z)over dot = g(t, z, v) with input v for the implicit system Delta is studied in this article. The main result shows necessary and sufficient conditions for the solution of this problem, under mild assumptions on the class of admissible state representations of Delta. These solvability conditions rely on an integrability test that is computed from the explicit system S. The approach of this article is the infinite-dimensional differential geometric setting of Fliess, Levine, Martin, and Rouchon (1999) (`A Lie-Backlund Approach to Equivalence and Flatness of Nonlinear Systems`, IEEE Transactions on Automatic Control, 44(5), (922-937)).

Spectral properties of chaotic signals with applications in communications

This paper investigates the characteristics of the Power Spectral Density (PSD) of chaotic signals generated by skew tent maps. The influence of the Lyapunov exponent on the autocorrelation sequence and on the PSD is evaluated via computational simulations. We conclude that the essential bandwidth of these signals is strongly related to this exponent and they can be low-pass or high-pass depending on the family`s parameter. This way, the PSD of a chaotic signal is a function of the generating map although this is not a one-to-one relationship. (C) 2009 Elsevier Ltd. All rights reserved.

A Geometric Approach for Turbo Decoding of Reed-Solomon Codes in QAM Modulation Schemes

This work studies the turbo decoding of Reed-Solomon codes in QAM modulation schemes for additive white Gaussian noise channels (AWGN) by using a geometric approach. Considering the relations between the Galois field elements of the Reed-Solomon code and the symbols combined with their geometric dispositions in the QAM constellation, a turbo decoding algorithm, based on the work of Chase and Pyndiah, is developed. Simulation results show that the performance achieved is similar to the one obtained with the pragmatic approach with binary decomposition and analysis.

Linear minimum mean square filter for discrete-time linear systems with Markov jumps and multiplicative noises

In this paper we obtain the linear minimum mean square estimator (LMMSE) for discrete-time linear systems subject to state and measurement multiplicative noises and Markov jumps on the parameters. It is assumed that the Markov chain is not available. By using geometric arguments we obtain a Kalman type filter conveniently implementable in a recurrence form. The stationary case is also studied and a proof for the convergence of the error covariance matrix of the LMMSE to a stationary value under the assumption of mean square stability of the system and ergodicity of the associated Markov chain is obtained. It is shown that there exists a unique positive semi-definite solution for the stationary Riccati-like filter equation and, moreover, this solution is the limit of the error covariance matrix of the LMMSE. The advantage of this scheme is that it is very easy to implement and all calculations can be performed offline. (c) 2011 Elsevier Ltd. All rights reserved.

Ovarian function in cows submitted to acute stress during proestrus

Studies in which ACTH was administrated in heifers after the occurrence of luteolysis showed a delay in the onset of estrus and the estrus duration was shortened. This study evaluated the effect of acute stress by road transportation on estrous behavior and ovulation, monitored by serum progesterone and cortisol concentrations in cows at the periovulatory period, using a crossover design. Eleven crossbred cows, divided into Control and Transport groups had their estrus cycle synchronized with GnRH, an intravaginal progesterone device, and cloprostenol. Thirty hours after withdrawal of the device, the animals of the Transport group were transported for 60 min by truck and those from the Control group remained at pasture. Ovarian ultrasound examination was performed every 12 h from device withdrawal until ovulation in every cow. From the day after removal of the device until ovulation estrous behavior was monitored 24 h a day. Blood samples for serum cortisol and progesterone concentrations were taken at -90, -60, 0, 30, 60 and 180 min in relation to the end of transportation. Transportation during the estrous period induced stress in cows as reflected by changes in serum concentrations of progesterone and cortisol. However, we did not detect impairment in estrus expression, estrus duration or ovulation (P>0.05). (C) 2010 Elsevier B.V. All rights reserved.

Nutrition in bean plants and anthracnose intensity in function of silicon and copper application.

Nutrition in bean plants and anthracnose intensity in function of silicon and copper application. The objective of this work was to evaluate the effect of calcium silicate and copper sulfate on anthracnose intensity and nutrition of bean plants. The experiment was conducted using an experimental design in randomized blocks following a 4 x 4 factorial arrangement , (four levels of calcium silicate and four levels of copper sulfate) and two additional treatments (plants without inoculation and plants sprinkled with Benomyl). Four evaluations of the incidence and severity of anthracnose were done, in addition to measuring, total leaf area. At the end of the evaluations, incidence: and data were integrated over time, obtaining the area under disease progress curve (AUDPC). Contents of N, P, K, Ca, Mg, B, Cu, Fe, Mn, Zn, Si and lignin were determined in the aerial Part. A linear decrease of the intensity AUDPC was observed with the increase of the doses of calcium silicate. The severity AUDPC was influenced by the doses of copper, obtaining a reduction of 35% on the higher dosage. The supply of silicon and copper altered the content of the K, mg, S, Zn, Ca and Si in the aerial part of the bean plants.

Brazil nut ingestion increased plasma selenium but had minimal effects on lipids, apolipoproteins, and high-density lipoprotein function in human subjects

The Brazil nut (Bertholletia excelsa) of the Amazon region is consumed worldwide. It is rich in both monounsaturated fatty acids and polyunsaturated fatty acids and is known for its high selenium content. This study tested the hypothesis whether the consumption of this nut could affect the plasma lipids and apolipoproteins and some functional properties of the antiatherogenic high-density lipoprotein (HDL). Fifteen normolipidemic subjects aged 27.3 +/- 3.9 years and with body mass index of 23.8 +/- 2.8 kg/m(2) consumed 45 g of Brazil nuts per day during a 15-day period. On days 0 and 15, blood was collected for biochemical analysis, determination of HDL particle size, paraoxonase 1 activity, and lipid transfer from a lipoprotein-like nanoparticle to the HDL fraction. Brazil nut ingestion did not alter HDL, low-density lipoprotein cholesterol, triacylglycerols, apolipoprotein A-1, or apolipoprotein B concentrations. HDL particle diameter and the activity of antioxidative paraoxonase 1, mostly found in the HDL fraction, Were also unaffected. Supplementation increased the reception of cholesteryl esters (P <.05) by the HDL yet did not alter the reception of phospholipids, free cholesterol, or triacylglycerols. As expected, plasma selenium was significantly increased. However, the consumption of Brazil nuts for short duration by normolipidemic subjects in comparable amounts to those tested for other nuts did not alter serum lipid profile. The only alteration in HDL function was the increase in cholesteryl ester transfer. This latter finding may be beneficial because it would improve the nonatherogenic reverse cholesterol transport pathway. (c) 2008 Elsevier Inc. All rights reserved.