Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.

Parallel Dynamic Optimization of Steel Risers

This paper addresses the use of optimization techniques in the design of a steel riser. Two methods are used: the genetic algorithm, which imitates the process of natural selection, and the simulated annealing, which is based on the process of annealing of a metal. Both of them are capable of searching a given solution space for the best feasible riser configuration according to predefined criteria. Optimization issues are discussed, such as problem codification, parameter selection, definition of objective function, and restrictions. A comparison between the results obtained for economic and structural objective functions is made for a case study. Optimization method parallelization is also addressed. [DOI: 10.1115/1.4001955]

Risk-based decision making method for maintenance policy selection of thermal power plant equipment

This study presents a decision-making method for maintenance policy selection of power plants equipment. The method is based on risk analysis concepts. The method first step consists in identifying critical equipment both for power plant operational performance and availability based on risk concepts. The second step involves the proposal of a potential maintenance policy that could be applied to critical equipment in order to increase its availability. The costs associated with each potential maintenance policy must be estimated, including the maintenance costs and the cost of failure that measures the critical equipment failure consequences for the power plant operation. Once the failure probabilities and the costs of failures are estimated, a decision-making procedure is applied to select the best maintenance policy. The decision criterion is to minimize the equipment cost of failure, considering the costs and likelihood of occurrence of failure scenarios. The method is applied to the analysis of a lubrication oil system used in gas turbines journal bearings. The turbine has more than 150 MW nominal output, installed in an open cycle thermoelectric power plant. A design modification with the installation of a redundant oil pump is proposed for lubricating oil system availability improvement. (C) 2009 Elsevier Ltd. All rights reserved.

Sequential decision making with partially ordered preferences

This paper presents new insights and novel algorithms for strategy selection in sequential decision making with partially ordered preferences; that is, where some strategies may be incomparable with respect to expected utility. We assume that incomparability amongst strategies is caused by indeterminacy/imprecision in probability values. We investigate six criteria for consequentialist strategy selection: Gamma-Maximin, Gamma-Maximax, Gamma-Maximix, Interval Dominance, Maximality and E-admissibility. We focus on the popular decision tree and influence diagram representations. Algorithms resort to linear/multilinear programming; we describe implementation and experiments. (C) 2010 Elsevier B.V. All rights reserved.

Failure assessments of corroded pipelines with axial defects using stress-based criteria: Numerical studies and verification analyses

Conventional procedures used to assess the integrity of corroded piping systems with axial defects generally employ simplified failure criteria based upon a plastic collapse failure mechanism incorporating the tensile properties of the pipe material. These methods establish acceptance criteria for defects based on limited experimental data for low strength structural steels which do not necessarily address specific requirements for the high grade steels currently used. For these cases, failure assessments may be overly conservative or provide significant scatter in their predictions, which lead to unnecessary repair or replacement of in-service pipelines. Motivated by these observations, this study examines the applicability of a stress-based criterion based upon plastic instability analysis to predict the failure pressure of corroded pipelines with axial defects. A central focus is to gain additional insight into effects of defect geometry and material properties on the attainment of a local limit load to support the development of stress-based burst strength criteria. The work provides an extensive body of results which lend further support to adopt failure criteria for corroded pipelines based upon ligament instability analyses. A verification study conducted on burst testing of large-diameter pipe specimens with different defect length shows the effectiveness of a stress-based criterion using local ligament instability in burst pressure predictions, even though the adopted burst criterion exhibits a potential dependence on defect geometry and possibly on material`s strain hardening capacity. Overall, the results presented here suggests that use of stress-based criteria based upon plastic instability analysis of the defect ligament is a valid engineering tool for integrity assessments of pipelines with axial corroded defects. (C) 2008 Elsevier Ltd. All rights reserved.

Miniaturized structures for removal or selection of particles from liquid flow

The aim of this work was the development of miniaturized structures useful for retention and/or selection of particles and viscous substances from a liquid flow. The proposed low costs structures are similar to macroscopic wastewater treatment systems, named baffles, and allow disassemble. They were simulated using FEMLAB 3.2b package and manufactured in acrylic with conventional tools. Tests for retention or selection of particles in water or air and viscous fluids in water were carried out. Either in air or water particles with 50 mu m diameter will be retained but not with 13 mu m diameter. In aqueous flow, it is also possible the retention of viscous samples, such as silicone 350 cSt. The simulated results showed good agreement with experimental measurements. These miniaturized structures can be useful in sample pretreatment for chemical analysis and microorganism manipulation. (C) 2007 Elsevier B.V. All rights reserved.

Obtaining and selection of hexokinases-less strains of Saccharomyces cerevisiae for production of ethanol and fructose from sucrose

Saccharomyces cerevisiae hexokinase-less strains were produced to study the production of ethanol and fructose from sucrose. These strains do not have the hexokinases A and B. Twenty-three double-mutant strains were produced, and then, three were selected for presenting a smaller growth in yeast extract-peptone-fructose. In fermentations with a medium containing sucrose (180.3 g L-1) and with cell recycles, simulating industrial conditions, the capacity of these mutant yeasts in inverting sucrose and fermenting only glucose was well characterized. Besides that, we could also see their great tolerance to the stresses of fermentative recycles, where fructose production (until 90 g L-1) and ethanol production (until 42.3 g L-1) occurred in cycles of 12 h, in which hexokinase-less yeasts performed high growth (51.2% of wet biomass) and viability rates (77% of viable cells) after nine consecutive cycles.

Spatial-temporal variation in coiled and straight morphotypes of Cylindrospermopsis raciborskii (Wolsz) Seenayya et Subba Raju (Cyanobacteria)

(Spatial-temporal variation in coiled and straight morphotypes of Cylindrospermopsis raciborskii (Wolsz) Seenayya et Subba Raju (Cyanobacteria)). This study reports the spatial and temporal behavior of straight and coiled morphotypes of C. raciborskii in a reservoir in Brazil`s semiarid region as well as the main factors responsible for the variability. Two set of samples were collected from the subsurface and bottom in the central region of a reservoir in two seasonal periods (dry January 2005; rainy June 2005) over 20-hour sampling periods during daylight (8 am, 12 pm and 4 pm) and dark (8 pm, 12 am and 4 am) hours. Measurements of abiotic parameters were determined concurrently to the sampling of biotic variables. Two C. raciborskii morphotypes were found in the reservoir: straight and coiled. There was no difference in density of the straight and coiled C. raciborskii morphotypes between the different sampling times for either season. Vertical differences were found in the distribution of both morphotypes in both seasons, with greater densities recorded at the subsurface. Densities of the two C. raciborskii morphotypes were greater in the dry season, with the density of the coiled morphotype at the surface two-fold greater than that of the straight morphotype and that found in the rainy season. The ecological success of the coiled morphotype was due to thermal stratification, whereas a mixed condition was determinant in the success of the straight morphotype.

Does using stepwise variable selection to build sequential path analysis models make sense?

Causal inference methods - mainly path analysis and structural equation modeling - offer plant physiologists information about cause-and-effect relationships among plant traits. Recently, an unusual approach to causal inference through stepwise variable selection has been proposed and used in various works on plant physiology. The approach should not be considered correct from a biological point of view. Here, it is explained why stepwise variable selection should not be used for causal inference, and shown what strange conclusions can be drawn based upon the former analysis when one aims to interpret cause-and-effect relationships among plant traits.

Cyanobacteria and Cyanotoxin in the Billings Reservoir (Sao Paulo, SP, Brazil)

The Billings Complex and the Guarapiranga System are important strategic reservoirs for the city of Sao Paulo and surrounding areas because the water is used among other things, for the public water supply. They produce 19,000 liters of water per second and Supply water to 5.4 million people. Crude water is transferred from the Taquacetuba branch of the Billings Complex to the Guarapiranga Reservoir to regulate the water level of the reservoir. The objective of this study was to evaluate the water quality in the Taquacetuba branch, focusing on cyanobacteria and cyanotoxins. Surface water samples were collected in February (summer) and July (winter) of 2007. Analyses were conducted of physical, chemical, and biological variables of he water, cyanobacteria richness and density, and the presence of cyanotoxins. The water was classified as eutrophic-hypereutrophic. Cyanobacteria blooms were observed in both collection periods. The cyanobacteria bloom was most significant in July, reflecting lower water transparency and higher levels of total solids, suspended organic matter, chlorophyll-a, and cyanobacteria density in the surface water. Low richness and elevated dominance of the cyanobacteria were found in both periods. Cylindrospermopsis raciborskii was dominant in February, with 352 661.0 cel mL(-1), and Microcystis panniformis was dominant in July, with 1 866 725.0 cel mL(-1). Three variants of microcystin were found in February (MC-RR, MC-LR, MC-YR), as well as saxitoxin. The same variants of microcystin were found in July, but no saxitoxin was detected. Anatoxin-a and cylindropermopsin were not detected in either period. These findings are of great concern because the water in the Taquacetuba branch, which is transferred into the Guarapiranga Reservoir, is not treated nor managed. It is recommended that monitoring be intensified and more effective measures be taken by the responsible agencies to prevent the process of eutrophication and the consequent development of the cyanobacteria and their toxins.

Vegetative and fruit characteristics of papaya trees obtained by mass selection

The present work had as purpose to evaluate some characteristics of papaya trees (Carica papaya L.), Golden cultivar, obtained trough plant mass selection, regarding plant and fruit quality in the first months of production. The samples were evaluated in a commercial crop at: 0, 20, 40, 70, 130, 180, 230, 260, 280, 310 and 340 days after the planting (DAP) and the first fruits were harvested at 230 DAP. The results showed the low height (199cm in 340 DAP) and low first flowering`s heigth (71cm), which is important to facilitate the harvest process. The plants presented good yield with high number of leafs (allowing a great area of fruit cover) and about 60 fruits per plant. The fruits kept similar features to cv. Golden. The fruit`s fresh weight ranged from 302.4 to 467.5g, which is in the range of the Brazilian market. The pulp thickness was 2.35cm, which is a feature of great economic interest. The pulp thickness showed close relation with climatic factors, and great variations of temperature and precipitation accelerated the pulp loss of thickness.

Trypanosoma cruzi: Effects of social stress in Calomys callosus a natural reservoir of infection

Social environment can represent a major source of stress affecting cortisol and/or corticosterone levels, thereby altering the immune response. We have investigated the effects of social isolation on the development of Trypanosoma cruzi infection in female Calomys callosus, a natural reservoir of this protozoan parasite. Animals were divided in groups of five animals each. The animals of one group were kept together in a single cage. In a second group, four females were kept together in a cage with one male. In the final group, five individuals were kept isolated in private cages. The isolated animals showed body weight reduction, decreased numbers of peritoneal macrophages, lower global leucocytes counts, smaller lytic antibody percentage and a significantly higher level of blood parasites compared to the other animals. Their behavior was also altered. They were more aggressive than grouped females, or females exposed to the presence of a male. These results suggest that isolation creates a distinct social behavior in which immunity is impaired and pathogenesis is enhanced. (C) 2008 Elsevier Inc. All rights reserved.

Quality of Reporting of Observational Longitudinal Research

Observational longitudinal research is particularly useful for assessing etiology and prognosis and for providing evidence for clinical decision making. However, there are no structured reporting requirements for studies of this design to assist authors, editors, and readers. The authors developed and tested a checklist of criteria related to threats to the internal and external validity of observational longitudinal studies. The checklist criteria concerned recruitment, data collection, biases, and data analysis and descriptive issues relevant to study rationale, study population, and generalizability. Two raters independently assessed 49 randomly selected articles describing stroke research published from 1999 to 2003 in six journals: American Journal of Epidemiology, Journal of Epidemiology and Community Health, Stroke, Annals of Neurology, Archives of Physical Medicine and Rehabilitation, and American Journal of Physical Medicine and Rehabilitation. On average, 17 of the 33 checklist criteria were reported. Criteria describing the study design were better reported than those related to internal validity. No relation was found between study type (etiologic or prognostic) or word count and quality of reporting. A flow diagram for summarizing participant flow through a study was developed. Editors and authors should consider using a checklist and flow diagram when reporting on observational longitudinal research.

Geographic variation and positive selection on M7 lysin, an acrosomal sperm protein in mussels (Mytilus spp.)

Successful fertilization in free-spawning marine organisms depends on the interactions between genes expressed on the surfaces of eggs and sperm. Positive selection frequently characterizes the molecular evolution of such genes, raising the possibility that some common deterministic process drives the evolution of gamete recognition genes and may even be important for understanding the evolution of prezygotic isolation and speciation in the marine realm. One hypothesis is that gamete recognition genes are subject to selection for prezygotic isolation, namely reinforcement. In a previous study, positive selection on the gene coding for the acrosomal sperm protein M7 lysin was demonstrated among allopatric populations of mussels in the Mytilus edulis species group (M. edulis, M. galloprovincialis, and M. trossulus). Here, we expand sampling to include M7 lysin haplotypes from populations where mussel species are sympatric and hybridize to determine whether there is a pattern of reproductive character displacement, which would be consistent with reinforcement driving selection on this gene. We do not detect a strong pattern of reproductive character displacement; there are no unique haplotypes in sympatry nor is there consistently greater population structure in comparisons involving sympatric populations. One distinct group of haplotypes, however, is strongly affected by natural selection and this group of haplotypes is found within M. galloprovincialis populations throughout the Northern Hemisphere concurrent with haplotypes common to M. galloprovincialis and M. edulis. We suggest that balancing selection, perhaps resulting from sexual conflicts between sperm and eggs, maintains old allelic diversity within M. galloprovincialis.

Weibull model selection for reliability modelling

A large number of models have been derived from the two-parameter Weibull distribution and are referred to as Weibull models. They exhibit a wide range of shapes for the density and hazard functions, which makes them suitable for modelling complex failure data sets. The WPP and IWPP plot allows one to determine in a systematic manner if one or more of these models are suitable for modelling a given data set. This paper deals with this topic.