Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Moderating effects of management control systems and innovation on performance. Simple methods for correcting the effects of measurement error for interaction effects in small samples

In the accounting literature, interaction or moderating effects are usually assessed by means of OLS regression and summated rating scales are constructed to reduce measurement error bias. Structural equation models and two-stage least squares regression could be used to completely eliminate this bias, but large samples are needed. Partial Least Squares are appropriate for small samples but do not correct measurement error bias. In this article, disattenuated regression is discussed as a small sample alternative and is illustrated on data of Bisbe and Otley (in press) that examine the interaction effect of innovation and style of use of budgets on performance. Sizeable differences emerge between OLS and disattenuated regression

Validation and performance comparison of two carbon isotope ratio methods to control the misuse of androgens in humans.

Carbon isotope ratio of androgens in urine specimens is routinely determined to exclude an abuse of testosterone or testosterone prohormones by athletes. Increasing application of gas chromatography/combustion/isotope ratio mass spectrometry (GC/C/IRMS) in the last years for target and systematic investigations on samples has resulted in the demand for rapid sample throughput as well as high selectivity in the extraction process particularly in the case of conspicuous samples. For that purpose, we present herein the complimentary use of an SPE-based assay and an HPLC fractionation method as a two-stage strategy for the isolation of testosterone metabolites and endogenous reference compounds prior to GC/C/IRMS analyses. Assays validation demonstrated acceptable performance in terms of intermediate precision (range: 0.1-0.4 per thousand) and Bland-Altman analyses revealed no significant bias (0.2 per thousand). For further validation of this two-stage analyses strategy, all the specimens (n=124) collected during a major sport event were processed.

Comparing simulation methods for modelling the errors of stand inventory data

Abstract

Plant-microbe interactions and the new biotechnological methods of plant disease control

Plants constitute an excellent ecosystem for microorganisms. The environmental conditions offered differ considerably between the highly variable aerial plant part and the more stable root system. Microbes interact with plant tissues and cells with different degrees of dependence. The most interesting from the microbial ecology point of view, however, are specific interactions developed by plant-beneficial (either non-symbiotic or symbiotic) and pathogenic microorganisms. Plants, like humans and other animals, also become sick, but they have evolved a sophisticated defense response against microbes, based on a combination of constitutive and inducible responses which can be localized or spread throughout plant organs and tissues. The response is mediated by several messenger molecules that activate pathogen-responsive genes coding for enzymes or antimicrobial compounds, and produces less sophisticated and specific compounds than immunoglobulins in animals. However, the response specifically detects intracellularly a type of protein of the pathogen based on a gene-for-gene interaction recognition system, triggering a biochemical attack and programmed cell death. Several implications for the management of plant diseases are derived from knowledge of the basis of the specificity of plant-bacteria interactions. New biotechnological products are currently being developed based on stimulation of the plant defense response, and on the use of plant-beneficial bacteria for biological control of plant diseases (biopesticides) and for plant growth promotion (biofertilizers)

Automated, adaptive methods for forest inventory

Forest inventories are used to estimate forest characteristics and the condition of forest for many different applications: operational tree logging for forest industry, forest health state estimation, carbon balance estimation, land-cover and land use analysis in order to avoid forest degradation etc. Recent inventory methods are strongly based on remote sensing data combined with field sample measurements, which are used to define estimates covering the whole area of interest. Remote sensing data from satellites, aerial photographs or aerial laser scannings are used, depending on the scale of inventory. To be applicable in operational use, forest inventory methods need to be easily adjusted to local conditions of the study area at hand. All the data handling and parameter tuning should be objective and automated as much as possible. The methods also need to be robust when applied to different forest types. Since there generally are no extensive direct physical models connecting the remote sensing data from different sources to the forest parameters that are estimated, mathematical estimation models are of "black-box" type, connecting the independent auxiliary data to dependent response data with linear or nonlinear arbitrary models. To avoid redundant complexity and over-fitting of the model, which is based on up to hundreds of possibly collinear variables extracted from the auxiliary data, variable selection is needed. To connect the auxiliary data to the inventory parameters that are estimated, field work must be performed. In larger study areas with dense forests, field work is expensive, and should therefore be minimized. To get cost-efficient inventories, field work could partly be replaced with information from formerly measured sites, databases. The work in this thesis is devoted to the development of automated, adaptive computation methods for aerial forest inventory. The mathematical model parameter definition steps are automated, and the cost-efficiency is improved by setting up a procedure that utilizes databases in the estimation of new area characteristics.

Suitability of LiDAR technology for forest inventory in Russia

LiDAR is an advanced remote sensing technology with many applications, including forest inventory. The most common type is ALS (airborne laser scanning). The method is successfully utilized in many developed markets, where it is replacing traditional forest inventory methods. However, it is innovative for Russian market, where traditional field inventory dominates. ArboLiDAR is a forest inventory solution that engages LiDAR, color infrared imagery, GPS ground control plots and field sample plots, developed by Arbonaut Ltd. This study is an industrial market research for LiDAR technology in Russia focused on customer needs. Russian forestry market is very attractive, because of large growing stock volumes. It underwent drastic changes in 2006, but it is still in transitional stage. There are several types of forest inventory, both with public and private funding. Private forestry enterprises basically need forest inventory in two cases – while making coupe demarcation before timber harvesting and as a part of forest management planning, that is supposed to be done every ten years on the whole leased territory. The study covered 14 companies in total that include private forestry companies with timber harvesting activities, private forest inventory providers, state subordinate companies and forestry software developer. The research strategy is multiple case studies with semi-structured interviews as the main data collection technique. The study focuses on North-West Russia, as it is the most developed Russian region in forestry. The research applies the Voice of the Customer (VOC) concept to elicit customer needs of Russian forestry actors and discovers how these needs are met. It studies forest inventory methods currently applied in Russia and proposes the model of method comparison, based on Multi-criteria decision making (MCDM) approach, mainly on Analytical Hierarchy Process (AHP). Required product attributes are classified in accordance with Kano model. The answer about suitability of LiDAR technology is ambiguous, since many details should be taken into account.

New statistical methods to assess the effect of time-dependent exposures in case-control studies

Contexte. Les études cas-témoins sont très fréquemment utilisées par les épidémiologistes pour évaluer l’impact de certaines expositions sur une maladie particulière. Ces expositions peuvent être représentées par plusieurs variables dépendant du temps, et de nouvelles méthodes sont nécessaires pour estimer de manière précise leurs effets. En effet, la régression logistique qui est la méthode conventionnelle pour analyser les données cas-témoins ne tient pas directement compte des changements de valeurs des covariables au cours du temps. Par opposition, les méthodes d’analyse des données de survie telles que le modèle de Cox à risques instantanés proportionnels peuvent directement incorporer des covariables dépendant du temps représentant les histoires individuelles d’exposition. Cependant, cela nécessite de manipuler les ensembles de sujets à risque avec précaution à cause du sur-échantillonnage des cas, en comparaison avec les témoins, dans les études cas-témoins. Comme montré dans une étude de simulation précédente, la définition optimale des ensembles de sujets à risque pour l’analyse des données cas-témoins reste encore à être élucidée, et à être étudiée dans le cas des variables dépendant du temps. Objectif: L’objectif général est de proposer et d’étudier de nouvelles versions du modèle de Cox pour estimer l’impact d’expositions variant dans le temps dans les études cas-témoins, et de les appliquer à des données réelles cas-témoins sur le cancer du poumon et le tabac. Méthodes. J’ai identifié de nouvelles définitions d’ensemble de sujets à risque, potentiellement optimales (le Weighted Cox model and le Simple weighted Cox model), dans lesquelles différentes pondérations ont été affectées aux cas et aux témoins, afin de refléter les proportions de cas et de non cas dans la population source. Les propriétés des estimateurs des effets d’exposition ont été étudiées par simulation. Différents aspects d’exposition ont été générés (intensité, durée, valeur cumulée d’exposition). Les données cas-témoins générées ont été ensuite analysées avec différentes versions du modèle de Cox, incluant les définitions anciennes et nouvelles des ensembles de sujets à risque, ainsi qu’avec la régression logistique conventionnelle, à des fins de comparaison. Les différents modèles de régression ont ensuite été appliqués sur des données réelles cas-témoins sur le cancer du poumon. Les estimations des effets de différentes variables de tabac, obtenues avec les différentes méthodes, ont été comparées entre elles, et comparées aux résultats des simulations. Résultats. Les résultats des simulations montrent que les estimations des nouveaux modèles de Cox pondérés proposés, surtout celles du Weighted Cox model, sont bien moins biaisées que les estimations des modèles de Cox existants qui incluent ou excluent simplement les futurs cas de chaque ensemble de sujets à risque. De plus, les estimations du Weighted Cox model étaient légèrement, mais systématiquement, moins biaisées que celles de la régression logistique. L’application aux données réelles montre de plus grandes différences entre les estimations de la régression logistique et des modèles de Cox pondérés, pour quelques variables de tabac dépendant du temps. Conclusions. Les résultats suggèrent que le nouveau modèle de Cox pondéré propose pourrait être une alternative intéressante au modèle de régression logistique, pour estimer les effets d’expositions dépendant du temps dans les études cas-témoins

Methods for short-term control of Imperata grass in Peruvian Amazon

The traditional control of Imperata brasiliensis grasslands used by farmers in the Peruvian Amazon is to burn the grass. The objective of this study was to compare different methods of short-term control. Biological, mechanical, chemical and traditional methods of control were compared. Herbicide spraying and manual weeding have shown to be very effective in reducing above- and below-ground biomass growth in the first 45 days after slashing the grass, with effects persisting in the longer term, but both are expensive methods. Shading seems to be less effective in the short-term, whereas it influences the Imperata growth in the longer term. After one year shading, glyphosate application and weeding significantly reduced aboveground biomass by 94, 67 and 53%; and belowground biomass by 76, 65 and 58%, respectively, compared to control. We also found a significant decrease of Imperata rhizomes in soil during time under shading. Burning has proved to have no significant effect on Imperata growth. The use of shade trees in a kind of agroforestry system could be a suitable method for small farmers to control Imperata grasslands.

Explorations of the Practical Issues of Learning Prediction-Control Tasks Using Temporal Difference Learning Methods

There has been recent interest in using temporal difference learning methods to attack problems of prediction and control. While these algorithms have been brought to bear on many problems, they remain poorly understood. It is the purpose of this thesis to further explore these algorithms, presenting a framework for viewing them and raising a number of practical issues and exploring those issues in the context of several case studies. This includes applying the TD(lambda) algorithm to: 1) learning to play tic-tac-toe from the outcome of self-play and of play against a perfectly-playing opponent and 2) learning simple one-dimensional segmentation tasks.

Moderating effects of management control systems and innovation on performance. Simple methods for correcting the effects of measurement error for interaction effects in small samples

In the accounting literature, interaction or moderating effects are usually assessed by means of OLS regression and summated rating scales are constructed to reduce measurement error bias. Structural equation models and two-stage least squares regression could be used to completely eliminate this bias, but large samples are needed. Partial Least Squares are appropriate for small samples but do not correct measurement error bias. In this article, disattenuated regression is discussed as a small sample alternative and is illustrated on data of Bisbe and Otley (in press) that examine the interaction effect of innovation and style of use of budgets on performance. Sizeable differences emerge between OLS and disattenuated regression

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Development of molecular monitoring methods and assessment of the environmental fate of the biological control agent of fire blight Pseudomonas fluorescens EPS62e

Pseudomonas fluorescens EPS62e es va seleccionar com a agent de biocontrol del foc bacterià per la seva eficàcia en el control de Erwinia amylovora. En aquest treball es van desenvolupar mètodes de traçabilitat que van permetre la seva detecció específica i quantificació. Mitjançant les tècniques RAPD i U-PCR es van obtenir fragments d'amplificació diferencial per EPS62e que es van seqüenciar i caracteritzar com marcadors SCAR per dissenyar una PCR en temps real. La PCR a temps real es va utilitzar simultàniament amb mètodes microbiològics per estudiar l'adaptabilitat epifítica de EPS62e en pomera i perera. L'ús combinat de mètodes microbiològics i moleculars va permetre la identificació de tres estats fisiològics de EPS62e: la colonització activa, l'entrada en un estat de viable però no cultivable, i la mort cel·lular. Aquest treball mostra que EPS62e està ben adaptada a la colonització de flors a camp, encoratjant la seva utilització dins d'una estratègia de control biològic contra el foc bacterià.