Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Positive contrast visualization of nitinol devices using susceptibility gradient mapping.

MRI visualization of devices is traditionally based on signal loss due to T(2)* effects originating from local susceptibility differences. To visualize nitinol devices with positive contrast, a recently introduced postprocessing method is adapted to map the induced susceptibility gradients. This method operates on regular gradient-echo MR images and maps the shift in k-space in a (small) neighborhood of every voxel by Fourier analysis followed by a center-of-mass calculation. The quantitative map of the local shifts generates the positive contrast image of the devices, while areas without susceptibility gradients render a background with noise only. The positive signal response of this method depends only on the choice of the voxel neighborhood size. The properties of the method are explained and the visualizations of a nitinol wire and two stents are shown for illustration.

Alternative ways to estimate change points in multinomial sequences. An application to an authorship attribution problem

The statistical analysis of literary style is the part of stylometry that compares measurable characteristicsin a text that are rarely controlled by the author, with those in other texts. When thegoal is to settle authorship questions, these characteristics should relate to the author’s style andnot to the genre, epoch or editor, and they should be such that their variation between authors islarger than the variation within comparable texts from the same author.For an overview of the literature on stylometry and some of the techniques involved, see for exampleMosteller and Wallace (1964, 82), Herdan (1964), Morton (1978), Holmes (1985), Oakes (1998) orLebart, Salem and Berry (1998).Tirant lo Blanc, a chivalry book, is the main work in catalan literature and it was hailed to be“the best book of its kind in the world” by Cervantes in Don Quixote. Considered by writterslike Vargas Llosa or Damaso Alonso to be the first modern novel in Europe, it has been translatedseveral times into Spanish, Italian and French, with modern English translations by Rosenthal(1996) and La Fontaine (1993). The main body of this book was written between 1460 and 1465,but it was not printed until 1490.There is an intense and long lasting debate around its authorship sprouting from its first edition,where its introduction states that the whole book is the work of Martorell (1413?-1468), while atthe end it is stated that the last one fourth of the book is by Galba (?-1490), after the death ofMartorell. Some of the authors that support the theory of single authorship are Riquer (1990),Chiner (1993) and Badia (1993), while some of those supporting the double authorship are Riquer(1947), Coromines (1956) and Ferrando (1995). For an overview of this debate, see Riquer (1990).Neither of the two candidate authors left any text comparable to the one under study, and thereforediscriminant analysis can not be used to help classify chapters by author. By using sample textsencompassing about ten percent of the book, and looking at word length and at the use of 44conjunctions, prepositions and articles, Ginebra and Cabos (1998) detect heterogeneities that mightindicate the existence of two authors. By analyzing the diversity of the vocabulary, Riba andGinebra (2000) estimates that stylistic boundary to be near chapter 383.Following the lead of the extensive literature, this paper looks into word length, the use of the mostfrequent words and into the use of vowels in each chapter of the book. Given that the featuresselected are categorical, that leads to three contingency tables of ordered rows and therefore tothree sequences of multinomial observations.Section 2 explores these sequences graphically, observing a clear shift in their distribution. Section 3describes the problem of the estimation of a suden change-point in those sequences, in the followingsections we propose various ways to estimate change-points in multinomial sequences; the methodin section 4 involves fitting models for polytomous data, the one in Section 5 fits gamma modelsonto the sequence of Chi-square distances between each row profiles and the average profile, theone in Section 6 fits models onto the sequence of values taken by the first component of thecorrespondence analysis as well as onto sequences of other summary measures like the averageword length. In Section 7 we fit models onto the marginal binomial sequences to identify thefeatures that distinguish the chapters before and after that boundary. Most methods rely heavilyon the use of generalized linear models

Online Robust 3D Mapping Using Structure from Motion Cues

This paper presents a complete solution for creating accurate 3D textured models from monocular video sequences. The methods are developed within the framework of sequential structure from motion, where a 3D model of the environment is maintained and updated as new visual information becomes available. The camera position is recovered by directly associating the 3D scene model with local image observations. Compared to standard structure from motion techniques, this approach decreases the error accumulation while increasing the robustness to scene occlusions and feature association failures. The obtained 3D information is used to generate high quality, composite visual maps of the scene (mosaics). The visual maps are used to create texture-mapped, realistic views of the scene

A survey addressing the fundamental matrix estimation problem

Epipolar geometry is a key point in computer vision and the fundamental matrix estimation is the only way to compute it. This article surveys several methods of fundamental matrix estimation which have been classified into linear methods, iterative methods and robust methods. All of these methods have been programmed and their accuracy analysed using real images. A summary, accompanied with experimental results, is given

Schistosomiasis risk mapping in the state of Minas Gerais, Brazil, using a decision tree approach, remote sensing data and sociological indicators

Schistosomiasis mansoni is not just a physical disease, but is related to social and behavioural factors as well. Snails of the Biomphalaria genus are an intermediate host for Schistosoma mansoni and infect humans through water. The objective of this study is to classify the risk of schistosomiasis in the state of Minas Gerais (MG). We focus on socioeconomic and demographic features, basic sanitation features, the presence of accumulated water bodies, dense vegetation in the summer and winter seasons and related terrain characteristics. We draw on the decision tree approach to infection risk modelling and mapping. The model robustness was properly verified. The main variables that were selected by the procedure included the terrain's water accumulation capacity, temperature extremes and the Human Development Index. In addition, the model was used to generate two maps, one that included risk classification for the entire of MG and another that included classification errors. The resulting map was 62.9% accurate.

Mapping the proteome of Leishmania Viannia parasites using two-dimensional polyacrylamide gel electrophoresis and associated technologies.

In this study we have demonstrated the potential of two-dimensional electrophoresis (2DE)-based technologies as tools for characterization of the Leishmania proteome (the expressed protein complement of the genome). Standardized neutral range (pH 5-7) proteome maps of Leishmania (Viannia) guyanensis and Leishmania (Viannia) panamensis promastigotes were reproducibly generated by 2DE of soluble parasite extracts, which were prepared using lysis buffer containing urea and nonidet P-40 detergent. The Coomassie blue and silver nitrate staining systems both yielded good resolution and representation of protein spots, enabling the detection of approximately 800 and 1,500 distinct proteins, respectively. Several reference protein spots common to the proteomes of all parasite species/strains studied were isolated and identified by peptide mass spectrometry (LC-ES-MS/MS), and bioinformatics approaches as members of the heat shock protein family, ribosomal protein S12, kinetoplast membrane protein 11 and a hypothetical Leishmania-specific 13 kDa protein of unknown function. Immunoblotting of Leishmania protein maps using a monoclonal antibody resulted in the specific detection of the 81.4 kDa and 77.5 kDa subunits of paraflagellar rod proteins 1 and 2, respectively. Moreover, differences in protein expression profiles between distinct parasite clones were reproducibly detected through comparative proteome analyses of paired maps using image analysis software. These data illustrate the resolving power of 2DE-based proteome analysis. The production and basic characterization of good quality Leishmania proteome maps provides an essential first step towards comparative protein expression studies aimed at identifying the molecular determinants of parasite drug resistance and virulence, as well as discovering new drug and vaccine targets.

Discovery and fine mapping of serum protein loci through transethnic meta-analysis.

Many disorders are associated with altered serum protein concentrations, including malnutrition, cancer, and cardiovascular, kidney, and inflammatory diseases. Although these protein concentrations are highly heritable, relatively little is known about their underlying genetic determinants. Through transethnic meta-analysis of European-ancestry and Japanese genome-wide association studies, we identified six loci at genome-wide significance (p < 5 × 10(-8)) for serum albumin (HPN-SCN1B, GCKR-FNDC4, SERPINF2-WDR81, TNFRSF11A-ZCCHC2, FRMD5-WDR76, and RPS11-FCGRT, in up to 53,190 European-ancestry and 9,380 Japanese individuals) and three loci for total protein (TNFRS13B, 6q21.3, and ELL2, in up to 25,539 European-ancestry and 10,168 Japanese individuals). We observed little evidence of heterogeneity in allelic effects at these loci between groups of European and Japanese ancestry but obtained substantial improvements in the resolution of fine mapping of potential causal variants by leveraging transethnic differences in the distribution of linkage disequilibrium. We demonstrated a functional role for the most strongly associated serum albumin locus, HPN, for which Hpn knockout mice manifest low plasma albumin concentrations. Other loci associated with serum albumin harbor genes related to ribosome function, protein translation, and proteasomal degradation, whereas those associated with serum total protein include genes related to immune function. Our results highlight the advantages of transethnic meta-analysis for the discovery and fine mapping of complex trait loci and have provided initial insights into the underlying genetic architecture of serum protein concentrations and their association with human disease.

Appraisal of Phlebotomus argentipes habitat suitability using a remotely sensed index in the kala-azar endemic focus of Bihar, India

Visceral leishmaniasis, or kala-azar, is recognised as a serious emerging public health problem in India. In this study, environmental parameters, such as land surface temperature (LST) and renormalised difference vegetation indices (RDVI), were used to delineate the association between environmental variables and Phlebotomus argentipes abundance in a representative endemic region of Bihar, India. The adult P. argentipes were collected between September 2009-February 2010 using the hand-held aspirator technique. The distribution of P. argentipes was analysed with the LST and RDVI of the peak and lean seasons. The association between environmental covariates and P. argentipes density was analysed a multivariate linear regression model. The sandfly density at its maximum in September, whereas the minimum density was recorded in January. The regression model indicated that the season, minimum LST, mean LST and mean RDVI were the best environmental covariates for the P. argentipes distribution. The final model indicated that nearly 74% of the variance of sandfly density could be explained by these environmental covariates. This approach might be useful for mapping and predicting the distribution of P. argentipes, which may help the health agencies that are involved in the kala-azar control programme focus on high-risk areas.

Fine mapping and functional analysis of the multiple sclerosis risk gene CD6.

CD6 has recently been identified and validated as risk gene for multiple sclerosis (MS), based on the association of a single nucleotide polymorphism (SNP), rs17824933, located in intron 1. CD6 is a cell surface scavenger receptor involved in T-cell activation and proliferation, as well as in thymocyte differentiation. In this study, we performed a haptag SNP screen of the CD6 gene locus using a total of thirteen tagging SNPs, of which three were non-synonymous SNPs, and replicated the recently reported GWAS SNP rs650258 in a Spanish-Basque collection of 814 controls and 823 cases. Validation of the six most strongly associated SNPs was performed in an independent collection of 2265 MS patients and 2600 healthy controls. We identified association of haplotypes composed of two non-synonymous SNPs [rs11230563 (R225W) and rs2074225 (A257V)] in the 2(nd) SRCR domain with susceptibility to MS (P max(T) permutation = 1×10(-4)). The effect of these haplotypes on CD6 surface expression and cytokine secretion was also tested. The analysis showed significantly different CD6 expression patterns in the distinct cell subsets, i.e. - CD4(+) naïve cells, P = 0.0001; CD8(+) naïve cells, P<0.0001; CD4(+) and CD8(+) central memory cells, P = 0.01 and 0.05, respectively; and natural killer T (NKT) cells, P = 0.02; with the protective haplotype (RA) showing higher expression of CD6. However, no significant changes were observed in natural killer (NK) cells, effector memory and terminally differentiated effector memory T cells. Our findings reveal that this new MS-associated CD6 risk haplotype significantly modifies expression of CD6 on CD4(+) and CD8(+) T cells.