Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Classification of sags according to their origin based on the waveform similarity

A statistical method for classification of sags their origin downstream or upstream from the recording point is proposed in this work. The goal is to obtain a statistical model using the sag waveforms useful to characterise one type of sags and to discriminate them from the other type. This model is built on the basis of multi-way principal component analysis an later used to project the available registers in a new space with lower dimension. Thus, a case base of diagnosed sags is built in the projection space. Finally classification is done by comparing new sags against the existing in the case base. Similarity is defined in the projection space using a combination of distances to recover the nearest neighbours to the new sag. Finally the method assigns the origin of the new sag according to the origin of their neighbours

A quantum molecular similarity analysis of changes in molecular electron density caused by basis set flotation and electric field application

Quantum molecular similarity (QMS) techniques are used to assess the response of the electron density of various small molecules to application of a static, uniform electric field. Likewise, QMS is used to analyze the changes in electron density generated by the process of floating a basis set. The results obtained show an interrelation between the floating process, the optimum geometry, and the presence of an external field. Cases involving the Le Chatelier principle are discussed, and an insight on the changes of bond critical point properties, self-similarity values and density differences is performed

A comparative analysis by means of quantum molecular similarity measures of density distributions derived from conventional ab initio and density functional methods

A procedure based on quantum molecular similarity measures (QMSM) has been used to compare electron densities obtained from conventional ab initio and density functional methodologies at their respective optimized geometries. This method has been applied to a series of small molecules which have experimentally known properties and molecular bonds of diverse degrees of ionicity and covalency. Results show that in most cases the electron densities obtained from density functional methodologies are of a similar quality than post-Hartree-Fock generalized densities. For molecules where Hartree-Fock methodology yields erroneous results, the density functional methodology is shown to yield usually more accurate densities than those provided by the second order Møller-Plesset perturbation theory

OpenSearch-geo: the simple standard for geographic web search engines

When publishing information on the web, one expects it to reach all the people that could be interested in. This is mainly achieved with general purpose indexing and search engines like Google which is the most used today. In the particular case of geographic information (GI) domain, exposing content to mainstream search engines is a complex task that needs specific actions. In many occasions it is convenient to provide a web site with a specially tailored search engine. Such is the case for on-line dictionaries (wikipedia, wordreference), stores (amazon, ebay), and generally all those holding thematic databases. Due to proliferation of these engines, A9.com proposed a standard interface called OpenSearch, used by modern web browsers to manage custom search engines. Geographic information can also benefit from the use of specific search engines. We can distinguish between two main approaches in GI retrieval information efforts: Classical OGC standardization on one hand (CSW, WFS filters), which are very complex for the mainstream user, and on the other hand the neogeographer’s approach, usually in the form of specific APIs lacking a common query interface and standard geographic formats. A draft ‘geo’ extension for OpenSearch has been proposed. It adds geographic filtering for queries and recommends a set of simple standard response geographic formats, such as KML, Atom and GeoRSS. This proposal enables standardization while keeping simplicity, thus covering a wide range of use cases, in both OGC and the neogeography paradigms. In this article we will analyze the OpenSearch geo extension in detail and its use cases, demonstrating its applicability to both the SDI and the geoweb. Open source implementations will be presented as well

First-order molecular descriptors for molecular steric similarity

En aquest treball s'analitza la contribució estèrica de les molècules a les seves propietats químiques i físiques, mitjançant l'avaluació del seu volum i de la seva mesura de semblança, a partir d'ara definits com a descriptors moleculars de primer ordre. La difeèsncia entre aquests dos conceptes ha estat aclarida: mentre que el volum és la magnitud de l'espai que ocupa la molècula com a entitat global, la mesura de semblança ens dóna una idea de com està distribuïda la densitat electrònica al llarg d'aquest volum, i reflecteix més les diferències locals existents. L'ús de diverses aproximacions per a l'obtenció d'ambdós valors ha estat analitzat sobre diferents classes d'isòmers

Quantum similarity topological indices definition, analysis and comparison with classical molecular topological indices

Es mostra que, gracies a una extensió en la definició dels Índexs Moleculars Topològics, s'arriba a la formulació d'índexs relacionats amb la teoria de la Semblança Molecular Quàntica. Es posa de manifest la connexió entre les dues metodologies: es revela que un marc de treball teòric sòlidament fonamentat sobre la teoria de la Mecànica Quàntica es pot connectar amb una de les tècniques més antigues relacionades amb els estudis de QSPR. Es mostren els resultats per a dos casos d'exemple d'aplicació d'ambdues metodologies

Practical applications of quantum molecular similarity measures (QMSM): programs and examples

Es descriu l'aproximació de Capes Atòmiques dins de la teoria de la Semblança Molecular Quàntica. Partint només de dades teòriques, s'ha trobat una relació entre estructura molecular i activitat biològica per a diversos conjunts de molècules. Es descriuen els aspectes teòrics de la Semblança Molecular Quàntica i alguns exemples d'aplicació

Molecular quantum similarity in QSAR: applications in computer-aided molecular design

La present tesi està centrada en l'ús de la Teoria de Semblança Quàntica per a calcular descriptors moleculars. Aquests descriptors s'utilitzen com a paràmetres estructurals per a derivar correlacions entre l'estructura i la funció o activitat experimental per a un conjunt de compostos. Els estudis de Relacions Quantitatives Estructura-Activitat són d'especial interès per al disseny racional de molècules assistit per ordinador i, en particular, per al disseny de fàrmacs. Aquesta memòria consta de quatre parts diferenciades. En els dos primers blocs es revisen els fonaments de la teoria de semblança quàntica, així com l'aproximació topològica basada en la teoria de grafs. Ambdues teories es fan servir per a calcular els descriptors moleculars. En el segon bloc, s'ha de remarcar la programació i implementació de programari per a calcular els anomenats índexs topològics de semblança quàntica. La tercera secció detalla les bases de les Relacions Quantitatives Estructura-Activitat i, finalment, el darrer apartat recull els resultats d'aplicació obtinguts per a diferents sistemes biològics.