Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Automatic generation of active coordinates for quantum dynamics calculations: application to the dynamics of benzene photochemistry

A new practical method to generate a subspace of active coordinates for quantum dynamics calculations is presented. These reduced coordinates are obtained as the normal modes of an analytical quadratic representation of the energy difference between excited and ground states within the complete active space self-consistent field method. At the Franck-Condon point, the largest negative eigenvalues of this Hessian correspond to the photoactive modes: those that reduce the energy difference and lead to the conical intersection; eigenvalues close to 0 correspond to bath modes, while modes with large positive eigenvalues are photoinactive vibrations, which increase the energy difference. The efficacy of quantum dynamics run in the subspace of the photoactive modes is illustrated with the photochemistry of benzene, where theoretical simulations are designed to assist optimal control experiments

A quantum molecular similarity analysis of changes in molecular electron density caused by basis set flotation and electric field application

Quantum molecular similarity (QMS) techniques are used to assess the response of the electron density of various small molecules to application of a static, uniform electric field. Likewise, QMS is used to analyze the changes in electron density generated by the process of floating a basis set. The results obtained show an interrelation between the floating process, the optimum geometry, and the presence of an external field. Cases involving the Le Chatelier principle are discussed, and an insight on the changes of bond critical point properties, self-similarity values and density differences is performed

A comparative analysis by means of quantum molecular similarity measures of density distributions derived from conventional ab initio and density functional methods

A procedure based on quantum molecular similarity measures (QMSM) has been used to compare electron densities obtained from conventional ab initio and density functional methodologies at their respective optimized geometries. This method has been applied to a series of small molecules which have experimentally known properties and molecular bonds of diverse degrees of ionicity and covalency. Results show that in most cases the electron densities obtained from density functional methodologies are of a similar quality than post-Hartree-Fock generalized densities. For molecules where Hartree-Fock methodology yields erroneous results, the density functional methodology is shown to yield usually more accurate densities than those provided by the second order Møller-Plesset perturbation theory

Electronic couplings and on-site energies for hole transfer in DNA: systematic quantum mechanical/molecular dynamic study

The electron hole transfer (HT) properties of DNA are substantially affected by thermal fluctuations of the π stack structure. Depending on the mutual position of neighboring nucleobases, electronic coupling V may change by several orders of magnitude. In the present paper, we report the results of systematic QM/molecular dynamic (MD) calculations of the electronic couplings and on-site energies for the hole transfer. Based on 15 ns MD trajectories for several DNA oligomers, we calculate the average coupling squares 〈 V2 〉 and the energies of basepair triplets X G+ Y and X A+ Y, where X, Y=G, A, T, and C. For each of the 32 systems, 15 000 conformations separated by 1 ps are considered. The three-state generalized Mulliken-Hush method is used to derive electronic couplings for HT between neighboring basepairs. The adiabatic energies and dipole moment matrix elements are computed within the INDO/S method. We compare the rms values of V with the couplings estimated for the idealized B -DNA structure and show that in several important cases the couplings calculated for the idealized B -DNA structure are considerably underestimated. The rms values for intrastrand couplings G-G, A-A, G-A, and A-G are found to be similar, ∼0.07 eV, while the interstrand couplings are quite different. The energies of hole states G+ and A+ in the stack depend on the nature of the neighboring pairs. The X G+ Y are by 0.5 eV more stable than X A+ Y. The thermal fluctuations of the DNA structure facilitate the HT process from guanine to adenine. The tabulated couplings and on-site energies can be used as reference parameters in theoretical and computational studies of HT processes in DNA

Análisis de adopción, actividad y participación en aplicaciones SIG Libres: un estudio sobre GRASS, Quantum GIS y gvSIG basado en indicadores

A través del análisis de información pública del proyecto se han elaborado una serie de indicadores que permiten comparar el grado de adopción, actividad y participación en proyectos de software libre. Estos indicadores han sido desarrollados en base a información obtenida desde herramientas habituales en los proyectos (listas de correo, repositorios de código, etc) y que nos permiten reconstruir los patrones de comportamiento en los mismos

OpenSearch-geo: the simple standard for geographic web search engines

When publishing information on the web, one expects it to reach all the people that could be interested in. This is mainly achieved with general purpose indexing and search engines like Google which is the most used today. In the particular case of geographic information (GI) domain, exposing content to mainstream search engines is a complex task that needs specific actions. In many occasions it is convenient to provide a web site with a specially tailored search engine. Such is the case for on-line dictionaries (wikipedia, wordreference), stores (amazon, ebay), and generally all those holding thematic databases. Due to proliferation of these engines, A9.com proposed a standard interface called OpenSearch, used by modern web browsers to manage custom search engines. Geographic information can also benefit from the use of specific search engines. We can distinguish between two main approaches in GI retrieval information efforts: Classical OGC standardization on one hand (CSW, WFS filters), which are very complex for the mainstream user, and on the other hand the neogeographer’s approach, usually in the form of specific APIs lacking a common query interface and standard geographic formats. A draft ‘geo’ extension for OpenSearch has been proposed. It adds geographic filtering for queries and recommends a set of simple standard response geographic formats, such as KML, Atom and GeoRSS. This proposal enables standardization while keeping simplicity, thus covering a wide range of use cases, in both OGC and the neogeography paradigms. In this article we will analyze the OpenSearch geo extension in detail and its use cases, demonstrating its applicability to both the SDI and the geoweb. Open source implementations will be presented as well

Fuzzy set Theory and Quantum Chemistry

Es discuteixen breument algunes consideracions sobre l'aplicació de la Teoria dels Conjunts difusos a la Química quàntica. Es demostra aqui que molts conceptes químics associats a la teoria són adequats per ésser connectats amb l'estructura dels Conjunts difusos. També s'explica com algunes descripcions teoriques dels observables quàntics es potencien tractant-les amb les eines associades als esmentats Conjunts difusos. La funció densitat es pren com a exemple de l'ús de distribucions de possibilitat al mateix temps que les distribucions de probabilitat quàntiques

Quantum similarity topological indices definition, analysis and comparison with classical molecular topological indices

Es mostra que, gracies a una extensió en la definició dels Índexs Moleculars Topològics, s'arriba a la formulació d'índexs relacionats amb la teoria de la Semblança Molecular Quàntica. Es posa de manifest la connexió entre les dues metodologies: es revela que un marc de treball teòric sòlidament fonamentat sobre la teoria de la Mecànica Quàntica es pot connectar amb una de les tècniques més antigues relacionades amb els estudis de QSPR. Es mostren els resultats per a dos casos d'exemple d'aplicació d'ambdues metodologies

Practical applications of quantum molecular similarity measures (QMSM): programs and examples

Es descriu l'aproximació de Capes Atòmiques dins de la teoria de la Semblança Molecular Quàntica. Partint només de dades teòriques, s'ha trobat una relació entre estructura molecular i activitat biològica per a diversos conjunts de molècules. Es descriuen els aspectes teòrics de la Semblança Molecular Quàntica i alguns exemples d'aplicació

Molecular quantum similarity in QSAR: applications in computer-aided molecular design

La present tesi està centrada en l'ús de la Teoria de Semblança Quàntica per a calcular descriptors moleculars. Aquests descriptors s'utilitzen com a paràmetres estructurals per a derivar correlacions entre l'estructura i la funció o activitat experimental per a un conjunt de compostos. Els estudis de Relacions Quantitatives Estructura-Activitat són d'especial interès per al disseny racional de molècules assistit per ordinador i, en particular, per al disseny de fàrmacs. Aquesta memòria consta de quatre parts diferenciades. En els dos primers blocs es revisen els fonaments de la teoria de semblança quàntica, així com l'aproximació topològica basada en la teoria de grafs. Ambdues teories es fan servir per a calcular els descriptors moleculars. En el segon bloc, s'ha de remarcar la programació i implementació de programari per a calcular els anomenats índexs topològics de semblança quàntica. La tercera secció detalla les bases de les Relacions Quantitatives Estructura-Activitat i, finalment, el darrer apartat recull els resultats d'aplicació obtinguts per a diferents sistemes biològics.