Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

The efficiency of direct payments versus tax reductions under uncertainty

This paper analyzes the optimal behavior of farmers in the presence of direct payments and uncertainty. In an empirical analysis for Switzerland, it confirms previously obtained theoretical results and determines the magnitude of the theoretical predicted effects. The results show that direct payments increase agricultural production between 3.7% to 4.8%. Alternatively to direct payments, the production effect of tax reductions is evaluated in order to determine its magnitude. The empirical analysis corroborates the theoretical results of the literature and demonstrates that tax reductions are also distorting, but to a substantially lesser degree if losses are not offset. However, tax reductions, independently whether losses are offset or not, lead to higher government spending than pure direct payments

Ka-Map: an interactive web-mapping application

ka-Map ("ka" as in ka-boom!) is an open source project that is aimed at providing a javascript API for developing highly interactive web-mapping interfaces using features available in modern web browsers. ka-Map currently has a number of interesting features. It sports the usual array of user interface elements such as: interactive, continuous panning without reloading the page; keyboard navigation options (zooming, panning); zooming to pre-set scales; interactive scalebar, legend and keymap support; optional layer control on client side; server side tile caching

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Online Robust 3D Mapping Using Structure from Motion Cues

This paper presents a complete solution for creating accurate 3D textured models from monocular video sequences. The methods are developed within the framework of sequential structure from motion, where a 3D model of the environment is maintained and updated as new visual information becomes available. The camera position is recovered by directly associating the 3D scene model with local image observations. Compared to standard structure from motion techniques, this approach decreases the error accumulation while increasing the robustness to scene occlusions and feature association failures. The obtained 3D information is used to generate high quality, composite visual maps of the scene (mosaics). The visual maps are used to create texture-mapped, realistic views of the scene

Mapping sub-flows to p2mp LSPs

In previous work we proposed a multi-objective traffic engineering scheme (MHDB-S model) using different distribution trees to multicast several flows. In this paper, we propose a heuristic algorithm to create multiple point-to-multipoint (p2mp) LSPs based on the optimum sub-flow values obtained with our MHDB-S model. Moreover, a general problem for supporting multicasting in MPLS networks is the lack of labels. To reduce the number of labels used, a label space reduction algorithm solution is also considered

Determination of the direct band-gap energy of InAlAs matched to InP by photoluminescence excitation spectroscopy

A series of InxAl1-xAs samples (0.51≪x≪0.55)coherently grown on InP was studied in order to measure the band-gap energy of the lattice matched composition. As the substrate is opaque to the relevant photon energies, a method is developed to calculate the optical absorption coefficient from the photoluminescence excitation spectra. The effect of strain on the band-gap energy has been taken into account. For x=0.532, at 14 K we have obtained Eg0=1549±6 meV

The mapping of the local contributions of Fermi and Coulomb correlation into intracule and extracule density distributions

The contributions of the correlated and uncorrelated components of the electron-pair density to atomic and molecular intracule I(r) and extracule E(R) densities and its Laplacian functions ∇2I(r) and ∇2E(R) are analyzed at the Hartree-Fock (HF) and configuration interaction (CI) levels of theory. The topologies of the uncorrelated components of these functions can be rationalized in terms of the corresponding one-electron densities. In contrast, by analyzing the correlated components of I(r) and E(R), namely, IC(r) and EC(R), the effect of electron Fermi and Coulomb correlation can be assessed at the HF and CI levels of theory. Moreover, the contribution of Coulomb correlation can be isolated by means of difference maps between IC(r) and EC(R) distributions calculated at the two levels of theory. As application examples, the He, Ne, and Ar atomic series, the C2-2, N2, O2+2 molecular series, and the C2H4 molecule have been investigated. For these atoms and molecules, it is found that Fermi correlation accounts for the main characteristics of IC(r) and EC(R), with Coulomb correlation increasing slightly the locality of these functions at the CI level of theory. Furthermore, IC(r), EC(R), and the associated Laplacian functions, reveal the short-ranged nature and high isotropy of Fermi and Coulomb correlation in atoms and molecules

A new efficient approach to the direct restricted active space self-consistent field method

An implicitly parallel method for integral-block driven restricted active space self-consistent field (RASSCF) algorithms is presented. The approach is based on a model space representation of the RAS active orbitals with an efficient expansion of the model subspaces. The applicability of the method is demonstrated with a RASSCF investigation of the first two excited states of indole

Casos de aplicaciones empresariales de Web Mapping y SIG Libre

La incorporación Software Libre para Geomática (FOSS4G), para la explotación de Información Geoespacial en Sistemas de Información (SI) empresariales es una tendencia inexorable. Aunque estas tecnologías se están difundiendo con rapidez en entornos de empresas especializadas, Universidades, Administraciones Públicas y Centros Tecnológicos, todavía es algo incipiente en grandes empresas, especialmente en aquellas no directamente relacionadas con las tecnologías de los SIG. El objetivo de esta presentación será mostrar cómo se está consiguiendo introducir el software de SIG libre en el mundo empresarial, con tres casos de éxito. El primero es un desarrollo tradicional para una compañía tipo ‘utility’ donde el cliente define una funcionalidad y contrata su desarrollo. El segundo es un modelo de contratación de servicios. La tercera es una aplicación para la administración pública. En los tres casos, el uso de software libre ha permitido ofrecer soluciones exitosas para los requerimientos de los clientes (tanto funcionales como de rendimiento), y óptimas en coste

Efficient topology estimation for large scale optical mapping

Large scale image mosaicing methods are in great demand among scientists who study different aspects of the seabed, and have been fostered by impressive advances in the capabilities of underwater robots in gathering optical data from the seafloor. Cost and weight constraints mean that lowcost Remotely operated vehicles (ROVs) usually have a very limited number of sensors. When a low-cost robot carries out a seafloor survey using a down-looking camera, it usually follows a predetermined trajectory that provides several non time-consecutive overlapping image pairs. Finding these pairs (a process known as topology estimation) is indispensable to obtaining globally consistent mosaics and accurate trajectory estimates, which are necessary for a global view of the surveyed area, especially when optical sensors are the only data source. This thesis presents a set of consistent methods aimed at creating large area image mosaics from optical data obtained during surveys with low-cost underwater vehicles. First, a global alignment method developed within a Feature-based image mosaicing (FIM) framework, where nonlinear minimisation is substituted by two linear steps, is discussed. Then, a simple four-point mosaic rectifying method is proposed to reduce distortions that might occur due to lens distortions, error accumulation and the difficulties of optical imaging in an underwater medium. The topology estimation problem is addressed by means of an augmented state and extended Kalman filter combined framework, aimed at minimising the total number of matching attempts and simultaneously obtaining the best possible trajectory. Potential image pairs are predicted by taking into account the uncertainty in the trajectory. The contribution of matching an image pair is investigated using information theory principles. Lastly, a different solution to the topology estimation problem is proposed in a bundle adjustment framework. Innovative aspects include the use of fast image similarity criterion combined with a Minimum spanning tree (MST) solution, to obtain a tentative topology. This topology is improved by attempting image matching with the pairs for which there is the most overlap evidence. Unlike previous approaches for large-area mosaicing, our framework is able to deal naturally with cases where time-consecutive images cannot be matched successfully, such as completely unordered sets. Finally, the efficiency of the proposed methods is discussed and a comparison made with other state-of-the-art approaches, using a series of challenging datasets in underwater scenarios

Análisis de la aplicabilidad de la técnica Value Stream Mapping en el rediseño de sistemas productivos

Los sistemas productivos de las empresas han de adaptarse a las exigencias de los mercados. El Value Stream Mapping (VSM) es una técnica desarrollada por la Producción Ajustada y orientada al rediseño de dichos sistemas productivos. Si bien existe divulgación teórica sobre la técnica así como publicaciones de casos prácticos exitosos, se detecta la carencia de un análisis que explore en profundidad la aplicabilidad de la técnica en entornos productivos relacionados con las lineas de flujo desconectadas. Así, el objetivo de la tesis es la evaluación de la aplicabilidad del VSM en dichos entornos. El método de investigación adoptado ha consistido en un estudio de casos múltiple sobre seis empresas. Los resultados confirman la validez práctica del VSM para el rediseño de sistemas productivos. No obstante, también se fijan aspectos de mejora y desarrollo para que la técnica pueda convertirse en la referencia base.