Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Visual SLAM for underwater vehicles using video velocity log and natural landmarks

A visual SLAM system has been implemented and optimised for real-time deployment on an AUV equipped with calibrated stereo cameras. The system incorporates a novel approach to landmark description in which landmarks are local sub maps that consist of a cloud of 3D points and their associated SIFT/SURF descriptors. Landmarks are also sparsely distributed which simplifies and accelerates data association and map updates. In addition to landmark-based localisation the system utilises visual odometry to estimate the pose of the vehicle in 6 degrees of freedom by identifying temporal matches between consecutive local sub maps and computing the motion. Both the extended Kalman filter and unscented Kalman filter have been considered for filtering the observations. The output of the filter is also smoothed using the Rauch-Tung-Striebel (RTS) method to obtain a better alignment of the sequence of local sub maps and to deliver a large-scale 3D acquisition of the surveyed area. Synthetic experiments have been performed using a simulation environment in which ray tracing is used to generate synthetic images for the stereo system

Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

Sensitivity and specificity of tympanometric gradient and middle ear resonance in young children and infants

This paper discusses the use of tympanometric gradient, middle ear resonance and static admittance as diagnostic tools for testing the hearing of children and infants.

Velocity discrimination of angular motion at different frequencies in individuals with normal vestibular function

The purpose of this study was to evaluate discrimination of angular velocity in individuals with normal vestibular function using a newly developed adaptive psychophysical measure. Vestibular psychophysical testing may complement existing clinical measures in diagnosing and treating patients with imbalance.

Declines of Aerial Insectivores in North America Follow a Geographic Gradient

North American birds that feed on aerial insects are experiencing widespread population declines. An analysis of the North American Breeding Bird Survey trend estimates for 1966 to 2006 suggests that declines in this guild are significantly stronger than in passerines in general. The pattern of decline also shows a striking geographical gradient, with aerial insectivore declines becoming more prevalent towards the northeast of North America. Declines are also more acute in species that migrate long distances compared to those that migrate short distances. The declines become manifest, almost without exception, in the mid 1980s. The taxonomic breadth of these downward trends suggests that declines in aerial insectivore populations are linked to changes in populations of flying insects, and these changes might be indicative of underlying ecosystem changes.

The counter-propagating Rossby-wave perspective on baroclinic instability. Part III: Primitive-equation disturbances on the sphere

Baroclinic instability of perturbations described by the linearized primitive quations, growing on steady zonal jets on the sphere, can be understood in terms of the interaction of pairs of counter-propagating Rossby waves (CRWs). The CRWs can be viewed as the basic components of the dynamical system where the Hamiltonian is the pseudoenergy and each CRW has a zonal coordinate and pseudomomentum. The theory holds for adiabatic frictionless flow to the extent that truncated forms of pseudomomentum and pseudoenergy are globally conserved. These forms focus attention on Rossby wave activity. Normal mode (NM) dispersion relations for realistic jets are explained in terms of the two CRWs associated with each unstable NM pair. Although derived from the NMs, CRWs have the conceptual advantage that their structure is zonally untilted, and can be anticipated given only the basic state. Moreover, their zonal propagation, phase-locking and mutual interaction can all be understood by ‘PV-thinking’ applied at only two ‘home-bases’—potential vorticity (PV) anomalies at one home-base induce circulation anomalies, both locally and at the other home-base, which in turn can advect the PV gradient and modify PV anomalies there. At short wavelengths the upper CRW is focused in the mid-troposphere just above the steering level of the NM, but at longer wavelengths the upper CRW has a second wave-activity maximum at the tropopause. In the absence of meridional shear, CRW behaviour is very similar to that of Charney modes, while shear results in a meridional slant with height of the air-parcel displacement-structures of CRWs in sympathy with basic-state zonal angular-velocity surfaces. A consequence of this slant is that baroclinically growing eddies (on jets broader than the Rossby radius) must tilt downshear in the horizontal, giving rise to up-gradient momentum fluxes that tend to accelerate the barotropic component of the jet.

The counter-propagating Rossby-wave perspective on baroclinic instability. Part IV: Nonlinear life cycles

Pairs of counter-propagating Rossby waves (CRWs) can be used to describe baroclinic instability in linearized primitive-equation dynamics, employing simple propagation and interaction mechanisms at only two locations in the meridional plane—the CRW ‘home-bases’. Here, it is shown how some CRW properties are remarkably robust as a growing baroclinic wave develops nonlinearly. For example, the phase difference between upper-level and lower-level waves in potential-vorticity contours, defined initially at the home-bases of the CRWs, remains almost constant throughout baroclinic wave life cycles, despite the occurrence of frontogenesis and Rossby-wave breaking. As the lower wave saturates nonlinearly the whole baroclinic wave changes phase speed from that of the normal mode to that of the self-induced phase speed of the upper CRW. On zonal jets without surface meridional shear, this must always act to slow the baroclinic wave. The direction of wave breaking when a basic state has surface meridional shear can be anticipated because the displacement structures of CRWs tend to be coherent along surfaces of constant basic-state angular velocity, U. This results in up-gradient horizontal momentum fluxes for baroclinically growing disturbances. The momentum flux acts to shift the jet meridionally in the direction of the increasing surface U, so that the upper CRW breaks in the same direction as occurred at low levels

The response of a uniform horizontal temperature gradient to heating

The response of a uniform horizontal temperature gradient to prescribed fixed heating is calculated in the context of an extended version of surface quasigeostrophic dynamics. It is found that for zero mean surface flow and weak cross-gradient structure the prescribed heating induces a mean temperature anomaly proportional to the spatial Hilbert transform of the heating. The interior potential vorticity generated by the heating enhances this surface response. The time-varying part is independent of the heating and satisfies the usual linearized surface quasigeostrophic dynamics. It is shown that the surface temperature tendency is a spatial Hilbert transform of the temperature anomaly itself. It then follows that the temperature anomaly is periodically modulated with a frequency proportional to the vertical wind shear. A strong local bound on wave energy is also found. Reanalysis diagnostics are presented that indicate consistency with key findings from this theory.