Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Dam wall detection and tracking using a Mechanically Scanned Imaging Sonar

In dam inspection tasks, an underwater robot has to grab images while surveying the wall meanwhile maintaining a certain distance and relative orientation. This paper proposes the use of an MSIS (mechanically scanned imaging sonar) for relative positioning of a robot with respect to the wall. An imaging sonar gathers polar image scans from which depth images (range & bearing) are generated. Depth scans are first processed to extract a line corresponding to the wall (with the Hough transform), which is then tracked by means of an EKF (Extended Kalman Filter) using a static motion model and an implicit measurement equation associating the sensed points to the candidate line. The line estimate is referenced to the robot fixed frame and represented in polar coordinates (rho&thetas) which directly corresponds to the actual distance and relative orientation of the robot with respect to the wall. The proposed system has been tested in simulation as well as in water tank conditions

SQualTrack: A Tool for Robust Fault Detection

One of the techniques used to detect faults in dynamic systems is analytical redundancy. An important difficulty in applying this technique to real systems is dealing with the uncertainties associated with the system itself and with the measurements. In this paper, this uncertainty is taken into account by the use of intervals for the parameters of the model and for the measurements. The method that is proposed in this paper checks the consistency between the system's behavior, obtained from the measurements, and the model's behavior; if they are inconsistent, then there is a fault. The problem of detecting faults is stated as a quantified real constraint satisfaction problem, which can be solved using the modal interval analysis (MIA). MIA is used because it provides powerful tools to extend the calculations over real functions to intervals. To improve the results of the detection of the faults, the simultaneous use of several sliding time windows is proposed. The result of implementing this method is semiqualitative tracking (SQualTrack), a fault-detection tool that is robust in the sense that it does not generate false alarms, i.e., if there are false alarms, they indicate either that the interval model does not represent the system adequately or that the interval measurements do not represent the true values of the variables adequately. SQualTrack is currently being used to detect faults in real processes. Some of these applications using real data have been developed within the European project advanced decision support system for chemical/petrochemical manufacturing processes and are also described in this paper

A Monte Carlo-Based Fiber Tracking Algorithm using Diffusion Tensor MRI

Diffusion tensor magnetic resonance imaging, which measures directional information of water diffusion in the brain, has emerged as a powerful tool for human brain studies. In this paper, we introduce a new Monte Carlo-based fiber tracking approach to estimate brain connectivity. One of the main characteristics of this approach is that all parameters of the algorithm are automatically determined at each point using the entropy of the eigenvalues of the diffusion tensor. Experimental results show the good performance of the proposed approach