Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

A Monte Carlo-Based Fiber Tracking Algorithm using Diffusion Tensor MRI

Diffusion tensor magnetic resonance imaging, which measures directional information of water diffusion in the brain, has emerged as a powerful tool for human brain studies. In this paper, we introduce a new Monte Carlo-based fiber tracking approach to estimate brain connectivity. One of the main characteristics of this approach is that all parameters of the algorithm are automatically determined at each point using the entropy of the eigenvalues of the diffusion tensor. Experimental results show the good performance of the proposed approach

Magnetic pill tracking : a novel non-invasive tool for investigation of human digestive motility

RESUME Les améliorations méthodologiques des dernières décennies ont permis une meilleure compréhension de la motilité gastro-intestinale. Il manque toutefois une méthode qui permette de suivre la progression du chyme le long du tube gastro-intestinal. Pour permettre l'étude de la motilité de tout le tractus digestif humain, une nouvelle technique, peu invasive, a été élaborée au Département de Physiologie, en collaboration avec l'EPFL. Appelée "Magnet Tracking", la technique est basée sur la détection du champ magnétique généré par des matériaux ferromagnétiques avalés. A cet usage, une pilule magnétique, une matrice de capteurs et un logiciel ont été développés. L'objet de ce travail est de démontrer la faisabilité d'un examen de la motilité gastro-intestinale chez l'Homme par cette méthode. L'aimant est un cylindre (ø 6x7 mm, 0.2 cm3) protégé par une gaine de silicone. Le système de mesure est constitué d'une matrice de 4x4 capteurs et d'un ordinateur portable. Les capteurs fonctionnent sur l'effet Hall. Grâce à l'interface informatique, l'évolution de la position de l'aimant est suivie en temps réel à travers tout le tractus digestif. Sa position est exprimée en fonction du temps ou reproduite en 3-D sous forme d'une trajectoire. Différents programmes ont été crées pour analyser la dynamique des mouvements de l'aimant et caractériser la motilité digestive. Dix jeunes volontaires en bonne santé ont participé à l'étude. L'aimant a été avalé après une nuit de jeûne et son séjour intra digestif suivi pendant 2 jours consécutifs. Le temps moyen de mesure était de 34 heures. Chaque sujet a été examiné une fois sauf un qui a répété sept fois l'expérience. Les sujets restaient en décubitus dorsal, tranquilles et pouvaient interrompre la mesure s'ils le désiraient. Ils sont restés à jeûne le premier jour. L'évacuation de l'aimant a été contrôlée chez tous les sujets. Tous les sujets ont bien supporté l'examen. Le marqueur a pu être détecté de l'oesophage au rectum. La trajectoire ainsi constituée représente une conformation de l'anatomie digestive : une bonne superposition de celle-ci à l'anatomie est obtenue à partir des images de radiologie conventionnelle (CT-scan, lavement à la gastrografine). Les mouvements de l'aimant ont été caractérisés selon leur périodicité, leur amplitude ou leur vitesse pour chaque segment du tractus digestif. Ces informations physiologiques sont bien corrélées à celles obtenues par des méthodes établies d'étude de la motilité gastro-intestinale. Ce travail démontre la faisabilité d'un examen de la motilité gastro-intestinal chez l'Homme par la méthode de Magnet Tracking. La technique fournit les données anatomiques et permet d'analyser en temps réel la dynamique des mouvements du tube digestif. Cette méthode peu invasive ouvre d'intéressantes perspectives pour l'étude de motilité dans des conditions physiologiques et pathologiques. Des expériences visant à valider cette approche en tant que méthode clinique sont en voie de réalisation dans plusieurs centres en Suisse et à l'étranger. SUMMARY Methodological improvements realised over the last decades have permitted a better understanding of gastrointestinal motility. Nevertheless, a method allowing a continuous following of lumina' contents is still lacking. In order to study the human digestive tract motility, a new minimally invasive technique was developed at the Department of Physiology in collaboration with Swiss Federal Institute of Technology. The method is based on the detection of magnetic field generated by swallowed ferromagnetic materials. The aim of our work was to demonstrate the feasibility of this new approach to study the human gastrointestinal motility. The magnet used was a cylinder (ø6x7mm, 0.2 cm3) coated with silicon. The magnet tracking system consisted of a 4x4 matrix of sensors based on the Hall effect Signals from the sensors were digitised and sent to a laptop computer for processing and storage. Specific software was conceived to analyse in real time the progression of the magnet through the gastrointestinal tube. Ten young and healthy volunteers were enrolled in the study. After a fasting period of 12 hours, they swallowed the magnet. The pill was then tracked for two consecutive days for 34 hours on average. Each subject was studied once except one who was studied seven times. Every subject laid on his back for the entire experiment but could interrupt it at anytime. Evacuation of the magnet was controlled in all subjects. The examination was well tolerated. The pill could be followed from the esophagus to the rectum. The trajectory of the magnet represented a "mould" of the anatomy of the digestive tube: a good superimposition with radiological anatomy (gastrografin contrast and CT) was obtained. Movements of the magnet were characterized by periodicity, velocity, and amplitude of displacements for every segment of the digestive tract. The physiological information corresponded well to data from current methods of studying gastrointestinal motility. This work demonstrates the feasibility of the new approach in studies of human gastrointestinal motility. The technique allows to correlate in real time the dynamics of digestive movements with the anatomical data. This minimally invasive method is ready for studies of human gastrointestinal motility under physiological as well as pathological conditions. Studies aiming at validation of this new approach as a clinically relevant tool are being realised in several centres in Switzerland and abroad. Abstract: A new minimally invasive technique allowing for anatomical mapping and motility studies along the entire human digestive system is presented. The technique is based on continuous tracking of a small magnet progressing through the digestive tract. The coordinates of the magnet are calculated from signals recorded by 16 magnetic field sensors located over the abdomen. The magnet position, orientation and trajectory are displayed in real time. Ten young healthy volunteers were followed during 34 h. The technique was well tolerated and no complication was encountered, The information obtained was 3-D con-figuration of the digestive tract and dynamics of the magnet displacement (velocity, transit time, length estimation, rhythms). In the same individual, repea-ted examination gave very reproducible results. The anatomical and physiological information obtained corresponded well to data from current methods and imaging. This simple, minimally invasive technique permits examination of the entire digestive tract and is suitable for both research and clinical studies. In combination with other methods, it may represent a useful tool for studies of Cl motility with respect to normal and pathological conditions.

Equivalence of piecewise-linear approximation and Lagrangian relaxation for network revenue management

The network revenue management (RM) problem arises in airline, hotel, media,and other industries where the sale products use multiple resources. It can be formulatedas a stochastic dynamic program but the dynamic program is computationallyintractable because of an exponentially large state space, and a number of heuristicshave been proposed to approximate it. Notable amongst these -both for their revenueperformance, as well as their theoretically sound basis- are approximate dynamic programmingmethods that approximate the value function by basis functions (both affinefunctions as well as piecewise-linear functions have been proposed for network RM)and decomposition methods that relax the constraints of the dynamic program to solvesimpler dynamic programs (such as the Lagrangian relaxation methods). In this paperwe show that these two seemingly distinct approaches coincide for the network RMdynamic program, i.e., the piecewise-linear approximation method and the Lagrangianrelaxation method are one and the same.

Three-dimensional magnetic resonance myocardial motion tracking from a single image plane.

Three-dimensional imaging for the quantification of myocardial motion is a key step in the evaluation of cardiac disease. A tagged magnetic resonance imaging method that automatically tracks myocardial displacement in three dimensions is presented. Unlike other techniques, this method tracks both in-plane and through-plane motion from a single image plane without affecting the duration of image acquisition. A small z-encoding gradient is subsequently added to the refocusing lobe of the slice-selection gradient pulse in a slice following CSPAMM acquisition. An opposite polarity z-encoding gradient is added to the orthogonal tag direction. The additional z-gradients encode the instantaneous through plane position of the slice. The vertical and horizontal tags are used to resolve in-plane motion, while the added z-gradients is used to resolve through-plane motion. Postprocessing automatically decodes the acquired data and tracks the three-dimensional displacement of every material point within the image plane for each cine frame. Experiments include both a phantom and in vivo human validation. These studies demonstrate that the simultaneous extraction of both in-plane and through-plane displacements and pathlines from tagged images is achievable. This capability should open up new avenues for the automatic quantification of cardiac motion and strain for scientific and clinical purposes.

Particle in an electromagnetic field: The Lorentz-Dirac equation

A new method to solve the Lorentz-Dirac equation in the presence of an external electromagnetic field is presented. The validity of the approximation is discussed, and the method is applied to a particle in the presence of a constant magnetic field.

Performance of a reciprocal shaker in mechanical dispersion of soil samples for particle-size analysis

The dispersion of the samples in soil particle-size analysis is a fundamental step, which is commonly achieved with a combination of chemical agents and mechanical agitation. The purpose of this study was to evaluate the efficiency of a low-speed reciprocal shaker for the mechanical dispersion of soil samples of different textural classes. The particle size of 61 soil samples was analyzed in four replications, using the pipette method to determine the clay fraction and sieving to determine coarse, fine and total sand fractions. The silt content was obtained by difference. To evaluate the performance, the results of the reciprocal shaker (RSh) were compared with data of the same soil samples available in reports of the Proficiency testing for Soil Analysis Laboratories of the Agronomic Institute of Campinas (Prolab/IAC). The accuracy was analyzed based on the maximum and minimum values defining the confidence intervals for the particle-size fractions of each soil sample. Graphical indicators were also used for data comparison, based on dispersion and linear adjustment. The descriptive statistics indicated predominantly low variability in more than 90 % of the results for sand, medium-textured and clay samples, and for 68 % of the results for heavy clay samples, indicating satisfactory repeatability of measurements with the RSh. Medium variability was frequently associated with silt, followed by the fine sand fraction. The sensitivity analyses indicated an accuracy of 100 % for the three main separates (total sand, silt and clay), in all 52 samples of the textural classes heavy clay, clay and medium. For the nine sand soil samples, the average accuracy was 85.2 %; highest deviations were observed for the silt fraction. In relation to the linear adjustments, the correlation coefficients of 0.93 (silt) or &gt; 0.93 (total sand and clay), as well as the differences between the angular coefficients and the unit < 0.16, indicated a high correlation between the reference data (Prolab/IAC) and results obtained with the RSh. In conclusion, the mechanical dispersion by the reciprocal shaker of soil samples of different textural classes was satisfactory. The results allowed recommending the use of the equipment at low agitation for particle size- analysis. The advantages of this Brazilian apparatus are its low cost, the possibility to simultaneously analyze a great number of samples using ordinary, easily replaceable glass or plastic bottles.

Particle dispersion in synthetic turbulent flows

We study particle dispersion advected by a synthetic turbulent flow from a Lagrangian perspective and focus on the two-particle and cluster dispersion by the flow. It has been recently reported that Richardson¿s law for the two-particle dispersion can stem from different dispersion mechanisms, and can be dominated by either diffusive or ballistic events. The nature of the Richardson dispersion depends on the parameters of our flow and is discussed in terms of the values of a persistence parameter expressing the relative importance of the two above-mentioned mechanisms. We support this analysis by studying the distribution of interparticle distances, the relative velocity correlation functions, as well as the relative trajectories.

Equivalence Between the Lagrangian and Hamiltonian Formalisms for Constrained Systems

The equivalence between the Lagrangian and Hamiltonian formalism is studied for constraint systems. A procedure to construct the Lagrangian constraints from the Hamiltonian constraints is given. Those Hamiltonian constraints that are first class with respect to the Hamiltonian constraints produce Lagrangian constraints that are FL-projectable.

Dirac and reduced quantization: A Lagrangian approach and Application to Coset Spaces

A Lagrangian treatment of the quantization of first class Hamiltonian systems with constraints and Hamiltonian linear and quadratic in the momenta, respectively, is performed. The first reduce and then quantize and the first quantize and then reduce (Diracs) methods are compared. A source of ambiguities in this latter approach is pointed out and its relevance on issues concerning self-consistency and equivalence with the first reduce method is emphasized. One of the main results is the relation between the propagator obtained la Dirac and the propagator in the full space. As an application of the formalism developed, quantization on coset spaces of compact Lie groups is presented. In this case it is shown that a natural selection of a Dirac quantization allows for full self-consistency and equivalence. Finally, the specific case of the propagator on a two-dimensional sphere S2 viewed as the coset space SU(2)/U(1) is worked out. 1995 American Institute of Physics.

Charged-particle interaction with liquids: Ripplon excitations

We calculate the ripplon field contribution to the self-energy of an electron exterior to a liquid for planar and spherical geometries. We compare the full dielectric calculation of the electron-liquid interaction with the simpler alternative method consisting of integrating the electron-atom static-induced-dipolar potential through the whole liquid volume. We obtain good agreement between both methods for a nonpolar liquid such as 4He but differences up to 40% for a polar liquid such as water. We study the conditions under which the ripplon contribution to the self-energy is a perturbation. For an electron moving parallel to a planar liquid surface, we calculate the ripplon contribution to its stopping power. For this dynamical case, we conclude that the alternative method is a good approximation even for polar liquids.

Interlaboratory comparison of size measurements on nanoparticles using nanoparticle tracking analysis (NTA)

One of the key challenges in the field of nanoparticle (NP) analysis is in producing reliable and reproducible characterisation data for nanomaterials. This study looks at the reproducibility using a relatively new, but rapidly adopted, technique, Nanoparticle Tracking Analysis (NTA) on a range of particle sizes and materials in several different media. It describes the protocol development and presents both the data and analysis of results obtained from 12 laboratories, mostly based in Europe, who are primarily QualityNano members. QualityNano is an EU FP7 funded Research Infrastructure that integrates 28 European analytical and experimental facilities in nanotechnology, medicine and natural sciences with the goal of developing and implementing best practice and quality in all aspects of nanosafety assessment. This study looks at both the development of the protocol and how this leads to highly reproducible results amongst participants. In this study, the parameter being measured is the modal particle size.

Bi-planar 2D-to-3D registration in Fourier domain for stereoscopic X-ray motion tracking

In this paper we present a new method to track bonemovements in stereoscopic X-ray image series of the kneejoint. The method is based on two different X-ray imagesets: a rotational series of acquisitions of the stillsubject knee that will allow the tomographicreconstruction of the three-dimensional volume (model),and a stereoscopic image series of orthogonal projectionsas the subject performs movements. Tracking the movementsof bones throughout the stereoscopic image series meansto determine, for each frame, the best pose of everymoving element (bone) previously identified in the 3Dreconstructed model. The quality of a pose is reflectedin the similarity between its simulated projections andthe actual radiographs. We use direct Fourierreconstruction to approximate the three-dimensionalvolume of the knee joint. Then, to avoid the expensivecomputation of digitally rendered radiographs (DRR) forpose recovery, we reformulate the tracking problem in theFourier domain. Under the hypothesis of parallel X-raybeams, we use the central-slice-projection theorem toreplace the heavy 2D-to-3D registration of projections inthe signal domain by efficient slice-to-volumeregistration in the Fourier domain. Focusing onrotational movements, the translation-relevant phaseinformation can be discarded and we only consider scalarFourier amplitudes. The core of our motion trackingalgorithm can be implemented as a classical frame-wiseslice-to-volume registration task. Preliminary results onboth synthetic and real images confirm the validity ofour approach.

Adaptive tracking of EEG oscillations.

Neuronal oscillations are an important aspect of EEG recordings. These oscillations are supposed to be involved in several cognitive mechanisms. For instance, oscillatory activity is considered a key component for the top-down control of perception. However, measuring this activity and its influence requires precise extraction of frequency components. This processing is not straightforward. Particularly, difficulties with extracting oscillations arise due to their time-varying characteristics. Moreover, when phase information is needed, it is of the utmost importance to extract narrow-band signals. This paper presents a novel method using adaptive filters for tracking and extracting these time-varying oscillations. This scheme is designed to maximize the oscillatory behavior at the output of the adaptive filter. It is then capable of tracking an oscillation and describing its temporal evolution even during low amplitude time segments. Moreover, this method can be extended in order to track several oscillations simultaneously and to use multiple signals. These two extensions are particularly relevant in the framework of EEG data processing, where oscillations are active at the same time in different frequency bands and signals are recorded with multiple sensors. The presented tracking scheme is first tested with synthetic signals in order to highlight its capabilities. Then it is applied to data recorded during a visual shape discrimination experiment for assessing its usefulness during EEG processing and in detecting functionally relevant changes. This method is an interesting additional processing step for providing alternative information compared to classical time-frequency analyses and for improving the detection and analysis of cross-frequency couplings.

Exact solution to the exit-time problem for an undamped free particle driven by Gaussian white noise

In a recent paper [Phys. Rev. Lett. 75, 189 (1995)] we have presented the exact analytical expression for the mean exit time, T(x,v), of a free inertial process driven by Gaussian white noise out of a region (0,L) in space. In this paper we give a detailed account of the method employed and present results on asymptotic properties and averages of T(x,v).