Curiosity driven reinforcement learning for motion planning on humanoids

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

Numerical investigation of motion and deformation of a single red blood cell in a stenosed capillary

It is generally assumed that influence of the red blood cells (RBCs) is predominant in blood rheology. The healthy RBCs are highly deformable and can thus easily squeeze through the smallest capillaries having internal diameter less than their characteristic size. On the other hand, RBCs infected by malaria or other diseases are stiffer and so less deformable. Thus it is harder for them to flow through the smallest capillaries. Therefore, it is very important to critically and realistically investigate the mechanical behavior of both healthy and infected RBCs which is a current gap in knowledge. The motion and the steady state deformed shape of the RBCs depend on many factors, such as the geometrical parameters of the capillary through which blood flows, the membrane bending stiffness and the mean velocity of the blood flow. In this study, motion and deformation of a single two-dimensional RBC in a stenosed capillary is explored by using smoothed particle hydrodynamics (SPH) method. An elastic spring network is used to model the RBC membrane, while the RBC's inside fluid and outside fluid are treated as SPH particles. The effect of RBC's membrane stiffness (kb), inlet pressure (P) and geometrical parameters of the capillary on the motion and deformation of the RBC is studied. The deformation index, RBC's mean velocity and the cell membrane energy are analyzed when the cell passes through the stenosed capillary. The simulation results demonstrate that the kb, P and the geometrical parameters of the capillary have a significant impact on the RBCs' motion and deformation in the stenosed section.

Predictive motion planning for AUVs subject to strong time-varying currents and forecasting uncertainties

This paper presents a novel path planning method for minimizing the energy consumption of an autonomous underwater vehicle subjected to time varying ocean disturbances and forecast model uncertainty. The algorithm determines 4-Dimensional path candidates using Nonlinear Robust Model Predictive Control (NRMPC) and solutions optimised using A*-like algorithms. Vehicle performance limits are incorporated into the algorithm with disturbances represented as spatial and temporally varying ocean currents with a bounded uncertainty in their predictions. The proposed algorithm is demonstrated through simulations using a 4-Dimensional, spatially distributed time-series predictive ocean current model. Results show the combined NRMPC and A* approach is capable of generating energy-efficient paths which are resistant to both dynamic disturbances and ocean model uncertainty.

Simulating droplet motion on virtual leaf surfaces

A curvilinear thin film model is used to simulate the motion of droplets on a virtual leaf surface, with a view to better understand the retention of agricultural sprays on plants. The governing model, adapted from Roy et al. (2002 J. Fluid Mech. 454, 235–261) with the addition of a disjoining pressure term, describes the gravity- and curvature driven flow of a small droplet on a complex substrate: a cotton leaf reconstructed from digitized scan data. Coalescence is the key mechanism behind spray coating of foliage, and our simulations demonstrate that various experimentally observed coalescence behaviours can be reproduced qualitatively. By varying the contact angle over the domain, we also demonstrate that the presence of a chemical defect can act as an obstacle to the droplet’s path, causing break-up. In simulations on the virtual leaf, it is found that the movement of a typical spray size droplet is driven almost exclusively by substrate curvature gradients. It is not until droplet mass is sufficiently increased via coalescence that gravity becomes the dominating force.

Made by motion: A conceptual framework for abstracted animation derived from motion-captured movements

Despite an increasing number of acclaimed abstract animations being created through the application of motion capture technologies there has been little detailed documentation and analysis of this approach for abstract animation production. More specifically, it is unclear what the key considerations are, and what issues practitioners might face, when integrating motion capture movement data into their practice. In response to this issue this study explored and documented the practice of generating abstract visual and temporal artefacts from motion captured dance movements that compose abstract animated short films. The study has resulted in a possible framework for this form of practice and outlines five key considerations which should be taken into account by practitioners who use motion capture in the production of abstract animated short films.

Development of a new ultrasound-based system for tracking motion of the human lumbar spine: Reliability, stability and repeatability during forward bending movement trials

The aim of this study was to develop a new method for quantifying intersegmental motion of the spine in an instrumented motion segment L4–L5 model using ultrasound image post-processing combined with an electromagnetic device. A prospective test–retest design was employed, combined with an evaluation of stability and within- and between-day intra-tester reliability during forward bending by 15 healthy male patients. The accuracy of the measurement system using the model was calculated to be ± 0.9° (standard deviation = 0.43) over a 40° range and ± 0.4 cm (standard deviation = 0.28) over 1.5 cm. The mean composite range of forward bending was 15.5 ± 2.04° during a single trial (standard error of the mean = 0.54, coefficient of variation = 4.18). Reliability (intra-class correlation coefficient = 2.1) was found to be excellent for both within-day measures (0.995–0.999) and between-day measures (0.996–0.999). Further work is necessary to explore the use of this approach in the evaluation of biomechanics, clinical assessments and interventions.

Heritability of head motion during resting state functional MRI in 462 healthy twins

Head motion (HM) is a critical confounding factor in functional MRI. Here we investigate whether HM during resting state functional MRI (RS-fMRI) is influenced by genetic factors in a sample of 462 twins (65% fema≤ 101 MZ (monozygotic) and 130 DZ (dizygotic) twin pairs; mean age: 21 (SD=3.16), range 16-29). Heritability estimates for three HM components-mean translation (MT), maximum translation (MAXT) and mean rotation (MR)-ranged from 37 to 51%. We detected a significant common genetic influence on HM variability, with about two-thirds (genetic correlations range 0.76-1.00) of the variance shared between MR, MT and MAXT. A composite metric (HM-PC1), which aggregated these three, was also moderately heritable (h2=42%). Using a sub-sample (N=35) of the twins we confirmed that mean and maximum translational and rotational motions were consistent "traits" over repeated scans (r=0.53-0.59); reliability was even higher for the composite metric (r=0.66). In addition, phenotypic and cross-trait cross-twin correlations between HM and resting state functional connectivities (RS-FCs) with Brodmann areas (BA) 44 and 45, in which RS-FCs were found to be moderately heritable (BA44: h2-=0.23 (sd=0.041), BA45: h2-=0.26 (sd=0.061)), indicated that HM might not represent a major bias in genetic studies using FCs. Even so, the HM effect on FC was not completely eliminated after regression. HM may be a valuable endophenotype whose relationship with brain disorders remains to be elucidated.

A time and motion study of peripheral venous catheter flushing practice using manually prepared and prefilled flush syringes

Peripheral venous catheters (PVCs) are the simplest and most frequently used method for drug, fluid, and blood product administration in the hospital setting. It is estimated that up to 90% of patients in acute care hospitals require a PVC; however, PVCs are associated with inherent complications, which can be mechanical or infectious. There have been a range of strategies to prevent or reduce PVC-related complications that include optimizing patency through the use of flushing. Little is known about the current status of flushing practice. This observational study quantified preparation and administration time and identified adherence to principles of Aseptic Non-Touch Technique and organizational protocol on PVC flushing by using both manually prepared and prefilled syringes.

Interplay of MLC, gantry and respiratory motion during DCAT delivery

This study investigated the possible interplay effects arising from the treatment of moving targets using the dynamic conformal arc therapy (DCAT) technique. Dose from a modulated test beam was measured, with and without phantom motion and with and without a 30o arc rotation, using a diode array placed on a sinusoidally moving platform. Measurements were repeated at five different collimator angles (0, 22.5, 45, 67.5 and 90o), at two different dose rates (300 and 600 MU/min). Results showed that the effect of respiratory motion on the measured dose distribution increased slightly when the beams were delivered as arcs, rather than with a static gantry angle, and that this effect increased substantially as the collimator angle was increased from 0o (MLC motion perpendicular to respiratory motion) to 90o (MLC motion parallel to respiratory motion). The dose oscillations arising from interplay between phantom and MLC motion were found to increase in magnitude when the dose rate was increased. These results led to the development of simple recommendations for minimizing the negative effects of motion interplay on DCAT dose distributions

Investigation of the relationship between BMI and prostate motion in radiation therapy treatment for prostate cancer

This thesis studied the influence of patient obesity on prostate motion during radiation therapy treatment delivery, an important consideration in the accurate treatment of prostate cancer. The study highlighted the importance of daily image guidance to correct for prostate motion, increasing radiation dose to the prostate while decreasing radiation dose to surrounding healthy tissues, thereby increasing patient quality of life.

The effect of LRP5 polymorphisms on bone mineral density is apparent in childhood

Bone mass acquired during childhood is the primary determinant of adult bone mineral density (BMD) and osteoporosis risk. Bone accrual is subject to genetic influences. Activating and inactivating LRP5 gene mutations elicit extreme bone phenotypes, while more common LRP5 polymorphisms are associated with normal variation of BMD. Our aim was to test the hypothesis that LRP5 gene polymorphisms influence bone mass acquisition during childhood. The association between LRP5 gene polymorphisms and bone size and mineralization was examined in 819 unrelated British Caucasian children (n = 429 boys) aged 9 years. Height, weight, pubertal status (where available), total-body and spinal bone area, bone mineral content (BMC), BMD, and area-adjusted BMC (aBMC) were assessed. Dual-energy X-ray absorptiometry (DXA)-gene associations were assessed by linear regression, with adjustment for age, gender, pubertal status, and body size parameters. There were 140, 79, 12, and 2 girls who achieved Tanner stages I-IV, respectively, and 179 and 32 boys who achieved Tanner stages I and II, respectively. The rs2306862 (N740N) coding polymorphism in exon 10 of the LRP5 gene was associated with spinal BMD and aBMC (each P = 0.01) and total-body BMD and aBMC (P = 0.04 and 0.03, respectively). Adjusting for pubertal stage strengthened associations between this polymorphism and spinal BMD and aBMC (P = 0.01 and 0.002, respectively). Individuals homozygous for the T allele had greater spinal BMD and aBMC scores than those homozygous for the C allele. A dose effect was apparent as the mean spinal BMD and aBMC of heterozygous TC individuals were intermediate between those of their TT and CC counterparts. The N740N polymorphism in exon 10 of LRP5 was associated with spinal BMD and aBMC in pre- and early pubertal children. These results indicate that LRP5 influences volumetric bone density in childhood, possibly through effects on trabecular bone formation.

Moffatt vortices induced by the motion of a contact line

A recent hydrodynamic theory of liquid slippage on a solid substrate (Kirkinis & Davis, Phys. Rev. Lett., vol. 110, 2013, 234503) gives rise to a sequence of eddies (Moffatt vortices) that co-move with a moving contact line (CL) in a liquid wedge. The presence of these vortices is established through secular equations that depend on the dynamic contact angle α and capillary number Ca. The limiting case α→O is associated with the appearance of such vortices in a channel. The vortices are generated by the relative motion of the interfaces, which in turn is due to the motion of the CL. This effect has yet to be observed in experiment.

In Perpetual Motion: Theories of Power, Educational History, and the Child

"In Perpetual Motion is an "historical choreography" of power, pedagogy, and the child from the 1600s to the early 1900s. It breaks new ground by historicizing the analytics of power and motion that have interpenetrated renditions of the young. Through a detailed examination of the works of John Locke, Jean-Jacques Rousseau, Johann Herbart, and G. Stanley Hall, this book maps the discursive shifts through which the child was given a unique nature, inscribed in relation to reason, imbued with an effectible interiority, and subjected to theories of power and motion. The book illustrates how developmentalist visions took hold in U.S. public school debates. It documents how particular theories of power became submerged and taken for granted as essences inside the human subject. In Perpetual Motion studiously challenges views of power as in or of the gaze, tracing how different analytics of power have been used to theorize what gazing could notice."--BOOK JACKET.

Control of Ship Roll Motion

The undesirable effects of roll motion of ships (rocking about the longitudinal axis) became noticeable in the mid-nineteenth century when significant changes were introduced to the design of ships as a result of sails being replaced by steam engines and the arrangement being changed from broad to narrow hulls. The combination of these changes led to lower transverse stability (lower restoring moment for a given angle of roll) with the consequence of larger roll motion. The increase in roll motion and its effect on cargo and human performance lead to the development several control devices that aimed at reducing and controlling roll motion. The control devices most commonly used today are fin stabilizers, rudder, anti-roll tanks, and gyrostabilizers. The use of different types of actuators for control of ship roll motion has been amply demonstrated for over 100 years. Performance, however, can still fall short of expectations because of difficulties associated with control system design, which have proven to be far from trivial due to fundamental performance limitations and large variations of the spectral characteristics of wave-induced roll motion. This short article provides an overview of the fundamentals of control design for ship roll motion reduction. The overview is limited to the most common control devices.

Energy-based motion control of a slender hull unmanned underwater vehicle

This paper presents a motion control system for tracking of attitude and speed of an underactuated slender-hull unmanned underwater vehicle. The feedback control strategy is developed using the Port-Hamiltonian theory. By shaping of the target dynamics (desired dynamic response in closed loop) with particular attention to the target mass matrix, the influence of the unactuated dynamics on the controlled system is suppressed. This results in achievable dynamics independent of stable uncontrolled states. Throughout the design, the insight of the physical phenomena involved is used to propose the desired target dynamics. Integral action is added to the system for robustness and to reject steady disturbances. This is achieved via a change of coordinates that result in input-to-state stable (ISS) target dynamics. As a final step in the design, an anti-windup scheme is implemented to account for limited actuator capacity, namely saturation. The performance of the design is demonstrated through simulation with a high-fidelity model.