Curiosity driven reinforcement learning for motion planning on humanoids

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

Static load bearing exercises during rehabilitation of individuals with transfemoral amputation fitted with osseointegrated implant: Kinetic analysis

The desire to solve problems caused by socket prostheses in transfemoral amputees and the acquired success of osseointegration in the dental application has led to the introduction of osseointegration in the orthopedic surgery. Since its first introduction in 1990 in Gothenburg Sweden the osseointegrated (OI) orthopedic fixation has proven several benefits[1]. The surgery consists of two surgical procedures followed by a lengthy rehabilitation program. The rehabilitation program after an OI implant includes a specific training period with a short training prosthesis. Since mechanical loading is considered to be one of the key factors that influence bone mass and the osseointegration of bone-anchored implants, the rehabilitation program will also need to include some form of load bearing exercises (LBE). To date there are two frequently used commercially available human implants. We can find proof in the literature that load bearing exercises are performed by patients with both types of OI implants. We refer to two articles, a first one written by Dr. Aschoff and all and published in 2010 in the Journal of Bone and Joint Surgery.[2] The second one presented by Hagberg et al in 2009 gives a very thorough description of the rehabilitation program of TFA fitted with an OPRA implant. The progression of the load however is determined individually according to the residual skeleton’s quality, pain level and body weight of the participant.[1] Patients are using a classical bathroom weighing scale to control the load on the implant during the course of their rehabilitation. The bathroom scale is an affordable and easy-to-use device but it has some important shortcomings. The scale provides instantaneous feedback to the patient only on the magnitude of the vertical component of the applied force. The forces and moments applied along and around the three axes of the implant are unknown. Although there are different ways to assess the load on the implant for instance through inverse dynamics in a motion analysis laboratory [3-6] this assessment is challenging. A recent proof- of-concept study by Frossard et al (2009) showed that the shortcomings of the weighing scale can be overcome by a portable kinetic system based on a commercial transducer[7].

Numerical investigation of motion and deformation of a single red blood cell in a stenosed capillary

It is generally assumed that influence of the red blood cells (RBCs) is predominant in blood rheology. The healthy RBCs are highly deformable and can thus easily squeeze through the smallest capillaries having internal diameter less than their characteristic size. On the other hand, RBCs infected by malaria or other diseases are stiffer and so less deformable. Thus it is harder for them to flow through the smallest capillaries. Therefore, it is very important to critically and realistically investigate the mechanical behavior of both healthy and infected RBCs which is a current gap in knowledge. The motion and the steady state deformed shape of the RBCs depend on many factors, such as the geometrical parameters of the capillary through which blood flows, the membrane bending stiffness and the mean velocity of the blood flow. In this study, motion and deformation of a single two-dimensional RBC in a stenosed capillary is explored by using smoothed particle hydrodynamics (SPH) method. An elastic spring network is used to model the RBC membrane, while the RBC's inside fluid and outside fluid are treated as SPH particles. The effect of RBC's membrane stiffness (kb), inlet pressure (P) and geometrical parameters of the capillary on the motion and deformation of the RBC is studied. The deformation index, RBC's mean velocity and the cell membrane energy are analyzed when the cell passes through the stenosed capillary. The simulation results demonstrate that the kb, P and the geometrical parameters of the capillary have a significant impact on the RBCs' motion and deformation in the stenosed section.

Methods for the illumination of multilevel buildings with vertical light pipes

This paper examines the feasibility of using vertical light pipes to naturally illuminate the central core of a multilevel building not reached by window light. The challenges addressed were finding a method to extract and distribute equal amounts of light at each level and designing collectors to improve the effectiveness of vertical light pipes in delivering low elevation sunlight to the interior. Extraction was achieved by inserting partially reflecting cones within transparent sections of the pipes at each floor level. Theory was formulated to estimate the partial reflectance necessary to provide equal light extraction at each level. Designs for daylight collectors formed from laser cut panels tilted above the light pipe were developed and the benefits and limitations of static collectors as opposed to collectors that follow the sun azimuth investigated. Performance was assessed with both basic and detailed mathematical simulation and by observations made with a five level model building under clear sky conditions.

Predictive motion planning for AUVs subject to strong time-varying currents and forecasting uncertainties

This paper presents a novel path planning method for minimizing the energy consumption of an autonomous underwater vehicle subjected to time varying ocean disturbances and forecast model uncertainty. The algorithm determines 4-Dimensional path candidates using Nonlinear Robust Model Predictive Control (NRMPC) and solutions optimised using A*-like algorithms. Vehicle performance limits are incorporated into the algorithm with disturbances represented as spatial and temporally varying ocean currents with a bounded uncertainty in their predictions. The proposed algorithm is demonstrated through simulations using a 4-Dimensional, spatially distributed time-series predictive ocean current model. Results show the combined NRMPC and A* approach is capable of generating energy-efficient paths which are resistant to both dynamic disturbances and ocean model uncertainty.

Simulating droplet motion on virtual leaf surfaces

A curvilinear thin film model is used to simulate the motion of droplets on a virtual leaf surface, with a view to better understand the retention of agricultural sprays on plants. The governing model, adapted from Roy et al. (2002 J. Fluid Mech. 454, 235–261) with the addition of a disjoining pressure term, describes the gravity- and curvature driven flow of a small droplet on a complex substrate: a cotton leaf reconstructed from digitized scan data. Coalescence is the key mechanism behind spray coating of foliage, and our simulations demonstrate that various experimentally observed coalescence behaviours can be reproduced qualitatively. By varying the contact angle over the domain, we also demonstrate that the presence of a chemical defect can act as an obstacle to the droplet’s path, causing break-up. In simulations on the virtual leaf, it is found that the movement of a typical spray size droplet is driven almost exclusively by substrate curvature gradients. It is not until droplet mass is sufficiently increased via coalescence that gravity becomes the dominating force.

Made by motion: A conceptual framework for abstracted animation derived from motion-captured movements

Despite an increasing number of acclaimed abstract animations being created through the application of motion capture technologies there has been little detailed documentation and analysis of this approach for abstract animation production. More specifically, it is unclear what the key considerations are, and what issues practitioners might face, when integrating motion capture movement data into their practice. In response to this issue this study explored and documented the practice of generating abstract visual and temporal artefacts from motion captured dance movements that compose abstract animated short films. The study has resulted in a possible framework for this form of practice and outlines five key considerations which should be taken into account by practitioners who use motion capture in the production of abstract animated short films.

Heat-producing element enrichment in granitic rocks, the role of crustal composition and evolution

This study greatly enhanced our knowledge of the potential for geothermal energy development in Queensland as a viable clean energy source in the coming decades. Key outcomes of the project were understanding the first-order controls on the concentration of the heat-producing elements: uranium, thorium and potassium in granitic rocks, and constraining where rocks with the greatest heat-producing potential lie at depth in Queensland. Importantly, new temperature and heat flow maps for southwest Queensland were developed that will greatly assist future exploration efforts.

Development of a new ultrasound-based system for tracking motion of the human lumbar spine: Reliability, stability and repeatability during forward bending movement trials

The aim of this study was to develop a new method for quantifying intersegmental motion of the spine in an instrumented motion segment L4–L5 model using ultrasound image post-processing combined with an electromagnetic device. A prospective test–retest design was employed, combined with an evaluation of stability and within- and between-day intra-tester reliability during forward bending by 15 healthy male patients. The accuracy of the measurement system using the model was calculated to be ± 0.9° (standard deviation = 0.43) over a 40° range and ± 0.4 cm (standard deviation = 0.28) over 1.5 cm. The mean composite range of forward bending was 15.5 ± 2.04° during a single trial (standard error of the mean = 0.54, coefficient of variation = 4.18). Reliability (intra-class correlation coefficient = 2.1) was found to be excellent for both within-day measures (0.995–0.999) and between-day measures (0.996–0.999). Further work is necessary to explore the use of this approach in the evaluation of biomechanics, clinical assessments and interventions.

Low δ18O zircons from the Bruneau-­Jarbidge eruptive center: A key to crustal anatexis along the track of the Yellowstone hotspot

The Bruneau-Jarbidge eruptive center (BJEC) in the central Snake River Plain, Idaho, USA consists of the Cougar Point Tuff (CPT), a series of ten, high-temperature (900-1000°C) voluminous ignimbrites produced over the explosive phase of volcanism (12.8-10.5 Ma) and more than a dozen equally high-temperature rhyolite lava flows produced during the effusive phase (10.5-8 Ma). Spot analyses by ion microprobe of oxygen isotope ratios in 210 zircons demonstrate that all of the eruptive units of the BJEC are characterized by zircon δ¹⁸O values ≤ 2.5‰, thus documenting the largest low δ¹⁸O silicic volcanic province known on Earth (>10⁴ km³). There is no evidence for voluminous normal δ¹⁸O magmatism at the BJEC that precedes generation of low δ¹⁸O magmas as there is at other volcanic centers that generate low δ¹⁸O magmas such as Heise and Yellowstone. At these younger volcanic centers of the hotspot track, such low δ¹⁸O magmas represent ~45 % and ~20% respectively of total eruptive volumes. Zircons in all BJEC tuffs and lavas studied (23 units) document strong δ¹⁸O depletion (median CPT δ¹⁸OZrc = 1.0‰, post-CPT lavas = 1.5‰) with the third member of the CPT recording an excursion to minimum δ¹⁸O values (δ¹⁸OZrc= -1.8‰) in a supereruption > 2‰ lower than other voluminous low δ¹⁸O rhyolites known worldwide (δ¹⁸OWR ≤0.9 vs. 3.4‰). Subsequent units of the CPT and lavas record a progressive recovery in δ¹⁸OZrc to ~2.5‰ over a ~ 4 m.y. interval (12 to 8 Ma). We present detailed evidence of unit-to-unit systematic patterns in O isotopic zoning in zircons (i.e. direction and magnitude of Δcore-rim), spectrum of δ¹⁸O in individual units, and zircon inheritance patterns established by re-analysis of spots for U-Th-Pb isotopes by LA-ICPMS and SHRIMP. In conjunction with mineral thermometry and magma compositions, these patterns are difficult to reconcile with the well-established model for "cannibalistic" low δ¹⁸O magma genesis at Heise and Yellowstone. We present an alternative model for the central Snake River Plain using the modeling results of Leeman et al. (2008) for ¹⁸O depletion as a function of depth in a mid-upper crustal protolith that was hydrothermally altered by infiltrating meteoric waters prior to the onset of silicic magmatism. The model proposes that BJEC silicic magmas were generated in response to the propagation of a melting front, driven by the incremental growth of a vast underlying mafic sill complex, over a ~5 m.y. interval through a crustal volume in which a vertically asymmetric δ¹⁸OWR gradient had previously developed that was sharply inflected from ~ -1 to 10‰ at mid-upper crustal depths. Within the context of the model, data from BJEC zircons are consistent with incremental melting and mixing events in roof zones of magma reservoirs that accompany surfaceward advance of the coupled mafic-silicic magmatic system.

Heritability of head motion during resting state functional MRI in 462 healthy twins

Head motion (HM) is a critical confounding factor in functional MRI. Here we investigate whether HM during resting state functional MRI (RS-fMRI) is influenced by genetic factors in a sample of 462 twins (65% fema≤ 101 MZ (monozygotic) and 130 DZ (dizygotic) twin pairs; mean age: 21 (SD=3.16), range 16-29). Heritability estimates for three HM components-mean translation (MT), maximum translation (MAXT) and mean rotation (MR)-ranged from 37 to 51%. We detected a significant common genetic influence on HM variability, with about two-thirds (genetic correlations range 0.76-1.00) of the variance shared between MR, MT and MAXT. A composite metric (HM-PC1), which aggregated these three, was also moderately heritable (h2=42%). Using a sub-sample (N=35) of the twins we confirmed that mean and maximum translational and rotational motions were consistent "traits" over repeated scans (r=0.53-0.59); reliability was even higher for the composite metric (r=0.66). In addition, phenotypic and cross-trait cross-twin correlations between HM and resting state functional connectivities (RS-FCs) with Brodmann areas (BA) 44 and 45, in which RS-FCs were found to be moderately heritable (BA44: h2-=0.23 (sd=0.041), BA45: h2-=0.26 (sd=0.061)), indicated that HM might not represent a major bias in genetic studies using FCs. Even so, the HM effect on FC was not completely eliminated after regression. HM may be a valuable endophenotype whose relationship with brain disorders remains to be elucidated.

A time and motion study of peripheral venous catheter flushing practice using manually prepared and prefilled flush syringes

Peripheral venous catheters (PVCs) are the simplest and most frequently used method for drug, fluid, and blood product administration in the hospital setting. It is estimated that up to 90% of patients in acute care hospitals require a PVC; however, PVCs are associated with inherent complications, which can be mechanical or infectious. There have been a range of strategies to prevent or reduce PVC-related complications that include optimizing patency through the use of flushing. Little is known about the current status of flushing practice. This observational study quantified preparation and administration time and identified adherence to principles of Aseptic Non-Touch Technique and organizational protocol on PVC flushing by using both manually prepared and prefilled syringes.

Interplay of MLC, gantry and respiratory motion during DCAT delivery

This study investigated the possible interplay effects arising from the treatment of moving targets using the dynamic conformal arc therapy (DCAT) technique. Dose from a modulated test beam was measured, with and without phantom motion and with and without a 30o arc rotation, using a diode array placed on a sinusoidally moving platform. Measurements were repeated at five different collimator angles (0, 22.5, 45, 67.5 and 90o), at two different dose rates (300 and 600 MU/min). Results showed that the effect of respiratory motion on the measured dose distribution increased slightly when the beams were delivered as arcs, rather than with a static gantry angle, and that this effect increased substantially as the collimator angle was increased from 0o (MLC motion perpendicular to respiratory motion) to 90o (MLC motion parallel to respiratory motion). The dose oscillations arising from interplay between phantom and MLC motion were found to increase in magnitude when the dose rate was increased. These results led to the development of simple recommendations for minimizing the negative effects of motion interplay on DCAT dose distributions

Investigation of the relationship between BMI and prostate motion in radiation therapy treatment for prostate cancer

This thesis studied the influence of patient obesity on prostate motion during radiation therapy treatment delivery, an important consideration in the accurate treatment of prostate cancer. The study highlighted the importance of daily image guidance to correct for prostate motion, increasing radiation dose to the prostate while decreasing radiation dose to surrounding healthy tissues, thereby increasing patient quality of life.

The path less travelled: Vertical circulation spaces in apartment buildings in the subtropics

This paper presents data on residents’ use of common stairways and lifts (vertical circulation spaces) in multi-storey apartment buildings (MSABs) in Brisbane, Australia. Vertical movement is a defining aspect of multi-storey living and the energy consumed by lifts contributes significantly to the energy budget of the typical MSAB. The purpose is to investigate whether a reappraisal of vertical circulation design, through the lens of residents’ requirements, might contribute to energy reductions in this building type. Data was gathered on a theoretical sample of MSAB ranging from five decades old to very recent schemes. 90 residents were surveyed about their day-to-day experiences of circulation and access systems. The results showed that residents mainly chose to use the stairs for convenience and exercise. Building management regimes that limited residents’ access to collective spaces were the main impediment to discretionary stair use. Only two buildings did not have fully enclosed stairwells and these had the highest stair usage, suggesting that stair design, and building governance are two areas that might be worthy of attention. The more that circulation design is focussed on limiting access, the less opportunities there are for personal choice, incidental social interaction and casual surveillance of collective spaces. The more that design of vertical circulation spaces in MSAB meets residents’ needs the less likely they are to be reliant on continuous energy supply for normal functioning.