Curiosity driven reinforcement learning for motion planning on humanoids

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

Numerical investigation of motion and deformation of a single red blood cell in a stenosed capillary

It is generally assumed that influence of the red blood cells (RBCs) is predominant in blood rheology. The healthy RBCs are highly deformable and can thus easily squeeze through the smallest capillaries having internal diameter less than their characteristic size. On the other hand, RBCs infected by malaria or other diseases are stiffer and so less deformable. Thus it is harder for them to flow through the smallest capillaries. Therefore, it is very important to critically and realistically investigate the mechanical behavior of both healthy and infected RBCs which is a current gap in knowledge. The motion and the steady state deformed shape of the RBCs depend on many factors, such as the geometrical parameters of the capillary through which blood flows, the membrane bending stiffness and the mean velocity of the blood flow. In this study, motion and deformation of a single two-dimensional RBC in a stenosed capillary is explored by using smoothed particle hydrodynamics (SPH) method. An elastic spring network is used to model the RBC membrane, while the RBC's inside fluid and outside fluid are treated as SPH particles. The effect of RBC's membrane stiffness (kb), inlet pressure (P) and geometrical parameters of the capillary on the motion and deformation of the RBC is studied. The deformation index, RBC's mean velocity and the cell membrane energy are analyzed when the cell passes through the stenosed capillary. The simulation results demonstrate that the kb, P and the geometrical parameters of the capillary have a significant impact on the RBCs' motion and deformation in the stenosed section.

Made by motion: A conceptual framework for abstracted animation derived from motion-captured movements

Despite an increasing number of acclaimed abstract animations being created through the application of motion capture technologies there has been little detailed documentation and analysis of this approach for abstract animation production. More specifically, it is unclear what the key considerations are, and what issues practitioners might face, when integrating motion capture movement data into their practice. In response to this issue this study explored and documented the practice of generating abstract visual and temporal artefacts from motion captured dance movements that compose abstract animated short films. The study has resulted in a possible framework for this form of practice and outlines five key considerations which should be taken into account by practitioners who use motion capture in the production of abstract animated short films.

Development of a new ultrasound-based system for tracking motion of the human lumbar spine: Reliability, stability and repeatability during forward bending movement trials

The aim of this study was to develop a new method for quantifying intersegmental motion of the spine in an instrumented motion segment L4–L5 model using ultrasound image post-processing combined with an electromagnetic device. A prospective test–retest design was employed, combined with an evaluation of stability and within- and between-day intra-tester reliability during forward bending by 15 healthy male patients. The accuracy of the measurement system using the model was calculated to be ± 0.9° (standard deviation = 0.43) over a 40° range and ± 0.4 cm (standard deviation = 0.28) over 1.5 cm. The mean composite range of forward bending was 15.5 ± 2.04° during a single trial (standard error of the mean = 0.54, coefficient of variation = 4.18). Reliability (intra-class correlation coefficient = 2.1) was found to be excellent for both within-day measures (0.995–0.999) and between-day measures (0.996–0.999). Further work is necessary to explore the use of this approach in the evaluation of biomechanics, clinical assessments and interventions.

Heritability of head motion during resting state functional MRI in 462 healthy twins

Head motion (HM) is a critical confounding factor in functional MRI. Here we investigate whether HM during resting state functional MRI (RS-fMRI) is influenced by genetic factors in a sample of 462 twins (65% fema≤ 101 MZ (monozygotic) and 130 DZ (dizygotic) twin pairs; mean age: 21 (SD=3.16), range 16-29). Heritability estimates for three HM components-mean translation (MT), maximum translation (MAXT) and mean rotation (MR)-ranged from 37 to 51%. We detected a significant common genetic influence on HM variability, with about two-thirds (genetic correlations range 0.76-1.00) of the variance shared between MR, MT and MAXT. A composite metric (HM-PC1), which aggregated these three, was also moderately heritable (h2=42%). Using a sub-sample (N=35) of the twins we confirmed that mean and maximum translational and rotational motions were consistent "traits" over repeated scans (r=0.53-0.59); reliability was even higher for the composite metric (r=0.66). In addition, phenotypic and cross-trait cross-twin correlations between HM and resting state functional connectivities (RS-FCs) with Brodmann areas (BA) 44 and 45, in which RS-FCs were found to be moderately heritable (BA44: h2-=0.23 (sd=0.041), BA45: h2-=0.26 (sd=0.061)), indicated that HM might not represent a major bias in genetic studies using FCs. Even so, the HM effect on FC was not completely eliminated after regression. HM may be a valuable endophenotype whose relationship with brain disorders remains to be elucidated.

Interplay of MLC, gantry and respiratory motion during DCAT delivery

This study investigated the possible interplay effects arising from the treatment of moving targets using the dynamic conformal arc therapy (DCAT) technique. Dose from a modulated test beam was measured, with and without phantom motion and with and without a 30o arc rotation, using a diode array placed on a sinusoidally moving platform. Measurements were repeated at five different collimator angles (0, 22.5, 45, 67.5 and 90o), at two different dose rates (300 and 600 MU/min). Results showed that the effect of respiratory motion on the measured dose distribution increased slightly when the beams were delivered as arcs, rather than with a static gantry angle, and that this effect increased substantially as the collimator angle was increased from 0o (MLC motion perpendicular to respiratory motion) to 90o (MLC motion parallel to respiratory motion). The dose oscillations arising from interplay between phantom and MLC motion were found to increase in magnitude when the dose rate was increased. These results led to the development of simple recommendations for minimizing the negative effects of motion interplay on DCAT dose distributions

Investigation of the relationship between BMI and prostate motion in radiation therapy treatment for prostate cancer

This thesis studied the influence of patient obesity on prostate motion during radiation therapy treatment delivery, an important consideration in the accurate treatment of prostate cancer. The study highlighted the importance of daily image guidance to correct for prostate motion, increasing radiation dose to the prostate while decreasing radiation dose to surrounding healthy tissues, thereby increasing patient quality of life.

An exploratory ergonomic study of musculoskeletal disorder prevention in the Queensland Ambulance Service

From 2008-09 to 2012-13, the most prevalent worker compensation claim in the Queensland Ambulance Service (QAS) was musculoskeletal injuries at >80%. This is consistent with literature that shows Musculoskeletal Disorders (MSD) was one of the front runners for workplace injuries among many professions. In an attempt to reduce the injury rate and related claims, the QAS created a selection criterion for their workers based on the Health Related Fitness Test. This method intended to select workers based upon their fitness level, instead of selecting for their ability to perform the tasks or modify the tasks to better suit the workers. With injury rates remaining high, further research produced the Patient Handling Equipment Project Report, which provided the background for the Manual Handling Program Book. The Manual Handling Program Book however lacks in accurately addressing musculoskeletal hazards; actions which cause or avoid injury, correct posture and motion for patient movement, muscular biomechanics, static and dynamic workload including activities causing strain, and equipment use in relation to musculoskeletal hazards. The exploratory research aims to better understand the ambulance service’s perception of Manual Materials Handling (MMH), how it relates to musculoskeletal injuries and how the service has attempted to reduce its prevalence. Based on a literature review and a critical analysis of the QAS Health Related Fitness Test, QAS Patient Handling Equipment Project Report and the QAS Manual Handling Program Book, an understanding of their shortfalls in the prevention of musculoskeletal injuries was gained. This entails understanding the work tasks, workloads, strains and workflow of paramedics. This research creates a starting point for further research into musculoskeletal injuries in paramedics. This study specifically looks at hazards related to musculoskeletal disorders. It identifies work system deficiencies that contribute to the prevalence of musculoskeletal injuries, and possible interventions to avoid them in paramedics.

A pilot study in different unstable designs on the biomechanical effect of gait characteristics

The purpose of this study was to compare kinematics and kinetics during walking for healthy subjects using unstable shoes with different designs. Ten subjects participated in this study, and foot biomechanical data during walking were quantified using motion analysis system and a force plate. Data were collected for unstable shoes condition after accommodation period of one week. With soft material added in the heel region, the peak impact force was effectively reduced when compared among similar shapes. In addition, the soft material added in the rocker bottom showed more to be in dorsiflexed position during the initial stance. The shoe with three rocker curves design reduced the contact area in the heel strike, which may result in increasing human body forward speed. Further studies shall be carried out after adapting to long periods of wearing unstable shoes.

A coupled SPH-DEM approach to model the interactions between multiple red blood cells in motion in capillaries

Red blood cells (RBCs) are the most common type of blood cells in the blood and 99% of the blood cells are RBCs. During the circulation of blood in the cardiovascular network, RBCs squeeze through the tiny blood vessels (capillaries). They exhibit various types of motions and deformed shapes, when flowing through these capillaries with diameters varying between 5 10 µm. RBCs occupy about 45 % of the whole blood volume and the interaction between the RBCs directly influences on the motion and the deformation of the RBCs. However, most of the previous numerical studies have explored the motion and deformation of a single RBC when the interaction between RBCs has been neglected. In this study, motion and deformation of two 2D (two-dimensional) RBCs in capillaries are comprehensively explored using a coupled smoothed particle hydrodynamics (SPH) and discrete element method (DEM) model. In order to clearly model the interactions between RBCs, only two RBCs are considered in this study even though blood with RBCs is continuously flowing through the blood vessels. A spring network based on the DEM is employed to model the viscoelastic membrane of the RBC while the inside and outside fluid of RBC is modelled by SPH. The effect of the initial distance between two RBCs, membrane bending stiffness (Kb) of one RBC and undeformed diameter of one RBC on the motion and deformation of both RBCs in a uniform capillary is studied. Finally, the deformation behavior of two RBCs in a stenosed capillary is also examined. Simulation results reveal that the interaction between RBCs has significant influence on their motion and deformation.

Stationary processes for object detection and non-rigid structure-from-motion

Stationary processes are random variables whose value is a signal and whose distribution is invariant to translation in the domain of the signal. They are intimately connected to convolution, and therefore to the Fourier transform, since the covariance matrix of a stationary process is a Toeplitz matrix, and Toeplitz matrices are the expression of convolution as a linear operator. This thesis utilises this connection in the study of i) efficient training algorithms for object detection and ii) trajectory-based non-rigid structure-from-motion.

The'cave' mineral oxammite: a high resolution thermogravimetry and Raman spectroscopic study

The thermal decomposition of natural ammonium oxalate known as oxammite has been studied using a combination of high resolution thermogravimetry coupled to an evolved gas mass spectrometer and Raman spectroscopy coupled to a thermal stage. Three mass loss steps were found at 57, 175 and 188°C attributed to dehydration, ammonia evolution and carbon dioxide evolution respectively. Raman spectroscopy shows two bands at 3235 and 3030 cm-1 attributed to the OH stretching vibrations and three bands at 2995, 2900 and 2879 cm-1, attributed to the NH vibrational modes. The thermal degradation of oxammite may be followed by the loss of intensity of these bands. No intensity remains in the OH stretching bands at 100°C and the NH stretching bands show no intensity at 200°C. Multiple CO symmetric stretching bands are observed at 1473, 1454, 1447 and 1431cm-1, suggesting that the mineral oxammite is composed of a mixture of chemicals including ammonium oxalate dihydrate, ammonium oxalate monohydrate and anhydrous ammonium oxalate.

Near-infrared spectroscopic study of basic aluminum sulfate and nitrate

The tridecameric Al-polymer [AlO4Al12(OH)24(H2O)12]7+ was prepared by forced hydrolysis of Al3+ up to an OH/Al molar ratio of 2.2. Under slow evaporation crystals were formed of Al13-nitrate. Upon addition of sulfate the tridecamer crystallised as the monoclinic Al13-sulfate. These crystals have been studied using near-infrared spectroscopy and compared to Al2(SO4)3.16H2O. Although the near-infrared spectra of the Al13-sulfate and nitrate are very similar indicating similar crystal structures, there are minor differences related to the strength with which the crystal water molecules are bonded to the salt groups. The interaction between crystal water and nitrate is stronger than with the sulfate as reflected by the shift of the crystal water band positions from 6213, 4874 and 4553 cm–1 for the Al13 sulfate towards 5925, 4848 and 4532 cm–1 for the nitrate. A reversed shift from 5079 and 5037 cm–1 for the sulfate towards 5238 and 5040 cm–1 for the nitrate for the water molecules in the Al13 indicate that the nitrate-Al13 bond is weakened due to the influence of the crystal water on the nitrate. The Al-OH bond in the Al13 complex is not influenced by changing the salt group due to the shielding by the water molecules of the Al13 complex.