Curiosity driven reinforcement learning for motion planning on humanoids

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

An improved computational method in structural dynamics

Purpose – In structural, earthquake and aeronautical engineering and mechanical vibration, the solution of dynamic equations for a structure subjected to dynamic loading leads to a high order system of differential equations. The numerical methods are usually used for integration when either there is dealing with discrete data or there is no analytical solution for the equations. Since the numerical methods with more accuracy and stability give more accurate results in structural responses, there is a need to improve the existing methods or develop new ones. The paper aims to discuss these issues. Design/methodology/approach – In this paper, a new time integration method is proposed mathematically and numerically, which is accordingly applied to single-degree-of-freedom (SDOF) and multi-degree-of-freedom (MDOF) systems. Finally, the results are compared to the existing methods such as Newmark’s method and closed form solution. Findings – It is concluded that, in the proposed method, the data variance of each set of structural responses such as displacement, velocity, or acceleration in different time steps is less than those in Newmark’s method, and the proposed method is more accurate and stable than Newmark’s method and is capable of analyzing the structure at fewer numbers of iteration or computation cycles, hence less time-consuming. Originality/value – A new mathematical and numerical time integration method is proposed for the computation of structural responses with higher accuracy and stability, lower data variance, and fewer numbers of iterations for computational cycles.

Dynamics of nitrogen and suspended solids removal in experimental stormwater biofilters under intermittent wetting and drying

This research study comprehensively analyses the dynamics of nitrogen and suspended solids removal in stormwater biofilters. The study focuses on pollutant removal during an event with time, rather than the conventional event-mean analysis. Antecedent dry days (number of days in between rainfall) during which biofilters remain dry and the inflow concentration of pollutants were two other important variables analysed in this study. The research outcome highlights the significance of dry-phase processes and the process of stabilization on filter performance and sets a paradigm shift from the current approach towards an innovative way of performance analysis of biofilters.

Static load bearing exercises during rehabilitation of individuals with transfemoral amputation fitted with osseointegrated implant: Kinetic analysis

The desire to solve problems caused by socket prostheses in transfemoral amputees and the acquired success of osseointegration in the dental application has led to the introduction of osseointegration in the orthopedic surgery. Since its first introduction in 1990 in Gothenburg Sweden the osseointegrated (OI) orthopedic fixation has proven several benefits[1]. The surgery consists of two surgical procedures followed by a lengthy rehabilitation program. The rehabilitation program after an OI implant includes a specific training period with a short training prosthesis. Since mechanical loading is considered to be one of the key factors that influence bone mass and the osseointegration of bone-anchored implants, the rehabilitation program will also need to include some form of load bearing exercises (LBE). To date there are two frequently used commercially available human implants. We can find proof in the literature that load bearing exercises are performed by patients with both types of OI implants. We refer to two articles, a first one written by Dr. Aschoff and all and published in 2010 in the Journal of Bone and Joint Surgery.[2] The second one presented by Hagberg et al in 2009 gives a very thorough description of the rehabilitation program of TFA fitted with an OPRA implant. The progression of the load however is determined individually according to the residual skeleton’s quality, pain level and body weight of the participant.[1] Patients are using a classical bathroom weighing scale to control the load on the implant during the course of their rehabilitation. The bathroom scale is an affordable and easy-to-use device but it has some important shortcomings. The scale provides instantaneous feedback to the patient only on the magnitude of the vertical component of the applied force. The forces and moments applied along and around the three axes of the implant are unknown. Although there are different ways to assess the load on the implant for instance through inverse dynamics in a motion analysis laboratory [3-6] this assessment is challenging. A recent proof- of-concept study by Frossard et al (2009) showed that the shortcomings of the weighing scale can be overcome by a portable kinetic system based on a commercial transducer[7].

Dynamic input to determine hip joint moments, power and work on the prosthetic limb of transfemoral amputees: ground reactions vs knee reactions

Study Design: Comparative analysis Background: Calculations of lower limbs kinetics are limited by floor-mounted force-plates. Objectives: Comparison of hip joint moments, power and mechanical work on the prosthetic limb of a transfemoral amputee calculated by inverse dynamics using either the ground reactions (force-plates) or knee reactions (transducer). Methods: Kinematics, ground reactions and knee reactions were collected using a motion analysis system, two force-plates and a multi-axial transducer mounted below the socket, respectively. Results: The inverse dynamics using ground reactions under-estimated the peaks of hip energy generation and absorption occurring at 63 % and 76 % of the gait cycle (GC) by 28 % and 54 %, respectively. This method over-estimated a phase of negative work at the hip (from 37 %GC to 56 %GC) by 24%. It under-estimated the phases of positive (from 57 %GC to 72 %GC) and negative (from 73 %GC to 98 %GC) work at the hip by 11 % and 58%, respectively. Conclusions: A transducer mounted within the prosthesis has the capacity to provide more realistic kinetics of the prosthetic limb because it enables assessment of multiple consecutive steps and a wide range of activities without issues of foot placement on force-plates. CLINICAL RELEVANCE The hip is the only joint that an amputee controls directly to set in motion the prosthesis. Hip joint kinetics are associated with joint degeneration, low back pain, risks of fall, etc. Therefore, realistic assessment of hip kinetics over multiple gait cycles and a wide range of activities is essential.

Direct measurement as a tool for assessing the accuracy of inverse dynamics

The understanding of the loads generated within the prosthetic leg can aid engineers in the design of components and clinicians in the process of rehabilitation. Traditional methods to assess these loads have relied on inverse dynamics. This indirect method estimates the applied load using video recordings and force-plates located at a distance from the region of interest, such as the base of the residuum. The well-known limitations of this method are related to the accuracy of this recursive model and the experimental conditions required (Frossard et al., 2003). Recent developments in sensors (Frossard et al., 2003) and prosthetic fixation (Brånemark et al., 2000) permit the direct measurement of the loads applied on the residuum of transfemoral amputees. In principle, direct measurement should be an appropriate tool for assessing the accuracy of inverse dynamics. The purpose of this paper is to determine the validity of this assumption. The comparative variable used in this study is the velocity of the relative body center of mass (VCOM(t)). The relativity is used to align the static (w.r.t. position) force plate measurement with the dynamic load cell measurement.

Thermal behavior of kaolinite–urea intercalation complex and molecular dynamics simulation for urea molecule orientation

The thermal behavior of kaolinite–urea intercalation complex was investigated by thermogravimetry–differential scanning calorimetry (TG–DSC), X-ray diffraction (XRD), and fourier transform infrared spectroscopy (FTIR). In addition, the interaction mode of urea molecules intercalated into the kaolinite gallery was studied by means of molecular dynamics simulation. Three main mass losses were observed at 136 °C, in the range of 210–270 °C, and at 500 °C in the TG–DSC curves, which were, respectively, attributed to (1) melting of the surface-adsorbed urea, (2) removal of the intercalated urea, and (3) dehydroxylation of the deintercalated kaolinite. The three DSC endothermic peaks at 218, 250, and 261 °C were related to the successive removals of intercalated urea with three different distribution structures. Based on the angle between the dipole moment vector of urea and the basal surface of kaolinite, the three urea models could be described as follows: (1) Type A, the dipole moment vector is nearly parallel to the basal surface of kaolinite; (2) Type B, the dipole moment vector points to the silica tetrahedron with the angle between it and the basal surface of kaolinite ranging from 20°to 40°; and (3) Type C, the dipole moment vector is nearly perpendicular to the basal surface of kaolinite. The three distribution structures of urea molecules were validated by the results of the molecular dynamics simulation. Furthermore, the thermal behavior of the kaolinite–urea intercalation complex investigated by TG–DSC was also supported by FTIR and XRD analyses.

Industry-school partnerships: An ecological case study to understand operational dynamics

This thesis is an ecological systems case study of an industry-school partnership. It examines a minerals and energy sector partnership with Queensland schools and explains the operational dynamics. In doing so, an original contribution to theory and practice was presented, together with implications for the impact of industry on education.

Numerical investigation of motion and deformation of a single red blood cell in a stenosed capillary

It is generally assumed that influence of the red blood cells (RBCs) is predominant in blood rheology. The healthy RBCs are highly deformable and can thus easily squeeze through the smallest capillaries having internal diameter less than their characteristic size. On the other hand, RBCs infected by malaria or other diseases are stiffer and so less deformable. Thus it is harder for them to flow through the smallest capillaries. Therefore, it is very important to critically and realistically investigate the mechanical behavior of both healthy and infected RBCs which is a current gap in knowledge. The motion and the steady state deformed shape of the RBCs depend on many factors, such as the geometrical parameters of the capillary through which blood flows, the membrane bending stiffness and the mean velocity of the blood flow. In this study, motion and deformation of a single two-dimensional RBC in a stenosed capillary is explored by using smoothed particle hydrodynamics (SPH) method. An elastic spring network is used to model the RBC membrane, while the RBC's inside fluid and outside fluid are treated as SPH particles. The effect of RBC's membrane stiffness (kb), inlet pressure (P) and geometrical parameters of the capillary on the motion and deformation of the RBC is studied. The deformation index, RBC's mean velocity and the cell membrane energy are analyzed when the cell passes through the stenosed capillary. The simulation results demonstrate that the kb, P and the geometrical parameters of the capillary have a significant impact on the RBCs' motion and deformation in the stenosed section.

Direct measurement of prosthetic loading for evidence-based practice

The demand for an evidence-based clinical practice involving lower limb amputees is increasing. Some of the critical care decisions are related to the loading applied on the residuum partially responsible for comfort and functional outcome. This loading can be assessed using inverse dynamics equations. Typically, this method requires a gait laboratory (e.g., 3D motion analysis system, force-plates). It is mainly suited for the analysis only few steps of walking while being expensive and labour intensive. However, recent scientific and industrial developments demonstrated that discrete and light portable sensors can be placed within the prosthesis to measure accurately the loading during an unlimited number of steps and activities of daily living. Several studies indicated that method based on direct measurements might provide more realistic results. Furthermore, it is a user-friendly method more accessible to clinicians, such as prosthetists. The purpose of this symposium will be to give an overview of these additional opportunities for clinicians to obtain relevant data for evidence-based practice. The three main aims will be: • To present some of the equipment used for direct measurements, • To propose ways to analyse some key data sets, • To give some practical example of data sets for transtibial and transfemoral amputees.

Predictive motion planning for AUVs subject to strong time-varying currents and forecasting uncertainties

This paper presents a novel path planning method for minimizing the energy consumption of an autonomous underwater vehicle subjected to time varying ocean disturbances and forecast model uncertainty. The algorithm determines 4-Dimensional path candidates using Nonlinear Robust Model Predictive Control (NRMPC) and solutions optimised using A*-like algorithms. Vehicle performance limits are incorporated into the algorithm with disturbances represented as spatial and temporally varying ocean currents with a bounded uncertainty in their predictions. The proposed algorithm is demonstrated through simulations using a 4-Dimensional, spatially distributed time-series predictive ocean current model. Results show the combined NRMPC and A* approach is capable of generating energy-efficient paths which are resistant to both dynamic disturbances and ocean model uncertainty.

Simulating droplet motion on virtual leaf surfaces

A curvilinear thin film model is used to simulate the motion of droplets on a virtual leaf surface, with a view to better understand the retention of agricultural sprays on plants. The governing model, adapted from Roy et al. (2002 J. Fluid Mech. 454, 235–261) with the addition of a disjoining pressure term, describes the gravity- and curvature driven flow of a small droplet on a complex substrate: a cotton leaf reconstructed from digitized scan data. Coalescence is the key mechanism behind spray coating of foliage, and our simulations demonstrate that various experimentally observed coalescence behaviours can be reproduced qualitatively. By varying the contact angle over the domain, we also demonstrate that the presence of a chemical defect can act as an obstacle to the droplet’s path, causing break-up. In simulations on the virtual leaf, it is found that the movement of a typical spray size droplet is driven almost exclusively by substrate curvature gradients. It is not until droplet mass is sufficiently increased via coalescence that gravity becomes the dominating force.

Microtubulin binding sites as target for developing anticancer agents

Microtubules (MTs) play important and diverse roles in eukaryotic cells. Their function and biophysical properties have made α−and β−tubulin, the main components of MTs, the subject of intense study. Interfering with normal MT dynamics, for example, by the addition of tubulin ligands, can cause the cell great distress and affect MT stability and functions, including mitosis, cell motion and intracellular organelle transport. It has been shown in the literature that tubulin is an important target molecule for developing anticancer drugs. Tubulin binding molecules have generated considerable interest after the successful introduction of the taxanes into clinical oncology and the widespread use of the vinca alkaloids vincristine and vinblastine. These compounds inhibit cell mitosis by binding to the protein tubulin in the mitotic spindle and preventing polymerization into the MTs. This mode of action is also shared with other natural agents eg colchicine and podophyllotoxin. However various tubulin isotypes have shown resistance to taxanes and other MT agents. Therefore, there is a strong need to design and develop new natural analogs as antimitotic agents to interact with tubulin at sites different from those of vinca alkaloids and taxanes. This minireview provides SAR on several classes of antimitotic agents reported in the literature. The structures and data given are essential to the scientists who are involved in drug design and development in the field of anticancer drugs.

Using culture to manage the transition into university: Conceptualising the dynamics of withdrawal and engagement

This conceptual paper explored the purposes of using culture in the process of coping with stress by looking how first year undergraduate students used cultural elements and activities to aid their transition into university. Results supported two key conceptualisations of the use of culture. Firstly, results indicated that students used culture either for withdrawal purposes, i.e., for escaping from the stressful situation, or for engagement purposes, i.e., for actively engaging with the stressful situation. Secondly, the results suggested three different forms of using culture to engage with stressful situations: mood management, learning, and personal interaction. While the results of the study resonate with the distinction between avoidance versus approach-oriented coping strategies that are widely explored in the stress and coping literature, they also suggest that the relationship between withdrawal and engagement might be dynamic with those two strategies serving distinct purposes in the process of coping with stress. The paper thus suggests that there is a need to develop process-oriented models of coping that would allow identifying patterns in the way people fluctuate between withdrawal and engagement that support and facilitate their personal growth and development.