Curiosity driven reinforcement learning for motion planning on humanoids

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

Remembering the past through picture books

In anticipation of the commemorations around the centenary of World War 1 (2014-2018) this chapter examines the ways in which war and its effects have been represented in picture books for children. It looks at the ways in which these picture books create “textual monuments” as points of reference through which younger generations can “develop a narrative of the past” and “explore different points of view”. The focus of the discussion centres on a number of recent picture books and their application to aspects of the F-2 Australian History curriculum.

ATAR scores only part of the picture for teaching

A common debate has resurfaced over teacher quality and the quality of teacher education in Australia. This time it was started by a leaked draft report into teacher education from the Australian Institute of Teaching and School Leadership (AITSL) which the public can’t expect to see for at least another month. Media reports have all focused on one aspect of the wide-ranging report: teaching students' ATARs.

Numerical investigation of motion and deformation of a single red blood cell in a stenosed capillary

It is generally assumed that influence of the red blood cells (RBCs) is predominant in blood rheology. The healthy RBCs are highly deformable and can thus easily squeeze through the smallest capillaries having internal diameter less than their characteristic size. On the other hand, RBCs infected by malaria or other diseases are stiffer and so less deformable. Thus it is harder for them to flow through the smallest capillaries. Therefore, it is very important to critically and realistically investigate the mechanical behavior of both healthy and infected RBCs which is a current gap in knowledge. The motion and the steady state deformed shape of the RBCs depend on many factors, such as the geometrical parameters of the capillary through which blood flows, the membrane bending stiffness and the mean velocity of the blood flow. In this study, motion and deformation of a single two-dimensional RBC in a stenosed capillary is explored by using smoothed particle hydrodynamics (SPH) method. An elastic spring network is used to model the RBC membrane, while the RBC's inside fluid and outside fluid are treated as SPH particles. The effect of RBC's membrane stiffness (kb), inlet pressure (P) and geometrical parameters of the capillary on the motion and deformation of the RBC is studied. The deformation index, RBC's mean velocity and the cell membrane energy are analyzed when the cell passes through the stenosed capillary. The simulation results demonstrate that the kb, P and the geometrical parameters of the capillary have a significant impact on the RBCs' motion and deformation in the stenosed section.

Encouraging Empathy through Picture Books about Migration

Australia has become one of the most highly multilingual and multicultural societies in the world today with people descending from 270 ancestries, who speak more than 260 languages (Commonwealth of Australia, 2011). Immigration is something that children encounter in their daily lives either through personal experience or through witnessing the lives of migrants at school, in the community, or through popular media, including children’s literature. Schools are frequently the initial interface for individuals who resettle in Australia and they ‘play a significant role in establishing meaningful connections to Australian society and a sense of belonging in Australia’ (Uptin, Wright, & Harwood, 2013, p. 1). Children's literature about cultural and ethnic diversity explores the impacts of migration and related issues creating ‘imaginary realms’ (Dudek & Ommundsen, 2007). These fictional interpretations of the migrant experience or the experience of migration are supported by distinctive “real life” cultural experiences. Picture books furnish teachers and students with an accessible means to investigate these complex issues through sensitive discussions. This chapter investigates how picture books about migration help deepen children’s perceptive understanding of migrants’ plights, and thereby nurture tolerance and empathy.

Predictive motion planning for AUVs subject to strong time-varying currents and forecasting uncertainties

This paper presents a novel path planning method for minimizing the energy consumption of an autonomous underwater vehicle subjected to time varying ocean disturbances and forecast model uncertainty. The algorithm determines 4-Dimensional path candidates using Nonlinear Robust Model Predictive Control (NRMPC) and solutions optimised using A*-like algorithms. Vehicle performance limits are incorporated into the algorithm with disturbances represented as spatial and temporally varying ocean currents with a bounded uncertainty in their predictions. The proposed algorithm is demonstrated through simulations using a 4-Dimensional, spatially distributed time-series predictive ocean current model. Results show the combined NRMPC and A* approach is capable of generating energy-efficient paths which are resistant to both dynamic disturbances and ocean model uncertainty.

Simulating droplet motion on virtual leaf surfaces

A curvilinear thin film model is used to simulate the motion of droplets on a virtual leaf surface, with a view to better understand the retention of agricultural sprays on plants. The governing model, adapted from Roy et al. (2002 J. Fluid Mech. 454, 235–261) with the addition of a disjoining pressure term, describes the gravity- and curvature driven flow of a small droplet on a complex substrate: a cotton leaf reconstructed from digitized scan data. Coalescence is the key mechanism behind spray coating of foliage, and our simulations demonstrate that various experimentally observed coalescence behaviours can be reproduced qualitatively. By varying the contact angle over the domain, we also demonstrate that the presence of a chemical defect can act as an obstacle to the droplet’s path, causing break-up. In simulations on the virtual leaf, it is found that the movement of a typical spray size droplet is driven almost exclusively by substrate curvature gradients. It is not until droplet mass is sufficiently increased via coalescence that gravity becomes the dominating force.

Made by motion: A conceptual framework for abstracted animation derived from motion-captured movements

Despite an increasing number of acclaimed abstract animations being created through the application of motion capture technologies there has been little detailed documentation and analysis of this approach for abstract animation production. More specifically, it is unclear what the key considerations are, and what issues practitioners might face, when integrating motion capture movement data into their practice. In response to this issue this study explored and documented the practice of generating abstract visual and temporal artefacts from motion captured dance movements that compose abstract animated short films. The study has resulted in a possible framework for this form of practice and outlines five key considerations which should be taken into account by practitioners who use motion capture in the production of abstract animated short films.

Development of a new ultrasound-based system for tracking motion of the human lumbar spine: Reliability, stability and repeatability during forward bending movement trials

The aim of this study was to develop a new method for quantifying intersegmental motion of the spine in an instrumented motion segment L4–L5 model using ultrasound image post-processing combined with an electromagnetic device. A prospective test–retest design was employed, combined with an evaluation of stability and within- and between-day intra-tester reliability during forward bending by 15 healthy male patients. The accuracy of the measurement system using the model was calculated to be ± 0.9° (standard deviation = 0.43) over a 40° range and ± 0.4 cm (standard deviation = 0.28) over 1.5 cm. The mean composite range of forward bending was 15.5 ± 2.04° during a single trial (standard error of the mean = 0.54, coefficient of variation = 4.18). Reliability (intra-class correlation coefficient = 2.1) was found to be excellent for both within-day measures (0.995–0.999) and between-day measures (0.996–0.999). Further work is necessary to explore the use of this approach in the evaluation of biomechanics, clinical assessments and interventions.

Heritability of head motion during resting state functional MRI in 462 healthy twins

Head motion (HM) is a critical confounding factor in functional MRI. Here we investigate whether HM during resting state functional MRI (RS-fMRI) is influenced by genetic factors in a sample of 462 twins (65% fema≤ 101 MZ (monozygotic) and 130 DZ (dizygotic) twin pairs; mean age: 21 (SD=3.16), range 16-29). Heritability estimates for three HM components-mean translation (MT), maximum translation (MAXT) and mean rotation (MR)-ranged from 37 to 51%. We detected a significant common genetic influence on HM variability, with about two-thirds (genetic correlations range 0.76-1.00) of the variance shared between MR, MT and MAXT. A composite metric (HM-PC1), which aggregated these three, was also moderately heritable (h2=42%). Using a sub-sample (N=35) of the twins we confirmed that mean and maximum translational and rotational motions were consistent "traits" over repeated scans (r=0.53-0.59); reliability was even higher for the composite metric (r=0.66). In addition, phenotypic and cross-trait cross-twin correlations between HM and resting state functional connectivities (RS-FCs) with Brodmann areas (BA) 44 and 45, in which RS-FCs were found to be moderately heritable (BA44: h2-=0.23 (sd=0.041), BA45: h2-=0.26 (sd=0.061)), indicated that HM might not represent a major bias in genetic studies using FCs. Even so, the HM effect on FC was not completely eliminated after regression. HM may be a valuable endophenotype whose relationship with brain disorders remains to be elucidated.

Top-down influences on lexical selection during spoken word production: A 4T fMRI investigation of refractory effects in picture naming

Spoken word production is assumed to involve stages of processing in which activation spreads through layers of units comprising lexical-conceptual knowledge and their corresponding phonological word forms. Using high-field (4T) functional magnetic resonance imagine (fMRI), we assessed whether the relationship between these stages is strictly serial or involves cascaded-interactive processing, and whether central (decision/control) processing mechanisms are involved in lexical selection. Participants performed the competitor priming paradigm in which distractor words, named from a definition and semantically related to a subsequently presented target picture, slow picture-naming latency compared to that with unrelated words. The paradigm intersperses two trials between the definition and the picture to be named, temporally separating activation in the word perception and production networks. Priming semantic competitors of target picture names significantly increased activation in the left posterior temporal cortex, and to a lesser extent the left middle temporal cortex, consistent with the predictions of cascaded-interactive models of lexical access. In addition, extensive activation was detected in the anterior cingulate and pars orbitalis of the inferior frontal gyrus. The findings indicate that lexical selection during competitor priming is biased by top-down mechanisms to reverse associations between primed distractor words and target pictures to select words that meet the current goal of speech.

Auditory context effects in picture naming investigated with event-related fMRI

Naming an object entails a number of processing stages, including retrieval of a target lexical concept and encoding of its phonological word form. We investigated these stages using the picture-word interference task in an fMRI experiment. Participants named target pictures in the presence of auditorily presented semantically related, phonologically related, or unrelated distractor words or in isolation. We observed BOLD signal changes in left-hemisphere regions associated with lexical-conceptual and phonological processing, including the midto-posterior lateral temporal cortex. However, these BOLD responses manifested as signal reductions for all distractor conditions relative to naming alone. Compared with unrelated words, phonologically related distractors showed further signal reductions, whereas only the pars orbitalis of the left inferior frontal cortex showed a selective reduction in response in the semantic condition. We interpret these findings as indicating that the word forms of lexical competitors are phonologically encoded and that competition during lexical selection is reduced by phonologically related distractors. Since the extended nature of auditory presentation requires a large portion of a word to be presented before its meaning is accessed, we attribute the BOLD signal reductions observed for semantically related and unrelated words to lateral inhibition mechanisms engaged after target name selection has occurred, as has been proposed in some production models.

Orthographic/phonological facilitation of naming responses in the picture-word task: An event-related fMRI study using overt vocal responding

In the picture-word interference task, naming responses are facilitated when a distractor word is orthographically and phonologically related to the depicted object as compared to an unrelated word. We used event-related functional magnetic resonance imaging (fMRI) to investigate the cerebral hemodynamic responses associated with this priming effect. Serial (or independent-stage) and interactive models of word production that explicitly account for picture-word interference effects assume that the locus of the effect is at the level of retrieving phonological codes, a role attributed recently to the left posterior superior temporal cortex (Wernicke's area). This assumption was tested by randomly presenting participants with trials from orthographically related and unrelated distractor conditions and acquiring image volumes coincident with the estimated peak hemodynamic response for each trial. Overt naming responses occurred in the absence of scanner noise, allowing reaction time data to be recorded. Analysis of this data confirmed the priming effect. Analysis of the fMRI data revealed blood oxygen level-dependent signal decreases in Wernicke's area and the right anterior temporal cortex, whereas signal increases were observed in the anterior cingulate, the right orbitomedial prefrontal, somatosensory, and inferior parietal cortices, and the occipital lobe. The results are interpreted as supporting the locus for the facilitation effect as assumed by both classes of theoretical model of word production. In addition, our results raise the possibilities that, counterintuitively, picture-word interference might be increased by the presentation of orthographically related distractors, due to competition introduced by activation of phonologically related word forms, and that this competition requires inhibitory processes to be resolved. The priming effect is therefore viewed as being sufficient to offset the increased interference. We conclude that information from functional imaging studies might be useful for constraining theoretical models of word production.

Independent distractor frequency and age-of-acquisition dffects in picture-word interference: FMRI evidence for post-lexical and lexical accounts according to distractor type

In two fMRI experiments, participants named pictures with superimposed distractors that were high or low in frequency or varied in terms of age of acquisition. Pictures superimposed with low-frequency words were named more slowly than those superimposed with high-frequency words, and late-acquired words interfered with picture naming to a greater extent than early-acquired words. The distractor frequency effect (Experiment 1) was associated with increased activity in left premotor and posterior superior temporal cortices, consistent with the operation of an articulatory response buffer and verbal selfmonitoring system. Conversely, the distractor age-of-acquisition effect (Experiment 2) was associated with increased activity in the left middle and posterior middle temporal cortex, consistent with the operation of lexical level processes such as lemma and phonological word form retrieval. The spatially dissociated patterns of activity across the two experiments indicate that distractor effects in picture-word interference may occur at lexical or postlexical levels of processing in speech production.

The semantic interference effect in the picture-word paradigm: An event-related fMRI study employing overt responses

We used event-related functional magnetic resonance imaging (fMRI) to investigate neural responses associated with the semantic interference (SI) effect in the picture-word task. Independent stage models of word production assume that the locus of the SI effect is at the conceptual processing level (Levelt et al. [1999]: Behav Brain Sci 22:1-75), whereas interactive models postulate that it occurs at phonological retrieval (Starreveld and La Heij [1996]: J Exp Psychol Learn Mem Cogn 22:896-918). In both types of model resolution of the SI effect occurs as a result of competitive, spreading activation without the involvement of inhibitory links. These assumptions were tested by randomly presenting participants with trials from semantically-related and lexical control distractor conditions and acquiring image volumes coincident with the estimated peak hemodynamic response for each trial. Overt vocalization of picture names occurred in the absence of scanner noise, allowing reaction time (RT) data to be collected. Analysis of the RT data confirmed the SI effect. Regions showing differential hemodynamic responses during the SI effect included the left mid section of the middle temporal gyrus, left posterior superior temporal gyrus, left anterior cingulate cortex, and bilateral orbitomedial prefrontal cortex. Additional responses were observed in the frontal eye fields, left inferior parietal lobule, and right anterior temporal and occipital cortex. The results are interpreted as indirectly supporting interactive models that allow spreading activation between both conceptual processing and phonological retrieval levels of word production. In addition, the data confirm that selective attention/response suppression has a role in resolving the SI effect similar to the way in which Stroop interference is resolved. We conclude that neuroimaging studies can provide information about the neuroanatomical organization of the lexical system that may prove useful for constraining theoretical models of word production.