Curiosity driven reinforcement learning for motion planning on humanoids

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

In silico biology : making the most of parallel computing

Biological systems are typically complex and adaptive, involving large numbers of entities, or organisms, and many-layered interactions between these. System behaviour evolves over time, and typically benefits from previous experience by retaining memory of previous events. Given the dynamic nature of these phenomena, it is non-trivial to provide a comprehensive description of complex adaptive systems and, in particular, to define the importance and contribution of low-level unsupervised interactions to the overall evolution process. In this chapter, the authors focus on the application of the agent-based paradigm in the context of the immune response to HIV. Explicit implementation of lymph nodes and the associated lymph network, including lymphatic chain structure, is a key objective, and requires parallelisation of the model. Steps taken towards an optimal communication strategy are detailed.

Template-based circuit understanding

When verifying or reverse-engineering digital circuits, one often wants to identify and understand small components in a larger system. A possible approach is to show that the sub-circuit under investigation is functionally equivalent to a reference implementation. In many cases, this task is difficult as one may not have full information about the mapping between input and output of the two circuits, or because the equivalence depends on settings of control inputs. We propose a template-based approach that automates this process. It extracts a functional description for a low-level combinational circuit by showing it to be equivalent to a reference implementation, while synthesizing an appropriate mapping of input and output signals and setting of control signals. The method relies on solving an exists/forall problem using an SMT solver, and on a pruning technique based on signature computation.

Development of a cost-effective and flexible vibration DAQ system for long-term continuous structural health monitoring

In the structural health monitoring (SHM) field, long-term continuous vibration-based monitoring is becoming increasingly popular as this could keep track of the health status of structures during their service lives. However, implementing such a system is not always feasible due to on-going conflicts between budget constraints and the need of sophisticated systems to monitor real-world structures under their demanding in-service conditions. To address this problem, this paper presents a comprehensive development of a cost-effective and flexible vibration DAQ system for long-term continuous SHM of a newly constructed institutional complex with a special focus on the main building. First, selections of sensor type and sensor positions are scrutinized to overcome adversities such as low-frequency and low-level vibration measurements. In order to economically tackle the sparse measurement problem, a cost-optimized Ethernet-based peripheral DAQ model is first adopted to form the system skeleton. A combination of a high-resolution timing coordination method based on the TCP/IP command communication medium and a periodic system resynchronization strategy is then proposed to synchronize data from multiple distributed DAQ units. The results of both experimental evaluations and experimental–numerical verifications show that the proposed DAQ system in general and the data synchronization solution in particular work well and they can provide a promising cost-effective and flexible alternative for use in real-world SHM projects. Finally, the paper demonstrates simple but effective ways to make use of the developed monitoring system for long-term continuous structural health evaluation as well as to use the instrumented building herein as a multi-purpose benchmark structure for studying not only practical SHM problems but also synchronization related issues.

Towards robust automatic affective classification of images using facial expressions for practical applications

Affect is an important feature of multimedia content and conveys valuable information for multimedia indexing and retrieval. Most existing studies for affective content analysis are limited to low-level features or mid-level representations, and are generally criticized for their incapacity to address the gap between low-level features and high-level human affective perception. The facial expressions of subjects in images carry important semantic information that can substantially influence human affective perception, but have been seldom investigated for affective classification of facial images towards practical applications. This paper presents an automatic image emotion detector (IED) for affective classification of practical (or non-laboratory) data using facial expressions, where a lot of “real-world” challenges are present, including pose, illumination, and size variations etc. The proposed method is novel, with its framework designed specifically to overcome these challenges using multi-view versions of face and fiducial point detectors, and a combination of point-based texture and geometry. Performance comparisons of several key parameters of relevant algorithms are conducted to explore the optimum parameters for high accuracy and fast computation speed. A comprehensive set of experiments with existing and new datasets, shows that the method is effective despite pose variations, fast, and appropriate for large-scale data, and as accurate as the method with state-of-the-art performance on laboratory-based data. The proposed method was also applied to affective classification of images from the British Broadcast Corporation (BBC) in a task typical for a practical application providing some valuable insights.

No acute and persistent DNA damage after an Ironman triathlon

During acute and strenuous exercise, the enhanced formation of reactive oxygen species can induce damage to lipids, proteins, and nucleic acids. The aim of this study was to investigate the effect of an Ironman triathlon (3.8 km swim, 180 km cycle, 42 km run), as a prototype of ultra-endurance exercise, on DNA stability. As biomarkers of genomic instability, the number of micronuclei, nucleoplasmic bridges, and nuclear buds were measured within the cytokinesis-block micronucleus cytome assay in once-divided peripheral lymphocytes of 20 male triathletes. Blood samples were taken 2 days before, within 20 min after the race, and 5 and 19 days post-race. Overall, the number of micronuclei decreased (P < 0.05) after the race, remained at a low level until 5 days post-race, and declined further to 19 days post-race (P < 0.01). The frequency of nucleoplasmic bridges and nuclear buds did not change immediately after the triathlon. The number of nucleoplasmic bridge declined from 2 days pre-race to 19 days post-exercise (P < 0.05). The frequency of nuclear buds increased after the triathlon, peaking 5 days post-race (P < 0.01) and decreased to basic levels 19 days after the race (P < 0.01). The results suggest that an Ironman triathlon does not cause long-lasting DNA damage in well-trained athletes.

The presence of executive deficits in patients with Obstructive Sleep Apnoea

Obstructive sleep apnoea (OSA) is a chronic condition in which the upper airways collapse repeatedly during sleep, completely or partially obstructing breathing. This obstruction leads to chronic intermittent hypoxia and severe sleep fragmentation, disrupting the restorative functions of sleep. Beebe and Gozal (2002)a developed a theory which hypothesises that disruption of the restorative functions of sleep lead to a chronic low level brain damage most evident in executive functions (EF). Neuropsychological testing of EF, volumetric MRI, magnetic resonance spectroscopy, event related potentials and CSF biomarkers all provide support for this theory. Little research has been done to explore the nature of the subjective complaint and it’s impact on the activities of daily living.

The Arctic Summer Cloud Ocean Study (ASCOS): overview and experimental design

The climate in the Arctic is changing faster than anywhere else on earth. Poorly understood feedback processes relating to Arctic clouds and aerosol–cloud interactions contribute to a poor understanding of the present changes in the Arctic climate system, and also to a large spread in projections of future climate in the Arctic. The problem is exacerbated by the paucity of research-quality observations in the central Arctic. Improved formulations in climate models require such observations, which can only come from measurements in situ in this difficult-to-reach region with logistically demanding environmental conditions. The Arctic Summer Cloud Ocean Study (ASCOS) was the most extensive central Arctic Ocean expedition with an atmospheric focus during the International Polar Year (IPY) 2007–2008. ASCOS focused on the study of the formation and life cycle of low-level Arctic clouds. ASCOS departed from Longyearbyen on Svalbard on 2 August and returned on 9 September 2008. In transit into and out of the pack ice, four short research stations were undertaken in the Fram Strait: two in open water and two in the marginal ice zone. After traversing the pack ice northward, an ice camp was set up on 12 August at 87°21' N, 01°29' W and remained in operation through 1 September, drifting with the ice. During this time, extensive measurements were taken of atmospheric gas and particle chemistry and physics, mesoscale and boundary-layer meteorology, marine biology and chemistry, and upper ocean physics. ASCOS provides a unique interdisciplinary data set for development and testing of new hypotheses on cloud processes, their interactions with the sea ice and ocean and associated physical, chemical, and biological processes and interactions. For example, the first-ever quantitative observation of bubbles in Arctic leads, combined with the unique discovery of marine organic material, polymer gels with an origin in the ocean, inside cloud droplets suggests the possibility of primary marine organically derived cloud condensation nuclei in Arctic stratocumulus clouds. Direct observations of surface fluxes of aerosols could, however, not explain observed variability in aerosol concentrations, and the balance between local and remote aerosols sources remains open. Lack of cloud condensation nuclei (CCN) was at times a controlling factor in low-level cloud formation, and hence for the impact of clouds on the surface energy budget. ASCOS provided detailed measurements of the surface energy balance from late summer melt into the initial autumn freeze-up, and documented the effects of clouds and storms on the surface energy balance during this transition. In addition to such process-level studies, the unique, independent ASCOS data set can and is being used for validation of satellite retrievals, operational models, and reanalysis data sets.

Psychological distress and pain reporting in Australian coal miners

BACKGROUND: Coal mining is of significant economic importance to the Australian economy. Despite this fact, the related workforce is subjected to a number of psychosocial risks and musculoskeletal injury, and various psychological disorders are common among this population group. Because only limited research has been conducted in this population group, we sought to examine the relationship between physical (pain) and psychological (distress) factors, as well as the effects of various demographic, lifestyle, and fatigue indicators on this relationship. METHODS: Coal miners (N = 231) participated in a survey of musculoskeletal pain and distress on-site during their work shifts. Participants also provided demographic information (job type, age, experience in the industry, and body mass index) and responded to questions about exercise and sleep quality (on- and off-shift) as well as physical and mental tiredness after work. RESULTS: A total of 177 workers (80.5%) reported experiencing pain in at least one region of their body. The majority of the sample population (61.9%) was classified as having low-level distress, 28.4% had scores indicating mild to moderate distress, and 9.6% had scores indicating high levels of distress. Both number of pain regions and job type (being an operator) significantly predicted distress. Higher distress score was also associated with greater absenteeism in workers who reported lower back pain. In addition, perceived sleep quality during work periods partially mediated the relationship between pain and distress. CONCLUSION: The study findings support the existence of widespread musculoskeletal pain among the coal-mining workforce, and this pain is associated with increased psychological distress. Operators (truck drivers) and workers reporting poor sleep quality during work periods are most likely to report increased distress, which highlights the importance of supporting the mining workforce for sustained productivity.

5-year Chlamydia vaccination programme could reverse disease-related koala population decline: Predictions from a mathematical model using field data

BACKGROUND Many koala populations around Australia are in serious decline, with a substantial component of this decline in some Southeast Queensland populations attributed to the impact of Chlamydia. A Chlamydia vaccine for koalas is in development and has shown promise in early trials. This study contributes to implementation preparedness by simulating vaccination strategies designed to reverse population decline and by identifying which age and sex category it would be most effective to target. METHODS We used field data to inform the development and parameterisation of an individual-based stochastic simulation model of a koala population endemic with Chlamydia. The model took into account transmission, morbidity and mortality caused by Chlamydia infections. We calibrated the model to characteristics of typical Southeast Queensland koala populations. As there is uncertainty about the effectiveness of the vaccine in real-world settings, a variety of potential vaccine efficacies, half-lives and dosing schedules were simulated. RESULTS Assuming other threats remain constant, it is expected that current population declines could be reversed in around 5-6 years if female koalas aged 1-2 years are targeted, average vaccine protective efficacy is 75%, and vaccine coverage is around 10% per year. At lower vaccine efficacies the immunological effects of boosting become important: at 45% vaccine efficacy population decline is predicted to reverse in 6 years under optimistic boosting assumptions but in 9 years under pessimistic boosting assumptions. Terminating a successful vaccination programme at 5 years would lead to a rise in Chlamydia prevalence towards pre-vaccination levels. CONCLUSION For a range of vaccine efficacy levels it is projected that population decline due to endemic Chlamydia can be reversed under realistic dosing schedules, potentially in just 5 years. However, a vaccination programme might need to continue indefinitely in order to maintain Chlamydia prevalence at a sufficiently low level for population growth to continue.

Examining novice education: What can we learn from a compulsory program delivered to both mature-age and young pre-learner licence drivers?

This paper reports results from a qualitative evaluation of a compulsory pre-Learner driver education program within the Australian Capital Territory(ACT), Australia. Two methods were used to obtain feedback from those involved in the delivery of the program as well as those who participated in programs. The first, semi-structured interviews, was undertaken with class room teachers who run the program in their schools, group facilitators running the program with more mature-age students at private facilities (n = 15 in total), and former participants in both school-based and private-based versions of the program (n = 19). The second method used an on-line survey for students (n = 79). Results from both methods were consistent with each other, indicating that strengths of the program were perceived as being its interactive components and the high level of engagement of the target audience. There was strong support from young and mature-age students for the program to remain compulsory. However, consistent with other findings on novice driver education, mature-age participants identified that the program was less relevant to them. It may be that to have greater relevance to mature-age learners, content could address and challenge perceptions about behaviours other than intentional high-risk behaviours (e.g. low level speeding, fatigue) as well as encourage planning/strategies to avoid them. While a longer term, outcome focussed, evaluation of the pre-learner education program is needed, this study suggests that the program is well received by pre-licence drivers and that teachers and facilitators perceive it as both effective and beneficial.

Latency, Power and Performance Trade-offs in Network-on-Chips by Link Microarchitecture Exploration

This paper presents a power, latency and throughput trade-off study on NoCs by varying microarchitectural (e.g. pipelining) and circuit level (e.g. frequency and voltage) parameters. We change pipelining depth, operating frequency and supply voltage for 3 example NoCs - 16 node 2D Torus, Tree network and Reduced 2D Torus. We use an in-house NoC exploration framework capable of topology generation and comparison using parameterized models of Routers and links developed in SystemC. The framework utilizes interconnect power and delay models from a low-level modelling tool called Intacte[1]1. We find that increased pipelining can actually reduce latency. We also find that there exists an optimal degree of pipelining which is the most energy efficient in terms of minimizing energy-delay product.

A spatially explicit multi-competitor coexistence model of penaeid (shrimp) distribution on the Australian Great Barrier Reef

A spatially explicit multi-competitor coexistence model was developed for meta-populations of prawns (shrimp) occupying habitat patches across the Great Barrier Reef, where dispersal was localised and dispersal rates varied between species. Prawns were modelled as individuals moving to and from patches or cells according to pre-set decision rules. The landscape was simulated as a matrix of cells with each cell having a spatially explicit survival index for each species. Mixed species prawn assemblages moved over this simplified spatially explicit landscape. A low level of chronic random environmental disturbance was assumed (cyclone and tropical storm damage) with additional acute spatially confined disturbance due to commercial trawling, modelled as an increase in mortality affecting inter-specific competition. The general form of the results was for increased disturbance to favour good-colonising "generalist" species at the expense of good-competitor "specialists". Increasing fishing mortality (local patch extinctions) combined with poor colonising ability resulted in low equilibrium abundance for even the best competitor, while in the same circumstances the poorest competitor but best coloniser could have the highest equilibrium abundance. This mimics the switch from high-value prawn species to lower-value prawn species as trawl effort increases, reflected in historic catch and effort logbook data and reported anecdotaly from the north Queensland trawl fleet. To match the observed distribution and behaviour of prawn assemblages, a combination inter-species competition, a spatially explicit landscape, and a defined pattern of disturbance (trawling) was required. Modelling this combination could simulate not only general trends in spatial distribution of each of prawn species but also localised concentrations observed in the survey data

Microbiological status of piggery effluent from 13 piggeries in the south east Queensland region of Australia

Aims: To assist in the development of safe piggery effluent re-use guidelines by determining the level of selected pathogens and indicator organisms in the effluent ponds of 13 south-east Queensland piggeries. Methods and Results: The numbers of thermotolerant coliforms, Campylobacter jejuni/coli, Erysipelothrix rhusiopathiae, Escherichia coli, Salmonella and rotavirus were determined in 29 samples derived from the 13 piggeries. The study demonstrated that the 13 final effluent ponds contained an average of 1Æ2 · 105 colony-forming units (CFU) 100 ml)1 of thermotolerant coliforms and 1Æ03 · 105 CFU 100 ml)1 of E. coli. The Campylobacter level varied from none detectable (two of 13 piggeries) to a maximum of 930 most probable number (MPN) 100 ml)1 (two of 13 piggeries). Salmonella was detected in the final ponds of only four of the 13 piggeries and then only at a low level (highest level being 51 MPN 100 ml)1). No rotavirus and no Erysip. rhusiopathiae were detected. The average log10 reductions across the ponding systems to the final irrigation pond were 1Æ77 for thermotolerant coliforms, 1Æ71 for E. coli and 1Æ04 for Campylobacter. Conclusions: This study has provided a baseline knowledge on the levels of indicator organisms and selected pathogens in piggery effluent. Significance and Impact of the Study: The knowledge gained in this study will assist in the development of guidelines to ensure the safe and sustainable re-use of piggery effluent.

Relationship Between Hardness Genes and Quality in Barley (Hordeum vulgare)

Barley (Hordeum vulgare) genotypes were sequenced for polymorphism in the hardness genes, these being the three hordoindoline (hin a, hin b1 and hin b2) genes. The variation in haplotype was determined by sequencing for single nucleotide polymorphisms (SNPs). Polymorphism between each gene was then compared to grain hardness (three methods), malt quality characteristics (hot water extract and friability) and cattle feed quality. Two haplotypes were found in a set of forty barley genotypes. For hin a, two alleles were present, namely hin a1 and hin a2. However, there was no specific hin a allele that was associated with grain hardness, malt and feed quality. Barley has two hin b genes, namely hin b1 and hin b2, and the genotypes tested here had one of two alleles for each gene. However, there were no obvious effects on hardness or quality from either of these hin b alleles. Unlike wheat, where a clear relationship has been demonstrated between a number of SNPs in the wheat hardness genes and quality (soft or hard wheat), there was no such relationship for barley. Despite the wide range in hardness, malt and feed quality, there were only two haplotypes for each of the hin a, hin b1 and hin b2 genes and there was no clear relationship between grain hardness, malt or feed quality. The genotypes used in this study demonstrated that there was a low level of polymorphism in hardness genes in current commercial varieties as well as breeding lines and these polymorphisms had no impact on quality.