From vision to actions: Towards adaptive and autonomous humanoid robots

Although robotics research has seen advances over the last decades robots are still not in widespread use outside industrial applications. Yet a range of proposed scenarios have robots working together, helping and coexisting with humans in daily life. In all these a clear need to deal with a more unstructured, changing environment arises. I herein present a system that aims to overcome the limitations of highly complex robotic systems, in terms of autonomy and adaptation. The main focus of research is to investigate the use of visual feedback for improving reaching and grasping capabilities of complex robots. To facilitate this a combined integration of computer vision and machine learning techniques is employed. From a robot vision point of view the combination of domain knowledge from both imaging processing and machine learning techniques, can expand the capabilities of robots. I present a novel framework called Cartesian Genetic Programming for Image Processing (CGP-IP). CGP-IP can be trained to detect objects in the incoming camera streams and successfully demonstrated on many different problem domains. The approach requires only a few training images (it was tested with 5 to 10 images per experiment) is fast, scalable and robust yet requires very small training sets. Additionally, it can generate human readable programs that can be further customized and tuned. While CGP-IP is a supervised-learning technique, I show an integration on the iCub, that allows for the autonomous learning of object detection and identification. Finally this dissertation includes two proof-of-concepts that integrate the motion and action sides. First, reactive reaching and grasping is shown. It allows the robot to avoid obstacles detected in the visual stream, while reaching for the intended target object. Furthermore the integration enables us to use the robot in non-static environments, i.e. the reaching is adapted on-the- fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. The second integration highlights the capabilities of these frameworks, by improving the visual detection by performing object manipulation actions.

A 3D numerical study of the collateral capacity of the circle of Willis with anatomical variation in the posterior circulation

Background The Circle of Willis (CoW) is the most important collateral pathway of the cerebral artery. The present study aims to investigate the collateral capacity of CoW with anatomical variation when unilateral internalcarotid artery (ICA) is occluded. Methods Basing on MRI data, we have reconstructed eight 3D models with variations in the posterior circulation of the CoW and set four different degrees of stenosis in the right ICA, namely 24%, 43%, 64% and 79%, respectively. Finally, a total of 40 models are performed with computational fluid dynamics simulations. All of the simulations share the same boundary condition with static pressure and the volume flow rate (VFR) are obtained to evaluate their collateral capacity. Results As for the middle cerebral artery (MCA) and the anterior cerebral artery (ACA), the transitional-type model possesses the best collateral capacity. But for the posterior cerebral artery (PCA), unilateral stenosis of ICA has the weakest influence on the unilateral posterior communicating artery (PCoA) absent model. We also find that the full fetal-type posterior circle of Willis is an utmost dangerous variation which must be paid more attention. Conclusion The results demonstrate that different models have different collateral capacities in coping stenosis of unilateral ICA and these differences can be reflected by different outlets. The study could be used as a reference for neurosurgeon in choosing the best treatment strategy.

3D numerical simulation of avascular tumour growth: effect of hypoxic micro-environment in host tissue

A three-dimensional (3D) mathematical model of tumour growth at the avascular phase and vessel remodelling in host tissues is proposed with emphasis on the study of the interactions of tumour growth and hypoxic micro-environment in host tissues. The hybrid based model includes the continuum part, such as the distributions of oxygen and vascular endothelial growth factors (VEGFs), and the discrete part of tumour cells (TCs) and blood vessel networks. The simulation shows the dynamic process of avascular tumour growth from a few initial cells to an equilibrium state with varied vessel networks. After a phase of rapidly increasing numbers of the TCs, more and more host vessels collapse due to the stress caused by the growing tumour. In addition, the consumption of oxygen expands with the enlarged tumour region. The study also discusses the effects of certain factors on tumour growth, including the density and configuration of preexisting vessel networks and the blood oxygen content. The model enables us to examine the relationship between early tumour growth and hypoxic micro-environment in host tissues, which can be useful for further applications, such as tumour metastasis and the initialization of tumour angiogenesis.

Three-dimensional 3d-4f heterometallic polymers containing both azide and carboxylate as co-ligands

The hydrothermal reaction of Ln(NO3)(3), Ni(NO3)(2), NaN3, and isonicotinic acid (L) yielded two novel 3-D coordination frameworks (1 and 2) of general formula [Ni(2)Ln(L)(5)(N-3)(2)(H2O)(3)] center dot 2H(2)O (Ln = Pr(III) for 1 and Nd(III) for 2), containing Ni-Pr or Ni-Nd hybrid extended three-dimensional networks containing both azido and carboxylate as co-ligands. Both the compounds are found to be isostructural and crystallize in monoclinic system having P2(1)/n space group. Here the lanthanide ions are found to be nonacoordinated. Both bidentate and monodentate modes of binding of the carboxylate with the lanthanides have been observed in the above complexes. Variable temperature magnetic studies of the above two complexes have been investigated in the temperature range 2-300 K which showed dominant antiferromagnetic interaction in both the cases and these experimental results are analyzed with the theoretical models. (c) 2008 Elsevier B.V. All rights reserved.

A Bayesian Framework for the Assessment of Vision-based Weed and Fruit Detection and Classification Algorithms

This paper proposes new metrics and a performance-assessment framework for vision-based weed and fruit detection and classification algorithms. In order to compare algorithms, and make a decision on which one to use fora particular application, it is necessary to take into account that the performance obtained in a series of tests is subject to uncertainty. Such characterisation of uncertainty seems not to be captured by the performance metrics currently reported in the literature. Therefore, we pose the problem as a general problem of scientific inference, which arises out of incomplete information, and propose as a metric of performance the(posterior) predictive probabilities that the algorithms will provide a correct outcome for target and background detection. We detail the framework through which these predicted probabilities can be obtained, which is Bayesian in nature. As an illustration example, we apply the framework to the assessment of performance of four algorithms that could potentially be used in the detection of capsicums (peppers).

Closed-form solutions for low-rank non-rigid reconstruction

Recovering the motion of a non-rigid body from a set of monocular images permits the analysis of dynamic scenes in uncontrolled environments. However, the extension of factorisation algorithms for rigid structure from motion to the low-rank non-rigid case has proved challenging. This stems from the comparatively hard problem of finding a linear “corrective transform” which recovers the projection and structure matrices from an ambiguous factorisation. We elucidate that this greater difficulty is due to the need to find multiple solutions to a non-trivial problem, casting a number of previous approaches as alleviating this issue by either a) introducing constraints on the basis, making the problems nonidentical, or b) incorporating heuristics to encourage a diverse set of solutions, making the problems inter-dependent. While it has previously been recognised that finding a single solution to this problem is sufficient to estimate cameras, we show that it is possible to bootstrap this partial solution to find the complete transform in closed-form. However, we acknowledge that our method minimises an algebraic error and is thus inherently sensitive to deviation from the low-rank model. We compare our closed-form solution for non-rigid structure with known cameras to the closed-form solution of Dai et al. [1], which we find to produce only coplanar reconstructions. We therefore make the recommendation that 3D reconstruction error always be measured relative to a trivial reconstruction such as a planar one.

A Survey of autonomous vision-based See and Avoid for Unmanned Aircraft Systems

This paper provides a comprehensive review of the vision-based See and Avoid problem for unmanned aircraft. The unique problem environment and associated constraints are detailed, followed by an in-depth analysis of visual sensing limitations. In light of such detection and estimation constraints, relevant human, aircraft and robot collision avoidance concepts are then compared from a decision and control perspective. Remarks on system evaluation and certification are also included to provide a holistic review approach. The intention of this work is to clarify common misconceptions, realistically bound feasible design expectations and offer new research directions. It is hoped that this paper will help us to unify design efforts across the aerospace and robotics communities.

Affective sensing + wearable imaging: Vision impairment and implications for spatial enquiry

Recent interest in affect and the body have mobilized a contemporary review of aesthetics and phenomenology within architecture to unpack how environments affect spatial experience. Emerging spatial studies within the neuro-sciences, and their implications for architectural research as raised by architectural theorists Juhani Pallasmaa (2014) and Harry Mallgrave (2013) has been well supported by a raft of scientists and institutions including the prestigious Salk Institute. Although there has been some headway in spatial studies of the vision impaired (Cattaneo et al, 2011) to understand the role of their non-visual systems in assisting navigation and location, little is discussed in terms of their other abilities in sensing particular qualities of space which impinge upon emotion. This paper reviews a collection of studies exploring face vision and echo-location, amongst others, which provide insight into what might be termed affective perception of the vision impaired, and how further interplay between this research and the architectural field can contribute new knowledge regarding space and affect. By engaging with themes from the Aesthetics, Phenomenology and indeed Neuro-science fields, the paper provides background of current and potential cross disciplinary research, and highlights the role wearable technologies can play in enhancing knowledge of affective spatial experience.

Estimation of light interception in research environments: A joint approach using directional light sensors and 3D virtual plants applied to sunflower (Helianthus annuus) and Arabidopsis thaliana in natural and artificial conditions

Light interception is a major factor influencing plant development and biomass production. Several methods have been proposed to determine this variable, but its calculation remains difficult in artificial environments with heterogeneous light. We propose a method that uses 3D virtual plant modelling and directional light characterisation to estimate light interception in highly heterogeneous light environments such as growth chambers and glasshouses. Intercepted light was estimated by coupling an architectural model and a light model for different genotypes of the rosette species Arabidopsis thaliana (L.) Heynh and a sunflower crop. The model was applied to plants of contrasting architectures, cultivated in isolation or in canopy, in natural or artificial environments, and under contrasting light conditions. The model gave satisfactory results when compared with observed data and enabled calculation of light interception in situations where direct measurements or classical methods were inefficient, such as young crops, isolated plants or artificial conditions. Furthermore, the model revealed that A. thaliana increased its light interception efficiency when shaded. To conclude, the method can be used to calculate intercepted light at organ, plant and plot levels, in natural and artificial environments, and should be useful in the investigation of genotype-environment interactions for plant architecture and light interception efficiency. This paper originates from a presentation at the 5th International Workshop on Functional–Structural Plant Models, Napier, New Zealand, November 2007.

Half-occlusion processing in stereoscopic vision

This study investigated questions related to half-occlusion processing in human stereoscopic vision: (1) How does the depth location of a half-occluding figure affect the depth localization of adjacent monocular objects? (2) Is three-dimensional slant around vertical axis (geometric effect) affected by half-occlusion constraints? and (3) How the half-occlusion constraints and surface formation processes are manifested in stereoscopic capture? Our results showed that the depth localization of binocular objects affects the depth localization of discrete monocular objects. We also showed that the visual system has a preference for a frontoparallel surface interpretation if the half-occlusion configuration allows multiple interpretation alternatives. When the surface formation was constrained by textures, our results showed that a process of rematching spreading determines the resulting perception and that the spreading can be limited by illusory contours that support the presence of binocularly unmatched figures. The unmatched figures could be present, if the inducing figures producing the illusory surface contained binocular image differences that provided cues for quantitative da Vinci stereopsis. These findings provide evidence of the significant role of half-occlusions in stereoscopic processing.

Distributed Robotic Vision as a service

The vision sense of standalone robots is limited by line of sight and onboard camera capabilities, but processing video from remote cameras puts a high computational burden on robots. This paper describes the Distributed Robotic Vision Service, DRVS, which implements an on-demand distributed visual object detection service. Robots specify visual information requirements in terms of regions of interest and object detection algorithms. DRVS dynamically distributes the object detection computation to remote vision systems with processing capabilities, and the robots receive high-level object detection information. DRVS relieves robots of managing sensor discovery and reduces data transmission compared to image sharing models of distributed vision. Navigating a sensorless robot from remote vision systems is demonstrated in simulation as a proof of concept.

Feature integration in human vision

The earliest stages of human cortical visual processing can be conceived as extraction of local stimulus features. However, more complex visual functions, such as object recognition, require integration of multiple features. Recently, neural processes underlying feature integration in the visual system have been under intensive study. A specialized mid-level stage preceding the object recognition stage has been proposed to account for the processing of contours, surfaces and shapes as well as configuration. This thesis consists of four experimental, psychophysical studies on human visual feature integration. In two studies, classification image a recently developed psychophysical reverse correlation method was used. In this method visual noise is added to near-threshold stimuli. By investigating the relationship between random features in the noise and observer s perceptual decision in each trial, it is possible to estimate what features of the stimuli are critical for the task. The method allows visualizing the critical features that are used in a psychophysical task directly as a spatial correlation map, yielding an effective "behavioral receptive field". Visual context is known to modulate the perception of stimulus features. Some of these interactions are quite complex, and it is not known whether they reflect early or late stages of perceptual processing. The first study investigated the mechanisms of collinear facilitation, where nearby collinear Gabor flankers increase the detectability of a central Gabor. The behavioral receptive field of the mechanism mediating the detection of the central Gabor stimulus was measured by the classification image method. The results show that collinear flankers increase the extent of the behavioral receptive field for the central Gabor, in the direction of the flankers. The increased sensitivity at the ends of the receptive field suggests a low-level explanation for the facilitation. The second study investigated how visual features are integrated into percepts of surface brightness. A novel variant of the classification image method with brightness matching task was used. Many theories assume that perceived brightness is based on the analysis of luminance border features. Here, for the first time this assumption was directly tested. The classification images show that the perceived brightness of both an illusory Craik-O Brien-Cornsweet stimulus and a real uniform step stimulus depends solely on the border. Moreover, the spatial tuning of the features remains almost constant when the stimulus size is changed, suggesting that brightness perception is based on the output of a single spatial frequency channel. The third and fourth studies investigated global form integration in random-dot Glass patterns. In these patterns, a global form can be immediately perceived, if even a small proportion of random dots are paired to dipoles according to a geometrical rule. In the third study the discrimination of orientation structure in highly coherent concentric and Cartesian (straight) Glass patterns was measured. The results showed that the global form was more efficiently discriminated in concentric patterns. The fourth study investigated how form detectability depends on the global regularity of the Glass pattern. The local structure was either Cartesian or curved. It was shown that randomizing the local orientation deteriorated the performance only with the curved pattern. The results give support for the idea that curved and Cartesian patterns are processed in at least partially separate neural systems.

Retinal photography screening programs to prevent vision loss from diabetic retinopathy in rural and urban Australia: A review

Purpose This review assessed the effectiveness of diabetic retinopathy (DR) screening programs, using retinal photography in Australian urban and rural settings, and considered implications for public health strategy and policy. Methods An electronic search of MEDLINE, PubMed, and Embase for studies published between 1 January 1996 and the 30 June 2013 was undertaken. Key search terms were “diabetic retinopathy,” “screening,” “retinal photography” and “Australia.” Results Twelve peer-reviewed publications were identified. The 14 DR screening programs identified from the 12 publications were successfully undertaken in urban, rural and remote communities across Australia. Locations included a pathology collection center, and Indigenous primary health care and Aboriginal community controlled organizations. Each intervention using retinal photography was highly effective at increasing the number of people who underwent screening for DR. The review identified that prior to commencement of the screening programs a median of 48% (range 16–85%) of those screened had not undergone a retinal examination within the recommended time frame (every year for Indigenous people and every 2 years for non-Indigenous people in Australia). A median of 16% (range 0–45%) of study participants had evidence of DR. Conclusions This review has shown there have been many pilot and demonstration projects in rural and urban Australia that confirm the effectiveness of retinal photography-based screening for DR

3d and 4d core level X-ray photoelectron spectroscopy of mixed-valence CePd3

3d and 4d core-level XPS spectra for CePd3, a mixed-valence system, have been measured. Each spectrum exhibits two sets of structures, each corresponding to one of the valence states of cerium. Thus the usefulness of XPS, which has so far not been used extensively to investigate the mixed-valence cerium systems, is pointed out.

Developing Queensland 3D Tropical Fruits and Nuts modelling capabilities

Development of 3D functional structural plant models for macadamias and other tropical fruit and nuts.