Improving robot vision models for object detection through interaction

We propose a method for learning specific object representations that can be applied (and reused) in visual detection and identification tasks. A machine learning technique called Cartesian Genetic Programming (CGP) is used to create these models based on a series of images. Our research investigates how manipulation actions might allow for the development of better visual models and therefore better robot vision. This paper describes how visual object representations can be learned and improved by performing object manipulation actions, such as, poke, push and pick-up with a humanoid robot. The improvement can be measured and allows for the robot to select and perform the `right' action, i.e. the action with the best possible improvement of the detector.

Reactive reaching and grasping on a humanoid: Towards closing the action-perception loop on the iCub

We propose a system incorporating a tight integration between computer vision and robot control modules on a complex, high-DOF humanoid robot. Its functionality is showcased by having our iCub humanoid robot pick-up objects from a table in front of it. An important feature is that the system can avoid obstacles - other objects detected in the visual stream - while reaching for the intended target object. Our integration also allows for non-static environments, i.e. the reaching is adapted on-the-fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. Furthermore we show that this system can be used both in autonomous and tele-operation scenarios.

From vision to actions: Towards adaptive and autonomous humanoid robots

Although robotics research has seen advances over the last decades robots are still not in widespread use outside industrial applications. Yet a range of proposed scenarios have robots working together, helping and coexisting with humans in daily life. In all these a clear need to deal with a more unstructured, changing environment arises. I herein present a system that aims to overcome the limitations of highly complex robotic systems, in terms of autonomy and adaptation. The main focus of research is to investigate the use of visual feedback for improving reaching and grasping capabilities of complex robots. To facilitate this a combined integration of computer vision and machine learning techniques is employed. From a robot vision point of view the combination of domain knowledge from both imaging processing and machine learning techniques, can expand the capabilities of robots. I present a novel framework called Cartesian Genetic Programming for Image Processing (CGP-IP). CGP-IP can be trained to detect objects in the incoming camera streams and successfully demonstrated on many different problem domains. The approach requires only a few training images (it was tested with 5 to 10 images per experiment) is fast, scalable and robust yet requires very small training sets. Additionally, it can generate human readable programs that can be further customized and tuned. While CGP-IP is a supervised-learning technique, I show an integration on the iCub, that allows for the autonomous learning of object detection and identification. Finally this dissertation includes two proof-of-concepts that integrate the motion and action sides. First, reactive reaching and grasping is shown. It allows the robot to avoid obstacles detected in the visual stream, while reaching for the intended target object. Furthermore the integration enables us to use the robot in non-static environments, i.e. the reaching is adapted on-the- fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. The second integration highlights the capabilities of these frameworks, by improving the visual detection by performing object manipulation actions.

Type extension and efficient AST manipulation

Oberon-2 is an object-oriented language with a class structure based on type extension. The runtime structure of Oberon-2 is described and the low-level mechanism for dynamic type checking explained. It is shown that the superior type-safety of the language, when used for programming styles based on heterogeneous, pointer-linked data structures, has an entirely negligible cost in runtime performance.

Towards automatic object segmentation with sequential multiple views

Object segmentation is one of the fundamental steps for a number of robotic applications such as manipulation, object detection, and obstacle avoidance. This paper proposes a visual method for incorporating colour and depth information from sequential multiview stereo images to segment objects of interest from complex and cluttered environments. Rather than segmenting objects using information from a single frame in the sequence, we incorporate information from neighbouring views to increase the reliability of the information and improve the overall segmentation result. Specifically, dense depth information of a scene is computed using multiple view stereo. Depths from neighbouring views are reprojected into the reference frame to be segmented compensating for imperfect depth computations for individual frames. The multiple depth layers are then combined with color information from the reference frame to create a Markov random field to model the segmentation problem. Finally, graphcut optimisation is employed to infer pixels belonging to the object to be segmented. The segmentation accuracy is evaluated over images from an outdoor video sequence demonstrating the viability for automatic object segmentation for mobile robots using monocular cameras as a primary sensor.

Semantic context and visual feature effects in object naming : an fMRI study using arterial spin labeling

Previous behavioral studies reported a robust effect of increased naming latencies when objects to be named were blocked within semantic category, compared to items blocked between category. This semantic context effect has been attributed to various mechanisms including inhibition or excitation of lexico-semantic representations and incremental learning of associations between semantic features and names, and is hypothesized to increase demands on verbal self-monitoring during speech production. Objects within categories also share many visual structural features, introducing a potential confound when interpreting the level at which the context effect might occur. Consistent with previous findings, we report a significant increase in response latencies when naming categorically related objects within blocks, an effect associated with increased perfusion fMRI signal bilaterally in the hippocampus and in the left middle to posterior superior temporal cortex. No perfusion changes were observed in the middle section of the left middle temporal cortex, a region associated with retrieval of lexical-semantic information in previous object naming studies. Although a manipulation of visual feature similarity did not influence naming latencies, we observed perfusion increases in the perirhinal cortex for naming objects with similar visual features that interacted with the semantic context in which objects were named. These results provide support for the view that the semantic context effect in object naming occurs due to an incremental learning mechanism, and involves increased demands on verbal self-monitoring.

Experiences with an Object-Oriented, Multi-Stage Language

Metaphor is a multi-stage programming language extension to an imperative, object-oriented language in the style of C# or Java. This paper discusses some issues we faced when applying multi-stage language design concepts to an imperative base language and run-time environment. The issues range from dealing with pervasive references and open code to garbage collection and implementing cross-stage persistence.

Security metrics for object-oriented class designs

Measuring quality attributes of object-oriented designs (e.g. maintainability and performance) has been covered by a number of studies. However, these studies have not considered security as much as other quality attributes. Also, most security studies focus at the level of individual program statements. This approach makes it hard and expensive to discover and fix vulnerabilities caused by design errors. In this work, we focus on the security design of an object oriented application and define a number of security metrics. These metrics allow designers to discover and fix security vulnerabilities at an early stage, and help compare the security of various alternative designs. In particular, we propose seven security metrics to measure Data Encapsulation (accessibility) and Cohesion (interactions) of a given object-oriented class from the point of view of potential information flow.

When the object speaks, a postcolonial encounter : anthropological, representations and Aboriginal's women's self-presentations

As an Aboriginal woman currently reviewing feminist literature in Australia, I have found that representations of Aboriginal women's gender have been generated predominantly by women anthropologists. Australian feminists utilise this literature in their writing and teaching and accept its truths without question; the most often quoted ethnographic text is Diane Bell's Daughters of the Dreaming (1983a).1 Feminists' lack of critical engagement with this literature implies that they are content to accept women anthropologists' representations because Aboriginal women are not central to their constructions of feminism.2 Instead the Aboriginal woman is positioned on the margins, a symbol of difference; a reminder that it is feminists who are the bearers of true womanhood.

Multi-edge decimation in multi-modal 3D collaborative applications

Multi-resolution modelling has become essential as modern 3D applications demand 3D objects with higher LODs (LOD). Multi-modal devices such as PDAs and UMPCs do not have sufficient resources to handle the original 3D objects. The increased usage of collaborative applications has created many challenges for remote manipulation working with 3D objects of different quality. This paper studies how we can improve multi-resolution techniques by performing multiedge decimation and using annotative commands. It also investigates how devices with poorer quality 3D object can participate in collaborative actions.

Human brain regions required for the dividing and switching of attention between two features of a single object

This combined PET and ERP study was designed to identify the brain regions activated in switching and divided attention between different features of a single object using matched sensory stimuli and motor response. The ERP data have previously been reported in this journal [64]. We now present the corresponding PET data. We identified partially overlapping neural networks with paradigms requiring the switching or dividing of attention between the elements of complex visual stimuli. Regions of activation were found in the prefrontal and temporal cortices and cerebellum. Each task resulted in different prefrontal cortical regions of activation lending support to the functional subspecialisation of the prefrontal and temporal cortices being based on the cognitive operations required rather than the stimuli themselves.

The manipulation of apoptosis-related genes to generate resistance to Fusarium wilt and water stress in banana

Bananas are susceptible to a diverse range of biotic and abiotic stresses, many of which cause serious production constraints worldwide. One of the most destructive banana diseases is Fusarium wilt caused by the soil-borne fungus, Fusarium oxysporum f. sp. cubense (Foc). No effective control strategy currently exists for this disease which threatens global banana production. Although disease resistance exists in some wild bananas, attempts to introduce resistance into commercially acceptable bananas by conventional breeding have been hampered by low fertility, long generation times and association of poor agronomical traits with resistance genes. With the advent of reliable banana transformation protocols, molecular breeding is now regarded as a viable alternative strategy to generate disease-resistant banana plants. Recently, a novel strategy involving the expression of anti-apoptosis genes in plants was shown to result in resistance against several necrotrophic fungi. Further, the transgenic plants showed increased resistance to a range of abiotic stresses. In this thesis, the use of anti-apoptosis genes to generate transgenic banana plants with resistance to Fusarium wilt was investigated. Since water stress is an important abiotic constraint to banana production, the resistance of the transgenic plants to water stress was also examined. Embryogenic cell suspensions (ECS) of two commercially important banana cultivars, Grand Naine (GN) and Lady Finger (LF), were transformed using Agrobacterium with the anti-apoptosis genes, Bcl-xL, Bcl-xL G138A, Ced-9 and Bcl- 2 3’ UTR. An interesting, and potentially important, outcome was that the use of anti-apoptosis genes resulted in up to a 50-fold increase in Agrobacterium-mediated transformation efficiency of both LF and GN cells over vector controls. Regenerated plants were subjected to a complete molecular characterisation in order to detect the presence of the transgene (PCR), transcript (RT-PCR) and gene product (Western blot) and to determine the gene copy number (Southern blot). A total of 36 independently-transformed GN lines (8 x Bcl-xL, 5 x Bcl-xL G138A, 15 x Ced-9 and 8 x Bcl-2 3’ UTR) and 41 independently-transformed LF lines (8 x Bcl-xL, 7 x BclxL G138A, 13 x Ced-9 and 13 x Bcl-2 3’ UTR) were identified. The 41 transgenic LF lines were multiplied and clones from each line were acclimatised and grown under glasshouse conditions for 8 weeks to allow monitoring for phenotypic abnormalities. Plants derived from 3 x Bcl-xL, 2 x Ced-9 and 5 x Bcl-2 3’ UTR lines displayed a variety of aberrant phenotypes. However, all but one of these abnormalities were off-types commonly observed in tissue-cultured, non-transgenic banana plants and were therefore unlikely to be transgene-related. Prior to determining the resistance of the transgenic plants to Foc race 1, the apoptotic effects of the fungus on both wild-type and Bcl-2 3’ UTR-transgenic LF banana cells were investigated using rapid in vitro root assays. The results from these assays showed that apoptotic-like cell death was elicited in wild-type banana root cells as early as 6 hours post-exposure to fungal spores. In contrast, these effects were attenuated in the root cells of Bcl-2 3’ UTR-transgenic lines that were exposed to fungal spores. Thirty eight of the 41 transgenic LF lines were subsequently assessed for resistance to Foc race 1 in small-plant glasshouse bioassays. To overcome inconsistencies in rating the internal (vascular discolouration) disease symptoms, a MatLab-based computer program was developed to accurately and reliably assess the level of vascular discolouration in banana corms. Of the transgenic LF banana lines challenged with Foc race 1, 2 x Bcl-xL, 3 x Ced-9, 2 x Bcl-2 3’ UTR and 1 x Bcl-xL G138A-transgenic line were found to show significantly less external and internal symptoms than wild-type LF banana plants used as susceptible controls at 12 weeks post-inoculation. Of these lines, Bcl-2 3’ UTR-transgenic line #6 appeared most resistant, displaying very mild symptoms similar to the wild-type Cavendish banana plants that were included as resistant controls. This line remained resistant for up to 23 weeks post-inoculation. Since anti-apoptosis genes have been shown to confer resistance to various abiotic stresses in other crops, the ability of these genes to confer resistance against water stress in banana was also investigated. Clonal plants derived from each of the 38 transgenic LF banana plants were subjected to water stress for a total of 32 days. Several different lines of transgenic plants transformed with either Bcl-xL, Bcl-xL G138A, Ced-9 or Bcl-2 3’ UTR showed a delay in visual water stress symptoms compared with the wild-type control plants. These plants all began producing new growth from the pseudostem following daily rewatering for one month. In an attempt to determine whether the protective effect of anti-apoptosis genes in transgenic banana plants was linked with reactive oxygen species (ROS)-associated programmed cell death (PCD), the effect of the chloroplast-targeting, ROS-inducing herbicide, Paraquat, on wild-type and transgenic LF was investigated. When leaf discs from wild-type LF banana plants were exposed to 10 ìM Paraquat, complete decolourisation occurred after 48 hours which was confirmed to be associated with cell death and ROS production by trypan blue and 3,3-diaminobenzidine (DAB) staining, respectively. When leaf discs from the transgenic lines were exposed to Paraquat, those derived from some lines showed a delay in decolourisation, suggesting only a weak protective effect from the transgenes. Finally, the protective effect of anti-apoptosis genes against juglone, a ROS-inducing phytotoxin produced by the causal agent of black Sigatoka, Mycosphaerella fijiensis, was investigated. When leaf discs from wild-type LF banana plants were exposed to 25 ppm juglone, complete decolourisation occurred after 48 hours which was again confirmed to be associated with cell death and ROS production by trypan blue and DAB staining, respectively. Further, TdT-mediated dUTP nick-end labelling (TUNEL) assays on these discs suggested that the cell death was apoptotic. When leaf discs from the transgenic lines were exposed to juglone, discs from some lines showed a clear delay in decolourisation, suggesting a protective effect. Whether these plants are resistant to black Sigatoka is unknown and will require future glasshouse and field trials. The work presented in this thesis provides the first report of the use of anti-apoptosis genes as a strategy to confer resistance to Fusarium wilt and water stress in a nongraminaceous monocot, banana. Such a strategy may be exploited to generate resistance to necrotrophic pathogens and abiotic stresses in other economically important crop plants.

Enhancing digital road map with lane details extracted from large-scale stereo aerial imagery using object-oriented image analysis

Precise, up-to-date and increasingly detailed road maps are crucial for various advanced road applications, such as lane-level vehicle navigation, and advanced driver assistant systems. With the very high resolution (VHR) imagery from digital airborne sources, it will greatly facilitate the data acquisition, data collection and updates if the road details can be automatically extracted from the aerial images. In this paper, we proposed an effective approach to detect road lane information from aerial images with employment of the object-oriented image analysis method. Our proposed algorithm starts with constructing the DSM and true orthophotos from the stereo images. The road lane details are detected using an object-oriented rule based image classification approach. Due to the affection of other objects with similar spectral and geometrical attributes, the extracted road lanes are filtered with the road surface obtained by a progressive two-class decision classifier. The generated road network is evaluated using the datasets provided by Queensland department of Main Roads. The evaluation shows completeness values that range between 76% and 98% and correctness values that range between 82% and 97%.