Vision-based topological mapping and localisation

Competent navigation in an environment is a major requirement for an autonomous mobile robot to accomplish its mission. Nowadays, many successful systems for navigating a mobile robot use an internal map which represents the environment in a detailed geometric manner. However, building, maintaining and using such environment maps for navigation is difficult because of perceptual aliasing and measurement noise. Moreover, geometric maps require the processing of huge amounts of data which is computationally expensive. This thesis addresses the problem of vision-based topological mapping and localisation for mobile robot navigation. Topological maps are concise and graphical representations of environments that are scalable and amenable to symbolic manipulation. Thus, they are well-suited for basic robot navigation applications, and also provide a representational basis for the procedural and semantic information needed for higher-level robotic tasks. In order to make vision-based topological navigation suitable for inexpensive mobile robots for the mass market we propose to characterise key places of the environment based on their visual appearance through colour histograms. The approach for representing places using visual appearance is based on the fact that colour histograms change slowly as the field of vision sweeps the scene when a robot moves through an environment. Hence, a place represents a region of the environment rather than a single position. We demonstrate in experiments using an indoor data set, that a topological map in which places are characterised using visual appearance augmented with metric clues provides sufficient information to perform continuous metric localisation which is robust to the kidnapped robot problem. Many topological mapping methods build a topological map by clustering visual observations to places. However, due to perceptual aliasing observations from different places may be mapped to the same place representative in the topological map. A main contribution of this thesis is a novel approach for dealing with the perceptual aliasing problem in topological mapping. We propose to incorporate neighbourhood relations for disambiguating places which otherwise are indistinguishable. We present a constraint based stochastic local search method which integrates the approach for place disambiguation in order to induce a topological map. Experiments show that the proposed method is capable of mapping environments with a high degree of perceptual aliasing, and that a small map is found quickly. Moreover, the method of using neighbourhood information for place disambiguation is integrated into a framework for topological off-line simultaneous localisation and mapping which does not require an initial categorisation of visual observations. Experiments on an indoor data set demonstrate the suitability of our method to reliably localise the robot while building a topological map.

Hippocampal models for simultaneous localisation and mapping on an autonomous robot

To navigate successfully in a novel environment a robot needs to be able to Simultaneously Localize And Map (SLAM) its surroundings. The most successful solutions to this problem so far have involved probabilistic algorithms, but there has been much promising work involving systems based on the workings of part of the rodent brain known as the hippocampus. In this paper we present a biologically plausible system called RatSLAM that uses competitive attractor networks to carry out SLAM in a probabilistic manner. The system can effectively perform parameter self-calibration and SLAM in one dimension. Tests in two dimensional environments revealed the inability of the RatSLAM system to maintain multiple pose hypotheses in the face of ambiguous visual input. These results support recent rat experimentation that suggest current competitive attractor models are not a complete solution to the hippocampal modelling problem.

Simultaneous localisation and mapping from natural landmarks using RatSLAM

This paper describes the current state of RatSLAM, a Simultaneous Localisation and Mapping (SLAM) system based on models of the rodent hippocampus. RatSLAM uses a competitive attractor network to fuse visual and odometry information. Energy packets in the network represent pose hypotheses, which are updated by odometry and can be enhanced or inhibited by visual input. This paper shows the effectiveness of the system in real robot tests in unmodified indoor environments using a learning vision system. Results are shown for two test environments; a large corridor loop and the complete floor of an office building.

Image acquisition and manipulation protocols for CT-PET fusion to improve the accuracy of gross tumour volume localisation for 3D conformal radiotherapy for lung cancer

Aims: To develop clinical protocols for acquiring PET images, performing CT-PET registration and tumour volume definition based on the PET image data, for radiotherapy for lung cancer patients and then to test these protocols with respect to levels of accuracy and reproducibility. Method: A phantom-based quality assurance study of the processes associated with using registered CT and PET scans for tumour volume definition was conducted to: (1) investigate image acquisition and manipulation techniques for registering and contouring CT and PET images in a radiotherapy treatment planning system, and (2) determine technology-based errors in the registration and contouring processes. The outcomes of the phantom image based quality assurance study were used to determine clinical protocols. Protocols were developed for (1) acquiring patient PET image data for incorporation into the 3DCRT process, particularly for ensuring that the patient is positioned in their treatment position; (2) CT-PET image registration techniques and (3) GTV definition using the PET image data. The developed clinical protocols were tested using retrospective clinical trials to assess levels of inter-user variability which may be attributed to the use of these protocols. A Siemens Somatom Open Sensation 20 slice CT scanner and a Philips Allegro stand-alone PET scanner were used to acquire the images for this research. The Philips Pinnacle3 treatment planning system was used to perform the image registration and contouring of the CT and PET images. Results: Both the attenuation-corrected and transmission images obtained from standard whole-body PET staging clinical scanning protocols were acquired and imported into the treatment planning system for the phantom-based quality assurance study. Protocols for manipulating the PET images in the treatment planning system, particularly for quantifying uptake in volumes of interest and window levels for accurate geometric visualisation were determined. The automatic registration algorithms were found to have sub-voxel levels of accuracy, with transmission scan-based CT-PET registration more accurate than emission scan-based registration of the phantom images. Respiration induced image artifacts were not found to influence registration accuracy while inadequate pre-registration over-lap of the CT and PET images was found to result in large registration errors. A threshold value based on a percentage of the maximum uptake within a volume of interest was found to accurately contour the different features of the phantom despite the lower spatial resolution of the PET images. Appropriate selection of the threshold value is dependant on target-to-background ratios and the presence of respiratory motion. The results from the phantom-based study were used to design, implement and test clinical CT-PET fusion protocols. The patient PET image acquisition protocols enabled patients to be successfully identified and positioned in their radiotherapy treatment position during the acquisition of their whole-body PET staging scan. While automatic registration techniques were found to reduce inter-user variation compared to manual techniques, there was no significant difference in the registration outcomes for transmission or emission scan-based registration of the patient images, using the protocol. Tumour volumes contoured on registered patient CT-PET images using the tested threshold values and viewing windows determined from the phantom study, demonstrated less inter-user variation for the primary tumour volume contours than those contoured using only the patient’s planning CT scans. Conclusions: The developed clinical protocols allow a patient’s whole-body PET staging scan to be incorporated, manipulated and quantified in the treatment planning process to improve the accuracy of gross tumour volume localisation in 3D conformal radiotherapy for lung cancer. Image registration protocols which factor in potential software-based errors combined with adequate user training are recommended to increase the accuracy and reproducibility of registration outcomes. A semi-automated adaptive threshold contouring technique incorporating a PET windowing protocol, accurately defines the geometric edge of a tumour volume using PET image data from a stand alone PET scanner, including 4D target volumes.

Experiments in visual localisation around underwater structures

Localisation of an AUV is challenging and a range of inspection applications require relatively accurate positioning information with respect to submerged structures. We have developed a vision based localisation method that uses a 3D model of the structure to be inspected. The system comprises a monocular vision system, a spotlight and a low-cost IMU. Previous methods that attempt to solve the problem in a similar way try and factor out the effects of lighting. Effects, such as shading on curved surfaces or specular reflections, are heavily dependent on the light direction and are difficult to deal with when using existing techniques. The novelty of our method is that we explicitly model the light source. Results are shown of an implementation on a small AUV in clear water at night.

Wide-baseline keypoint detection and matching with wide-angle images for vision based localisation

This thesis addresses the problem of detecting and describing the same scene points in different wide-angle images taken by the same camera at different viewpoints. This is a core competency of many vision-based localisation tasks including visual odometry and visual place recognition. Wide-angle cameras have a large field of view that can exceed a full hemisphere, and the images they produce contain severe radial distortion. When compared to traditional narrow field of view perspective cameras, more accurate estimates of camera egomotion can be found using the images obtained with wide-angle cameras. The ability to accurately estimate camera egomotion is a fundamental primitive of visual odometry, and this is one of the reasons for the increased popularity in the use of wide-angle cameras for this task. Their large field of view also enables them to capture images of the same regions in a scene taken at very different viewpoints, and this makes them suited for visual place recognition. However, the ability to estimate the camera egomotion and recognise the same scene in two different images is dependent on the ability to reliably detect and describe the same scene points, or ‘keypoints’, in the images. Most algorithms used for this purpose are designed almost exclusively for perspective images. Applying algorithms designed for perspective images directly to wide-angle images is problematic as no account is made for the image distortion. The primary contribution of this thesis is the development of two novel keypoint detectors, and a method of keypoint description, designed for wide-angle images. Both reformulate the Scale- Invariant Feature Transform (SIFT) as an image processing operation on the sphere. As the image captured by any central projection wide-angle camera can be mapped to the sphere, applying these variants to an image on the sphere enables keypoints to be detected in a manner that is invariant to image distortion. Each of the variants is required to find the scale-space representation of an image on the sphere, and they differ in the approaches they used to do this. Extensive experiments using real and synthetically generated wide-angle images are used to validate the two new keypoint detectors and the method of keypoint description. The best of these two new keypoint detectors is applied to vision based localisation tasks including visual odometry and visual place recognition using outdoor wide-angle image sequences. As part of this work, the effect of keypoint coordinate selection on the accuracy of egomotion estimates using the Direct Linear Transform (DLT) is investigated, and a simple weighting scheme is proposed which attempts to account for the uncertainty of keypoint positions during detection. A word reliability metric is also developed for use within a visual ‘bag of words’ approach to place recognition.

Structural health monitoring of bridges : source localisation in acoustic emission technique

Managing the sustainability of urban infrastructure requires regular health monitoring of key infrastructure such as bridges. The process of structural health monitoring involves monitoring a structure over a period of time using appropriate sensors, extracting damage sensitive features from the measurements made by the sensors, and analysing these features to determine the current state of the structure. Various techniques are available for structural health monitoring of structures, and acoustic emission is one technique that is finding an increasing use in the monitoring of civil infrastructures such as bridges. Acoustic emission technique is based on the recording of stress waves generated by rapid release of energy inside a material, followed by analysis of recorded signals to locate and identify the source of emission and assess its severity. This chapter first provides a brief background of the acoustic emission technique and the process of source localization. Results from laboratory experiments conducted to explore several aspects of the source localization process are also presented. The findings from the study can be expected to enhance knowledge of the acoustic emission process, and to aid the development of effective bridge structure diagnostics systems.

Damage localisation in beams and plates using vibration characteristics

Dynamic computer simulation techniques are used to develop and apply a multi-criteria procedure, incorporating changes in natural frequencies, modal flexibility and the modal strain energy, for damage localisation in beams and plates. Numerically simulated modal data obtained through finite element analyses are used to develop algorithms based on changes of modal flexibility and modal strain energy before and after damage and used as the indices for assessment of the state of structural health. The proposed procedure is illustrated through its application to flexural members under different damage scenarios and the results confirm its feasibility for damage assessment.

Application of acoustic emission technology in monitoring structure integrity of bridges

Bridges are an important part of a nation’s infrastructure and reliable monitoring methods are necessary to ensure their safety and efficiency. Most bridges in use today were built decades ago and are now subjected to changes in load patterns that can cause localized distress, which can result in bridge failure if not corrected. Early detection of damage helps in prolonging lives of bridges and preventing catastrophic failures. This paper briefly reviews the various technologies currently used in health monitoring of bridge structures and in particular discusses the application and challenges of acoustic emission (AE) technology. Some of the results from laboratory experiments on a bridge model are also presented. The main objectives of these experiments are source localisation and assessment. The findings of the study can be expected to enhance the knowledge of acoustic emission process and thereby aid in the development of an effective bridge structure diagnostics system.

The synthesis of novel nitroxides and their application as small-molecule antioxidants and profluorescent probes for oxidative stress

The detection and potential treatment of oxidative stress in biological systems has been explored using isoindoline-based nitroxide radicals. A novel tetraethyl-fluorescein nitroxide was synthesised for its use as a profluorescent probe for redox processes in biological systems. This tetraethyl system, as well as a tetramethyl-fluorescein nitroxide, were shown to be sensitive and selective probes for superoxide in vitro. The redox environment of cellular systems was also explored using the tetramethylfluorescein species based on its reduction to the hydroxylamine. Flow cytometry was employed to assess the extent of nitroxide reduction, reflecting the overall cellular redox environment. Treatment of normal fibroblasts with rotenone and 2-deoxyglucose resulted in an oxidising cellular environment as shown by the lack of reduction of the fluorescein-nitroxide system. Assessment of the tetraethyl-fluorescein nitroxide system in the same way demonstrated its enhanced resistance to reduction and offers the potential to detect and image biologically relevant reactive oxygen species directly. Importantly, these profluorescent nitroxide compounds were shown to be more effective than the more widely used and commercially available probes for reactive oxygen species such as 2’,7’-dichlorodihydrofluorescein diacetate. Fluorescence imaging of the tetramethyl-fluorescein nitroxide and a number of other rhodamine-nitroxide derivatives was undertaken, revealing the differential cellular localisation of these systems and thus their potential for the detection of redox changes in specific cellular compartments. As well as developing novel methods for the detection of oxidative stress, a number of novel isoindoline nitroxides were synthesised for their potential application as small-molecule antioxidants. These compounds incorporated known pharmacophores into the isoindoline-nitroxide structure in an attempt to increase their efficacy in biological systems. A primary and a secondary amine nitroxide were synthesised which incorporated the phenethylamine backbone of the sympathomimetic amine class of drugs. Initial assessment of the novel primary amine derivative indicated a protective effect comparable to that of 5-carboxy-1,1,3,3- tetramethylisoindolin-2-yloxyl. Methoxy-substituted nitroxides were also synthesised as potential antioxidants for their structural similarity to some amphetamine type stimulants. A copper-catalysed methodology provided access to both the mono- and di-substituted methoxy-nitroxides. Deprotection of the ethers in these compounds using boron tribromide successfully produced a phenolnitroxide, however the catechol moiety in the disubstituted derivative appeared to undergo reaction with the nitroxide to produce quinone-like degradation products. A novel fluoran-nitroxide was also synthesised from the methoxy-substituted nitroxide, providing a pH-sensitive spin probe. An amino-acid precursor containing a nitroxide moiety was also synthesised for its application as a dual-action antioxidant. N-Acetyl protection of the nitroxide radical was necessary prior to the Erlenmeyer reaction with N-acetyl glycine. Hydrolysis and reduction of the azlactone intermediate produced a novel amino acid precursor with significant potential as an effective antioxidant.

Fashion localisation in a global aesthetic market

This paper investigates how fashion circulates globally and is adapted and localised by consumers. The rise of fashion blogs, social networking, on-line retail and on-line streaming of fashion shows has exponentially increased the availability of fashion images globally, enabling a further multiplication of styles and looks. The geographical dispersion of production systems in third world countries, and the concentration of management and finance in first world countries are increasingly acknowledged as having an uneven social and economic effect. However, processes of hibridisation and creolisation give rise to new cultural forms where the local and the foreign are mixed in interesting ways. I argue that the current circulation of fashion must be understood as adaptation in which “outside aesthetic influence is integrated into and becomes part of an existing style tradition” (Lynch and Strauss, 2007, p. 154). This emergence of new local and eclectic styles denies assumptions in which consumers are disengaged while duped by a system of commodification. The paper argues that, through a process of “deterritorialisation”, “displacement” and “repatriation” (Appadurai 1996, p. 32), creative ordinary consumers are able to engage with fashion, reinventing it in the context of their local cultures.

Outdoor simultaneous localisation and mapping using RatSLAM

In this paper an existing method for indoor Simultaneous Localisation and Mapping (SLAM) is extended to operate in large outdoor environments using an omnidirectional camera as its principal external sensor. The method, RatSLAM, is based upon computational models of the area in the rat brain that maintains the rodent’s idea of its position in the world. The system uses the visual appearance of different locations to build hybrid spatial-topological maps of places it has experienced that facilitate relocalisation and path planning. A large dataset was acquired from a dynamic campus environment and used to verify the system’s ability to construct representations of the world and simultaneously use these representations to maintain localisation.

Gaussian process models for sensor-centric robot localisation

This paper presents an approach to building an observation likelihood function from a set of sparse, noisy training observations taken from known locations by a sensor with no obvious geometric model. The basic approach is to fit an interpolant to the training data, representing the expected observation, and to assume additive sensor noise. This paper takes a Bayesian view of the problem, maintaining a posterior over interpolants rather than simply the maximum-likelihood interpolant, giving a measure of uncertainty in the map at any point. This is done using a Gaussian process framework. To validate the approach experimentally, a model of an environment is built using observations from an omni-directional camera. After a model has been built from the training data, a particle filter is used to localise while traversing this environment

CAT-SLAM : probabilistic localisation and mapping using a continuous appearance-based trajectory

This paper describes a new system, dubbed Continuous Appearance-based Trajectory Simultaneous Localisation and Mapping (CAT-SLAM), which augments sequential appearance-based place recognition with local metric pose filtering to improve the frequency and reliability of appearance-based loop closure. As in other approaches to appearance-based mapping, loop closure is performed without calculating global feature geometry or performing 3D map construction. Loop-closure filtering uses a probabilistic distribution of possible loop closures along the robot’s previous trajectory, which is represented by a linked list of previously visited locations linked by odometric information. Sequential appearance-based place recognition and local metric pose filtering are evaluated simultaneously using a Rao–Blackwellised particle filter, which weights particles based on appearance matching over sequential frames and the similarity of robot motion along the trajectory. The particle filter explicitly models both the likelihood of revisiting previous locations and exploring new locations. A modified resampling scheme counters particle deprivation and allows loop-closure updates to be performed in constant time for a given environment. We compare the performance of CAT-SLAM with FAB-MAP (a state-of-the-art appearance-only SLAM algorithm) using multiple real-world datasets, demonstrating an increase in the number of correct loop closures detected by CAT-SLAM.

In vitro and in vivo MMP gene expression localisation by In Situ-RT-PCR in cell culture and paraffin embedded human breast cancer cell line xenografts

Background Members of the matrix metalloproteinase (MMP) family of proteases are required for the degradation of the basement membrane and extracellular matrix in both normal and pathological conditions. In vitro, MT1-MMP (MMP-14, membrane type-1-MMP) expression is higher in more invasive human breast cancer (HBC) cell lines, whilst in vivo its expression has been associated with the stroma surrounding breast tumours. MMP-1 (interstitial collagenase) has been associated with MDA-MB-231 invasion in vitro, while MMP-3 (stromelysin-1) has been localised around invasive cells of breast tumours in vivo. As MMPs are not stored intracellularly, the ability to localise their expression to their cells of origin is difficult. Methods We utilised the unique in situ-reverse transcription-polymerase chain reaction (IS-RT-PCR) methodology to localise the in vitro and in vivo gene expression of MT1-MMP, MMP-1 and MMP-3 in human breast cancer. In vitro, MMP induction was examined in the MDA-MB-231 and MCF-7 HBC cell lines following exposure to Concanavalin A (Con A). In vivo, we examined their expression in archival paraffin embedded xenografts derived from a range of HBC cell lines of varied invasive and metastatic potential. Mouse xenografts are heterogenous, containing neoplastic human parenchyma with mouse stroma and vasculature and provide a reproducible in vivo model system correlated to the human disease state. Results In vitro, exposure to Con A increased MT1-MMP gene expression in MDA-MB-231 cells and decreased MT1-MMP gene expression in MCF-7 cells. MMP-1 and MMP-3 gene expression remained unchanged in both cell lines. In vivo, stromal cells recruited into each xenograft demonstrated differences in localised levels of MMP gene expression. Specifically, MDA-MB-231, MDA-MB-435 and Hs578T HBC cell lines are able to influence MMP gene expression in the surrounding stroma. Conclusion We have demonstrated the applicability and sensitivity of IS-RT-PCR for the examination of MMP gene expression both in vitro and in vivo. Induction of MMP gene expression in both the epithelial tumour cells and surrounding stromal cells is associated with increased metastatic potential. Our data demonstrate the contribution of the stroma to epithelial MMP gene expression, and highlight the complexity of the role of MMPs in the stromal-epithelial interactions within breast carcinoma.