Experiments in visual localisation around underwater structures

Localisation of an AUV is challenging and a range of inspection applications require relatively accurate positioning information with respect to submerged structures. We have developed a vision based localisation method that uses a 3D model of the structure to be inspected. The system comprises a monocular vision system, a spotlight and a low-cost IMU. Previous methods that attempt to solve the problem in a similar way try and factor out the effects of lighting. Effects, such as shading on curved surfaces or specular reflections, are heavily dependent on the light direction and are difficult to deal with when using existing techniques. The novelty of our method is that we explicitly model the light source. Results are shown of an implementation on a small AUV in clear water at night.

Wide-baseline keypoint detection and matching with wide-angle images for vision based localisation

This thesis addresses the problem of detecting and describing the same scene points in different wide-angle images taken by the same camera at different viewpoints. This is a core competency of many vision-based localisation tasks including visual odometry and visual place recognition. Wide-angle cameras have a large field of view that can exceed a full hemisphere, and the images they produce contain severe radial distortion. When compared to traditional narrow field of view perspective cameras, more accurate estimates of camera egomotion can be found using the images obtained with wide-angle cameras. The ability to accurately estimate camera egomotion is a fundamental primitive of visual odometry, and this is one of the reasons for the increased popularity in the use of wide-angle cameras for this task. Their large field of view also enables them to capture images of the same regions in a scene taken at very different viewpoints, and this makes them suited for visual place recognition. However, the ability to estimate the camera egomotion and recognise the same scene in two different images is dependent on the ability to reliably detect and describe the same scene points, or ‘keypoints’, in the images. Most algorithms used for this purpose are designed almost exclusively for perspective images. Applying algorithms designed for perspective images directly to wide-angle images is problematic as no account is made for the image distortion. The primary contribution of this thesis is the development of two novel keypoint detectors, and a method of keypoint description, designed for wide-angle images. Both reformulate the Scale- Invariant Feature Transform (SIFT) as an image processing operation on the sphere. As the image captured by any central projection wide-angle camera can be mapped to the sphere, applying these variants to an image on the sphere enables keypoints to be detected in a manner that is invariant to image distortion. Each of the variants is required to find the scale-space representation of an image on the sphere, and they differ in the approaches they used to do this. Extensive experiments using real and synthetically generated wide-angle images are used to validate the two new keypoint detectors and the method of keypoint description. The best of these two new keypoint detectors is applied to vision based localisation tasks including visual odometry and visual place recognition using outdoor wide-angle image sequences. As part of this work, the effect of keypoint coordinate selection on the accuracy of egomotion estimates using the Direct Linear Transform (DLT) is investigated, and a simple weighting scheme is proposed which attempts to account for the uncertainty of keypoint positions during detection. A word reliability metric is also developed for use within a visual ‘bag of words’ approach to place recognition.

Structural health monitoring of bridges : source localisation in acoustic emission technique

Managing the sustainability of urban infrastructure requires regular health monitoring of key infrastructure such as bridges. The process of structural health monitoring involves monitoring a structure over a period of time using appropriate sensors, extracting damage sensitive features from the measurements made by the sensors, and analysing these features to determine the current state of the structure. Various techniques are available for structural health monitoring of structures, and acoustic emission is one technique that is finding an increasing use in the monitoring of civil infrastructures such as bridges. Acoustic emission technique is based on the recording of stress waves generated by rapid release of energy inside a material, followed by analysis of recorded signals to locate and identify the source of emission and assess its severity. This chapter first provides a brief background of the acoustic emission technique and the process of source localization. Results from laboratory experiments conducted to explore several aspects of the source localization process are also presented. The findings from the study can be expected to enhance knowledge of the acoustic emission process, and to aid the development of effective bridge structure diagnostics systems.

Damage localisation in beams and plates using vibration characteristics

Dynamic computer simulation techniques are used to develop and apply a multi-criteria procedure, incorporating changes in natural frequencies, modal flexibility and the modal strain energy, for damage localisation in beams and plates. Numerically simulated modal data obtained through finite element analyses are used to develop algorithms based on changes of modal flexibility and modal strain energy before and after damage and used as the indices for assessment of the state of structural health. The proposed procedure is illustrated through its application to flexural members under different damage scenarios and the results confirm its feasibility for damage assessment.

Fashion localisation in a global aesthetic market

This paper investigates how fashion circulates globally and is adapted and localised by consumers. The rise of fashion blogs, social networking, on-line retail and on-line streaming of fashion shows has exponentially increased the availability of fashion images globally, enabling a further multiplication of styles and looks. The geographical dispersion of production systems in third world countries, and the concentration of management and finance in first world countries are increasingly acknowledged as having an uneven social and economic effect. However, processes of hibridisation and creolisation give rise to new cultural forms where the local and the foreign are mixed in interesting ways. I argue that the current circulation of fashion must be understood as adaptation in which “outside aesthetic influence is integrated into and becomes part of an existing style tradition” (Lynch and Strauss, 2007, p. 154). This emergence of new local and eclectic styles denies assumptions in which consumers are disengaged while duped by a system of commodification. The paper argues that, through a process of “deterritorialisation”, “displacement” and “repatriation” (Appadurai 1996, p. 32), creative ordinary consumers are able to engage with fashion, reinventing it in the context of their local cultures.

Outdoor simultaneous localisation and mapping using RatSLAM

In this paper an existing method for indoor Simultaneous Localisation and Mapping (SLAM) is extended to operate in large outdoor environments using an omnidirectional camera as its principal external sensor. The method, RatSLAM, is based upon computational models of the area in the rat brain that maintains the rodent’s idea of its position in the world. The system uses the visual appearance of different locations to build hybrid spatial-topological maps of places it has experienced that facilitate relocalisation and path planning. A large dataset was acquired from a dynamic campus environment and used to verify the system’s ability to construct representations of the world and simultaneously use these representations to maintain localisation.

Gaussian process models for sensor-centric robot localisation

This paper presents an approach to building an observation likelihood function from a set of sparse, noisy training observations taken from known locations by a sensor with no obvious geometric model. The basic approach is to fit an interpolant to the training data, representing the expected observation, and to assume additive sensor noise. This paper takes a Bayesian view of the problem, maintaining a posterior over interpolants rather than simply the maximum-likelihood interpolant, giving a measure of uncertainty in the map at any point. This is done using a Gaussian process framework. To validate the approach experimentally, a model of an environment is built using observations from an omni-directional camera. After a model has been built from the training data, a particle filter is used to localise while traversing this environment

CAT-SLAM : probabilistic localisation and mapping using a continuous appearance-based trajectory

This paper describes a new system, dubbed Continuous Appearance-based Trajectory Simultaneous Localisation and Mapping (CAT-SLAM), which augments sequential appearance-based place recognition with local metric pose filtering to improve the frequency and reliability of appearance-based loop closure. As in other approaches to appearance-based mapping, loop closure is performed without calculating global feature geometry or performing 3D map construction. Loop-closure filtering uses a probabilistic distribution of possible loop closures along the robot’s previous trajectory, which is represented by a linked list of previously visited locations linked by odometric information. Sequential appearance-based place recognition and local metric pose filtering are evaluated simultaneously using a Rao–Blackwellised particle filter, which weights particles based on appearance matching over sequential frames and the similarity of robot motion along the trajectory. The particle filter explicitly models both the likelihood of revisiting previous locations and exploring new locations. A modified resampling scheme counters particle deprivation and allows loop-closure updates to be performed in constant time for a given environment. We compare the performance of CAT-SLAM with FAB-MAP (a state-of-the-art appearance-only SLAM algorithm) using multiple real-world datasets, demonstrating an increase in the number of correct loop closures detected by CAT-SLAM.

Matrix metalloproteinase localisation by in situ-RT-PCR in archival human breast biopsy material

Utilising archival human breast cancer biopsy material we examined the stromal/epithelial interactions of several matrix metalloproteinases (MMPs) using in situ-RT-PCR (IS-RT-PCR). In breast cancer, the stromal/epithelial interactions that occur, and the site of production of these proteases, are central to understanding their role in invasive and metastatic processes. We examined MT1-MMP (MMP-14, membrane type-1-MMP), MMP-1 (interstitial collagenase) and MMP-3 (stromelysin-1) for their localisation profile in progressive breast cancer biopsy material (poorly differentiated invasive breast carcinoma (PDIBC), invasive breast carcinomas (IBC) and lymph node metastases (LNM)). Expression of MT1-MMP, MMP-1 and MMP-3 was observed in both the tumour epithelial and surrounding stromal cells in most tissue sections examined. MT1-MMP expression was predominantly localised to the tumour component in the pre-invasive lesions. MMP-1 gene expression was relatively well distributed between both tissue compartments, while MMP-3 demonstrated highest expression levels in the stromal tissue surrounding the epithelial tumour cells. The results demonstrate the ability to distinguish compartmental gene expression profiles using IS-RT-PCR. Further, we suggest a role for MT1-MMP in early tumour progression, expression of MMP-1 during metastasis and focal expression pattern of MMP-3 in areas of expansion. These expression profiles may provide markers for early breast cancer diagnoses and present potential therapeutic targets.

In vitro and in vivo MMP gene expression localisation by In Situ-RT-PCR in cell culture and paraffin embedded human breast cancer cell line xenografts

Background Members of the matrix metalloproteinase (MMP) family of proteases are required for the degradation of the basement membrane and extracellular matrix in both normal and pathological conditions. In vitro, MT1-MMP (MMP-14, membrane type-1-MMP) expression is higher in more invasive human breast cancer (HBC) cell lines, whilst in vivo its expression has been associated with the stroma surrounding breast tumours. MMP-1 (interstitial collagenase) has been associated with MDA-MB-231 invasion in vitro, while MMP-3 (stromelysin-1) has been localised around invasive cells of breast tumours in vivo. As MMPs are not stored intracellularly, the ability to localise their expression to their cells of origin is difficult. Methods We utilised the unique in situ-reverse transcription-polymerase chain reaction (IS-RT-PCR) methodology to localise the in vitro and in vivo gene expression of MT1-MMP, MMP-1 and MMP-3 in human breast cancer. In vitro, MMP induction was examined in the MDA-MB-231 and MCF-7 HBC cell lines following exposure to Concanavalin A (Con A). In vivo, we examined their expression in archival paraffin embedded xenografts derived from a range of HBC cell lines of varied invasive and metastatic potential. Mouse xenografts are heterogenous, containing neoplastic human parenchyma with mouse stroma and vasculature and provide a reproducible in vivo model system correlated to the human disease state. Results In vitro, exposure to Con A increased MT1-MMP gene expression in MDA-MB-231 cells and decreased MT1-MMP gene expression in MCF-7 cells. MMP-1 and MMP-3 gene expression remained unchanged in both cell lines. In vivo, stromal cells recruited into each xenograft demonstrated differences in localised levels of MMP gene expression. Specifically, MDA-MB-231, MDA-MB-435 and Hs578T HBC cell lines are able to influence MMP gene expression in the surrounding stroma. Conclusion We have demonstrated the applicability and sensitivity of IS-RT-PCR for the examination of MMP gene expression both in vitro and in vivo. Induction of MMP gene expression in both the epithelial tumour cells and surrounding stromal cells is associated with increased metastatic potential. Our data demonstrate the contribution of the stroma to epithelial MMP gene expression, and highlight the complexity of the role of MMPs in the stromal-epithelial interactions within breast carcinoma.

Continuous appearance-based localisation and mapping

This thesis presents a novel approach to mobile robot navigation using visual information towards the goal of long-term autonomy. A novel concept of a continuous appearance-based trajectory is proposed in order to solve the limitations of previous robot navigation systems, and two new algorithms for mobile robots, CAT-SLAM and CAT-Graph, are presented and evaluated. These algorithms yield performance exceeding state-of-the-art methods on public benchmark datasets and large-scale real-world environments, and will help enable widespread use of mobile robots in everyday applications.

Dealing with shadows : capturing intrinsic scene appearance for image-based outdoor localisation

In outdoor environments shadows are common. These typically strong visual features cause considerable change in the appearance of a place, and therefore confound vision-based localisation approaches. In this paper we describe how to convert a colour image of the scene to a greyscale invariant image where pixel values are a function of underlying material property not lighting. We summarise the theory of shadow invariant images and discuss the modelling and calibration issues which are important for non-ideal off-the-shelf colour cameras. We evaluate the technique with a commonly used robotic camera and an autonomous car operating in an outdoor environment, and show that it can outperform the use of ordinary greyscale images for the task of visual localisation.

Low cost localisation for agricultural robotics

This paper presents a pose estimation approach that is resilient to typical sensor failure and suitable for low cost agricultural robots. Guiding large agricultural machinery with highly accurate GPS/INS systems has become standard practice, however these systems are inappropriate for smaller, lower-cost robots. Our positioning system estimates pose by fusing data from a low-cost global positioning sensor, low-cost inertial sensors and a new technique for vision-based row tracking. The results first demonstrate that our positioning system will accurately guide a robot to perform a coverage task across a 6 hectare field. The results then demonstrate that our vision-based row tracking algorithm improves the performance of the positioning system despite long periods of precision correction signal dropout and intermittent dropouts of the entire GPS sensor.

Combining multiple sensor modalities for a localisation robust to smoke

This paper proposes an approach to obtain a localisation that is robust to smoke by exploiting multiple sensing modalities: visual and infrared (IR) cameras. This localisation is based on a state-of-the-art visual SLAM algorithm. First, we show that a reasonably accurate localisation can be obtained in the presence of smoke by using only an IR camera, a sensor that is hardly affected by smoke, contrary to a visual camera (operating in the visible spectrum). Second, we demonstrate that improved results can be obtained by combining the information from the two sensor modalities (visual and IR cameras). Third, we show that by detecting the impact of smoke on the visual images using a data quality metric, we can anticipate and mitigate the degradation in performance of the localisation by discarding the most affected data. The experimental validation presents multiple trajectories estimated by the various methods considered, all thoroughly compared to an accurate dGPS/INS reference.