Designing a Computer Vision System for Underwater Robotic Interventions

This thesis deals with the challenging problem of designing systems able to perceive objects in underwater environments. In the last few decades research activities in robotics have advanced the state of art regarding intervention capabilities of autonomous systems. State of art in fields such as localization and navigation, real time perception and cognition, safe action and manipulation capabilities, applied to ground environments (both indoor and outdoor) has now reached such a readiness level that it allows high level autonomous operations. On the opposite side, the underwater environment remains a very difficult one for autonomous robots. Water influences the mechanical and electrical design of systems, interferes with sensors by limiting their capabilities, heavily impacts on data transmissions, and generally requires systems with low power consumption in order to enable reasonable mission duration. Interest in underwater applications is driven by needs of exploring and intervening in environments in which human capabilities are very limited. Nowadays, most underwater field operations are carried out by manned or remotely operated vehicles, deployed for explorations and limited intervention missions. Manned vehicles, directly on-board controlled, expose human operators to risks related to the stay in field of the mission, within a hostile environment. Remotely Operated Vehicles (ROV) currently represent the most advanced technology for underwater intervention services available on the market. These vehicles can be remotely operated for long time but they need support from an oceanographic vessel with multiple teams of highly specialized pilots. Vehicles equipped with multiple state-of-art sensors and capable to autonomously plan missions have been deployed in the last ten years and exploited as observers for underwater fauna, seabed, ship wrecks, and so on. On the other hand, underwater operations like object recovery and equipment maintenance are still challenging tasks to be conducted without human supervision since they require object perception and localization with much higher accuracy and robustness, to a degree seldom available in Autonomous Underwater Vehicles (AUV). This thesis reports the study, from design to deployment and evaluation, of a general purpose and configurable platform dedicated to stereo-vision perception in underwater environments. Several aspects related to the peculiar environment characteristics have been taken into account during all stages of system design and evaluation: depth of operation and light conditions, together with water turbidity and external weather, heavily impact on perception capabilities. The vision platform proposed in this work is a modular system comprising off-the-shelf components for both the imaging sensors and the computational unit, linked by a high performance ethernet network bus. The adopted design philosophy aims at achieving high flexibility in terms of feasible perception applications, that should not be as limited as in case of a special-purpose and dedicated hardware. Flexibility is required by the variability of underwater environments, with water conditions ranging from clear to turbid, light backscattering varying with daylight and depth, strong color distortion, and other environmental factors. Furthermore, the proposed modular design ensures an easier maintenance and update of the system over time. Performance of the proposed system, in terms of perception capabilities, has been evaluated in several underwater contexts taking advantage of the opportunity offered by the MARIS national project. Design issues like energy power consumption, heat dissipation and network capabilities have been evaluated in different scenarios. Finally, real-world experiments, conducted in multiple and variable underwater contexts, including open sea waters, have led to the collection of several datasets that have been publicly released to the scientific community. The vision system has been integrated in a state of the art AUV equipped with a robotic arm and gripper, and has been exploited in the robot control loop to successfully perform underwater grasping operations.

Spatially-multiplexed fibre-optic Bragg grating strain and temperature sensor system based on interferometric wavelength-shift detection

A prototype fibre-optic system using interferometric wavelength-shift detection, capable of multiplexing up to 32 fibre-optic Bragg grating strain and temperature sensors with identical characteristics, has been demonstrated. This system is based on a spatially multiplexed scheme for use with fibre-based low-coherence interferometric sensors, reported previously. Four fibre-optic Bragg grating channels using the same fibre grating have been demonstrated for measuring quasi-static strain and temperature.

Fiber Bragg grating sensor interrogation system using a CCD side detection method with Superimposed blazed gratings

We present, for the first time to our knowledge, experimental evidence showing that superimposed blazed fiber Bragg gratings may be fabricated and used to extend the dynamic range of a grating-based spectrometer. Blazed gratings of 4° and 8° were superimposed in germanosilicate fiber by ultraviolet inscription and used in conjunction with a coated charged-coupled device array to interrogate a wavelength-division-multiplexing sensor array. We show that the system can be used to monitor strain and temperature sensors simultaneously with an employable bandwidth which is extendable to 70 nm.

In-fibre Bragg grating temperature sensor system for medical applications

A novel quasidistributed in-fiber Bragg grating (FBG) temperature sensor system has been developed for temperature proving in vivo in the human body for medical applications, e.g., hyperthermia treatment. This paper provides the operating principle of FBG temperature sensors and then the design of the sensor head. High-resolution detection of the wavelength-shifts induced by temperature changes are achieved using drift-compensated interferometric detection while the return signals from the FBG sensor array are demultiplexed with a simple monochromator which offers crosstalk-free wavelength-division-multiplexing (WDM). A “strain-free” probe is designed by enclosing the FBG sensor array in a protection sleeve. A four FBG sensor system is demonstrated and the experimental results are in good agreement with those obtained by traditional electrical thermocouple sensors. A resolution of 0.1°C and an accuracy of ±0.2°C over a temperature range of 30-60°C have been achieved, which meet established medical requirements.

Optical fibre sensors and their interrogation

This thesis describes novel developments in the fabrication and understanding of type IA fibre Bragg gratings, the uses of said gratings as optical sensors and the interrogation of optical sensors using tilted fibre Bragg gratings. This thesis presents the most detailed study of type IA gratings performed to date and provides the basis of a dual grating optical sensor capable of independently measuring strain and temperature. Until this work it was not known how to reliably fabricate type IA gratings or how they would react to high ambient temperatures, nor was it known what effect external parameters such as fibre type, dopant levels, inscription laser intensity, or hydrogenation levels would have on the physical properties of the grating. This comprehensive study has yielded answers to all of these unknowns and produced several unexpected uses for type IA gratings, such as the use of the previously unreported strong loss band at 1400nm to locally heat fibres by optical absorption and thereby fabricate optically tuneable gratings which do not affect directly adjacent standard gratings. Blazed fibre Bragg gratings have been studied in detail and used to produce several high quality prototype sensor interrogation systems yielding stability an accuracy values unsurpassed by similar devices reported in literature. An accurate distribution map of light radiated by blazed gratings is shown for the first time and has been studied in respect of polarisation state showing that for certain easily achievable conditions a blazed grating spectrometer may be deemed to be polarisation insensitive. In a novel implementation of the system, it is shown that the dynamic wavelength range of a blazed grating spectrometer may be at least doubled by superimposing blazed gratings.

A three dimensional surface measurement and reconstruction system for respiratory function analysis

Respiration is a complex activity. If the relationship between all neurological and skeletomuscular interactions was perfectly understood, an accurate dynamic model of the respiratory system could be developed and the interaction between different inputs and outputs could be investigated in a straightforward fashion. Unfortunately, this is not the case and does not appear to be viable at this time. In addition, the provision of appropriate sensor signals for such a model would be a considerable invasive task. Useful quantitative information with respect to respiratory performance can be gained from non-invasive monitoring of chest and abdomen motion. Currently available devices are not well suited in application for spirometric measurement for ambulatory monitoring. A sensor matrix measurement technique is investigated to identify suitable sensing elements with which to base an upper body surface measurement device that monitors respiration. This thesis is divided into two main areas of investigation; model based and geometrical based surface plethysmography. In the first instance, chapter 2 deals with an array of tactile sensors that are used as progression of existing and previously investigated volumetric measurement schemes based on models of respiration. Chapter 3 details a non-model based geometrical approach to surface (and hence volumetric) profile measurement. Later sections of the thesis concentrate upon the development of a functioning prototype sensor array. To broaden the application area the study has been conducted as it would be fore a generically configured sensor array. In experimental form the system performance on group estimation compares favourably with existing system on volumetric performance. In addition provides continuous transient measurement of respiratory motion within an acceptable accuracy using approximately 20 sensing elements. Because of the potential size and complexity of the system it is possible to deploy it as a fully mobile ambulatory monitoring device, which may be used outside of the laboratory. It provides a means by which to isolate coupled physiological functions and thus allows individual contributions to be analysed separately. Thus facilitating greater understanding of respiratory physiology and diagnostic capabilities. The outcome of the study is the basis for a three-dimensional surface contour sensing system that is suitable for respiratory function monitoring and has the prospect with future development to be incorporated into a garment based clinical tool.

Interrogation of fibre optic sensors

The aim of the research work described in this thesis was to investigate the interrogation of fibre optic sensors using "off the shelf optical components and equipment developed mainly for the telecommunications industry. This provides a cost effective way of bringing fibre optic sensor systems to within the price range of their electro-mechanical counterparts. The research work focuses on the use of an arrayed waveguide grating, an acousto-optic tuneable filter and low-coherence interferometry to measure dynamic strain and displacement using fibre Bragg grating and interferometric sensors. Based on the intrinsic properties of arrayed waveguide gratings and acousto-optic tuneable filters used in conjunction with interferometry, fibre Bragg gratings and interferometric sensors a number of novel fibre optic sensor interrogation systems have been realised. Special single mode fibre, namely, high-birefringence fibre has been employed to implement a dual-beam interrogating interferometer. The first interrogation scheme is based on an optical channel monitor, which is an arrayed waveguide grating with integral photo-detectors providing a number of amplified electrical outputs. It is used to interrogate fibre Bragg grating and interferometric sensors. Using the properties of polarisation maintainability in high-birefringent fibre an interrogating interferometer was realised by winding a length of the fibre around a piezoelectric modulator generating a low-frequency carrier signal. The system was used to interrogate both fibre Bragg grating and interferometric sensors. Finally, the use of an acousto-optic tuneable filter is employed to interrogate fibre Bragg gratings. The device is used to generate a very high frequency carrier signal at the output of an optical interferometer.

Low-cost interrogation of optical fibre Bragg grating sensors

Through the application of novel signal processing techniques we are able to measure physical measurands with both high accuracy and low noise susceptibility. The first interrogation scheme is based upon a CCD spectrometer. We compare different algorithms for resolving the Bragg wavelength from a low resolution discrete representation of the reflected spectrum, and present optimal processing methods for providing a high integrity measurement from the reflection image. Our second sensing scheme uses a novel network of sensors to measure the distributive strain response of a mechanical system. Using neural network processing methods we demonstrate the measurement capabilities of a scalable low-cost fibre Bragg grating sensor network. This network has been shown to be comparable with the performance of existing fibre Bragg grating sensing techniques, at a greatly reduced implementation cost.

Vector bending sensors based on fibre Bragg gratings inscribed by infrared femtosecond laser

A novel, direction-sensitive bending sensor based on an asymmetric fiber Bragg grating (FBG) inscribed by an infrared femtosecond laser was demonstrated. The technique is based on tight transverse confinement of the femto-inscribed structures and can be directly applied in conventional, untreated singlemode fibers. The FBG structure was inscribed by an amplified, titanium sapphire laser system. The grating cross-section was elongated along the direction of the laser beam with the transverse dimensions of approximately 1 by 2 μm. It was suggested that the sensitivity of the device can be improved by inscribing smaller spatial features and by implementing more complex grating designs aimed at maximizing the effect of strain.

Long period gratings written into a photonic crystal fibre by a femtosecond laser as directional bend sensors

A series of symmetric and asymmetric LPGs were inscribed in photonic crystal fibre by a low repetition rate femtosecond laser system. The asymmetric LPGs were found to be spectrally sensitive to bend orientation, with some of the attenuation bands producing both red and blue wavelength shifts, whilst the symmetric devices produced only a unidirectional wavelength shift. Both sets of devices displayed strong polarisation dependence.

800 nm WDM interrogation system for strain, temperature, and refractive index sensing based on tilted fiber Bragg grating

A low-cost high-resolution wavelength-division-multiplexing (WDM) interrogation system operating around 800 nm region with operational bandwidth up to 60 nm and resolution of 12.7 pm utilizing a tilted fiber Bragg grating (TFBG) and a CCDarray detector has been implemented. The system has been evaluated for interrogating fiber Bragg grating based strain, temperature sensors, giving sensitivities of 0.59 pm/µe and 5.6 pm/°C, which are in good agreement with previously reported values. Furthermore, the system has been utilized to detect the refractive index change of sample liquids, demonstrating a capability of measuring index change as small as 10¯5. In addition, the vectorial expression of phase match condition and fabrication of TFBG have been discussed.

Modified hydrogel matrices in fibre optic sensors

Common problems encountered in clinical sensing are those of non-biocompatibility, and slow response time of the device. The latter, also applying to chemical sensors, is possibly due to a lack of understanding of polymer support or membrane properties and hence failure to optimise membranes chosen for specific sensor applications. Hydrogels can be described as polymers which swell in water. In addition to this, the presence of water in the polymer matrix offers some control of biocompatibility. They thus provide a medium through which rapid transport of a sensed species to an incorporated reagent could occur. This work considers the feasibility of such a system, leading to the design and construction of an optical sensor test bed. The development of suitable membrane systems and of suitable coating techniques in order to apply them to the fibre optics is described. Initial results obtained from hydrogel coatings implied that the refractive index change in the polymer matrix, due to a change in water content with pH is the major factor contributing to the sensor response. However the presence of the colourimetric reagent was also altering the output signal obtained. An analysis of factors contributing to the overall response, such as colour change and membrane composition were made on both the test bed, via optical response, and on whole membranes via measurement of water content change. The investigation of coatings with low equilibrium water contents, of less than 10% was carried out and in fact a clearer signal response from the test bed was noted. Again these membranes were suprisingly responding via refractive index change, with the reagent playing a primary role in obtaining a sensible or non-random response, although not in a colourimetric fashion. A photographic study of these coatings revealed some clues as to the physical nature of these coatings and hence partially explained this phenomenon. A study of the transport properties of the most successful membrane, on a coated wire electrode and also on the fibre optic test bed, in a series of test environments, indicated that the reagent was possibly acting as an ion exchanger and hence having a major influence on transport and therefore sensor characteristics.

Fibre Bragg grating sensors for distributive tactile sensing

Distributive tactile sensing is a method of tactile sensing in which a small number of sensors monitors the behaviour of a flexible substrate which is in contact with the object being sensed. This paper describes the first use of fibre Bragg grating sensors in such a system. Two systems are presented: the first is a one-dimensional metal strip with an array of four sensors, which is capable of detecting the magnitude and position of a contacting load. This system is favourably compared experimentally with a similar system using resistive strain gauges. The second system is a two-dimensional steel plate with nine sensors which is able to distinguish the position and shape of a contacting load, or the positions of two loads simultaneously. This system is compared with a similar system using 16 infrared displacement sensors. Each system uses neural networks to process the sensor data to give information concerning the type of contact. Issues and limitations of the systems are discussed, along with proposed solutions to some of the difficulties.

Optical fibre sensors and their applications in the industrial weighing and aerospace industries

This thesis presents the design, fabrication and testing of novel grating based Optical Fibre Sensor (OFS) systems being interrogated using “off the shelf” interrogation systems, with the eventual development of marketable commercial systems at the forefront of the research. Both in the industrial weighing and aerospace industries, there has been a drive to investigate the feasibility of using optical fibre sensors being deployed where traditionally their electrical or mechanical counterparts would traditionally have been. Already, in the industrial weighing industry, commercial operators are deploying OFS-based Weigh-In-Motion (WIM) systems. Likewise, in the aerospace industry, OFS have been deployed to monitor such parameters as load history, impact detection, structural damage, overload detection, centre of gravity and the determination of blade shape. Based on the intrinsic properties of fibre Bragg gratings (FBGs) and Long Period Fibre Gratings (LPFGs), a number of novel OFS-based systems have been realised. Experimental work has shown that in the case of static industrial weighing, FBGs can be integrated with current commercial products and used to detect applied loads. The work has also shown that embedding FBGs in e-glass, to form a sensing patch, can result in said patches being bonded to rail track, forming the basis of an FBG-based WIM system. The results obtained have been sufficiently encouraging to the industrial partner that this work will be progressed beyond the scope of the work presented in this thesis. Likewise, and to the best of the author’s knowledge, a novel Bragg grating based systems for aircraft fuel parameter sensing has been presented. FBG-based pressure sensors have been shown to demonstrate good sensitivity, linearity and repeatability, whilst LPFG-based systems have demonstrated a far greater sensitivity when compared to FBGs, as well the advantage of being potentially able to detect causes of fuel adulteration based on their sensitivity to refractive index (RI). In the case of the LPFG-based system, considerable work remains to be done on the mechanical strengthening to improve its survivability in a live aircraft fuel tank environment. The FBG system has already been developed to an aerospace compliant prototype and is due to be tested at the fuel testing facility based at Airbus, Filton, UK. It is envisaged by the author that in both application areas, continued research in this area will lead to the eventual development of marketable commercial products.

Fibre Bragg grating strain sensors

The fabrication of in-fibre Bragg gratings (FBGs) and their application as sensors is reported. The strain and temperature characteristic results for a number of chirped and uniform gratings written into three different host fibres are presented. The static and dynamic temperature response of a commercially available temperature compensated grating is reported. A five sensor wavelength division multiplexed fibre Bragg grating strain measurement system with an interrogation rate of 25 Hz and resolution of 10 was constructed. The results from this system are presented. A novel chirped FBG interrogation method was implemented in both the 1.3 and 1.5 m telecommunication windows. Several single and dual strain sensor systems, employing this method, were constructed and the results obtained from each are reported and discussed. These systems are particularly suitable for the measurement of large strain. The results from a system measuring up to 12 m and with a potential measurement range of 30 m are reported. This technique is also shown to give an obtainable resolution of 20 over a measurement range of 5 000 for a dual sensor system. These systems are simple, robust, passive and easy to implement. They offer low cost, high speed and, in the case of multiple sensors, truly simultaneous interrogation. These advantages make this technique ideal for strain sensing in SMART structures. Systems based on this method have been installed in the masts of four superyachts. A system, based on this technique, is currently being developed for the measurement of acoustic waves in carbon composite panels. The results from an alternative method for interrogating uniform FBG sensors are also discussed. Interrogation of the gratings was facilitated by a specifically written asymmetric grating which had a 15 nm long linearly sloped spectral edge. This technique was employed to interrogate a single sensor over a measurement range of 6 m and two sensors over a range of 4.5 me. The results obtained indicated achievable resolutions of 47 and 38 respectively.