Perception Quality Evaluation with Visual and Infrared Cameras in Challenging Environmental Conditions

This work aims to contribute to the reliability and integrity of perceptual systems of unmanned ground vehicles (UGV). A method is proposed to evaluate the quality of sensor data prior to its use in a perception system by utilising a quality metric applied to heterogeneous sensor data such as visual and infrared camera images. The concept is illustrated specifically with sensor data that is evaluated prior to the use of the data in a standard SIFT feature extraction and matching technique. The method is then evaluated using various experimental data sets that were collected from a UGV in challenging environmental conditions, represented by the presence of airborne dust and smoke. In the first series of experiments, a motionless vehicle is observing a ’reference’ scene, then the method is extended to the case of a moving vehicle by compensating for its motion. This paper shows that it is possible to anticipate degradation of a perception algorithm by evaluating the input data prior to any actual execution of the algorithm.

Towards discrimination of challenging conditions for UGVs with visual and infrared sensors

This work aims to contribute to reliability and integrity in perceptual systems of autonomous ground vehicles. Information theoretic based metrics to evaluate the quality of sensor data are proposed and applied to visual and infrared camera images. The contribution of the proposed metrics to the discrimination of challenging conditions is discussed and illustrated with the presence of airborne dust and smoke.

Automatic detection of diabetic foot complications with infrared thermography by asymmetric analysis

Early identification of diabetic foot complications and their precursors is essential in preventing their devastating consequences, such as foot infection and amputation. Frequent, automatic risk assessment by an intelligent telemedicine system might be feasible and cost effective. Infrared thermography is a promising modality for such a system. The temperature differences between corresponding areas on contralateral feet are the clinically significant parameters. This asymmetric analysis is hindered by (1) foot segmentation errors, especially when the foot temperature and the ambient temperature are comparable, and by (2) different shapes and sizes between contralateral feet due to deformities or minor amputations. To circumvent the first problem, we used a color image and a thermal image acquired synchronously. Foot regions, detected in the color image, were rigidly registered to the thermal image. This resulted in 97.8% ± 1.1% sensitivity and 98.4% ± 0.5% specificity over 76 high-risk diabetic patients with manual annotation as a reference. Nonrigid landmark-based registration with Bsplines solved the second problem. Corresponding points in the two feet could be found regardless of the shapes and sizes of the feet. With that, the temperature difference of the left and right feet could be obtained.

Applications of finite element computer modelling and thermal infrared remote sensing to the study of geothermal anomalies

Buried heat sources can be investigated by examining thermal infrared images and comparing these with the results of theoretical models which predict the thermal anomaly a given heat source may generate. Key factors influencing surface temperature include the geometry and temperature of the heat source, the surface meteorological environment, and the thermal conductivity and anisotropy of the rock. In general, a geothermal heat flux of greater than 2% of solar insolation is required to produce a detectable thermal anomaly in a thermal infrared image. A heat source of, for example, 2-300K greater than the average surface temperature must be a t depth shallower than 50m for the detection of the anomaly in a thermal infrared image, for typical terrestrial conditions. Atmospheric factors are of critical importance. While the mean atmospheric temperature has little significance, the convection is a dominant factor, and can act to swamp the thermal signature entirely. Given a steady state heat source that produces a detectable thermal anomaly, it is possible to loosely constrain the physical properties of the heat source and surrounding rock, using the surface thermal anomaly as a basis. The success of this technique is highly dependent on the degree to which the physical properties of the host rock are known. Important parameters include the surface thermal properties and thermal conductivity of the rock. Modelling of transient thermal situations was carried out, to assess the effect of time dependant thermal fluxes. One-dimensional finite element models can be readily and accurately applied to the investigation of diurnal heat flow, as with thermal inertia models. Diurnal thermal models of environments on Earth, the Moon and Mars were carried out using finite elements and found to be consistent with published measurements. The heat flow from an injection of hot lava into a near surface lava tube was considered. While this approach was useful for study, and long term monitoring in inhospitable areas, it was found to have little hazard warning utility, as the time taken for the thermal energy to propagate to the surface in dry rock (several months) in very long. The resolution of the thermal infrared imaging system is an important factor. Presently available satellite based systems such as Landsat (resolution of 120m) are inadequate for detailed study of geothermal anomalies. Airborne systems, such as TIMS (variable resolution of 3-6m) are much more useful for discriminating small buried heat sources. Planned improvements in the resolution of satellite based systems will broaden the potential for application of the techniques developed in this thesis. It is important to note, however, that adequate spatial resolution is a necessary but not sufficient condition for successful application of these techniques.

Visualization of water vapour flow in a packed bed adsorber by near-infrared diffused transmittance tomography

This work presents a procedure based on spatially-resolved near-infrared imaging, in order to observe temperature and composition maps in gas-solid packed beds subjected to effects of aspect ratio and non-isothermal conditions. The technique was applied to the water vapour flow in a packed bed adsorber of low aspect ratio, filled with silica gel, using a tuneable diode laser, focal planar array detector and tomographic reconstruction. The 2D projected images from parallel scanning permitted data to be retrieved from the packing and above the packing sections of 12.0×12.0×18.2mm at a volume-resolution of 0.15×0.15×0.026mm and a time-resolution of less than 3min. The technique revealed uneven temperature and composition maps in the core packed bed and in the vicinity of the wall due to flow maldistribution. In addition, the heat uptake from the packed bed and local cross-mixing were experimentally ascertained by local profiles of the water vapour composition and temperature under various aspect ratios and feed flow rates. The relative deviations in temperature and compositions were 11.1% and 9.3%, respectively. The deviation in composition, which covers the packing and above the packing sections, was slightly higher than the deviation of 8% obtained up-to-date but was limited to the exit of a packed bed adsorber. © 2011.

Nile blue-based nano-sized pH sensors for simultaneous far-red and near-infrared live bioimaging

Diblock copolymer vesicles are tagged with pH-responsive Nile Blue-based labels and used as a new type of pH-responsive colorimetric/fluorescent biosensor for far-red and near-infrared imaging of live cells. The diblock copolymer vesicles described herein are based on poly(2-(methacryloyloxy)ethyl phosphorylcholine-block-2-(diisopropylamino)ethyl methacrylate) [PMPC-PDPA]: the biomimetic PMPC block is known to facilitate rapid cell uptake for a wide range of cell lines, while the PDPA block constitutes the pH-responsive component that enables facile vesicle self-assembly in aqueous solution. These biocompatible vesicles can be utilized to detect interstitial hypoxic/acidic regions in a tumor model via a pH-dependent colorimetric shift. In addition, they are also useful for selective intracellular staining of lysosomes and early endosomes via subtle changes in fluorescence emission. Such nanoparticles combine efficient cellular uptake with a pH-responsive Nile Blue dye label to produce a highly versatile dual capability probe. This is in marked contrast to small molecule dyes, which are usually poorly uptaken by cells, frequently exhibit cytotoxicity, and are characterized by intracellular distributions invariably dictated by their hydrophilic/hydrophobic balance.

Mid-infrared photodetectors based on GaAsSb-capped InAs quantum dots

Quantum dot infrared photodetectors (QDIPs) are very attractive for many applications such as infrared imaging, remote sensing and gas sensing, thanks to its promising features such as high temperature operation, normal incidence response and low dark current [1]. However, the key issue is to obtain a high-quality active region which requires an optimization of the nanostructure. By using GaAsSb capping layer, InAs QDs have improved their optical emission in the range between 1.15 and 1.3 m (at Sb composition of 14 %), due to a reduction of a compressive strain in QD and an increment of a QD height [2]. In this work, we have demonstrated strong and narrow intraband photoresponses at ~ 5 m from GaAsSb-capped InAs/GaAs QDIPs under normal light-incidence.

Optically triggered infrared photodetector

We demonstrate a new class of semiconductor device: the optically triggered infrared photodetector (OTIP). This photodetector is based on a new physical principle that allows the detection of infrared light to be switched ON and OFF by means of an external light. Our experimental device, fabricated using InAs/AlGaAs quantum-dot technology, demonstrates normal incidence infrared detection in the 2−6 μm range. The detection is optically triggered by a 590 nm light-emitting diode. Furthermore, the detection gain is achieved in our device without an increase of the noise level. The novel characteristics of OTIPs open up new possibilities for third generation infrared imaging systems

Optically triggered infrared photodetector

We demonstrate a new class of semiconductor device: the optically triggered infrared photodetector (OTIP). This photodetector is based on a new physical principle that allows the detection of infrared light to be switched ON and OFF by means of an external light. Our experimental device, fabricated using InAs/AlGaAs quantum-dot technology, demonstrates normal incidence infrared detection in the 2−6 μm range. The detection is optically triggered by a 590 nm light-emitting diode. Furthermore, the detection gain is achieved in our device without an increase of the noise level. The novel characteristics of OTIPs open up new possibilities for third generation infrared imaging systems

Infrared microspectroscopy of inflammatory process and colon tumors.

According to the last global burden of disease published by the World Health Organization, tumors were the third leading cause of death worldwide in 2004. Among the different types of tumors, colorectal cancer ranks as the fourth most lethal. To date, tumor diagnosis is based mainly on the identification of morphological changes in tissues. Considering that these changes appears after many biochemical reactions, the development of vibrational techniques may contribute to the early detection of tumors, since they are able to detect such reactions. The present study aimed to develop a methodology based on infrared microspectroscopy to characterize colon samples, providing complementary information to the pathologist and facilitating the early diagnosis of tumors. The study groups were composed by human colon samples obtained from paraffin-embedded biopsies. The groups are divided in normal (n=20), inflammation (n=17) and tumor (n=18). Two adjacent slices were acquired from each block. The first one was subjected to chemical dewaxing and H&E staining. The infrared imaging was performed on the second slice, which was not dewaxed or stained. A computational preprocessing methodology was employed to identify the paraffin in the images and to perform spectral baseline correction. Such methodology was adapted to include two types of spectral quality control. Afterwards the preprocessing step, spectra belonging to the same image were analyzed and grouped according to their biochemical similarities. One pathologist associated each obtained group with some histological structure based on the H&E stained slice. Such analysis highlighted the biochemical differences between the three studied groups. Results showed that severe inflammation presents biochemical features similar to the tumors ones, indicating that tumors can develop from inflammatory process. A spectral database was constructed containing the biochemical information identified in the previous step. Spectra obtained from new samples were confronted with the database information, leading to their classification into one of the three groups: normal, inflammation or tumor. Internal and external validation were performed based on the classification sensitivity, specificity and accuracy. Comparison between the classification results and H&E stained sections revealed some discrepancies. Some regions histologically normal were identified as inflammation by the classification algorithm. Similarly, some regions presenting inflammatory lesions in the stained section were classified into the tumor group. Such differences were considered as misclassification, but they may actually evidence that biochemical changes are in course in the analyzed sample. In the latter case, the method developed throughout this thesis would have proved able to identify early stages of inflammatory and tumor lesions. It is necessary to perform additional experiments to elucidate this discrepancy between the classification results and the morphological features. One solution would be the use of immunohistochemistry techniques with specific markers for tumor and inflammation. Another option includes the recovering of the medical records of patients who participated in this study in order to check, in later times to the biopsy collection, whether they actually developed the lesions supposedly detected in this research.

Atmospheric infrared sounder (AIRS) : science and measurement requirements.

"D-6665."

Thermal imaging as a biometrics approach to facial signature authentication

This dissertation develops an image processing framework with unique feature extraction and similarity measurements for human face recognition in the thermal mid-wave infrared portion of the electromagnetic spectrum. The goals of this research is to design specialized algorithms that would extract facial vasculature information, create a thermal facial signature and identify the individual. The objective is to use such findings in support of a biometrics system for human identification with a high degree of accuracy and a high degree of reliability. This last assertion is due to the minimal to no risk for potential alteration of the intrinsic physiological characteristics seen through thermal infrared imaging. The proposed thermal facial signature recognition is fully integrated and consolidates the main and critical steps of feature extraction, registration, matching through similarity measures, and validation through testing our algorithm on a database, referred to as C-X1, provided by the Computer Vision Research Laboratory at the University of Notre Dame. Feature extraction was accomplished by first registering the infrared images to a reference image using the functional MRI of the Brain’s (FMRIB’s) Linear Image Registration Tool (FLIRT) modified to suit thermal infrared images. This was followed by segmentation of the facial region using an advanced localized contouring algorithm applied on anisotropically diffused thermal images. Thermal feature extraction from facial images was attained by performing morphological operations such as opening and top-hat segmentation to yield thermal signatures for each subject. Four thermal images taken over a period of six months were used to generate thermal signatures and a thermal template for each subject, the thermal template contains only the most prevalent and consistent features. Finally a similarity measure technique was used to match signatures to templates and the Principal Component Analysis (PCA) was used to validate the results of the matching process. Thirteen subjects were used for testing the developed technique on an in-house thermal imaging system. The matching using an Euclidean-based similarity measure showed 88% accuracy in the case of skeletonized signatures and templates, we obtained 90% accuracy for anisotropically diffused signatures and templates. We also employed the Manhattan-based similarity measure and obtained an accuracy of 90.39% for skeletonized and diffused templates and signatures. It was found that an average 18.9% improvement in the similarity measure was obtained when using diffused templates. The Euclidean- and Manhattan-based similarity measure was also applied to skeletonized signatures and templates of 25 subjects in the C-X1 database. The highly accurate results obtained in the matching process along with the generalized design process clearly demonstrate the ability of the thermal infrared system to be used on other thermal imaging based systems and related databases. A novel user-initialization registration of thermal facial images has been successfully implemented. Furthermore, the novel approach at developing a thermal signature template using four images taken at various times ensured that unforeseen changes in the vasculature did not affect the biometric matching process as it relied on consistent thermal features.

Datasets for the evaluation of multi-sensor perception in natural environments with challenging conditions

This paper presents large, accurately calibrated and time-synchronised datasets, gathered outdoors in controlled environmental conditions, using an unmanned ground vehicle (UGV), equipped with a wide variety of sensors. It discusses how the data collection process was designed, the conditions in which these datasets have been gathered, and some possible outcomes of their exploitation, in particular for the evaluation of performance of sensors and perception algorithms for UGVs.

Combining multiple sensor modalities for a localisation robust to smoke

This paper proposes an approach to obtain a localisation that is robust to smoke by exploiting multiple sensing modalities: visual and infrared (IR) cameras. This localisation is based on a state-of-the-art visual SLAM algorithm. First, we show that a reasonably accurate localisation can be obtained in the presence of smoke by using only an IR camera, a sensor that is hardly affected by smoke, contrary to a visual camera (operating in the visible spectrum). Second, we demonstrate that improved results can be obtained by combining the information from the two sensor modalities (visual and IR cameras). Third, we show that by detecting the impact of smoke on the visual images using a data quality metric, we can anticipate and mitigate the degradation in performance of the localisation by discarding the most affected data. The experimental validation presents multiple trajectories estimated by the various methods considered, all thoroughly compared to an accurate dGPS/INS reference.

Heated Nebulizer Microchips for Mass Spectrometry

Miniaturized analytical devices, such as heated nebulizer (HN) microchips studied in this work, are of increasing interest owing to benefits like faster operation, better performance, and lower cost relative to conventional systems. HN microchips are microfabricated devices that vaporize liquid and mix it with gas. They are used with low liquid flow rates, typically a few µL/min, and have previously been utilized as ion sources for mass spectrometry (MS). Conventional ion sources are seldom feasible at such low flow rates. In this work HN chips were developed further and new applications were introduced. First, a new method for thermal and fluidic characterization of the HN microchips was developed and used to study the chips. Thermal behavior of the chips was also studied by temperature measurements and infrared imaging. An HN chip was applied to the analysis of crude oil – an extremely complex sample – by microchip atmospheric pressure photoionization (APPI) high resolution mass spectrometry. With the chip, the sample flow rate could be reduced significantly without loss of performance and with greatly reduced contamination of the MS instrument. Thanks to its suitability to high temperature, microchip APPI provided efficient vaporization of nonvolatile compounds in crude oil. The first microchip version of sonic spray ionization (SSI) was presented. Ionization was achieved by applying only high (sonic) speed nebulizer gas to an HN microchip. SSI significantly broadens the range of analytes ionizable with the HN chips, from small stable molecules to labile biomolecules. The analytical performance of the microchip SSI source was confirmed to be acceptable. The HN microchips were also used to connect gas chromatography (GC) and capillary liquid chromatography (LC) to MS, using APPI for ionization. Microchip APPI allows efficient ionization of both polar and nonpolar compounds whereas with the most popular electrospray ionization (ESI) only polar and ionic molecules are ionized efficiently. The combination of GC with MS showed that, with HN microchips, GCs can easily be used with MS instruments designed for LC-MS. The presented analytical methods showed good performance. The first integrated LC–HN microchip was developed and presented. In a single microdevice, there were structures for a packed LC column and a heated nebulizer. Nonpolar and polar analytes were efficiently ionized by APPI. Ionization of nonpolar and polar analytes is not possible with previously presented chips for LC–MS since they rely on ESI. Preliminary quantitative performance of the new chip was evaluated and the chip was also demonstrated with optical detection. A new ambient ionization technique for mass spectrometry, desorption atmospheric pressure photoionization (DAPPI), was presented. The DAPPI technique is based on an HN microchip providing desorption of analytes from a surface. Photons from a photoionization lamp ionize the analytes via gas-phase chemical reactions, and the ions are directed into an MS. Rapid analysis of pharmaceuticals from tablets was successfully demonstrated as an application of DAPPI.