An exploration of feature detector performance in the thermal-infrared modality

Thermal-infrared images have superior statistical properties compared with visible-spectrum images in many low-light or no-light scenarios. However, a detailed understanding of feature detector performance in the thermal modality lags behind that of the visible modality. To address this, the first comprehensive study on feature detector performance on thermal-infrared images is conducted. A dataset is presented which explores a total of ten different environments with a range of statistical properties. An investigation is conducted into the effects of several digital and physical image transformations on detector repeatability in these environments. The effect of non-uniformity noise, unique to the thermal modality, is analyzed. The accumulation of sensor non-uniformities beyond the minimum possible level was found to have only a small negative effect. A limiting of feature counts was found to improve the repeatability performance of several detectors. Most other image transformations had predictable effects on feature stability. The best-performing detector varied considerably depending on the nature of the scene and the test.

A mask-based approach for the geometric calibration of thermal-infrared cameras

Accurate and efficient thermal-infrared (IR) camera calibration is important for advancing computer vision research within the thermal modality. This paper presents an approach for geometrically calibrating individual and multiple cameras in both the thermal and visible modalities. The proposed technique can be used to correct for lens distortion and to simultaneously reference both visible and thermal-IR cameras to a single coordinate frame. The most popular existing approach for the geometric calibration of thermal cameras uses a printed chessboard heated by a flood lamp and is comparatively inaccurate and difficult to execute. Additionally, software toolkits provided for calibration either are unsuitable for this task or require substantial manual intervention. A new geometric mask with high thermal contrast and not requiring a flood lamp is presented as an alternative calibration pattern. Calibration points on the pattern are then accurately located using a clustering-based algorithm which utilizes the maximally stable extremal region detector. This algorithm is integrated into an automatic end-to-end system for calibrating single or multiple cameras. The evaluation shows that using the proposed mask achieves a mean reprojection error up to 78% lower than that using a heated chessboard. The effectiveness of the approach is further demonstrated by using it to calibrate two multiple-camera multiple-modality setups. Source code and binaries for the developed software are provided on the project Web site.

Hand-held monocular SLAM in thermal-infrared

Thermal-infrared imagery is relatively robust to many of the failure conditions of visual and laser-based SLAM systems, such as fog, dust and smoke. The ability to use thermal-infrared video for localization is therefore highly appealing for many applications. However, operating in thermal-infrared is beyond the capacity of existing SLAM implementations. This paper presents the first known monocular SLAM system designed and tested for hand-held use in the thermal-infrared modality. The implementation includes a flexible feature detection layer able to achieve robust feature tracking in high-noise, low-texture thermal images. A novel approach for structure initialization is also presented. The system is robust to irregular motion and capable of handling the unique mechanical shutter interruptions common to thermal-infrared cameras. The evaluation demonstrates promising performance of the algorithm in several environments.

Ad hoc radiometric calibration of a thermal-infrared camera

Many applications can benefit from the accurate surface temperature estimates that can be made using a passive thermal-infrared camera. However, the process of radiometric calibration which enables this can be both expensive and time consuming. An ad hoc approach for performing radiometric calibration is proposed which does not require specialized equipment and can be completed in a fraction of the time of the conventional method. The proposed approach utilizes the mechanical properties of the camera to estimate scene temperatures automatically, and uses these target temperatures to model the effect of sensor temperature on the digital output. A comparison with a conventional approach using a blackbody radiation source shows that the accuracy of the method is sufficient for many tasks requiring temperature estimation. Furthermore, a novel visualization method is proposed for displaying the radiometrically calibrated images to human operators. The representation employs an intuitive coloring scheme and allows the viewer to perceive a large variety of temperatures accurately.

Towards robust night and day place recognition using visible and thermal imaging

The chief challenge facing persistent robotic navigation using vision sensors is the recognition of previously visited locations under different lighting and illumination conditions. The majority of successful approaches to outdoor robot navigation use active sensors such as LIDAR, but the associated weight and power draw of these systems makes them unsuitable for widespread deployment on mobile robots. In this paper we investigate methods to combine representations for visible and long-wave infrared (LWIR) thermal images with time information to combat the time-of-day-based limitations of each sensing modality. We calculate appearance-based match likelihoods using the state-of-the-art FAB-MAP [1] algorithm to analyse loop closure detection reliability across different times of day. We present preliminary results on a dataset of 10 successive traverses of a combined urban-parkland environment, recorded in 2-hour intervals from before dawn to after dusk. Improved location recognition throughout an entire day is demonstrated using the combined system compared with methods which use visible or thermal sensing alone.

3D thermal mapping of building interiors using an RGB-D and thermal camera

The building sector is the dominant consumer of energy and therefore a major contributor to anthropomorphic climate change. The rapid generation of photorealistic, 3D environment models with incorporated surface temperature data has the potential to improve thermographic monitoring of building energy efficiency. In pursuit of this goal, we propose a system which combines a range sensor with a thermal-infrared camera. Our proposed system can generate dense 3D models of environments with both appearance and temperature information, and is the first such system to be developed using a low-cost RGB-D camera. The proposed pipeline processes depth maps successively, forming an ongoing pose estimate of the depth camera and optimizing a voxel occupancy map. Voxels are assigned 4 channels representing estimates of their true RGB and thermal-infrared intensity values. Poses corresponding to each RGB and thermal-infrared image are estimated through a combination of timestamp-based interpolation and a pre-determined knowledge of the extrinsic calibration of the system. Raycasting is then used to color the voxels to represent both visual appearance using RGB, and an estimate of the surface temperature. The output of the system is a dense 3D model which can simultaneously represent both RGB and thermal-infrared data using one of two alternative representation schemes. Experimental results demonstrate that the system is capable of accurately mapping difficult environments, even in complete darkness.

Multi-sensor tracking using a scalable condensation filter

Surveillance and tracking systems typically use a single colour modality for their input. These systems work well in controlled conditions but often fail with low lighting, shadowing, smoke, dust, unstable backgrounds or when the foreground object is of similar colouring to the background. With advances in technology and manufacturing techniques, sensors that allow us to see into the thermal infrared spectrum are becoming more affordable. By using modalities from both the visible and thermal infrared spectra, we are able to obtain more information from a scene and overcome the problems associated with using visible light only for surveillance and tracking. Thermal images are not affected by lighting or shadowing and are not overtly affected by smoke, dust or unstable backgrounds. We propose and evaluate three approaches for fusing visual and thermal images for person tracking. We also propose a modified condensation filter to track and aid in the fusion of the modalities. We compare the proposed fusion schemes with using the visual and thermal domains on their own, and demonstrate that significant improvements can be achieved by using multiple modalities.

Multi-spectral fusion for surveillance systems

Surveillance systems such as object tracking and abandoned object detection systems typically rely on a single modality of colour video for their input. These systems work well in controlled conditions but often fail when low lighting, shadowing, smoke, dust or unstable backgrounds are present, or when the objects of interest are a similar colour to the background. Thermal images are not affected by lighting changes or shadowing, and are not overtly affected by smoke, dust or unstable backgrounds. However, thermal images lack colour information which makes distinguishing between different people or objects of interest within the same scene difficult. ----- By using modalities from both the visible and thermal infrared spectra, we are able to obtain more information from a scene and overcome the problems associated with using either modality individually. We evaluate four approaches for fusing visual and thermal images for use in a person tracking system (two early fusion methods, one mid fusion and one late fusion method), in order to determine the most appropriate method for fusing multiple modalities. We also evaluate two of these approaches for use in abandoned object detection, and propose an abandoned object detection routine that utilises multiple modalities. To aid in the tracking and fusion of the modalities we propose a modified condensation filter that can dynamically change the particle count and features used according to the needs of the system. ----- We compare tracking and abandoned object detection performance for the proposed fusion schemes and the visual and thermal domains on their own. Testing is conducted using the OTCBVS database to evaluate object tracking, and data captured in-house to evaluate the abandoned object detection. Our results show that significant improvement can be achieved, and that a middle fusion scheme is most effective.

Real-time mobile 3D temperature mapping

The ability to measure surface temperature and represent it on a metrically accurate 3D model has proven applications in many areas such as medical imaging, building energy auditing, and search and rescue. A system is proposed that enables this task to be performed with a handheld sensor, and for the first time with results able to be visualized and analyzed in real-time. A device comprising a thermal-infrared camera and range sensor is calibrated geometrically and used for data capture. The device is localized using a combination of ICP and video-based pose estimation from the thermal-infrared video footage which is shown to reduce the occurrence of failure modes. Furthermore, the problem of misregistration which can introduce severe distortions in assigned surface temperatures is avoided through the use of a risk-averse neighborhood weighting mechanism. Results demonstrate that the system is more stable and accurate than previous approaches, and can be used to accurately model complex objects and environments for practical tasks.

Handheld 3D thermography using range sensing and computer vision

This thesis developed a method for real-time and handheld 3D temperature mapping using a combination of off-the-shelf devices and efficient computer algorithms. It contributes a new sensing and data processing framework to the science of 3D thermography, unlocking its potential for application areas such as building energy auditing and industrial monitoring. New techniques for the precise calibration of multi-sensor configurations were developed, along with several algorithms that ensure both accurate and comprehensive surface temperature estimates can be made for rich 3D models as they are generated by a non-expert user.

Thermal Treatments of Fe—Mn Alkoxide of Glycerol: Infrared absorption and emission

Synthetic Fe—Mn alkoxide of glycerol samples are submitted to controlled heating conditions and examined by IR absorption spectroscopy. On the other hand, the same sample is studied by infrared emission spectroscopy (IRES), upon heating in situ from 100 to 600°C. The spectral techniques employed in this contribution, especially IRES, show that as a result of the thermal treatments ferromagnetic oxides (manganese ferrite) are formed between 350 and 400°C. Some further spectral changes are seen at higher temperatures.

Thermal treatment of weddellite—a Raman and infrared emission spectroscopic study

Thermal transformations of natural calcium oxalate dihydrate known in mineralogy as weddellite have been undertaken using a combination of Raman microscopy and infrared emission spectroscopy. The vibrational spectroscopic data was complimented with high resolution thermogravimetric analysis combined with evolved gas mass spectrometry. TG–MS identified three mass loss steps at 114, 422 and 592 °C. In the first mass loss step water is evolved only, in the second and third steps carbon dioxide is evolved. The combination of Raman microscopy and a thermal stage clearly identifies the changes in the molecular structure with thermal treatment. Weddellite is the phase in the temperature range up to the pre-dehydration temperature of 97 °C. At this temperature, the phase formed is whewellite (calcium oxalate monohydrate) and above 114 °C the phase is the anhydrous calcium oxalate. Above 422 °C, calcium carbonate is formed. Infrared emission spectroscopy shows that this mineral decomposes at around 650 °C. Changes in the position and intensity of the C=O and C---C stretching vibrations in the Raman spectra indicate the temperature range at which these phase changes occur.

Thermal analysis and infrared emission spectroscopic study of halloysite-potassium acetate intercalation compound

The thermal decomposition of halloysite-potassium acetate intercalation compound was investigated by thermogravimetric analysis and infrared emission spectroscopy. The X-ray diffraction patterns indicated that intercalation of potassium acetate into halloysite caused an increase of the basal spacing from 1.00 to 1.41 nm. The thermogravimetry results show that the mass losses of intercalation the compound occur in main three main steps, which correspond to (a) the loss of adsorbed water (b) the loss of coordination water and (c) the loss of potassium acetate and dehydroxylation. The temperature of dehydroxylation and dehydration of halloysite is decreased about 100 °C. The infrared emission spectra clearly show the decomposition and dehydroxylation of the halloysite intercalation compound when the temperature is raised. The dehydration of the intercalation compound is followed by the loss of intensity of the stretching vibration bands at region 3600-3200 cm-1. Dehydroxylation is followed by the decrease in intensity in the bands between 3695 and 3620 cm-1. Dehydration was completed by 300 °C and partial dehydroxylation by 350 °C. The inner hydroxyl group remained until around 500 °C.

Application of infrared emission spectroscopy to the thermal transition of indium hydroxide to indium oxide nanocubes

Cubic indium hydroxide nanomaterials were obtained by a low temperature soft-chemical method without any surfactants. The transition of nano-cubic indium hydroxide to cubic indium oxide during dehydroxylation has been studied by infrared emission spectroscopy. The spectra are related to the structure of the materials and the changes in the structure upon thermal treatment. The infrared absorption spectrum of In(OH)3 is characterised by an intense OH deformation band at 1150 cm-1 and two O-H stretching bands at 3107 and 3221 cm-1. In the infrared emission spectra, the hydroxyl-stretching and hydroxyl-bending bands diminish dramatically upon heating, and no intensity remains after 200 °C. However, new low intensity bands are found in the OH deformation region at 915 cm-1 and in OH stretching region at 3437 cm-1. These bands are attributed to the vibrations of newly formed InOH bonds because of the release and transfer of protons during calcination of the nanomaterial. The use of infrared emission spectroscopy enables the low-temperature phase transition brought about through dehydration of In(OH)3 nanocubes to be studied.

Thermal analysis and infrared emission spectroscopic study of kaolinite-potassium acetate intercalate complex

The thermal behavior and decomposition of kaolinite-potassium acetate intercalation complex was investigated through a combination of thermogravimetric analysis and infrared emission spectroscopy. Three main changes were observed at 48, 280, 323 and 460 °C which were attributed to (a) the loss of adsorbed water (b) loss of the water coordinated to acetate ion in the layer of kaolinite (c) loss of potassium acetate in the complex and (d) water through dehydroxylation. It is proposed that the KAc intercalation complex is stability except heating at above 300 °C. The infrared emission spectra clearly show the decomposition and dehydroxylation of the kaolinite intercalation complex when the temperature is raised. The dehydration of the intercalation complex is followed by the loss of intensity of the stretching vibration bands at region 3600-3200 cm-1. Dehydroxylation is followed by the decrease in intensity in the bands between 3695 and 3620 cm-1. Dehydration is completed by 400 °C and partial dehydroxylation by 650 °C. The inner hydroxyl group remained until around 700 °C.