Handheld 3D thermography using range sensing and computer vision

This thesis developed a method for real-time and handheld 3D temperature mapping using a combination of off-the-shelf devices and efficient computer algorithms. It contributes a new sensing and data processing framework to the science of 3D thermography, unlocking its potential for application areas such as building energy auditing and industrial monitoring. New techniques for the precise calibration of multi-sensor configurations were developed, along with several algorithms that ensure both accurate and comprehensive surface temperature estimates can be made for rich 3D models as they are generated by a non-expert user.

Tracking people in 3D using position, size and shape

This paper presents a prototype tracking system for tracking people in enclosed indoor environments where there is a high rate of occlusions. The system uses a stereo camera for acquisition, and is capable of disambiguating occlusions using a combination of depth map analysis, a two step ellipse fitting people detection process, the use of motion models and Kalman filters and a novel fit metric, based on computationally simple object statistics. Testing shows that our fit metric outperforms commonly used position based metrics and histogram based metrics, resulting in more accurate tracking of people.

Developing a vision for an nD modelling tool

The design of a building is a complicated process, having to formulate diverse components through unique tasks involving different personalities and organisations in order to satisfy multi-faceted client requirements. To do this successfully, the project team must encapsulate an integrated design that accommodates various social, economic and legislative factors. Therefore, in this era of increasing global competition integrated design has been increasingly recognised as a solution to deliver value to clients.----- The ‘From 3D to nD modelling’ project at the University of Salford aims to support integrated design; to enable and equip the design and construction industry with a tool that allows users to create, share, contemplate and apply knowledge from multiple perspectives of user requirements (accessibility, maintainability, sustainability, acoustics, crime, energy simulation, scheduling, costing etc.). Thus taking the concept of 3-dimensional computer modelling of the built environment to an almost infinite number of dimensions, to cope with whole-life construction and asset management issues in the design of modern buildings. This paper reports on the development of a vision for how integrated environments that will allow nD-enabled construction and asset management to be undertaken. The project is funded by a four-year platform grant from the Engineering and Physical Sciences Research Council (EPSRC) in the UK; thus awarded to a multi-disciplinary research team, to enable flexibility in the research strategy and to produce leading innovation. This paper reports on the development of a business process and IT vision for how integrated environments will allow nD-enabled construction and asset management to be undertaken. It further develops many of the key issues of a future vision arising from previous CIB W78 conferences.

Measuring Visual Consistency in 3D Rendering Systems

One of the major challenges facing a present day game development company is the removal of bugs from such complex virtual environments. This work presents an approach for measuring the correctness of synthetic scenes generated by a rendering system of a 3D application, such as a computer game. Our approach builds a database of labelled point clouds representing the spatiotemporal colour distribution for the objects present in a sequence of bug-free frames. This is done by converting the position that the pixels take over time into the 3D equivalent points with associated colours. Once the space of labelled points is built, each new image produced from the same game by any rendering system can be analysed by measuring its visual inconsistency in terms of distance from the database. Objects within the scene can be relocated (manually or by the application engine); yet the algorithm is able to perform the image analysis in terms of the 3D structure and colour distribution of samples on the surface of the object. We applied our framework to the publicly available game RacingGame developed for Microsoft(R) Xna(R). Preliminary results show how this approach can be used to detect a variety of visual artifacts generated by the rendering system in a professional quality game engine.

High dynamic range stereo vision for outdoor mobile robotics

We present a technique for high-dynamic range stereo for outdoor mobile robot applications. Stereo pairs are captured at a number of different exposures (exposure bracketing), and combined by projecting the 3D points into a common coordinate frame, and building a 3D occupancy map. We present experimental results for static scenes with constant and dynamic lighting as well as outdoor operation with variable and high contrast lighting conditions.

Vision only pose estimation and scene reconstruction on airborne platforms

We aim to demonstrate unaided visual 3D pose estimation and map reconstruction using both monocular and stereo vision techniques. To date, our work has focused on collecting data from Unmanned Aerial Vehicles, which generates a number of significant issues specific to the application. Such issues include scene reconstruction degeneracy from planar data, poor structure initialisation for monocular schemes and difficult 3D reconstruction due to high feature covariance. Most modern Visual Odometry (VO) and related SLAM systems make use of a number of sensors to inform pose and map generation, including laser range-finders, radar, inertial units and vision [1]. By fusing sensor inputs, the advantages and deficiencies of each sensor type can be handled in an efficient manner. However, many of these sensors are costly and each adds to the complexity of such robotic systems. With continual advances in the abilities, small size, passivity and low cost of visual sensors along with the dense, information rich data that they provide our research focuses on the use of unaided vision to generate pose estimates and maps from robotic platforms. We propose that highly accurate (�5cm) dense 3D reconstructions of large scale environments can be obtained in addition to the localisation of the platform described in other work [2]. Using images taken from cameras, our algorithm simultaneously generates an initial visual odometry estimate and scene reconstruction from visible features, then passes this estimate to a bundle-adjustment routine to optimise the solution. From this optimised scene structure and the original images, we aim to create a detailed, textured reconstruction of the scene. By applying such techniques to a unique airborne scenario, we hope to expose new robotic applications of SLAM techniques. The ability to obtain highly accurate 3D measurements of an environment at a low cost is critical in a number of agricultural and urban monitoring situations. We focus on cameras as such sensors are small, cheap and light-weight and can therefore be deployed in smaller aerial vehicles. This, coupled with the ability of small aerial vehicles to fly near to the ground in a controlled fashion, will assist in increasing the effective resolution of the reconstructed maps.

Construction 2020 - A vision for Australia's Property and Construction Industry

Construction 2020 is a national initiative undertaken by CRC for Construction Innovation to focus its ongoing leadership of the Australian property and construction industry in applied research and best contribute to the industry's national and international growth and competitiveness. It is the first major report on the long-term outlook for the industry since the late 1990s. The report identifies nine key themes for the future of the property and construction industry. These visions describe the major concerns of the industry and the improved future working environment favoured by its stakeholders. The first and clearest vision, agreed across the industry, is that environmentally sustainable construction the creation of buildings and infrastructure that minimise their impact on the natural environment is an area of huge potential. Here technologies like Construction Innovation's LCADesign can make a big difference. This is a calculator that works out automatically from 3D computer-aided design the environmental costs of materials in a building all at the push of a button. By working with industry, we'd expect to have a comprehensive set of eco-design tools for all stages of the construction life cycle, to minimise energy use, greenhouse and other forms of waste or pollution. Other significant areas of focus in the report include the development of nationally uniform codes of practice, new tools to evaluate design and product performance, comparisons with overseas industries, and a worldwide research network to ensure that Australian technology is at the cutting edge.

Frequency decomposition techniques for increased discriminative 3D facial information capture

Eigen-based techniques and other monolithic approaches to face recognition have long been a cornerstone in the face recognition community due to the high dimensionality of face images. Eigen-face techniques provide minimal reconstruction error and limit high-frequency content while linear discriminant-based techniques (fisher-faces) allow the construction of subspaces which preserve discriminatory information. This paper presents a frequency decomposition approach for improved face recognition performance utilising three well-known techniques: Wavelets; Gabor / Log-Gabor; and the Discrete Cosine Transform. Experimentation illustrates that frequency domain partitioning prior to dimensionality reduction increases the information available for classification and greatly increases face recognition performance for both eigen-face and fisher-face approaches.

PTZ camera pose estimation by tracking a 3D target

We present a technique for estimating the 6DOF pose of a PTZ camera by tracking a single moving target in the image with known 3D position. This is useful in situations where it is not practical to measure the camera pose directly. Our application domain is estimating the pose of a PTZ camerso so that it can be used for automated GPS-based tracking and filming of UAV flight trials. We present results which show the technique is able to localize a PTZ after a short vision-tracked flight, and that the estimated pose is sufficiently accurate for the PTZ to then actively track a UAV based on GPS position data.

3D ellipsoid fitting for multi-view gait recognition

Gait recognition approaches continue to struggle with challenges including view-invariance, low-resolution data, robustness to unconstrained environments, and fluctuating gait patterns due to subjects carrying goods or wearing different clothes. Although computationally expensive, model based techniques offer promise over appearance based techniques for these challenges as they gather gait features and interpret gait dynamics in skeleton form. In this paper, we propose a fast 3D ellipsoidal-based gait recognition algorithm using a 3D voxel model derived from multi-view silhouette images. This approach directly solves the limitations of view dependency and self-occlusion in existing ellipse fitting model-based approaches. Voxel models are segmented into four components (left and right legs, above and below the knee), and ellipsoids are fitted to each region using eigenvalue decomposition. Features derived from the ellipsoid parameters are modeled using a Fourier representation to retain the temporal dynamic pattern for classification. We demonstrate the proposed approach using the CMU MoBo database and show that an improvement of 15-20% can be achieved over a 2D ellipse fitting baseline.

Large scale monocular vision-only mapping from a fixed-wing sUAS

This paper presents the application of a monocular visual SLAMon a fixed-wing small Unmanned Aerial System (sUAS) capable of simultaneous estimation of aircraft pose and scene structure. We demonstrate the robustness of unconstrained vision alone in producing reliable pose estimates of a sUAS, at altitude. It is ultimately capable of online state estimation feedback for aircraft control and next-best-view estimation for complete map coverage without the use of additional sensors.We explore some of the challenges of visual SLAM from a sUAS including dealing with planar structure, distant scenes and noisy observations. The developed techniques are applied on vision data gathered from a fast-moving fixed-wing radio control aircraft flown over a 1×1km rural area at an altitude of 20-100m.We present both raw Structure from Motion results and a SLAM solution that includes FAB-MAP based loop-closures and graph-optimised pose. Timing information is also presented to demonstrate near online capabilities. We compare the accuracy of the 6-DOF pose estimates to an off-the-shelfGPS aided INS over a 1.7kmtrajectory.We also present output 3D reconstructions of the observed scene structure and texture that demonstrates future applications in autonomous monitoring and surveying.

GrabCutSFM : how 3D information improves unsupervised object segmentation

In this paper, we present an unsupervised graph cut based object segmentation method using 3D information provided by Structure from Motion (SFM), called Grab- CutSFM. Rather than focusing on the segmentation problem using a trained model or human intervention, our approach aims to achieve meaningful segmentation autonomously with direct application to vision based robotics. Generally, object (foreground) and background have certain discriminative geometric information in 3D space. By exploring the 3D information from multiple views, our proposed method can segment potential objects correctly and automatically compared to conventional unsupervised segmentation using only 2D visual cues. Experiments with real video data collected from indoor and outdoor environments verify the proposed approach.