Generating the Sparse Point Cloud of a Civil Infrastructure Scene Using a Single Video Camera under Practical Constraints

Automating the model generation process of infrastructure can substantially reduce the modeling time and cost. This paper presents a method to generate a sparse point cloud of an infrastructure scene using a single video camera under practical constraints. It is the first step towards establishing an automatic framework for object-oriented as-built modeling. Motion blur and key frame selection criteria are considered. Structure from motion and bundle adjustment are explored. The method is demonstrated in a case study where the scene of a reinforced concrete bridge is videotaped, reconstructed, and metrically validated. The result indicates the applicability, efficiency, and accuracy of the proposed method.

Three-Dimensional Tracking of Construction Resources using an On-Site Camera System

Vision trackers have been proposed as a promising alternative for tracking at large-scale, congested construction sites. They provide the location of a large number of entities in a camera view across frames. However, vision trackers provide only two-dimensional (2D) pixel coordinates, which are not adequate for construction applications. This paper proposes and validates a method that overcomes this limitation by employing stereo cameras and converting 2D pixel coordinates to three-dimensional (3D) metric coordinates. The proposed method consists of four steps: camera calibration, camera pose estimation, 2D tracking, and triangulation. Given that the method employs fixed, calibrated stereo cameras with a long baseline, appropriate algorithms are selected for each step. Once the first two steps reveal camera system parameters, the third step determines 2D pixel coordinates of entities in subsequent frames. The 2D coordinates are triangulated on the basis of the camera system parameters to obtain 3D coordinates. The methodology presented in this paper has been implemented and tested with data collected from a construction site. The results demonstrate the suitability of this method for on-site tracking purposes.

Augmenting depth camera output using photometric stereo

We present a system for augmenting depth camera output using multispectral photometric stereo. The technique is demonstrated using a Kinect sensor and is able to produce geometry independently for each frame. Improved reconstruction is demonstrated using the Kinect's inbuilt RGB camera and further improvements are achieved by introducing an additional high resolution camera. As well as qualitative improvements in reconstruction a quantitative reduction in temporal noise is shown. As part of the system an approach is presented for relaxing the assumption of multispectral photometric stereo that scenes are of constant chromaticity to the assumption that scenes contain multiple piecewise constant chromaticities.

Single camera pose estimation using Bayesian filtering and Kinect motion priors

Traditional approaches to upper body pose estimation using monocular vision rely on complex body models and a large variety of geometric constraints. We argue that this is not ideal and somewhat inelegant as it results in large processing burdens, and instead attempt to incorporate these constraints through priors obtained directly from training data. A prior distribution covering the probability of a human pose occurring is used to incorporate likely human poses. This distribution is obtained offline, by fitting a Gaussian mixture model to a large dataset of recorded human body poses, tracked using a Kinect sensor. We combine this prior information with a random walk transition model to obtain an upper body model, suitable for use within a recursive Bayesian filtering framework. Our model can be viewed as a mixture of discrete Ornstein-Uhlenbeck processes, in that states behave as random walks, but drift towards a set of typically observed poses. This model is combined with measurements of the human head and hand positions, using recursive Bayesian estimation to incorporate temporal information. Measurements are obtained using face detection and a simple skin colour hand detector, trained using the detected face. The suggested model is designed with analytical tractability in mind and we show that the pose tracking can be Rao-Blackwellised using the mixture Kalman filter, allowing for computational efficiency while still incorporating bio-mechanical properties of the upper body. In addition, the use of the proposed upper body model allows reliable three-dimensional pose estimates to be obtained indirectly for a number of joints that are often difficult to detect using traditional object recognition strategies. Comparisons with Kinect sensor results and the state of the art in 2D pose estimation highlight the efficacy of the proposed approach.

Convective heat dissipation with lattice-frame materials

This paper presents experimental results on heat transfer and pressure drop for a compact heat sink made of fully triangulated, lightweight (porosity∼0.938), aluminum lattice-frame materials (LFMs). Due to the inherent structural anisotropy of the LFMs, two mutually perpendicular orientations were selected for the measurements. Constant heat flux was applied to the heat sink under steady state conditions, and dissipated by forced air convection. The experimental data were compared with those predicted from an analytical model based on fin analogy. The experimental results revealed that pressure drop is strongly dependent upon the orientation of the structure, due mainly to the flow blockage effect. For heat transfer measurements, typical local temperature distributions on the substrate under constant heat flux conditions were captured with infrared camera. The thermal behavior of LFMs was found to follow closely that of cylinder banks, with early transition Reynolds number (based on strut diameter) equal to about 300. The Nusselt number prediction from the fin-analogy correlates well with experimental measurements, except at low Reynolds numbers where a slightly underestimation is observed. Comparisons with empty channels and commonly used heat exchanger media show that the present LFM heat sink can remove heat approximately seven times more efficient than an empty channel and as efficient as a bank of cylinders at the same porosity level. The aluminum LFMs are extremely stiff and strong, making them ideal candidates for multifunctional structures requiring both heat dissipation and mechanical load carrying capabilities. © 2003 Elsevier Ltd. All rights reserved.

Coarse-to-fine vision-based localization by indexing scale-invariant features

This paper presents a novel coarse-to-fine global localization approach inspired by object recognition and text retrieval techniques. Harris-Laplace interest points characterized by scale-invariant transformation feature descriptors are used as natural landmarks. They are indexed into two databases: a location vector space model (LVSM) and a location database. The localization process consists of two stages: coarse localization and fine localization. Coarse localization from the LVSM is fast, but not accurate enough, whereas localization from the location database using a voting algorithm is relatively slow, but more accurate. The integration of coarse and fine stages makes fast and reliable localization possible. If necessary, the localization result can be verified by epipolar geometry between the representative view in the database and the view to be localized. In addition, the localization system recovers the position of the camera by essential matrix decomposition. The localization system has been tested in indoor and outdoor environments. The results show that our approach is efficient and reliable. © 2006 IEEE.

Image-based localization and pose recovery using scale invariant features

In this paper, we propose a vision based mobile robot localization strategy. Local scale-invariant features are used as natural landmarks in unstructured and unmodified environment. The local characteristics of the features we use prove to be robust to occlusion and outliers. In addition, the invariance of the features to viewpoint change makes them suitable landmarks for mobile robot localization. Scale-invariant features detected in the first exploration are indexed into a location database. Indexing and voting allow efficient recognition of global localization. The localization result is verified by epipolar geometry between the representative view in database and the view to be localized, thus the probability of false localization will be decreased. The localization system can recover the pose of the camera mounted on the robot by essential matrix decomposition. Then the position of the robot can be computed easily. Both calibrated and un-calibrated cases are discussed and relative position estimation based on calibrated camera turns out to be the better choice. Experimental results show that our approach is effective and reliable in the case of illumination changes, similarity transformations and extraneous features. © 2004 IEEE.

A novel non-intrusive sampling technique for CO2 laser on-line beam monitoring utilising a silicon mirror

The measurement of high speed laser beam parameters during processing is a topic that has seen growing attention over the last few years as quality assurance places greater demand on the monitoring of the manufacturing process. The targets for any monitoring system is to be non-intrusive, low cost, simple to operate, high speed and capable of operation in process. A new ISO compliant system is presented based on the integration of an imaging plate and camera located behind a proprietary mirror sampling device. The general layout of the device is presented along with the thermal and optical performance of the sampling optic. Diagnostic performance of the system is compared with industry standard devices, demonstrating the high quality high speed data which has been generated using this system.

Spray visualisation and acoustic analysis in a gas-turbine type burner

This paper presents the characterisation of self-excited oscillations in a kerosene burner. The combustion instability exhibits two different modes and frequencies depending on the air flow rate. Experimental results reveal the influence of the spray to shift between these two modes. Pressure and heat release fluctuations have been measured simultaneously and the flame transfer function has been calculated from these measurements. The Mie scattering technique has been used to record spray fluctuations in reacting conditions with a high speed camera. Innovative image processing has enabled us to obtain fluctuations of the Mie scattered light from the spray as a temporal signal acquired simultaneously with pressure fluctuations. This has been used to determine a transfer function relating the image intensity and hence the spray fluctuations to changes in air velocity. This function has identified the different role the spray plays in the two modes of instability. At low air flow rates, the spray responds to an unsteady air flow rate and the time varying spray characteristics lead to unsteady combustion. At higher air flow rates, effective evaporation means that the spray dynamics are less important, leading to a different flame transfer function and frequency of self-excited oscillation. In conclusion, the combustion instabilities observed are closely related with the fluctuations of the spray motion and evaporation.

Reconstruction of surfaces of revolution from single uncalibrated views

This paper addresses the problem of recovering the 3D shape of a surface of revolution from a single uncalibrated perspective view. The algorithm introduced here makes use of the invariant properties of a surface of revolution and its silhouette to locate the image of the revolution axis, and to calibrate the focal length of the camera. The image is then normalized and rectified such that the resulting silhouette exhibits bilateral symmetry. Such a rectification leads to a simpler differential analysis of the silhouette, and yields a simple equation for depth recovery. It is shown that under a general camera configuration, there will be a 2-parameter family of solutions for the reconstruction. The first parameter corresponds to an unknown scale, whereas the second one corresponds to an unknown attitude of the object. By identifying the image of a latitude circle, the ambiguity due to the unknown attitude can be resolved. Experimental results on real images are presented, which demonstrate the quality of the reconstruction. © 2004 Elsevier B.V. All rights reserved.

Modelling of waterfront cantilever sheet pile walls subjected to earthquake loading

The seismic performance of waterfront cantilever sheet pile retaining walls is of continuing interest to geotechnical engineers as these structures suffer severe damage and even complete failure during earthquakes. This is often precipitated by liquefaction of the surrounding soil, either in the backfill or in front of the wall. This paper presents results from a series of small-scale plane strain models that were tested on a 1-g shaking table and recorded using a high-speed, high-resolution digital camera. The technique of Particle Image Velocimetry (PIV) was applied in order to allow the failure mechanisms to be visualised. It is shown that using PIV analyses it is possible to obtain failure mechanisms for a cantilever wall in liquefiable soil. These failure mechanisms are compared with those obtained for a cantilever wall in dry soil, previously carried out at a similar scale. It was observed that seismic liquefaction causes significant displacement in much larger zones of soil near the retaining wall compared to an equivalent dry case. The failure mechanism for a cantilever wall with liquefiable backfill, but with a remediated zone designed not to liquefy, is also presented and compared to the unremediated case.

A closely integrated digital image capture device for embedded applications

A modular image capture system with close integration to CCD cameras has been developed. The aim is to produce a system capable of integrating CCD sensor, image capture and image processing into a single compact unit. This close integration provides a direct mapping between CCD pixels and digital image pixels. The system has been interfaced to a digital signal processor board for the development and control of image processing tasks. These have included characterization and enhancement of noisy images from an intensified camera and measurement to subpixel resolutions. A highly compact form of the image capture system is in an advanced stage of development. This consists of a single FPGA device and a single VRAM providing a two chip image capturing system capable of being integrated into a CCD camera. A miniature compact PC has been developed using a novel modular interconnection technique, providing a processing unit in a three dimensional format highly suited to integration into a CCD camera unit. Work is under way to interface the compact capture system to the PC using this interconnection technique, combining CCD sensor, image capture and image processing into a single compact unit. ©2005 Copyright SPIE - The International Society for Optical Engineering.

Fourier analysis and gabor filtering for texture analysis and local reconstruction of general shapes

Since the pioneering work of Gibson in 1950, Shape- From-Texture has been considered by researchers as a hard problem, mainly due to restrictive assumptions which often limit its applicability. We assume a very general stochastic homogeneity and perspective camera model, for both deterministic and stochastic textures. A multi-scale distortion is efficiently estimated with a previously presented method based on Fourier analysis and Gabor filters. The novel 3D reconstruction method that we propose applies to general shapes, and includes non-developable and extensive surfaces. Our algorithm is accurate, robust and compares favorably to the present state of the art of Shape-From- Texture. Results show its application to non-invasively study shape changes with laid-on textures, while rendering and retexturing of cloth is suggested for future work. © 2009 IEEE.

A vision-based remote control

This Chapter presents a vision-based system for touch-free interaction with a display at a distance. A single camera is fixed on top of the screen and is pointing towards the user. An attention mechanism allows the user to start the interaction and control a screen pointer by moving their hand in a fist pose directed at the camera. On-screen items can be chosen by a selection mechanism. Current sample applications include browsing video collections as well as viewing a gallery of 3D objects, which the user can rotate with their hand motion. We have included an up-to-date review of hand tracking methods, and comment on the merits and shortcomings of previous approaches. The proposed tracker uses multiple cues, appearance, color, and motion, for robustness. As the space of possible observation models is generally too large for exhaustive online search, we select models that are suitable for the particular tracking task at hand. During a training stage, various off-the-shelf trackers are evaluated. From this data differentmethods of fusing them online are investigated, including parallel and cascaded tracker evaluation. For the case of fist tracking, combining a small number of observers in a cascade results in an efficient algorithm that is used in our gesture interface. The system has been on public display at conferences where over a hundred users have engaged with it. © 2010 Springer-Verlag Berlin Heidelberg.