Segmentation and recognition using structure from motion point clouds

We propose an algorithm for semantic segmentation based on 3D point clouds derived from ego-motion. We motivate five simple cues designed to model specific patterns of motion and 3D world structure that vary with object category. We introduce features that project the 3D cues back to the 2D image plane while modeling spatial layout and context. A randomized decision forest combines many such features to achieve a coherent 2D segmentation and recognize the object categories present. Our main contribution is to show how semantic segmentation is possible based solely on motion-derived 3D world structure. Our method works well on sparse, noisy point clouds, and unlike existing approaches, does not need appearance-based descriptors. Experiments were performed on a challenging new video database containing sequences filmed from a moving car in daylight and at dusk. The results confirm that indeed, accurate segmentation and recognition are possible using only motion and 3D world structure. Further, we show that the motion-derived information complements an existing state-of-the-art appearance-based method, improving both qualitative and quantitative performance. © 2008 Springer Berlin Heidelberg.

Scale-invariant vote-based 3D recognition and registration from point clouds

This chapter presents a method for vote-based 3D shape recognition and registration, in particular using mean shift on 3D pose votes in the space of direct similarity transformations for the first time. We introduce a new distance between poses in this spacethe SRT distance. It is left-invariant, unlike Euclidean distance, and has a unique, closed-form mean, in contrast to Riemannian distance, so is fast to compute. We demonstrate improved performance over the state of the art in both recognition and registration on a (real and) challenging dataset, by comparing our distance with others in a mean shift framework, as well as with the commonly used Hough voting approach. © 2013 Springer-Verlag Berlin Heidelberg.

Scaling the iHMM: Parallelization versus Hadoop

This paper compares parallel and distributed implementations of an iterative, Gibbs sampling, machine learning algorithm. Distributed implementations run under Hadoop on facility computing clouds. The probabilistic model under study is the infinite HMM [1], in which parameters are learnt using an instance blocked Gibbs sampling, with a step consisting of a dynamic program. We apply this model to learn part-of-speech tags from newswire text in an unsupervised fashion. However our focus here is on runtime performance, as opposed to NLP-relevant scores, embodied by iteration duration, ease of development, deployment and debugging. © 2010 IEEE.

Automated sparse 3D point cloud generation of infrastructure using its distinctive visual features

The commercial far-range (>10 m) spatial data collection methods for acquiring infrastructure’s geometric data are not completely automated because of the necessary manual pre- and/or post-processing work. The required amount of human intervention and, in some cases, the high equipment costs associated with these methods impede their adoption by the majority of infrastructure mapping activities. This paper presents an automated stereo vision-based method, as an alternative and inexpensive solution, to producing a sparse Euclidean 3D point cloud of an infrastructure scene utilizing two video streams captured by a set of two calibrated cameras. In this process SURF features are automatically detected and matched between each pair of stereo video frames. 3D coordinates of the matched feature points are then calculated via triangulation. The detected SURF features in two successive video frames are automatically matched and the RANSAC algorithm is used to discard mismatches. The quaternion motion estimation method is then used along with bundle adjustment optimization to register successive point clouds. The method was tested on a database of infrastructure stereo video streams. The validity and statistical significance of the results were evaluated by comparing the spatial distance of randomly selected feature points with their corresponding tape measurements.

Optimized selection of key frames for monocular videogrammetric surveying of civil infrastructure

Videogrammetry is an inexpensive and easy-to-use technology for spatial 3D scene recovery. When applied to large scale civil infrastructure scenes, only a small percentage of the collected video frames are required to achieve robust results. However, choosing the right frames requires careful consideration. Videotaping a built infrastructure scene results in large video files filled with blurry, noisy, or redundant frames. This is due to frame rate to camera speed ratios that are often higher than necessary; camera and lens imperfections and limitations that result in imaging noise; and occasional jerky motions of the camera that result in motion blur; all of which can significantly affect the performance of the videogrammetric pipeline. To tackle these issues, this paper proposes a novel method for automating the selection of an optimized number of informative, high quality frames. According to this method, as the first step, blurred frames are removed using the thresholds determined based on a minimum level of frame quality required to obtain robust results. Then, an optimum number of key frames are selected from the remaining frames using the selection criteria devised by the authors. Experimental results show that the proposed method outperforms existing methods in terms of improved 3D reconstruction results, while maintaining the optimum number of extracted frames needed to generate high quality 3D point clouds.© 2012 Elsevier Ltd. All rights reserved.

Classical plume theory: 1937-2010 and beyond

Developing a theoretical description of turbulent plumes, the likes of which may be seen rising above industrial chimneys, is a daunting thought. Plumes are ubiquitous on a wide range of scales in both the natural and the man-made environments. Examples that immediately come to mind are the vapour plumes above industrial smoke stacks or the ash plumes forming particle-laden clouds above an erupting volcano. However, plumes also occur where they are less visually apparent, such as the rising stream of warmair above a domestic radiator, of oil from a subsea blowout or, at a larger scale, of air above the so-called urban heat island. In many instances, not only the plume itself is of interest but also its influence on the environment as a whole through the process of entrainment. Zeldovich (1937, The asymptotic laws of freely-ascending convective flows. Zh. Eksp. Teor. Fiz., 7, 1463-1465 (in Russian)), Batchelor (1954, Heat convection and buoyancy effects in fluids. Q. J. R. Meteor. Soc., 80, 339-358) and Morton et al. (1956, Turbulent gravitational convection from maintained and instantaneous sources. Proc. R. Soc. Lond. A, 234, 1-23) laid the foundations for classical plume theory, a theoretical description that is elegant in its simplicity and yet encapsulates the complex turbulent engulfment of ambient fluid into the plume. Testament to the insight and approach developed in these early models of plumes is that the essential theory remains unchanged and is widely applied today. We describe the foundations of plume theory and link the theoretical developments with the measurements made in experiments necessary to close these models before discussing some recent developments in plume theory, including an approach which generalizes results obtained separately for the Boussinesq and the non-Boussinesq plume cases. The theory presented - despite its simplicity - has been very successful at describing and explaining the behaviour of plumes across the wide range of scales they are observed. We present solutions to the coupled set of ordinary differential equations (the plume conservation equations) that Morton et al. (1956) derived from the Navier-Stokes equations which govern fluid motion. In order to describe and contrast the bulk behaviour of rising plumes from general area sources, we present closed-form solutions to the plume conservation equations that were achieved by solving for the variation with height of Morton's non-dimensional flux parameter Γ - this single flux parameter gives a unique representation of the behaviour of steady plumes and enables a characterization of the different types of plume. We discuss advantages of solutions in this form before describing extensions to plume theory and suggesting directions for new research. © 2010 The Author. Published by Oxford University Press on behalf of the Institute of Mathematics and its Applications. All rights reserved.

The evolution of a stratified turbulent cloud

Localized regions of turbulence, or turbulent clouds, in a stratified fluid are the subject of this study, which focuses on the edge dynamics occurring between the turbulence and the surrounding quiescent region. Through laboratory experiments and numerical simulations of stratified turbulent clouds, we confirm that the edge dynamics can be subdivided into materially driven intrusions and horizontally travelling internal wave-packets. Three-dimensional visualizations show that the internal gravity wave-packets are in fact large-scale pancake structures that grow out of the turbulent cloud into the adjacent quiescent region. The wave-packets were tracked in time, and it is found that their speed obeys the group speed relation for linear internal gravity waves. The energetics of the propagating waves, which include waveforms that are inclined with respect to the horizontal, are also considered and it is found that, after a period of two eddy turnover times, the internal gravity waves carry up to 16 % of the cloud kinetic energy into the initially quiescent region. Turbulent events in nature are often in the form of decaying turbulent clouds, and it is therefore suggested that internal gravity waves radiated from an initial cloud could play a significant role in the reorganization of energy and momentum in the atmosphere and oceans.©2013 Cambridge University Press.