Live 3D shape reconstruction, recognition and registration

We present a video-based system which interactively captures the geometry of a 3D object in the form of a point cloud, then recognizes and registers known objects in this point cloud in a matter of seconds (fig. 1). In order to achieve interactive speed, we exploit both efficient inference algorithms and parallel computation, often on a GPU. The system can be broken down into two distinct phases: geometry capture, and object inference. We now discuss these in further detail. © 2011 IEEE.

A new distance for scale-invariant 3D shape recognition and registration

This paper presents a method for vote-based 3D shape recognition and registration, in particular using mean shift on 3D pose votes in the space of direct similarity transforms for the first time. We introduce a new distance between poses in this spacethe SRT distance. It is left-invariant, unlike Euclidean distance, and has a unique, closed-form mean, in contrast to Riemannian distance, so is fast to compute. We demonstrate improved performance over the state of the art in both recognition and registration on a real and challenging dataset, by comparing our distance with others in a mean shift framework, as well as with the commonly used Hough voting approach. © 2011 IEEE.

Molecular docking and 3D-QSAR study of pyranmycin derivatives against 16S rRNA A site

To understand pharmacophore properties of pyranmycin derivatives and to design novel inhibitors of 16S rRNA A site, comparative molecular field analysis (CoMFA) approach was applied to analyze three-dimensional quantitative structure-activity relationship (3D-QSAR) of 17 compounds. AutoDock 3.0.5 program was employed to locate the orientations and conformations of the inhibitors interacting with 16S rRNA A site. The interaction mode was demonstrated in the aspects of inhibitor conformation, hydrogen bonding and electrostatic interaction. Similar binding conformations of these inhibitors and good correlations between the calculated binding free energies and experimental biological activities suggest that the binding conformations of these inhibitors derived from docking procedure were reasonable. Robust and predictive 3D-QSAR model was obtained by CoMFA with q(2) values of 0.723 and 0.993 for cross-validated and noncross-validated, respectively. The 3D-QSAR model built here will provide clear guidelines for novel inhibitors design based on the Pyranmycin derivatives against 16S rRNA A site. (c) 2005 Elsevier B.V. All rights reserved.

Shifting atomic patterns: on the origin of the different atomic-scale patterns of graphite as observed using scanning tunnelling microscopy.

We present an in-depth study of the myriad atomically resolved patterns observed on graphite using the scanning tunnelling microscope (STM) over the past three decades. Through the use of highly resolved atomic resolution images, we demonstrate how the interactions between the different graphene layers comprising graphite affect the local surface atomic charge density and its resulting symmetry orientation, with particular emphasis on interactions that are thermodynamically unstable. Moreover, the interlayer graphene coupling is controlled experimentally by varying the tip-surface interaction, leading to associated changes in the atomic patterns. The images are corroborated by first-principles calculations, further validating our claim that surface graphene displacement, coming both from lateral and vertical displacement of the top graphene layer, forms the basis of the rich variety of atomic patterns observed in STM experiments on graphite.

A method to introduce accommodation cues into 3D integral imaging system using Fresnel holograms

Conventional 3D Integral imaging suffers from limited image depth range due to the fixed distance between the display panel and the lens array, while digital Fresnel holography suffers from a narrow viewing angle due to the lack of a high resolution spatial light modulator. This paper proposes an original system which combines the advantages of these two techniques to provide an integral imaging system of a reasonable viewing angle with accommodation cues. © 2012 Copyright Society of Photo-Optical Instrumentation Engineers (SPIE).

Hydrothermally-grown ZnO nanowire tips for scanning tunnelling microscopy.

The probe tip is pivotal in determining the resolution and nature of features observed in the Scanning Tunnelling Microscope (STM). We have augmented a conventional Pt/Ir metallic tip with a hydrothermally grown ZnO nanowire (NW). Atomic resolution imaging of graphite is attained. Current-voltage (IV) characteristics demonstrate an asymmetry stemming from the unintentional n-type doping of the ZnO NW, whereas the expected Schottky barrier at the ZnO-Pt/Ir interface is shown to have negligible effect. Moreover the photoconductivity of the system is investigated, paving the way towards a photodetector capable of atomic resolution.

Silhouette-based object phenotype recognition using 3D shape priors

This paper tackles the novel challenging problem of 3D object phenotype recognition from a single 2D silhouette. To bridge the large pose (articulation or deformation) and camera viewpoint changes between the gallery images and query image, we propose a novel probabilistic inference algorithm based on 3D shape priors. Our approach combines both generative and discriminative learning. We use latent probabilistic generative models to capture 3D shape and pose variations from a set of 3D mesh models. Based on these 3D shape priors, we generate a large number of projections for different phenotype classes, poses, and camera viewpoints, and implement Random Forests to efficiently solve the shape and pose inference problems. By model selection in terms of the silhouette coherency between the query and the projections of 3D shapes synthesized using the galleries, we achieve the phenotype recognition result as well as a fast approximate 3D reconstruction of the query. To verify the efficacy of the proposed approach, we present new datasets which contain over 500 images of various human and shark phenotypes and motions. The experimental results clearly show the benefits of using the 3D priors in the proposed method over previous 2D-based methods. © 2011 IEEE.

The influence of matrix integrity on stress-fiber remodeling in 3D.

Matrix anisotropy is important for long term in vivo functionality. However, it is not fully understood how to guide matrix anisotropy in vitro. Experiments suggest actin-mediated cell traction contributes. Although F-actin in 2D displays a stretch-avoidance response, 3D data are lacking. We questioned how cyclic stretch influences F-actin and collagen orientation in 3D. Small-scale cell-populated fibrous tissues were statically constrained and/or cyclically stretched with or without biochemical agents. A rectangular array of silicone posts attached to flexible membranes constrained a mixture of cells, collagen I and matrigel. F-actin orientation was quantified using fiber-tracking software, fitted using a bi-model distribution function. F-actin was biaxially distributed with static constraint. Surprisingly, uniaxial cyclic stretch, only induced a strong stretch-avoidance response (alignment perpendicular to stretching) at tissue surfaces and not in the core. Surface alignment was absent when a ROCK-inhibitor was added, but also when tissues were only statically constrained. Stretch-avoidance was also observed in the tissue core upon MMP1-induced matrix perturbation. Further, a strong stretch-avoidance response was obtained for F-actin and collagen, for immediate cyclic stretching, i.e. stretching before polymerization of the collagen. Results suggest that F-actin stress-fibers avoid cyclic stretch in 3D, unless collagen contact guidance dictates otherwise.

Progressive 3D reconstruction of infrastructure with videogrammetry

A number of methods are commonly used today to collect infrastructure's spatial data (time-of-flight, visual triangulation, etc.). However, current practice lacks a solution that is accurate, automatic, and cost-efficient at the same time. This paper presents a videogrammetric framework for acquiring spatial data of infrastructure which holds the promise to address this limitation. It uses a calibrated set of low-cost high resolution video cameras that is progressively traversed around the scene and aims to produce a dense 3D point cloud which is updated in each frame. It allows for progressive reconstruction as opposed to point-and-shoot followed by point cloud stitching. The feasibility of the framework is studied in this paper. Required steps through this process are presented and the unique challenges of each step are identified. Results specific to each step are also presented.

Automated sparse 3D point cloud generation of infrastructure using its distinctive visual features

The commercial far-range (>10 m) spatial data collection methods for acquiring infrastructure’s geometric data are not completely automated because of the necessary manual pre- and/or post-processing work. The required amount of human intervention and, in some cases, the high equipment costs associated with these methods impede their adoption by the majority of infrastructure mapping activities. This paper presents an automated stereo vision-based method, as an alternative and inexpensive solution, to producing a sparse Euclidean 3D point cloud of an infrastructure scene utilizing two video streams captured by a set of two calibrated cameras. In this process SURF features are automatically detected and matched between each pair of stereo video frames. 3D coordinates of the matched feature points are then calculated via triangulation. The detected SURF features in two successive video frames are automatically matched and the RANSAC algorithm is used to discard mismatches. The quaternion motion estimation method is then used along with bundle adjustment optimization to register successive point clouds. The method was tested on a database of infrastructure stereo video streams. The validity and statistical significance of the results were evaluated by comparing the spatial distance of randomly selected feature points with their corresponding tape measurements.

Toward automated generation of parametric BIMs based on hybrid video and laser scanning data

Only very few constructed facilities today have a complete record of as-built information. Despite the growing use of Building Information Modelling and the improvement in as-built records, several more years will be required before guidelines that require as-built data modelling will be implemented for the majority of constructed facilities, and this will still not address the stock of existing buildings. A technical solution for scanning buildings and compiling Building Information Models is needed. However, this is a multidisciplinary problem, requiring expertise in scanning, computer vision and videogrammetry, machine learning, and parametric object modelling. This paper outlines the technical approach proposed by a consortium of researchers that has gathered to tackle the ambitious goal of automating as-built modelling as far as possible. The top level framework of the proposed solution is presented, and each process, input and output is explained, along with the steps needed to validate them. Preliminary experiments on the earlier stages (i.e. processes) of the framework proposed are conducted and results are shown; the work toward implementation of the remainder is ongoing.

Automated 3D Vision Tracking for Project Control Support

On-site tracking in open construction sites is often difficult because of the large amounts of items that are present and need to be tracked. Additionally, the amounts of occlusions/obstructions present create a highly complex tracking environment. Existing tracking methods are based mainly on Radio Frequency technologies, including Global Positioning Systems (GPS), Radio Frequency Identification (RFID), Bluetooth and Wireless Fidelity (Wi-Fi, Ultra-Wideband, etc). These methods require considerable amounts of pre-processing time since they need to manually deploy tags and keep record of the items they are placed on. In construction sites with numerous entities, tags installation, maintenance and decommissioning become an issue since it increases the cost and time needed to implement these tracking methods. This paper presents a novel method for open site tracking with construction cameras based on machine vision. According to this method, video feed is collected from on site video cameras, and the user selects the entity he wishes to track. The entity is tracked in each video using 2D vision tracking. Epipolar geometry is then used to calculate the depth of the marked area to provide the 3D location of the entity. This method addresses the limitations of radio frequency methods by being unobtrusive and using inexpensive, and easy to deploy equipment. The method has been implemented in a C++ prototype and preliminary results indicate its effectiveness