Live 3D shape reconstruction, recognition and registration

We present a video-based system which interactively captures the geometry of a 3D object in the form of a point cloud, then recognizes and registers known objects in this point cloud in a matter of seconds (fig. 1). In order to achieve interactive speed, we exploit both efficient inference algorithms and parallel computation, often on a GPU. The system can be broken down into two distinct phases: geometry capture, and object inference. We now discuss these in further detail. © 2011 IEEE.

A new distance for scale-invariant 3D shape recognition and registration

This paper presents a method for vote-based 3D shape recognition and registration, in particular using mean shift on 3D pose votes in the space of direct similarity transforms for the first time. We introduce a new distance between poses in this spacethe SRT distance. It is left-invariant, unlike Euclidean distance, and has a unique, closed-form mean, in contrast to Riemannian distance, so is fast to compute. We demonstrate improved performance over the state of the art in both recognition and registration on a real and challenging dataset, by comparing our distance with others in a mean shift framework, as well as with the commonly used Hough voting approach. © 2011 IEEE.

Molecular docking and 3D-QSAR study of pyranmycin derivatives against 16S rRNA A site

To understand pharmacophore properties of pyranmycin derivatives and to design novel inhibitors of 16S rRNA A site, comparative molecular field analysis (CoMFA) approach was applied to analyze three-dimensional quantitative structure-activity relationship (3D-QSAR) of 17 compounds. AutoDock 3.0.5 program was employed to locate the orientations and conformations of the inhibitors interacting with 16S rRNA A site. The interaction mode was demonstrated in the aspects of inhibitor conformation, hydrogen bonding and electrostatic interaction. Similar binding conformations of these inhibitors and good correlations between the calculated binding free energies and experimental biological activities suggest that the binding conformations of these inhibitors derived from docking procedure were reasonable. Robust and predictive 3D-QSAR model was obtained by CoMFA with q(2) values of 0.723 and 0.993 for cross-validated and noncross-validated, respectively. The 3D-QSAR model built here will provide clear guidelines for novel inhibitors design based on the Pyranmycin derivatives against 16S rRNA A site. (c) 2005 Elsevier B.V. All rights reserved.

A method to introduce accommodation cues into 3D integral imaging system using Fresnel holograms

Conventional 3D Integral imaging suffers from limited image depth range due to the fixed distance between the display panel and the lens array, while digital Fresnel holography suffers from a narrow viewing angle due to the lack of a high resolution spatial light modulator. This paper proposes an original system which combines the advantages of these two techniques to provide an integral imaging system of a reasonable viewing angle with accommodation cues. © 2012 Copyright Society of Photo-Optical Instrumentation Engineers (SPIE).

Silhouette-based object phenotype recognition using 3D shape priors

This paper tackles the novel challenging problem of 3D object phenotype recognition from a single 2D silhouette. To bridge the large pose (articulation or deformation) and camera viewpoint changes between the gallery images and query image, we propose a novel probabilistic inference algorithm based on 3D shape priors. Our approach combines both generative and discriminative learning. We use latent probabilistic generative models to capture 3D shape and pose variations from a set of 3D mesh models. Based on these 3D shape priors, we generate a large number of projections for different phenotype classes, poses, and camera viewpoints, and implement Random Forests to efficiently solve the shape and pose inference problems. By model selection in terms of the silhouette coherency between the query and the projections of 3D shapes synthesized using the galleries, we achieve the phenotype recognition result as well as a fast approximate 3D reconstruction of the query. To verify the efficacy of the proposed approach, we present new datasets which contain over 500 images of various human and shark phenotypes and motions. The experimental results clearly show the benefits of using the 3D priors in the proposed method over previous 2D-based methods. © 2011 IEEE.

The influence of matrix integrity on stress-fiber remodeling in 3D.

Matrix anisotropy is important for long term in vivo functionality. However, it is not fully understood how to guide matrix anisotropy in vitro. Experiments suggest actin-mediated cell traction contributes. Although F-actin in 2D displays a stretch-avoidance response, 3D data are lacking. We questioned how cyclic stretch influences F-actin and collagen orientation in 3D. Small-scale cell-populated fibrous tissues were statically constrained and/or cyclically stretched with or without biochemical agents. A rectangular array of silicone posts attached to flexible membranes constrained a mixture of cells, collagen I and matrigel. F-actin orientation was quantified using fiber-tracking software, fitted using a bi-model distribution function. F-actin was biaxially distributed with static constraint. Surprisingly, uniaxial cyclic stretch, only induced a strong stretch-avoidance response (alignment perpendicular to stretching) at tissue surfaces and not in the core. Surface alignment was absent when a ROCK-inhibitor was added, but also when tissues were only statically constrained. Stretch-avoidance was also observed in the tissue core upon MMP1-induced matrix perturbation. Further, a strong stretch-avoidance response was obtained for F-actin and collagen, for immediate cyclic stretching, i.e. stretching before polymerization of the collagen. Results suggest that F-actin stress-fibers avoid cyclic stretch in 3D, unless collagen contact guidance dictates otherwise.

Progressive 3D reconstruction of infrastructure with videogrammetry

A number of methods are commonly used today to collect infrastructure's spatial data (time-of-flight, visual triangulation, etc.). However, current practice lacks a solution that is accurate, automatic, and cost-efficient at the same time. This paper presents a videogrammetric framework for acquiring spatial data of infrastructure which holds the promise to address this limitation. It uses a calibrated set of low-cost high resolution video cameras that is progressively traversed around the scene and aims to produce a dense 3D point cloud which is updated in each frame. It allows for progressive reconstruction as opposed to point-and-shoot followed by point cloud stitching. The feasibility of the framework is studied in this paper. Required steps through this process are presented and the unique challenges of each step are identified. Results specific to each step are also presented.

Automated sparse 3D point cloud generation of infrastructure using its distinctive visual features

The commercial far-range (>10 m) spatial data collection methods for acquiring infrastructure’s geometric data are not completely automated because of the necessary manual pre- and/or post-processing work. The required amount of human intervention and, in some cases, the high equipment costs associated with these methods impede their adoption by the majority of infrastructure mapping activities. This paper presents an automated stereo vision-based method, as an alternative and inexpensive solution, to producing a sparse Euclidean 3D point cloud of an infrastructure scene utilizing two video streams captured by a set of two calibrated cameras. In this process SURF features are automatically detected and matched between each pair of stereo video frames. 3D coordinates of the matched feature points are then calculated via triangulation. The detected SURF features in two successive video frames are automatically matched and the RANSAC algorithm is used to discard mismatches. The quaternion motion estimation method is then used along with bundle adjustment optimization to register successive point clouds. The method was tested on a database of infrastructure stereo video streams. The validity and statistical significance of the results were evaluated by comparing the spatial distance of randomly selected feature points with their corresponding tape measurements.

Automated 3D Vision Tracking for Project Control Support

On-site tracking in open construction sites is often difficult because of the large amounts of items that are present and need to be tracked. Additionally, the amounts of occlusions/obstructions present create a highly complex tracking environment. Existing tracking methods are based mainly on Radio Frequency technologies, including Global Positioning Systems (GPS), Radio Frequency Identification (RFID), Bluetooth and Wireless Fidelity (Wi-Fi, Ultra-Wideband, etc). These methods require considerable amounts of pre-processing time since they need to manually deploy tags and keep record of the items they are placed on. In construction sites with numerous entities, tags installation, maintenance and decommissioning become an issue since it increases the cost and time needed to implement these tracking methods. This paper presents a novel method for open site tracking with construction cameras based on machine vision. According to this method, video feed is collected from on site video cameras, and the user selects the entity he wishes to track. The entity is tracked in each video using 2D vision tracking. Epipolar geometry is then used to calculate the depth of the marked area to provide the 3D location of the entity. This method addresses the limitations of radio frequency methods by being unobtrusive and using inexpensive, and easy to deploy equipment. The method has been implemented in a C++ prototype and preliminary results indicate its effectiveness

Correlating Multiple 2D Vision Trackers for 3D Object Tracking on Construction Sites

Tracking methods have the potential to retrieve the spatial location of project related entities such as personnel and equipment at construction sites, which can facilitate several construction management tasks. Existing tracking methods are mainly based on Radio Frequency (RF) technologies and thus require manual deployment of tags. On construction sites with numerous entities, tags installation, maintenance and decommissioning become an issue since it increases the cost and time needed to implement these tracking methods. To address these limitations, this paper proposes an alternate 3D tracking method based on vision. It operates by tracking the designated object in 2D video frames and correlating the tracking results from multiple pre-calibrated views using epipolar geometry. The methodology presented in this paper has been implemented and tested on videos taken in controlled experimental conditions. Results are compared with the actual 3D positions to validate its performance.

The influence of matrix integrity on stress-fiber remodeling in 3D

Matrix anisotropy is important for long term in vivo functionality. However, it is not fully understood how to guide matrix anisotropy in vitro. Experiments suggest actin-mediated cell traction contributes. Although F-actin in 2D displays a stretch-avoidance response, 3D data are lacking. We questioned how cyclic stretch influences F-actin and collagen orientation in 3D. Small-scale cell-populated fibrous tissues were statically constrained and/or cyclically stretched with or without biochemical agents. A rectangular array of silicone posts attached to flexible membranes constrained a mixture of cells, collagen I and matrigel. F-actin orientation was quantified using fiber-tracking software, fitted using a bi-model distribution function. F-actin was biaxially distributed with static constraint. Surprisingly, uniaxial cyclic stretch, only induced a strong stretch-avoidance response (alignment perpendicular to stretching) at tissue surfaces and not in the core. Surface alignment was absent when a ROCK-inhibitor was added, but also when tissues were only statically constrained. Stretch-avoidance was also observed in the tissue core upon MMP1-induced matrix perturbation. Further, a strong stretch-avoidance response was obtained for F-actin and collagen, for immediate cyclic stretching, i.e. stretching before polymerization of the collagen. Results suggest that F-actin stress-fibers avoid cyclic stretch in 3D, unless collagen contact guidance dictates otherwise. © 2012 Elsevier Ltd.