Automated 3D Structure Inference of Civil Infrastructure Using a Stereo Camera Set

The commercial far-range (>10m) infrastructure spatial data collection methods are not completely automated. They need significant amount of manual post-processing work and in some cases, the equipment costs are significant. This paper presents a method that is the first step of a stereo videogrammetric framework and holds the promise to address these issues. Under this method, video streams are initially collected from a calibrated set of two video cameras. For each pair of simultaneous video frames, visual feature points are detected and their spatial coordinates are then computed. The result, in the form of a sparse 3D point cloud, is the basis for the next steps in the framework (i.e., camera motion estimation and dense 3D reconstruction). A set of data, collected from an ongoing infrastructure project, is used to show the merits of the method. Comparison with existing tools is also shown, to indicate the performance differences of the proposed method in the level of automation and the accuracy of results.

Recovering 3D Structure of Poorly Textured Infrastructure Scenes Using Point and Line Features

Most of the existing automated machine vision-based techniques for as-built documentation of civil infrastructure utilize only point features to recover the 3D structure of a scene. However it is often the case in man-made structures that not enough point features can be reliably detected (e.g. buildings and roofs); this can potentially lead to the failure of these techniques. To address the problem, this paper utilizes the prominence of straight lines in infrastructure scenes. It presents a hybrid approach that benefits from both point and line features. A calibrated stereo set of video cameras is used to collect data. Point and line features are then detected and matched across video frames. Finally, the 3D structure of the scene is recovered by finding 3D coordinates of the matched features. The proposed approach has been tested on realistic outdoor environments and preliminary results indicate its capability to deal with a variety of scenes.

Inferring 3D Structure with a Statistical Image-Based Shape Model

We present an image-based approach to infer 3D structure parameters using a probabilistic "shape+structure'' model. The 3D shape of a class of objects may be represented by sets of contours from silhouette views simultaneously observed from multiple calibrated cameras. Bayesian reconstructions of new shapes can then be estimated using a prior density constructed with a mixture model and probabilistic principal components analysis. We augment the shape model to incorporate structural features of interest; novel examples with missing structure parameters may then be reconstructed to obtain estimates of these parameters. Model matching and parameter inference are done entirely in the image domain and require no explicit 3D construction. Our shape model enables accurate estimation of structure despite segmentation errors or missing views in the input silhouettes, and works even with only a single input view. Using a dataset of thousands of pedestrian images generated from a synthetic model, we can perform accurate inference of the 3D locations of 19 joints on the body based on observed silhouette contours from real images.

A Statistical Image-Based Shape Model for Visual Hull Reconstruction and 3D Structure Inference

We present a statistical image-based shape + structure model for Bayesian visual hull reconstruction and 3D structure inference. The 3D shape of a class of objects is represented by sets of contours from silhouette views simultaneously observed from multiple calibrated cameras. Bayesian reconstructions of new shapes are then estimated using a prior density constructed with a mixture model and probabilistic principal components analysis. We show how the use of a class-specific prior in a visual hull reconstruction can reduce the effect of segmentation errors from the silhouette extraction process. The proposed method is applied to a data set of pedestrian images, and improvements in the approximate 3D models under various noise conditions are shown. We further augment the shape model to incorporate structural features of interest; unknown structural parameters for a novel set of contours are then inferred via the Bayesian reconstruction process. Model matching and parameter inference are done entirely in the image domain and require no explicit 3D construction. Our shape model enables accurate estimation of structure despite segmentation errors or missing views in the input silhouettes, and works even with only a single input view. Using a data set of thousands of pedestrian images generated from a synthetic model, we can accurately infer the 3D locations of 19 joints on the body based on observed silhouette contours from real images.

Modeling camera orientation and 3D structure from a sequence of images taken by a perambulating commercial video camera

In this paper we report the degree of reliability of image sequences taken by off-the-shelf TV cameras for modeling camera rotation and reconstructing 3D structure using computer vision techniques. This is done in spite of the fact that computer vision systems usually use imaging devices that are specifically designed for the human vision. Our scenario consists of a static scene and a mobile camera moving through the scene. The scene is any long axial building dominated by features along the three principal orientations and with at least one wall containing prominent repetitive planar features such as doors, windows bricks etc. The camera is an ordinary commercial camcorder moving along the axial axis of the scene and is allowed to rotate freely within the range +/- 10 degrees in all directions. This makes it possible that the camera be held by a walking unprofessional cameraman with normal gait, or to be mounted on a mobile robot. The system has been tested successfully on sequence of images of a variety of structured, but fairly cluttered scenes taken by different walking cameramen. The potential application areas of the system include medicine, robotics and photogrammetry.

The 3D structure and function of digestive cathepsin L-like proteinases of Tenebrio molitor larval midgut

Cathepsin L-like proteinases (CAL) are major digestive proteinases in the beetle Tenebrio molitor. Procathepsin Ls 2 (pCAL2) and 3 (pCAL3) were expressed as recombinant proteins in Escherichia coil, purified and activated under acidic conditions. Immunoblot analyses of different T. molitor larval tissues demonstrated that a polyclonal antibody to pCAL3 recognized pCAL3 and cathepsin L 3 (CAD) only in the anterior two-thirds of midgut tissue and midgut luminal contents of T. molitor larvae. Furthermore, immunocytolocalization data indicated that pCAL3 occurs in secretory vesicles and microvilli in anterior midgut Therefore CAL3, like cathepsin L 2 (CAL2), is a digestive enzyme secreted by T. molitor anterior midgut CAD hydrolyses Z-FR-MCA and Z-RR-MCA (typical cathepsin substrates), whereas CAL2 hydrolyses only Z-FR-MCA. Active site mutants (pCAL2C25S and pCAL3C265) were constructed by replacing the catalytic cysteine with serine to prevent autocatalytic processing. Recombinant pCAL2 and pCAL3 mutants (pCAL2C25S and pCAL3C26S) were prepared, crystallized and their 3D structures determined at 1.85 and 2.1 angstrom, respectively. While the overall structure of these enzymes is similar to other members of the papain superfamily, structural differences in the S2 subsite explain their substrate specificities. The data also supported models for CAL trafficking to lysosomes and to secretory vesicles to be discharged into midgut contents. (C) 2012 Elsevier Ltd. All rights reserved.

Learning 3D structure from 2D images using LBP features

An automatic machine learning strategy for computing the 3D structure of monocular images from a single image query using Local Binary Patterns is presented. The 3D structure is inferred through a training set composed by a repository of color and depth images, assuming that images with similar structure present similar depth maps. Local Binary Patterns are used to characterize the structure of the color images. The depth maps of those color images with a similar structure to the query image are adaptively combined and filtered to estimate the final depth map. Using public databases, promising results have been obtained outperforming other state-of-the-art algorithms and with a computational cost similar to the most efficient 2D-to-3D algorithms.

Predicting function from structure: 3D structure studies of the mammalian golgi complex

3D electron tomography studies of the structure of the mammalian Golgi complex have led to four functional predictions (1). The sorting and exit site from the Golgi comprises two or three distinct trans-cisternae (2). The docking of vesicular-tubular clusters at the cis-face and the fragmentation of trans-cisternae are coordinated (3). The mechanisms of transport through, and exit from, the Golgi vary with physiological state, and in different cells and tissues (4). Specialized trans-ER functions in the delivery of ceramide to sphingomyelin synthase in the trans-Golgi membrane, for the regulated sorting via sphingolipid-cholesterol-rich domains. These structure-based predictions can now be tested using a variety of powerful cell and molecular tools.

Human Ezrin: Prediction of the 3D structure and interactions with mTOR

The protein Ezrin, is a member of the ERM family (Ezrin, Radixin and Moesin) that links the F-actin to the plasma membrane. The protein is made of three domains namely the FERM domain, a central α-helical domain and the CERMAD domain. The residues in Ezrin such as Ser66, Tyr145, Tyr353 and Tyr477 regulate the function of the protein through phosphorylation. The protein is found in two distinct conformations of active and dormant (inactive) state. The initial step during the conformation change is the breakage of intramolecular interaction in dormant Ezrin by phosphorylation of residue Thr567. The dormant structure of human Ezrin was predicted computationally since only partial active form structure was available. The validation analysis showed that 99.7% residues were positioned in favored, allowed and generously allowed regions of the Ramachandran plot. The Z-score of Ezrin was −7.36, G-factor was 0.1, and the QMEAN score of the model was 0.61 indicating a good model for human Ezrin. The comparison of the conformations of the activated and dormant Ezrin showed a major shift in the F2 lobe (residues 142-149 and 161-177) while changes in the conformation induced mobility shifts in lobe F3 (residues 261 to 267). The 3D positions of the phosphorylation sites Tyr145, Tyr353, Tyr477, Tyr482 and Thr567 were also located. Using targeted molecular dynamic simulation, the molecular movements during conformational change from active to dormant were visualized. The dormant Ezrin auto-inhibits itself by a head-to-tail interaction of the N-terminal and C-terminal residues. The trajectory shows the breakage of the interactions and mobility of the CERMAD domain away from the FERM domain. Protein docking and clustering analysis were used to predict the residues involved in the interaction between dormant Ezrin and mTOR. Residues Tyr477 and Tyr482 were found to be involved in interaction with mTOR.

Measuring Visual Consistency in 3D Rendering Systems

One of the major challenges facing a present day game development company is the removal of bugs from such complex virtual environments. This work presents an approach for measuring the correctness of synthetic scenes generated by a rendering system of a 3D application, such as a computer game. Our approach builds a database of labelled point clouds representing the spatiotemporal colour distribution for the objects present in a sequence of bug-free frames. This is done by converting the position that the pixels take over time into the 3D equivalent points with associated colours. Once the space of labelled points is built, each new image produced from the same game by any rendering system can be analysed by measuring its visual inconsistency in terms of distance from the database. Objects within the scene can be relocated (manually or by the application engine); yet the algorithm is able to perform the image analysis in terms of the 3D structure and colour distribution of samples on the surface of the object. We applied our framework to the publicly available game RacingGame developed for Microsoft(R) Xna(R). Preliminary results show how this approach can be used to detect a variety of visual artifacts generated by the rendering system in a professional quality game engine.

Liveness detection based on 3D face shape analysis

In recent years face recognition systems have been applied in various useful applications, such as surveillance, access control, criminal investigations, law enforcement, and others. However face biometric systems can be highly vulnerable to spoofing attacks where an impostor tries to bypass the face recognition system using a photo or video sequence. In this paper a novel liveness detection method, based on the 3D structure of the face, is proposed. Processing the 3D curvature of the acquired data, the proposed approach allows a biometric system to distinguish a real face from a photo, increasing the overall performance of the system and reducing its vulnerability. In order to test the real capability of the methodology a 3D face database has been collected simulating spoofing attacks, therefore using photographs instead of real faces. The experimental results show the effectiveness of the proposed approach.

Efficiently capturing large, complex cultural heritage sites with a handheld mobile 3D laser mapping system

Accurate three-dimensional representations of cultural heritage sites are highly valuable for scientific study, conservation, and educational purposes. In addition to their use for archival purposes, 3D models enable efficient and precise measurement of relevant natural and architectural features. Many cultural heritage sites are large and complex, consisting of multiple structures spatially distributed over tens of thousands of square metres. The process of effectively digitising such geometrically complex locations requires measurements to be acquired from a variety of viewpoints. While several technologies exist for capturing the 3D structure of objects and environments, none are ideally suited to complex, large-scale sites, mainly due to their limited coverage or acquisition efficiency. We explore the use of a recently developed handheld mobile mapping system called Zebedee in cultural heritage applications. The Zebedee system is capable of efficiently mapping an environment in three dimensions by continually acquiring data as an operator holding the device traverses through the site. The system was deployed at the former Peel Island Lazaret, a culturally significant site in Queensland, Australia, consisting of dozens of buildings of various sizes spread across an area of approximately 400 × 250 m. With the Zebedee system, the site was scanned in half a day, and a detailed 3D point cloud model (with over 520 million points) was generated from the 3.6 hours of acquired data in 2.6 hours. We present results demonstrating that Zebedee was able to accurately capture both site context and building detail comparable in accuracy to manual measurement techniques, and at a greatly increased level of efficiency and scope. The scan allowed us to record derelict buildings that previously could not be measured because of the scale and complexity of the site. The resulting 3D model captures both interior and exterior features of buildings, including structure, materials, and the contents of rooms.

Structure of Sesbania mosaic virus at 3 angstrom resolution

Sesbania mosaic virus (SMV) is an isometric, ss-RNA plant virus found infecting Sesbania grandiflora plants in fields near Tirupathi, South India. The virus particles, which sediment at 116 S at pH 5.5, swell upon treatment with EDTA at pH 7.5 resulting in the reduction of the sedimentation coefficient to 108 S. SMV coat protein amino acid sequence was determined and found to have approximately 60% amino acid sequence identity with that of southern bean mosaic virus (SBMV). The amino terminal 60 residue segment, which contains a number of positively charged residues, is less well conserved between SMV and SBMV when compared to the rest of the sequence. The 3D structure of SMV was determined at 3.0 Å resolution by molecular replacement techniques using SBMV structure as the initial phasing model. The icosahedral asymmetric unit was found to contain four calcium ions occurring in inter subunit interfaces and three protein subunits, designated A, B and C. The conformation of the C subunit appears to be different from those of A and B in several segments of the polypeptide. These observations coupled with structural studies on SMV partially depleted of calcium suggest a plausible mechanisms for the initiation of the disassembly of the virus capsid.