3D Hand Pose Reconstruction Using Specialized Mappings

A system for recovering 3D hand pose from monocular color sequences is proposed. The system employs a non-linear supervised learning framework, the specialized mappings architecture (SMA), to map image features to likely 3D hand poses. The SMA's fundamental components are a set of specialized forward mapping functions, and a single feedback matching function. The forward functions are estimated directly from training data, which in our case are examples of hand joint configurations and their corresponding visual features. The joint angle data in the training set is obtained via a CyberGlove, a glove with 22 sensors that monitor the angular motions of the palm and fingers. In training, the visual features are generated using a computer graphics module that renders the hand from arbitrary viewpoints given the 22 joint angles. We test our system both on synthetic sequences and on sequences taken with a color camera. The system automatically detects and tracks both hands of the user, calculates the appropriate features, and estimates the 3D hand joint angles from those features. Results are encouraging given the complexity of the task.

Creating replicas of scultural heritage through 3D reconstruction and low cost 3D printer in Education

The process of making replicas of heritage has traditionally been developed by public agencies, corporations and museums and is not commonly used in schools. Currently there are technologies that allow creating cheap replicas. The new 3D reconstruction software, based on photographs and low cost 3D printers allow to make replicas at a cost much lower than traditional. This article describes the process of creating replicas of the sculpture Goslar Warrior of artist Henry Moore, located in Santa Cruz de Tenerife. To make this process, first, a digital model have been created using Autodesk Recap 360, Autodesk 123D Catch and Autodesk Meshmixer MarkerBot MakerWare applications. Physical replication, has been reproduced in polylactic acid (PLA) by MakerBot Replicator 2 3D printer. In addition, a cost analysis using, in one hand, the printer mentioned, and in the other hand, 3D printing services both online and local, is included. Finally, there has been a specific action with 141 students and 12 high school teachers, who filled a questionnary about the use of sculptural replicas in education.

The heart of a dragon: 3D anatomical reconstruction of the 'scaly-foot gastropod' (Mollusca: Gastropoda: Neomphalina) reveals its extraordinary circulatory system

Introduction: The 'scaly-foot gastropod' (Chrysomallon squamiferum Chen et al., 2015) from deep-sea hydrothermal vent ecosystems of the Indian Ocean is an active mobile gastropod occurring in locally high densities, and it is distinctive for the dermal scales covering the exterior surface of its foot. These iron-sulfide coated sclerites, and its nutritional dependence on endosymbiotic bacteria, are both noted as adaptations to the extreme environment in the flow of hydrogen sulfide. We present evidence for other adaptations of the 'scaly-foot gastropod' to life in an extreme environment, investigated through dissection and 3D tomographic reconstruction of the internal anatomy.

Results: Our anatomical investigations of juvenile and adult specimens reveal a large unganglionated nervous system, a simple and reduced digestive system, and that the animal is a simultaneous hermaphrodite. We show that Chrysomallon squamiferum relies on endosymbiotic bacteria throughout post-larval life. Of particular interest is the circulatory system: Chrysomallon has a very large ctenidium supported by extensive blood sinuses filled with haemocoel. The ctenidium provides oxygen for the host but the circulatory system is enlarged beyond the scope of other similar vent gastropods. At the posterior of the ctenidium is a remarkably large and well-developed heart. Based on the volume of the auricle and ventricle, the heart complex represents approximately 4 % of the body volume. This proportionally giant heart primarily sucks blood through the ctenidium and supplies the highly vascularised oesophageal gland. Thus we infer the elaborate cardiovascular system most likely evolved to oxygenate the endosymbionts in an oxygen poor environment and/or to supply hydrogen sulfide to the endosymbionts.

Conclusions: This study exemplifies how understanding the autecology of an organism can be enhanced by detailed investigation of internal anatomy. This gastropod is a large and active species that is abundant in its hydrothermal vent field ecosystem. Yet all of its remarkable features-protective dermal sclerites, circulatory system, high fecundity-can be viewed as adaptations beneficial to its endosymbiont microbes. We interpret these results to show that, as a result of specialisation to resolve energetic needs in an extreme chemosynthetic environment, this dramatic dragon-like species has become a carrying vessel for its bacteria.

3D Reconstruction of Medical Images from Slices Automatically Landmarked with Growing Neural Models

In this study, we utilise a novel approach to segment out the ventricular system in a series of high resolution T1-weighted MR images. We present a brain ventricles fast reconstruction method. The method is based on the processing of brain sections and establishing a fixed number of landmarks onto those sections to reconstruct the ventricles 3D surface. Automated landmark extraction is accomplished through the use of the self-organising network, the growing neural gas (GNG), which is able to topographically map the low dimensionality of the network to the high dimensionality of the contour manifold without requiring a priori knowledge of the input space structure. Moreover, our GNG landmark method is tolerant to noise and eliminates outliers. Our method accelerates the classical surface reconstruction and filtering processes. The proposed method offers higher accuracy compared to methods with similar efficiency as Voxel Grid.

Multi-view 3D reconstruction using virtual cameras

Ce mémoire s'intéresse à la reconstruction d'un modèle 3D à partir de plusieurs images. Le modèle 3D est élaboré avec une représentation hiérarchique de voxels sous la forme d'un octree. Un cube englobant le modèle 3D est calculé à partir de la position des caméras. Ce cube contient les voxels et il définit la position de caméras virtuelles. Le modèle 3D est initialisé par une enveloppe convexe basée sur la couleur uniforme du fond des images. Cette enveloppe permet de creuser la périphérie du modèle 3D. Ensuite un coût pondéré est calculé pour évaluer la qualité de chaque voxel à faire partie de la surface de l'objet. Ce coût tient compte de la similarité des pixels provenant de chaque image associée à la caméra virtuelle. Finalement et pour chacune des caméras virtuelles, une surface est calculée basée sur le coût en utilisant la méthode de SGM. La méthode SGM tient compte du voisinage lors du calcul de profondeur et ce mémoire présente une variation de la méthode pour tenir compte des voxels précédemment exclus du modèle par l'étape d'initialisation ou de creusage par une autre surface. Par la suite, les surfaces calculées sont utilisées pour creuser et finaliser le modèle 3D. Ce mémoire présente une combinaison innovante d'étapes permettant de créer un modèle 3D basé sur un ensemble d'images existant ou encore sur une suite d'images capturées en série pouvant mener à la création d'un modèle 3D en temps réel.

Personalized 3D reconstruction of the rib cage for clinical assessment of trunk deformities

Scoliosis is a 3D deformity of the spine and rib cage. Extensive validation of 3D reconstruction methods of the spine from biplanar radiography has already been published. In this article, we propose a novel method to reconstruct the rib cage, using the same biplanar views as for the 3D reconstruction of the spine, to allow clinical assessment of whole trunk deformities. This technique uses a semi-automatic segmentation of the ribs in the postero-anterior X-ray view and an interactive segmentation of partial rib edges in the lateral view. The rib midlines are automatically extracted in 2D and reconstructed in 3D using the epipolar geometry. For the ribs not visible in the lateral view, the method predicts their 3D shape. The accuracy of the proposed method has been assessed using data obtained from a synthetic bone model as a gold standard and has also been evaluated using data of real patients with scoliotic deformities. Results show that the reconstructed ribs enable a reliable evaluation of the rib axial rotation, which will allow a 3D clinical assessment of the spine and rib cage deformities.

A Statistical Image-Based Shape Model for Visual Hull Reconstruction and 3D Structure Inference

We present a statistical image-based shape + structure model for Bayesian visual hull reconstruction and 3D structure inference. The 3D shape of a class of objects is represented by sets of contours from silhouette views simultaneously observed from multiple calibrated cameras. Bayesian reconstructions of new shapes are then estimated using a prior density constructed with a mixture model and probabilistic principal components analysis. We show how the use of a class-specific prior in a visual hull reconstruction can reduce the effect of segmentation errors from the silhouette extraction process. The proposed method is applied to a data set of pedestrian images, and improvements in the approximate 3D models under various noise conditions are shown. We further augment the shape model to incorporate structural features of interest; unknown structural parameters for a novel set of contours are then inferred via the Bayesian reconstruction process. Model matching and parameter inference are done entirely in the image domain and require no explicit 3D construction. Our shape model enables accurate estimation of structure despite segmentation errors or missing views in the input silhouettes, and works even with only a single input view. Using a data set of thousands of pedestrian images generated from a synthetic model, we can accurately infer the 3D locations of 19 joints on the body based on observed silhouette contours from real images.

Optimal 3D surface reconstruction from multiview photographic images

This paper describes a new method for reconstructing 3D surface using a small number, e.g. 10, of 2D photographic images. The images are taken at different viewing directions by a perspective camera with full prior knowledge of the camera configurations. The reconstructed object's surface is represented a set of triangular facets. We empirically demonstrate that if the viewing directions are uniformly distributed around the object's viewing sphere, then the reconstructed 3D points optimally cluster closely on a highly curved part of the surface and are widely, spread on smooth or fat parts. The advantage of this property is that the reconstructed points along a surface or a contour generator are not undersampled or underrepresented because surfaces or contours should be sampled or represented with more densely points where their curvatures are high. The more complex the contour's shape, the greater is the number of points required, but the greater the number of points is automatically generated by the proposed method Given that the viewing directions are uniformly distributed, the number and distribution of the reconstructed points depend on the shape or the curvature of the surface regardless of the size of the surface or the size of the object.

3D surface point and wireframe reconstruction from multiview photographic images

This paper describes a new method for reconstructing 3D surface points and a wireframe on the surface of a freeform object using a small number, e.g. 10, of 2D photographic images. The images are taken at different viewing directions by a perspective camera with full prior knowledge of the camera configurations. The reconstructed surface points are frontier points and the wireframe is a network of contour generators. Both of them are reconstructed by pairing apparent contours in the 2D images. Unlike previous works, we empirically demonstrate that if the viewing directions are uniformly distributed around the object's viewing sphere, then the reconstructed 3D points automatically cluster closely on a highly curved part of the surface and are widely spread on smooth or flat parts. The advantage of this property is that the reconstructed points along a surface or a contour generator are not under-sampled or under-represented because surfaces or contours should be sampled or represented with more densely points where their curvatures are high. The more complex the contour's shape, the greater is the number of points required, but the greater the number of points is automatically generated by the proposed method. Given that the viewing directions are uniformly distributed, the number and distribution of the reconstructed points depend on the shape or the curvature of the surface regardless of the size of the surface or the size of the object. The unique pattern of the reconstructed points and contours may be used in 31) object recognition and measurement without computationally intensive full surface reconstruction. The results are obtained from both computer-generated and real objects. (C) 2007 Elsevier B.V. All rights reserved.

Optimal 3D surface reconstruction from a small number of conventional 2D X-ray images

This paper describes a method for reconstructing 3D frontier points, contour generators and surfaces of anatomical objects or smooth surfaces from a small number, e. g. 10, of conventional 2D X-ray images. The X-ray images are taken at different viewing directions with full prior knowledge of the X-ray source and sensor configurations. Unlike previous works, we empirically demonstrate that if the viewing directions are uniformly distributed around the object's viewing sphere, then the reconstructed 3D points automatically cluster closely on a highly curved part of the surface and are widely spread on smooth or flat parts. The advantage of this property is that the reconstructed points along a surface or a contour generator are not under-sampled or under-represented because surfaces or contours should be sampled or represented with more densely points where their curvatures are high. The more complex the contour's shape, the greater is the number of points required, but the greater the number of points is automatically generated by the proposed method. Given that the number of viewing directions is fixed and the viewing directions are uniformly distributed, the number and distribution of the reconstructed points depend on the shape or the curvature of the surface regardless of the size of the surface or the size of the object. The technique may be used not only in medicine but also in industrial applications.

Stereo vision and wavelets/multiwavelets analysis : 3d reconstruction using wavelets/multiwavelets theory and stereo vision

During the image formation process of the camera, explicit 3D information about the scene or objects in the scene are lost. Therefore, 3D structure or depth information has to be inferred implicitly from the 2D intensity images. This problem is com- monly referred to as 3D reconstruction. In this work a complete 3D reconstruction algorithm is presented, capable of reconstructing dimensionally accurate 3D models of the objects, based on stereo vision and multi-resolution analysis. The developed system uses a reference depth model of the objects under observation to improve the disparity maps, estimated. Only a few features are extracted from that reference model, which are the relative location of the discontinuities and the z-dimensional extremes of objects depth. The maximum error deviation of the estimated depth along the surfaces is less than 0.5mm and along the discontinuities is less than 1.5mm. The developed system is invariant to illuminative variations, and orientation, location and scaling of the objects under consideration, which makes the developed system highly robust.