3D facial merging for virtual human reconstruction

There is an increasing need of easy and affordable technologies to automatically generate virtual 3D models from their real counterparts. In particular, 3D human reconstruction has driven the creation of many clever techniques, most of them based on the visual hull (VH) concept. Such techniques do not require expensive hardware; however, they tend to yield 3D humanoids with realistic bodies but mediocre faces, since VH cannot handle concavities. On the other hand, structured light projectors allow to capture very accurate depth data, and thus to reconstruct realistic faces, but they are too expensive to use several of them. We have developed a technique to merge a VH-based 3D mesh of a reconstructed humanoid and the depth data of its face, captured by a single structured light projector. By combining the advantages of both systems in a simple setting, we are able to reconstruct realistic 3D human models with believable faces.

3D Maize Plant Reconstruction Based on Georeferenced Overlapping LiDAR Point Clouds

3D crop reconstruction with a high temporal resolution and by the use of non-destructive measuring technologies can support the automation of plant phenotyping processes. Thereby, the availability of such 3D data can give valuable information about the plant development and the interaction of the plant genotype with the environment. This article presents a new methodology for georeferenced 3D reconstruction of maize plant structure. For this purpose a total station, an IMU, and several 2D LiDARs with different orientations were mounted on an autonomous vehicle. By the multistep methodology presented, based on the application of the ICP algorithm for point cloud fusion, it was possible to perform the georeferenced point clouds overlapping. The overlapping point cloud algorithm showed that the aerial points (corresponding mainly to plant parts) were reduced to 1.5%–9% of the total registered data. The remaining were redundant or ground points. Through the inclusion of different LiDAR point of views of the scene, a more realistic representation of the surrounding is obtained by the incorporation of new useful information but also of noise. The use of georeferenced 3D maize plant reconstruction at different growth stages, combined with the total station accuracy could be highly useful when performing precision agriculture at the crop plant level.

3D reconstruction of medical images from slices automatically landmarked with growing neural models

In this study, we utilise a novel approach to segment out the ventricular system in a series of high resolution T1-weighted MR images. We present a brain ventricles fast reconstruction method. The method is based on the processing of brain sections and establishing a fixed number of landmarks onto those sections to reconstruct the ventricles 3D surface. Automated landmark extraction is accomplished through the use of the self-organising network, the growing neural gas (GNG), which is able to topographically map the low dimensionality of the network to the high dimensionality of the contour manifold without requiring a priori knowledge of the input space structure. Moreover, our GNG landmark method is tolerant to noise and eliminates outliers. Our method accelerates the classical surface reconstruction and filtering processes. The proposed method offers higher accuracy compared to methods with similar efficiency as Voxel Grid.

Improving the Accuracy of 3-D Reconstruction in Robotic Vision Applications

Thesis (Ph.D.)--University of Washington, 2016-08

From point cloud to HBIM: an integrated workflow for 3D reconstruction of heritage building with model accuracy evaluation

Modern society is now facing significant difficulties in attempting to preserve its architectural heritage. Numerous challenges arise consequently when it comes to documentation, preservation and restoration. Fortunately, new perspectives on architectural heritage are emerging owing to the rapid development of digitalization. Therefore, this presents new challenges for architects, restorers and specialists. Additionally, this has changed the way they approach the study of existing heritage, changing from conventional 2D drawings in response to the increasing requirement for 3D representations. Recently, Building Information Modelling for historic buildings (HBIM) has escalated as an emerging trend to interconnect geometrical and informational data. Currently, the latest 3D geomatics techniques based on 3D laser scanners with enhanced photogrammetry along with the continuous improvement in the BIM industry allow for an enhanced 3D digital reconstruction of historical and existing buildings. This research study aimed to develop an integrated workflow for the 3D digital reconstruction of heritage buildings starting from a point cloud. The Pieve of San Michele in Acerboli’s Church in Santarcangelo Di Romagna (6th century) served as the test bed. The point cloud was utilized as an essential referential to model the BIM geometry using Autodesk Revit® 2022. To validate the accuracy of the model, Deviation Analysis Method was employed using CloudCompare software to determine the degree of deviation between the HBIM model and the point cloud. The acquired findings showed a very promising outcome in the average distance between the HBIM model and the point cloud. The conducted approach in this study demonstrated the viability of producing a precise BIM geometry from point clouds.

Embedded Voxel Colouring

The reconstruction of a complex scene from multiple images is a fundamental problem in the field of computer vision. Volumetric methods have proven to be a strong alternative to traditional correspondence-based methods due to their flexible visibility models. In this paper we analyse existing methods for volumetric reconstruction and identify three key properties of voxel colouring algorithms: a water-tight surface model, a monotonic carving order, and causality. We present a new Voxel Colouring algorithm which embeds all reconstructions of a scene into a single output. While modelling exact visibility for arbitrary camera locations, Embedded Voxel Colouring removes the need for a priori threshold selection present in previous work. An efficient implementation is given along with results demonstrating the advantages of posteriori threshold selection.

Dynamic three-dimensional reconstruction of the heart by transesophageal echocardiography

OBJECTIVE: To evaluate echocardiography accuracy in performing and obtaining images for dynamical three-dimensional (3D) reconstruction. METHODS: Three-dimensional (3D) image reconstruction was obtained in 20 consecutive patients who underwent transesophageal echocardiography. A multiplanar 5 MHz transducer was used for 3D reconstruction. RESULTS: Twenty patients were studied consecutively. The following cardiac diseases were present: valvar prostheses-6 (2 mitral, 2 aortic and 2 mitral and aortic); mitral valve prolapse- 3; mitral and aortic disease - 2; aortic valve disease- 5; congenital heart disease- 3 (2 atrial septal defect- ASD - and 1 transposition of the great arteries -TGA); arteriovenous fistula- 1. In 7 patients, color Doppler was also obtained and used for 3D flow reconstruction. Twenty five cardiac structures were acquired and 60 reconstructions generated (28 of mitral valves, 14 of aortic valves, 4 of mitral prostheses, 7 of aortic prostheses and 7 of the ASD). Fifty five of 60 (91.6%) reconstructions were considered of good quality by 2 independent observers. The 11 reconstructed mitral valves/prostheses and the 2 reconstructed ASDs provided more anatomical information than two dimensional echocardiography (2DE) alone. CONCLUSION: 3D echocardiography using a transesophageal transducer is a feasible technique, which improves detection of anatomical details of cardiac structures, particularly of the mitral valve and atrial septum.

Estimating root–soil contact from 3D X-ray microtomographs

Adequate contact with the soil is essential for water and nutrient adsorption by plant roots, but the determination of root–soil contact is a challenging task because it is difficult to visualize roots in situ and quantify their interactions with the soil at the scale of micrometres. A method to determine root–soil contact using X-ray microtomography was developed. Contact areas were determined from 3D volumetric images using segmentation and iso-surface determination tools. The accuracy of the method was tested with physical model systems of contact between two objects (phantoms). Volumes, surface areas and contact areas calculated from the measured phantoms were compared with those estimated from image analysis. The volume was accurate to within 0.3%, the surface area to within 2–4%, and the contact area to within 2.5%. Maize and lupin roots were grown in soil (<2 mm) and vermiculite at matric potentials of −0.03 and −1.6 MPa and in aggregate fractions of 4–2, 2–1, 1–0.5 and < 0.5 mm at a matric potential of −0.03 MPa. The contact of the roots with their growth medium was determined from 3D volumetric images. Macroporosity (>70 µm) of the soil sieved to different aggregate fractions was calculated from binarized data. Root-soil contact was greater in soil than in vermiculite and increased with decreasing aggregate or particle size. The differences in root–soil contact could not be explained solely by the decrease in porosity with decreasing aggregate size but may also result from changes in particle and aggregate packing around the root.

3D face computational photography using PCA spaces

In this paper, we present a 3D face photography system based on a facial expression training dataset, composed of both facial range images (3D geometry) and facial texture (2D photography). The proposed system allows one to obtain a 3D geometry representation of a given face provided as a 2D photography, which undergoes a series of transformations through the texture and geometry spaces estimated. In the training phase of the system, the facial landmarks are obtained by an active shape model (ASM) extracted from the 2D gray-level photography. Principal components analysis (PCA) is then used to represent the face dataset, thus defining an orthonormal basis of texture and another of geometry. In the reconstruction phase, an input is given by a face image to which the ASM is matched. The extracted facial landmarks and the face image are fed to the PCA basis transform, and a 3D version of the 2D input image is built. Experimental tests using a new dataset of 70 facial expressions belonging to ten subjects as training set show rapid reconstructed 3D faces which maintain spatial coherence similar to the human perception, thus corroborating the efficiency and the applicability of the proposed system.

Representação e armazenamento de dados cadastrais 3D em banco de dados geográficos

Pós-graduação em Ciências Cartográficas - FCT

Assessment of spline-based 2D-3D registration for image-guided spine surgery

2D-3D registration of pre-operative 3D volumetric data with a series of calibrated and undistorted intra-operative 2D projection images has shown great potential in CT-based surgical navigation because it obviates the invasive procedure of the conventional registration methods. In this study, a recently introduced spline-based multi-resolution 2D-3D image registration algorithm has been adapted together with a novel least-squares normalized pattern intensity (LSNPI) similarity measure for image guided minimally invasive spine surgery. A phantom and a cadaver together with their respective ground truths were specially designed to experimentally assess possible factors that may affect the robustness, accuracy, or efficiency of the registration. Our experiments have shown that it is feasible for the assessed 2D-3D registration algorithm to achieve sub-millimeter accuracy in a realistic setup in less than one minute.

Electroanatomic reconstruction of the left atrium, pulmonary veins, and esophagus compared with the "true anatomy" on multislice computed tomography in patients undergoing catheter ablation of atrial fibrillation

BACKGROUND: Current concepts of catheter ablation for atrial fibrillation (AF) commonly use three-dimensional (3D) reconstructions of the left atrium (LA) for orientation, catheter navigation, and ablation line placement. OBJECTIVES: The purpose of this study was to compare the 3D electroanatomic reconstruction (Carto) of the LA, pulmonary veins (PVs), and esophagus with the true anatomy displayed on multislice computed tomography (CT). METHODS: In this prospective study, 100 patients undergoing AF catheter ablation underwent contrast-enhanced spiral CT scan with barium swallow and subsequent multiplanar and 3D reconstructions. Using Carto, circumferential plus linear LA lesions were placed. The esophagus was tagged and integrated into the Carto map. RESULTS: Compared with the true anatomy on CT, the electroanatomic reconstruction accurately displayed the true distance between the lower PVs; the distances between left upper PV, left lower PV, right lower PV, and center of the esophagus; the longitudinal diameter of the encircling line around the funnel of the left PVs; and the length of the mitral isthmus line. Only the distances between the upper PVs, the distance between the right upper PV and esophagus, and the diameter of the right encircling line were significantly shorter on the electroanatomic reconstructions. Furthermore, electroanatomic tagging of the esophagus reliably visualized the true anatomic relationship to the LA. On multiple tagging and repeated CT scans, the LA and esophagus showed a stable anatomic relationship, without relevant sideward shifting of the esophagus. CONCLUSION: Electroanatomic reconstruction can display with high accuracy the true 3D anatomy of the LA and PVs in most of the regions of interest for AF catheter ablation. In addition, Carto was able to visualize the true anatomic relationship between the esophagus and LA. Both structures showed a stable anatomic relationship on Carto and CT without relevant sideward shifting of the esophagus.

Computerized Three-Dimmensional Craniofacial Reconstruction from Skulls Based on Landmarks.

Human identification from a skull is a critical process in legal and forensic medicine, specially when no other means are available. Traditional clay-based methods attempt to generate the human face, in order to identify the corresponding person. However, these reconstructions lack of objectivity and consistence, since they depend on the practitioner. Current computerized techniques are based on facial models, which introduce undesired facial features when the final reconstruction is built. This paper presents an objective 3D craniofacial reconstruction technique, implemented in a graphic application, without using any facial template. The only information required by the software tool is the 3D image of the target skull and three parameters: age, gender and Body Mass Index (BMI) of the individual. Complexity is minimized, since the application database only consists of the anthropological information provided by soft tissue depth values in a set of points of the skull.

Efficient methodologies for system matrix modelling in iterative image reconstruction for rotating high-resolution PET

A fully 3D iterative image reconstruction algorithm has been developed for high-resolution PET cameras composed of pixelated scintillator crystal arrays and rotating planar detectors, based on the ordered subsets approach. The associated system matrix is precalculated with Monte Carlo methods that incorporate physical effects not included in analytical models, such as positron range effects and interaction of the incident gammas with the scintillator material. Custom Monte Carlo methodologies have been developed and optimized for modelling of system matrices for fast iterative image reconstruction adapted to specific scanner geometries, without redundant calculations. According to the methodology proposed here, only one-eighth of the voxels within two central transaxial slices need to be modelled in detail. The rest of the system matrix elements can be obtained with the aid of axial symmetries and redundancies, as well as in-plane symmetries within transaxial slices. Sparse matrix techniques for the non-zero system matrix elements are employed, allowing for fast execution of the image reconstruction process. This 3D image reconstruction scheme has been compared in terms of image quality to a 2D fast implementation of the OSEM algorithm combined with Fourier rebinning approaches. This work confirms the superiority of fully 3D OSEM in terms of spatial resolution, contrast recovery and noise reduction as compared to conventional 2D approaches based on rebinning schemes. At the same time it demonstrates that fully 3D methodologies can be efficiently applied to the image reconstruction problem for high-resolution rotational PET cameras by applying accurate pre-calculated system models and taking advantage of the system's symmetries.

3D Maps Representation Using GNG

Current RGB-D sensors provide a big amount of valuable information for mobile robotics tasks like 3D map reconstruction, but the storage and processing of the incremental data provided by the different sensors through time quickly become unmanageable. In this work, we focus on 3D maps representation and propose the use of the Growing Neural Gas (GNG) network as a model to represent 3D input data. GNG method is able to represent the input data with a desired amount of neurons or resolution while preserving the topology of the input space. Experiments show how GNG method yields a better input space adaptation than other state-of-the-art 3D map representation methods.