Towards a real-time 3D object recognition pipeline on mobile GPGPU computing platforms using low-cost RGB-D sensors

In this project, we propose the implementation of a 3D object recognition system which will be optimized to operate under demanding time constraints. The system must be robust so that objects can be recognized properly in poor light conditions and cluttered scenes with significant levels of occlusion. An important requirement must be met: the system must exhibit a reasonable performance running on a low power consumption mobile GPU computing platform (NVIDIA Jetson TK1) so that it can be integrated in mobile robotics systems, ambient intelligence or ambient assisted living applications. The acquisition system is based on the use of color and depth (RGB-D) data streams provided by low-cost 3D sensors like Microsoft Kinect or PrimeSense Carmine. The range of algorithms and applications to be implemented and integrated will be quite broad, ranging from the acquisition, outlier removal or filtering of the input data and the segmentation or characterization of regions of interest in the scene to the very object recognition and pose estimation. Furthermore, in order to validate the proposed system, we will create a 3D object dataset. It will be composed by a set of 3D models, reconstructed from common household objects, as well as a handful of test scenes in which those objects appear. The scenes will be characterized by different levels of occlusion, diverse distances from the elements to the sensor and variations on the pose of the target objects. The creation of this dataset implies the additional development of 3D data acquisition and 3D object reconstruction applications. The resulting system has many possible applications, ranging from mobile robot navigation and semantic scene labeling to human-computer interaction (HCI) systems based on visual information.

3D geometry of growth strata in a fault-propagation fold: insights into space–time evolution of the Crevillente Fault (Abanilla-Alicante sector), Betic Cordillera, Spain

This work presents a 3D geometric model of growth strata cropping out in a fault-propagation fold associated with the Crevillente Fault (Abanilla-Alicante sector) from the Bajo Segura Basin (eastern Betic Cordillera, southern Spain). The analysis of this 3D model enables us to unravel the along-strike and along-section variations of the growth strata, providing constraints to assess the fold development, and hence, the fault kinematic evolution in space and time. We postulate that the observed along-strike dip variations are related to lateral variation in fault displacement. Along-section variations of the progressive unconformity opening angles indicate greater fault slip in the upper Tortonian–Messinian time span; from the Messinian on, quantitative analysis of the unconformity indicate a constant or lower tectonic activity of the Crevillente Fault (Abanilla-Alicante sector); the minor abundance of striated pebbles in the Pliocene-Quaternary units could be interpreted as a decrease in the stress magnitude and consequently in the tectonic activity of the fault. At a regional scale, comparison of the growth successions cropping out in the northern and southern limits of the Bajo Segura Basin points to a southward migration of deformation in the basin. This means that the Bajo Segura Fault became active after the Crevillente Fault (Abanilla-Alicante sector), for which activity on the latter was probably decreasing according to our data. Consequently, we propose that the seismic hazard at the northern limit of the Bajo Segura Basin should be lower than at the southern limit.

3D models created from ROV videos and their corresponding subtransect dimensions

Underwater video transects have become a common tool for quantitative analysis of the seafloor. However a major difficulty remains in the accurate determination of the area surveyed as underwater navigation can be unreliable and image scaling does not always compensate for distortions due to perspective and topography. Depending on the camera set-up and available instruments, different methods of surface measurement are applied, which make it difficult to compare data obtained by different vehicles. 3-D modelling of the seafloor based on 2-D video data and a reference scale can be used to compute subtransect dimensions. Focussing on the length of the subtransect, the data obtained from 3-D models created with the software PhotoModeler Scanner are compared with those determined from underwater acoustic positioning (ultra short baseline, USBL) and bottom tracking (Doppler velocity log, DVL). 3-D model building and scaling was successfully conducted on all three tested set-ups and the distortion of the reference scales due to substrate roughness was identified as the main source of imprecision. Acoustic positioning was generally inaccurate and bottom tracking unreliable on rough terrain. Subtransect lengths assessed with PhotoModeler were on average 20% longer than those derived from acoustic positioning due to the higher spatial resolution and the inclusion of slope. On a high relief wall bottom tracking and 3-D modelling yielded similar results. At present, 3-D modelling is the most powerful, albeit the most time-consuming, method for accurate determination of video subtransect dimensions.

Multiview photometric stereo

This paper addresses the problem of obtaining complete, detailed reconstructions of textureless shiny objects. We present an algorithm which uses silhouettes of the object, as well as images obtained under changing illumination conditions. In contrast with previous photometric stereo techniques, ours is not limited to a single viewpoint but produces accurate reconstructions in full 3D. A number of images of the object are obtained from multiple viewpoints, under varying lighting conditions. Starting from the silhouettes, the algorithm recovers camera motion and constructs the object's visual hull. This is then used to recover the illumination and initialize a multiview photometric stereo scheme to obtain a closed surface reconstruction. There are two main contributions in this paper: First, we describe a robust technique to estimate light directions and intensities and, second, we introduce a novel formulation of photometric stereo which combines multiple viewpoints and, hence, allows closed surface reconstructions. The algorithm has been implemented as a practical model acquisition system. Here, a quantitative evaluation of the algorithm on synthetic data is presented together with complete reconstructions of challenging real objects. Finally, we show experimentally how, even in the case of highly textured objects, this technique can greatly improve on correspondence-based multiview stereo results.

Self-calibrated, multi-spectral photometric stereo for 3D face capture

This paper addresses the problem of obtaining 3d detailed reconstructions of human faces in real-time and with inexpensive hardware. We present an algorithm based on a monocular multi-spectral photometric-stereo setup. This system is known to capture high-detailed deforming 3d surfaces at high frame rates and without having to use any expensive hardware or synchronized light stage. However, the main challenge of such a setup is the calibration stage, which depends on the lights setup and how they interact with the specific material being captured, in this case, human faces. For this purpose we develop a self-calibration technique where the person being captured is asked to perform a rigid motion in front of the camera, maintaining a neutral expression. Rigidity constrains are then used to compute the head's motion with a structure-from-motion algorithm. Once the motion is obtained, a multi-view stereo algorithm reconstructs a coarse 3d model of the face. This coarse model is then used to estimate the lighting parameters with a stratified approach: In the first step we use a RANSAC search to identify purely diffuse points on the face and to simultaneously estimate this diffuse reflectance model. In the second step we apply non-linear optimization to fit a non-Lambertian reflectance model to the outliers of the previous step. The calibration procedure is validated with synthetic and real data.

Interactive Visualization of the Building of University of Economics – Varna via 3D Modeling

The object of this paper is presenting the University of Economics – Varna, using a 3D model with 3Ds MAX. Created in 1920, May 14, University of Economics - Varna is a cultural institution with a place and style of its own. With the emergence of the three-dimensional modeling we entered a new stage of the evolution of computer graphics. The main target is to preserve the historical vision, to demonstrate forward-thinking and using of future-oriented approaches.

The dynamics of neutrino-driven supernova explosions after shock revival in 2D and 3D

We study the growth of the explosion energy after shock revival in neutrino-driven explosions in two and three dimensions (2D/3D) using multi-group neutrino hydrodynamics simulations of an 11.2 M⊙ star. The 3D model shows a faster and steadier growth of the explosion energy and already shows signs of subsiding accretion after one second. By contrast, the growth of the explosion energy in 2D is unsteady, and accretion lasts for several seconds as confirmed by additional long-time simulations of stars of similar masses. Appreciable explosion energies can still be reached, albeit at the expense of rather high neutron star masses. In 2D, the binding energy at the gain radius is larger because the strong excitation of downward-propagating g modes removes energy from the freshly accreted material in the downflows. Consequently, the mass outflow rate is considerably lower in 2D than in 3D. This is only partially compensated by additional heating by outward-propagating acoustic waves in 2D. Moreover, the mass outflow rate in 2D is reduced because much of the neutrino energy deposition occurs in downflows or bubbles confined by secondary shocks without driving outflows. Episodic constriction of outflows and vertical mixing of colder shocked material and hot, neutrino-heated ejecta due to Rayleigh–Taylor instability further hamper the growth of the explosion energy in 2D. Further simulations will be necessary to determine whether these effects are generic over a wider range of supernova progenitors.

3D modelling of mango fruit skin blush in the tree canopy

The fruit of certain mango cultivars (e.g., 'Honey Gold') can develop blush on their skin. Skin blush due to red pigmentation is from the accumulation of anthocyanins. Anthocyanin biosynthesis is related to environmental determinants, including light received by the fruit. It has been observed that mango skin blush varies with position in the tree canopy. However, little investigation into this spatial relationship has been conducted. The objective of this preliminary study was to describe a 'Honey Gold' mango tree by capturing its three-dimensional (3D) architecture. A light path tracing model QuasiMC was then used to predict light received by fruit. The use of this 3D model was to better understand the relationship between mango fruit skin blush and fruit position in the canopy. The digitised mango tree mimicked the real tree at a high level of detail. Observations on mango skin blush distribution supported the proposition that sunlight exposure is an absolute requirement for anthocyanin development. No blush development occurred on shaded skin. It was affirmed that 3D mapping could allow for virtual experiments. For example, for virtual canopy thinning (e.g., 'window pruning') to admit more sunlight with a view to improve fruit blush. Improvements to 3D modelling of mango skin blush could focus on increasing accuracy, e.g., measurement of leaf light reflectance and transmission and the inclusion of the effect shading by branches.

3D modelling of the transmissivity of granitic rocks surrounding the old Quinta do Bispo mine | Modelação 3D da transmissividade do maciço granítico envolvente à antiga mina da Quinta do Bispo

Apresenta-se uma metodologia para caracterizar a transmissividade dos Granitos Hercínicos e Metasedimentos do Complexo Xisto-Grauváquico do maciço envolvente e subjacente à antiga área mineira de urânio da Quinta do Bispo. Inicia-se com a modelação das litologias e grau de alteração a que se segue a simulação condicional da densidade de fracturação. No final, a densidade de fracturação é convertida num modelo 3D de transmissividade por relação com os resultados dos ensaios de bombagem. The purpose of this work is to present a methodology for characterizing the transmissivity of the Hercynian granites and complex schist–greywacke metasediment rocks surrounding and underlying the old Quinta do Bispo uranium mining site. The methodology encompasses modelling of lithologies and weathering levels, followed by a conditional simulation of fracture density. Fracture density is then converted into a 3D model of transmissivity via a relationship with pumping tests.

Improved model of an electric calciner for carbon materials

Teknova have 2D steady-state models of the calciner but wish, in the long term, to have a 3D model that can also cover unsteady conditions, and can can model the loss of axisymmetry that someties occurs. Teknova also wish to understand the processes happening around the tip of the upper electrode, in particular the formation of a lip on it and the the shape of the empty region below it. The Study Group proposed potential models for the degree of graphitization, and for the granular flow. Also the Study Group considered the upper electrode in detail. The proposed model for the lip formation is by sublimation of carbon from the hottest parts of the furnace with redeposition in the region around the electrode, which may stick particles onto the electrode surface. In this model the region below the electrode would be a void, roughly a vertex-down conical cavity. The electric field near the lower rim of the electrode will then have a singularity and so the most intense heating of the charge will be around the rim. We conjecture that the reason why the lower electrode lasts so much longer than the upper is that it is not adjacent to a cavity like this, and therefore does not have a singularity in the field.

3D Surveying and Data Management towards the Realization of a Knowledge System for Cultural Heritage

The research activities involved the application of the Geomatic techniques in the Cultural Heritage field, following the development of two themes: Firstly, the application of high precision surveying techniques for the restoration and interpretation of relevant monuments and archaeological finds. The main case regards the activities for the generation of a high-fidelity 3D model of the Fountain of Neptune in Bologna. In this work, aimed to the restoration of the manufacture, both the geometrical and radiometrical aspects were crucial. The final product was the base of a 3D information system representing a shared tool where the different figures involved in the restoration activities shared their contribution in a multidisciplinary approach. Secondly, the arrangement of 3D databases for a Building Information Modeling (BIM) approach, in a process which involves the generation and management of digital representations of physical and functional characteristics of historical buildings, towards a so-called Historical Building Information Model (HBIM). A first application was conducted for the San Michele in Acerboli’s church in Santarcangelo di Romagna. The survey was performed by the integration of the classical and modern Geomatic techniques and the point cloud representing the church was used for the development of a HBIM model, where the relevant information connected to the building could be stored and georeferenced. A second application regards the domus of Obellio Firmo in Pompeii, surveyed by the integration of the classical and modern Geomatic techniques. An historical analysis permitted the definitions of phases and the organization of a database of materials and constructive elements. The goal is the obtaining of a federate model able to manage the different aspects: documental, analytic and reconstructive ones.

Sviluppo di un sistema di Realta Aumentata applicato a un modello 3D della prostata, sviluppato a partire da immagini RM, al fine di facilitare la dissezione del tumore e la conservazione dei fasci neuro vascolari durante l'intervento chirurgico di prostatectomia radicale robot-assistita

La realtà aumentata (AR) è una nuova tecnologia adottata in chirurgia prostatica con l'obiettivo di migliorare la conservazione dei fasci neurovascolari (NVB) ed evitare i margini chirurgici positivi (PSM). Abbiamo arruolato prospetticamente pazienti con diagnosi di cancro alla prostata (PCa) sul base di biopsia di fusione mirata con mpMRI positiva. Prima dell'intervento, i pazienti arruolati sono stati indirizzati a sottoporsi a ricostruzione del modello virtuale 3D basato su mpMRI preoperatoria immagini. Infine, il chirurgo ha eseguito la RARP con l'ausilio del modello 3D proiettato in AR all'interno della console robotica (RARP guidata AR-3D). I pazienti sottoposti a AR RARP sono stati confrontati con quelli sottoposti a "RARP standard" nello stesso periodo. Nel complesso, i tassi di PSM erano comparabili tra i due gruppi; I PSM a livello della lesione indice erano significativamente più bassi nei pazienti riferiti al gruppo AR-3D (5%) rispetto a quelli nel gruppo di controllo (20%; p = 0,01). La nuova tecnica di guida AR-3D per l'analisi IFS può consentono di ridurre i PSM a livello della lesione dell'indice

Digital 3D reconstruction as a research environment in art and architecture history: uncertainty classification and visualisation

The dissertation addresses the still not solved challenges concerned with the source-based digital 3D reconstruction, visualisation and documentation in the domain of archaeology, art and architecture history. The emerging BIM methodology and the exchange data format IFC are changing the way of collaboration, visualisation and documentation in the planning, construction and facility management process. The introduction and development of the Semantic Web (Web 3.0), spreading the idea of structured, formalised and linked data, offers semantically enriched human- and machine-readable data. In contrast to civil engineering and cultural heritage, academic object-oriented disciplines, like archaeology, art and architecture history, are acting as outside spectators. Since the 1990s, it has been argued that a 3D model is not likely to be considered a scientific reconstruction unless it is grounded on accurate documentation and visualisation. However, these standards are still missing and the validation of the outcomes is not fulfilled. Meanwhile, the digital research data remain ephemeral and continue to fill the growing digital cemeteries. This study focuses, therefore, on the evaluation of the source-based digital 3D reconstructions and, especially, on uncertainty assessment in the case of hypothetical reconstructions of destroyed or never built artefacts according to scientific principles, making the models shareable and reusable by a potentially wide audience. The work initially focuses on terminology and on the definition of a workflow especially related to the classification and visualisation of uncertainty. The workflow is then applied to specific cases of 3D models uploaded to the DFG repository of the AI Mainz. In this way, the available methods of documenting, visualising and communicating uncertainty are analysed. In the end, this process will lead to a validation or a correction of the workflow and the initial assumptions, but also (dealing with different hypotheses) to a better definition of the levels of uncertainty.

Three Dimensional Finite Element Analyses of Oral Structures by Computerized Tomography

Our aim was to document the benefits of three dimensional finite element model generations from computed tomography data as well as the realistic creation of all oral structures in a patient. The stresses resulting from the applied load in our study did not exceed the structure limitations, suggesting a clinically acceptable physiological condition.

Thermal Characteristics of Polybia scutellaris Nests (Hymenoptera: Vespidae) Using Computational Fluid Dynamics: A Possible Adaptation to Tropical Climates

Polybia scutellaris constructs huge nests characterized by numerous spinal projections on the surface. We investigated the thermal characteristics of P scutellaris nests in order to determine whether their nest temperature is homeothermically maintained and whether the spines play a role in the thermoregulation of the nests. In order to examine these hypotheses, we measured the nest temperature in a active nest and in an abandoned nest. The temperature in the active nest was almost stable at 27 degrees C, whereas that of the abandoned nest varied with changes in the ambient temperature, suggesting that nest temperature was maintained by the thermogenesis of colony individuals. In order to predict the thermal properties of the spines, a numerical simulation was employed. To construct a 3D-model of a P scutellaris nest, the nest architecture was simplified into an outer envelope and the surface spines, for both of which the initial temperature was set at 27 degrees C. The physical properties of the simulated nest were regarded to be those of wood since the nest of this species is constructed from plant materials. When the model was exposed to cool air (12 degrees C), the temperature was lower in the models with more spines. On the other hand, when the nest was heated (42 degrees C), the temperature increase was smaller in models with more spines. It is suggested that the spines act as a heat radiator, not as an insulator, against the changes in ambient temperature.