POSE ESTIMATION AND 3D RECONSTRUCTION USING SENSOR FUSION

A camera maps 3-dimensional (3D) world space to a 2-dimensional (2D) image space. In the process it loses the depth information, i.e., the distance from the camera focal point to the imaged objects. It is impossible to recover this information from a single image. However, by using two or more images from different viewing angles this information can be recovered, which in turn can be used to obtain the pose (position and orientation) of the camera. Using this pose, a 3D reconstruction of imaged objects in the world can be computed. Numerous algorithms have been proposed and implemented to solve the above problem; these algorithms are commonly called Structure from Motion (SfM). State-of-the-art SfM techniques have been shown to give promising results. However, unlike a Global Positioning System (GPS) or an Inertial Measurement Unit (IMU) which directly give the position and orientation respectively, the camera system estimates it after implementing SfM as mentioned above. This makes the pose obtained from a camera highly sensitive to the images captured and other effects, such as low lighting conditions, poor focus or improper viewing angles. In some applications, for example, an Unmanned Aerial Vehicle (UAV) inspecting a bridge or a robot mapping an environment using Simultaneous Localization and Mapping (SLAM), it is often difficult to capture images with ideal conditions. This report examines the use of SfM methods in such applications and the role of combining multiple sensors, viz., sensor fusion, to achieve more accurate and usable position and reconstruction information. This project investigates the role of sensor fusion in accurately estimating the pose of a camera for the application of 3D reconstruction of a scene. The first set of experiments is conducted in a motion capture room. These results are assumed as ground truth in order to evaluate the strengths and weaknesses of each sensor and to map their coordinate systems. Then a number of scenarios are targeted where SfM fails. The pose estimates obtained from SfM are replaced by those obtained from other sensors and the 3D reconstruction is completed. Quantitative and qualitative comparisons are made between the 3D reconstruction obtained by using only a camera versus that obtained by using the camera along with a LIDAR and/or an IMU. Additionally, the project also works towards the performance issue faced while handling large data sets of high-resolution images by implementing the system on the Superior high performance computing cluster at Michigan Technological University.

Software for Designing Custom Orbital-Craniofacial Implant Plans

Purpose: Custom cranio-orbital implants have been shown to achieve better performance than their hand-shaped counterparts by restoring skull anatomy more accurately and by reducing surgery time. Designing a custom implant involves reconstructing a model of the patient's skull using their computed tomography (CT) scan. The healthy side of the skull model, contralateral to the damaged region, can then be used to design an implant plan. Designing implants for areas of thin bone, such as the orbits, is challenging due to poor CT resolution of bone structures. This makes preoperative design time-intensive since thin bone structures in CT data must be manually segmented. The objective of this thesis was to research methods to accurately and efficiently design cranio-orbital implant plans, with a focus on the orbits, and to develop software that integrates these methods. Methods: The software consists of modules that use image and surface restoration approaches to enhance both the quality of CT data and the reconstructed model. It enables users to input CT data, and use tools to output a skull model with restored anatomy. The skull model can then be used to design the implant plan. The software was designed using 3D Slicer, an open-source medical visualization platform. It was tested on CT data from thirteen patients. Results: The average time it took to create a skull model with restored anatomy using our software was 0.33 hours ± 0.04 STD. In comparison, the design time of the manual segmentation method took between 3 and 6 hours. To assess the structural accuracy of the reconstructed models, CT data from the thirteen patients was used to compare the models created using our software with those using the manual method. When registering the skull models together, the difference between each set of skulls was found to be 0.4 mm ± 0.16 STD. Conclusions: We have developed a software to design custom cranio-orbital implant plans, with a focus on thin bone structures. The method described decreases design time, and is of similar accuracy to the manual method.

Interface virtual com sensor 3D para interação de pessoas acamadas

A interação homem-máquina tem evoluído significativamente nos últimos anos, a ponto de permitir desenvolver soluções adequadas para apoio a pessoas que possuem um certo tipo de limitação física ou cognitiva. O desenvolvimento de técnicas naturais e intuitivas de interação, as chamadas Natural User Interface (NUI), permitem, hoje, que pessoas que estejam acamadas e/ou com incapacidade motora possam executar um conjunto de ações por intermédio de gestos, aumentando assim a sua qualidade de vida. A solução implementada neste projecto é baseada em processamento de imagem e visão por computador através do sensor 3D Kinect e consiste numa interface natural para o desenvolvimento de uma aplicação que reconheça gestos efetuados por uma mão humana. Os gestos identificados pela aplicação acionam um conjunto de ações adequados a uma pessoa acamada, como, por exemplo, acionar a emergência, ligar ou desligar a TV ou controlar a inclinação da cama. O processo de desenvolvimento deste projeto implicou várias etapas. Inicialmente houve um trabalho intenso de investigação sobre as técnicas e tecnologias consideradas importantes para a realização do trabalho - a etapa de investigação, a qual acompanhou praticamente todo o processo. A segunda etapa consistiu na configuração do sistema ao nível do hardware e do software. Após a configuração do sistema, obtiveram-se os primeiros dados do sensor 3D Kinect, os quais foram convertidos num formato mais apropriado ao seu posterior tratamento. A segmentação da mão permitiu posteriormente o reconhecimento de gestos através da técnica de matching para os seis gestos implementados. Os resultados obtidos são satisfatórios, tendo-se contabilizado cerca de 96% de resultados válidos. A área da saúde e bem-estar tem necessidade de aplicações que melhorem a qualidade de vida de pessoas acamadas, nesse sentido, o protótipo desenvolvido faz todo o sentido na sociedade actual, onde se verifica o envelhecimento da população.

‘Crows’ vs. ‘Avatar’, or: 3D vs. total-dimensional immersion

3D film’s explicit new space depth arguably provides both an enhanced realistic quality to the image and a wealth of more acute visual and haptic sensations (a ‘montage of attractions’) to the increasingly involved spectator. But David Cronenberg’s related ironic remark that ‘cinema as such is from the outset a «special effect»’ should warn us against the geometrical naiveté of such assumptions, within a Cartesian ocularcentric tradition for long overcome by Merleau-Ponty’s embodiment of perception and Deleuze’s notion of the self-consistency of the artistic sensation and space. Indeed, ‘2D’ traditional cinema already provides the accomplished «fourth wall effect», enclosing the beholder behind his back within a space that no longer belongs to the screen (nor to ‘reality’) as such, and therefore is no longer ‘illusorily’ two-dimensional. This kind of totally absorbing, ‘dream-like’ space, metaphorical for both painting and cinema, is illustrated by the episode ‘Crows’ in Kurosawa’s Dreams. Such a space requires the actual effacement of the empirical status of spectator, screen and film as separate dimensions, and it is precisely the 3D caracteristic unfolding of merely frontal space layers (and film events) out of the screen towards us (and sometimes above the heads of the spectators before us) that reinstalls at the core of the film-viewing phenomenon a regressive struggle with reality and with different degrees of realism, originally overcome by film since the Lumière’s Arrival of a Train at Ciotat seminal demonstration. Through an analysis of crucial aspects in Avatar and the recent Cave of Forgotten Dreams, both dealing with historical and ontological deepening processes of ‘going inside’, we shall try to show how the formal and technically advanced component of those 3D-depth films impairs, on the contrary, their apparent conceptual purpose on the level of contents, and we will assume, drawing on Merleau-Ponty and Deleuze, that this technological mistake is due to a lack of recognition of the nature of perception and sensation in relation to space and human experience.

Development of 3D Cancer Cell Models to Test Metabolic Interventions as an Anticancer Strategy

In recent years, it has become evident that the role of mitochondria in the metabolic rewiring is essential for cancer development and progression. The metabolic profile during tumorigenesis has been performed mainly in traditional 2D cell models, including cell lines of various lineages and phenotypes. Although useful in many ways, their relevance can be often debatable, as they lack the interactions between different cells of the tumour microenvironment and/or interaction with the extracellular matrix 1,2. Improved models are now being developed using 3D cell culture technology, contributing with increased physiological relevance 3,4. In this work, we improved a method for the generation of 3D models from healthy and tumour colon tissue, based on organoid technology, and performed their molecular and biochemical characterization and validation. Further, in-plate cryopreservation was applied to these models, and optimal results were obtained in terms of cell viability and functionality of the cryopreserved models. We also cryopreserved colon fibroblasts with the aim to introduce them in a co-culture cryopreserved model with organoids. This technology allows the conversion of cell models into “plug and play” formats. Therefore, cryopreservation in-plate facilitates the accessibility of specialized cell models to cell-based research and application, in cases where otherwise such specialized models would be out of reach. Finally, we briefly explored the field of bioprinting, by testing a new matrix to support the growth of colon tumour organoids, which revealed promising preliminary results. To facilitate the reader, we organized this thesis into chapters, divided by the main points of work which include development, characterization and validation of the model, commercial output, and associated applications. Each chapter has a brief introduction, followed by results and discussion and a final conclusion. The thesis has also a general discussion and conclusion section in the end, which covers the main results obtained during this work.

Studio del metodo di preparazione di elettrocatalizzatori 3D Ag/Cu nella riduzione selettiva del 5-idrossimetilfurfurale

In questo lavoro di tesi sono stati preparati elettrocatalizzatori metallici 3D di argento supportato su schiume di rame a cella aperta, impiegandoli nella riduzione selettiva del 5-idrossimetilfurfurale (HMF) a 2,5-bisidrossimetilfurano (BHMF). L’utilizzo di questi supporti consente di incrementare l’area superficiale a disposizione, rispetto ai supporti 2D. La preparazione delle schiume è stata effettuata utilizzando due metodi di deposizione: spostamento galvanico ed elettrodeposizione. Lo scopo del lavoro è valutare l’influenza del metodo di deposizione sull’attività catalitica, per questo motivo le schiume bimetalliche ottenute sono state confrontate a quelle monometalliche di rame e argento. Inoltre è stato studiato l’effetto della concentrazione di HMF sulle prestazioni dei catalizzatori, con l’obiettivo finale di ottenere un sistema attivo a concentrazioni industriali. Le schiume sono state sottoposte a cicli di prove catalitiche a concentrazione di HMF crescente 0,02 M, 0,05 M e 0,10 M e caratterizzate prima e dopo la sequenza di prove con analisi SEM-EDS, XRD, spettroscopia raman. Le soluzioni sono state analizzate mediante ICP, GC-MS, ESI-MS. Con lo spostamento galvanico si ha la crescita di dendriti, formate prevalentemente sui bordi, e agglomerati di particelle nelle zone interne, mentre per elettrodeposizione si ottiene minore formazione di dendriti e particelle mediamente più piccole. La presenza di argento come fase attiva non migliora la conversione rispetto alle schiume monometalliche, ma causa un aumento di selettività ed efficienza faradica. Incrementando la concentrazione di HMF tutti i campioni presentano un aumento di conversione e un decremento di selettività ed efficienza faradica. Il catalizzatore ottenuto per spostamento galvanico, mostra prestazioni migliori a concentrazioni elevate e nelle prove di stabilità non dà segni di disattivazione, al contrario della schiuma preparata per elettrodeposizione che risulta leggermente disattivata.

Economic and Environmental Optimization of Ornamental Stone Block Cutting with a 3D Algorithm

This thesis focuses on finding the optimum block cutting dimensions in terms of the environmental and economic factors by using a 3D algorithm for a limestone quarry in Foggia, Italy. The environmental concerns of quarrying operations are mainly: energy consumption, material waste, and pollution. The main economic concerns are the block recovery, the selling prices, and the production costs. Fractures adversely affect the block recovery ratio. With a fracture model, block production can be optimized. In this research, the waste volume produced by quarrying was minimised to increase the recovery ratio and ensure economic benefits. SlabCutOpt is a software developed at DICAM–University of Bologna for block cutting optimization which tests different cutting angles on the x-y-z planes to offer up alternative cutting methods. The program tests several block sizes and outputs the optimal result for each entry. By using SlabCutOpt, ten different block dimensions were analysed, the results indicated the maximum number of non-intersecting blocks for each dimension. After analysing the outputs, the block named number 1 with the dimensions ‘1mx1mx1m’ had the highest recovery ratio as 43% and the total Relative Money Value (RMV) with a value of 22829. Dimension number 1, also had the lowest waste volume, with a value of 3953.25 m3, for the total bench. For cutting the total bench volume of 6932.25m3, the diamond wire cutter had the lowest dust emission values for the block with the dimension ‘2mx2mx2m’, with a value of 24m3. When compared with the Eco-Label standards, block dimensions having surface area values lower than 15m2, were found to fit the natural resource waste criteria of the label, as the threshold required 25% of minimum recovery [1]. Due to the relativity of production costs, together with the Eco-Label threshold, the research recommends the selection of the blocks with a surface area value between 6m2 and 14m2.

Laying the path to consumer-level immersive simulation environment

Since the end of the long winter of virtual reality (VR) at the beginning of the 2010 decade, many improvements have been made in terms of hardware technologies and software platforms performances and costs. Many expect such trend will continue, pushing the penetration rate of virtual reality headsets to skyrocket at some point in the future, just as mobile platforms did before. In the meantime, virtual reality is slowly transitioning from a specialized laboratory-only technology, to a consumer electronics appliance, opening interesting opportunities and challenges. In this transition, two interesting research questions amount to how 2D-based content and applications may benefit (or be hurt) by the adoption of 3D-based immersive environments and to how to proficiently support such integration. Acknowledging the relevance of the former, we here consider the latter question, focusing our attention on the diversified family of PC-based simulation tools and platforms. VR-based visualization is, in fact, widely understood and appreciated in the simulation arena, but mainly confined to high performance computing laboratories. Our contribution here aims at characterizing the simulation tools which could benefit from immersive interfaces, along with a general framework and a preliminary implementation which may be put to good use to support their transition from uniquely 2D to blended 2D/3D environments.

Understanding 3D structures from sketches

Sketches are a unique way to communicate: drawing a simple sketch does not require any training, sketches convey information that is hard to describe with words, they are powerful enough to represent almost any concept, and nowadays, it is possible to draw directly from mobile devices. Motivated from the unique characteristics of sketches and fascinated by the human ability to imagine 3D objects from drawings, this thesis focuses on automatically associating geometric information to sketches. The main research directions of the thesis can be summarized as obtaining geometric information from freehand scene sketches to improve 2D sketch-based tasks and investigating Vision-Language models to overcome 3D sketch-based tasks limitations. The first part of the thesis concerns geometric information prediction from scene sketches improving scene sketch to image generation and unlocking new creativity effects. The thesis proceeds showing a study conducted on the Vision-Language models embedding space considering sketches, line renderings and RGB renderings of 3D shape to overcome the use of supervised datasets for 3D sketch-based tasks, that are limited and hard to acquire. Following the obtained observations and results, Vision-Language models are applied to Sketch Based Shape Retrieval without the need of training on supervised datasets. We then analyze the use of Vision-Language models for sketch based 3D reconstruction in an unsupervised manner. In the final chapter we report the results obtained in an additional project carried during the PhD, which has lead to the development of a framework to learn an embedding space of neural networks that can be navigated to get ready-to-use models with desired characteristics.

Progettazione di una pinza demolitrice idraulica

Questo elaborato ha come obiettivo quello di progettare una pinza demolitrice idraulica per macchine operatrici che sia effettivamente producibile, industrializzabile, performante, affidabile ed economicamente competitiva per il mercato internazionale odierno. Dopo una prima parte di introduzione all’oggetto di studio e di comparazione con la concorrenza, vengono mostrati i primi studi sulla cinematica e sugli ingombri dell’attrezzatura eseguiti con l’ausilio del software 2D “DraftSight”. A questo punto si passa alla modellazione in 3D eseguita mediante il software “Solidworks” di tutti i componenti analizzando e giustificando le scelte progettuali che sono state fatte. Vengono quindi riportati tutti i calcoli analitici eseguiti, nonché le verifiche strutturali e l’ottimizzazione topologica realizzate mediante l’ausilio del pacchetto FEM “Simulation” di Solidworks. Infine si valutano i risultati ottenuti dalla progettazione e possibili sviluppi futuri.

Optimized methods to digital survey and 3D modelling of cultural heritage assets

Currently making digital 3D models and replicas of the cultural heritage assets play an important role in the preservation and having a high detail source for future research and intervention. In this dissertation, it is tried to assess different methods for digital surveying and making 3D replicas of cultural heritage assets in different scales of size. The methodologies vary in devices, software, workflow, and the amount of skill that is required. The three phases of the 3D modelling process are data acquisition, modelling, and model presentation. Each of these sections is divided into sub-sections and there are several approaches, methods, devices, and software that may be employed, furthermore, the selection process should be based on the operation's goal, available facilities, the scale and properties of the object or structure to be modeled, as well as the operators' expertise and experience. The most key point to remember is that the 3D modelling operation should be properly accurate, precise, and reliable; therefore, there are so many instructions and pieces of advice on how to perform 3D modelling effectively. It is an attempt to compare and evaluate the various ways of each phase in order to explain and demonstrate their differences, benefits, and drawbacks in order to serve as a simple guide for new and/or inexperienced users.

Tolerance optimization with statistical approach in the CAT environment

Nowadays, product development in all its phases plays a fundamental role in the industrial chain. The need for a company to compete at high levels, the need to be quick in responding to market demands and therefore to be able to engineer the product quickly and with a high level of quality, has led to the need to get involved in new more advanced methods/ processes. In recent years, we are moving away from the concept of 2D-based design and production and approaching the concept of Model Based Definition. By using this approach, increasingly complex systems turn out to be easier to deal with but above all cheaper in obtaining them. Thanks to the Model Based Definition it is possible to share data in a lean and simple way to the entire engineering and production chain of the product. The great advantage of this approach is precisely the uniqueness of the information. In this specific thesis work, this approach has been exploited in the context of tolerances with the aid of CAD / CAT software. Tolerance analysis or dimensional variation analysis is a way to understand how sources of variation in part size and assembly constraints propagate between parts and assemblies and how that range affects the ability of a project to meet its requirements. It is critically important to note how tolerance directly affects the cost and performance of products. Worst Case Analysis (WCA) and Statistical analysis (RSS) are the two principal methods in DVA. The thesis aims to show the advantages of using statistical dimensional analysis by creating and examining various case studies, using PTC CREO software for CAD modeling and CETOL 6σ for tolerance analysis. Moreover, it will be provided a comparison between manual and 3D analysis, focusing the attention to the information lost in the 1D case. The results obtained allow us to highlight the need to use this approach from the early stages of the product design cycle.

Acquisizione e modellazione 3D in ambito museale per nuove forme di accessibilità per ipovedenti

Il tema approfondito in questo elaborato è relativo a nuove forme di esperienze museali in grado di favorire una maggiore fruizione dei musei da parte dei visitatori portatori di disabilità sensoriali. L’obiettivo dello studio, reso possibile grazie alla collaborazione con il Comune di Riccione, l’Università di Bologna e il Museo del Territorio di Riccione, riguarda la riproduzione tattile 3D di alcuni manufatti di epoca romana rinvenuti nel territorio e l’implementazione di soluzioni di esplorazione dei modelli fisici con tecniche di realtà aumentata e feedback sonori per fornire al visitatore un’esperienza tattile/sensoriale sugli oggetti. La parte di competenza di questo elaborato riguarda principalmente due aspetti: il rilievo 3D effettuato su quattro diversi frammenti di due serie di lastre fittili, mediante l’utilizzo di scanner ad alta risoluzione e fotogrammetria digitale, e la successiva modellazione in ambiente digitale. Quest’ultima fase comporta l’elaborazione delle scansioni tramite software dedicati e la creazione di mesh per generare la stampa di copie fisiche 3D. In una fase successiva alla stampa 3D le copie saranno poi gestite con particolari tecnologie di esplorazione sensoriale che accompagneranno l’utente durante l’esperienza tattile con feedback sonori. La sperimentazione condotta in questa tesi vuole dimostrare come sia possibile portare ad una trasformazione del ruolo dei musei, da quello tradizionale di magazzino statico a quello di ambiente di apprendimento attivo a disposizione di tutti offrendo esperienze immersive ed educative.

Low-density Three-dimensional Foam Using Self-reinforced Hybrid Two-dimensional Atomic Layers.

Low-density nanostructured foams are often limited in applications due to their low mechanical and thermal stabilities. Here we report an approach of building the structural units of three-dimensional (3D) foams using hybrid two-dimensional (2D) atomic layers made of stacked graphene oxide layers reinforced with conformal hexagonal boron nitride (h-BN) platelets. The ultra-low density (1/400 times density of graphite) 3D porous structures are scalably synthesized using solution processing method. A layered 3D foam structure forms due to presence of h-BN and significant improvements in the mechanical properties are observed for the hybrid foam structures, over a range of temperatures, compared with pristine graphene oxide or reduced graphene oxide foams. It is found that domains of h-BN layers on the graphene oxide framework help to reinforce the 2D structural units, providing the observed improvement in mechanical integrity of the 3D foam structure.

Spinal Cord Atrophy Correlates With Disease Duration And Severity In Amyotrophic Lateral Sclerosis.

Our objective was to investigate spinal cord (SC) atrophy in amyotrophic lateral sclerosis (ALS) patients, and to determine whether it correlates with clinical parameters. Forty-three patients with ALS (25 males) and 43 age- and gender-matched healthy controls underwent MRI on a 3T scanner. We used T1-weighted 3D images covering the whole brain and the cervical SC to estimate cervical SC area and eccentricity at C2/C3 level using validated software (SpineSeg). Disease severity was quantified with the ALSFRS-R and ALS Severity scores. SC areas of patients and controls were compared with a Mann-Whitney test. We used linear regression to investigate association between SC area and clinical parameters. Results showed that mean age of patients and disease duration were 53.1 ± 12.2 years and 34.0 ± 29.8 months, respectively. The two groups were significantly different regarding SC areas (67.8 ± 6.8 mm² vs. 59.5 ± 8.4 mm², p < 0.001). Eccentricity values were similar in both groups (p = 0.394). SC areas correlated with disease duration (r = - 0.585, p < 0.001), ALSFRS-R score (r = 0.309, p = 0.044) and ALS Severity scale (r = 0.347, p = 0.022). In conclusion, patients with ALS have SC atrophy, but no flattening. In addition, SC areas correlated with disease duration and functional status. These data suggest that quantitative MRI of the SC may be a useful biomarker in the disease.