Self-calibrated, multi-spectral photometric stereo for 3D face capture

This paper addresses the problem of obtaining 3d detailed reconstructions of human faces in real-time and with inexpensive hardware. We present an algorithm based on a monocular multi-spectral photometric-stereo setup. This system is known to capture high-detailed deforming 3d surfaces at high frame rates and without having to use any expensive hardware or synchronized light stage. However, the main challenge of such a setup is the calibration stage, which depends on the lights setup and how they interact with the specific material being captured, in this case, human faces. For this purpose we develop a self-calibration technique where the person being captured is asked to perform a rigid motion in front of the camera, maintaining a neutral expression. Rigidity constrains are then used to compute the head's motion with a structure-from-motion algorithm. Once the motion is obtained, a multi-view stereo algorithm reconstructs a coarse 3d model of the face. This coarse model is then used to estimate the lighting parameters with a stratified approach: In the first step we use a RANSAC search to identify purely diffuse points on the face and to simultaneously estimate this diffuse reflectance model. In the second step we apply non-linear optimization to fit a non-Lambertian reflectance model to the outliers of the previous step. The calibration procedure is validated with synthetic and real data.

Integration of a 3D hydrogel matrix within a hollow core photonic crystal fibre for DNA probe immobilization

In this paper, we demonstrate the integration of a 3D hydrogel matrix within a hollow core photonic crystal fibre (HC-PCF). In addition, we also show the fluorescence of Cy5-labelled DNA molecules immobilized within the hydrogel formed in two different types of HC-PCF. The 3D hydrogel matrix is designed to bind with the amino groups of biomolecules using an appropriate cross-linker, providing higher sensitivity and selectivity than the standard 2D coverage, enabling a greater number of probe molecules to be available per unit area. The HC-PCFs, on the other hand, can be designed to maximize the capture of fluorescence to improve sensitivity and provide longer interaction lengths. This could enable the development of fibre-based point-of-care and remote systems, where the enhanced sensitivity would relax the constraints placed on sources and detectors. In this paper, we will discuss the formation of such polyethylene glycol diacrylate (PEGDA) hydrogels within a HC-PCF, including their optical properties such as light propagation and auto-fluorescence.

An integrative approach to face and expression recognition from 3D scans

Biometrics is afield of study which pursues the association of a person's identity with his/her physiological or behavioral characteristics.^ As one aspect of biometrics, face recognition has attracted special attention because it is a natural and noninvasive means to identify individuals. Most of the previous studies in face recognition are based on two-dimensional (2D) intensity images. Face recognition based on 2D intensity images, however, is sensitive to environment illumination and subject orientation changes, affecting the recognition results. With the development of three-dimensional (3D) scanners, 3D face recognition is being explored as an alternative to the traditional 2D methods for face recognition.^ This dissertation proposes a method in which the expression and the identity of a face are determined in an integrated fashion from 3D scans. In this framework, there is a front end expression recognition module which sorts the incoming 3D face according to the expression detected in the 3D scans. Then, scans with neutral expressions are processed by a corresponding 3D neutral face recognition module. Alternatively, if a scan displays a non-neutral expression, e.g., a smiling expression, it will be routed to an appropriate specialized recognition module for smiling face recognition.^ The expression recognition method proposed in this dissertation is innovative in that it uses information from 3D scans to perform the classification task. A smiling face recognition module was developed, based on the statistical modeling of the variance between faces with neutral expression and faces with a smiling expression.^ The proposed expression and face recognition framework was tested with a database containing 120 3D scans from 30 subjects (Half are neutral faces and half are smiling faces). It is shown that the proposed framework achieves a recognition rate 10% higher than attempting the identification with only the neutral face recognition module.^

Codificação compatível de vídeo 3D com o algoritmo HEVC

Esta dissertação apresenta um trabalho sobre codificação de vídeo 3D compatível com vídeo 2D. Tem por base o desenvolvimento de um método para melhorar, no descodificador, a reconstrução de uma vista subamostrada resultante de uma transmissão simulcast usando a norma de codificação de vídeo H.265 (informalmente denominada de High Efficiency Video Coding (HEVC)). Apesar de manter a compatibilidade com vídeo 2D a transmissão simulcast normalmente requer uma taxa de transmissão elevada. Na ausência de ferramentas de codificação 3D adequadas é possível reduzir a taxa de transmissão utilizando compressão assimétrica do vídeo, onde a vista base é codificada com a resolução espacial original, enquanto que a vista auxiliar é codificada com uma resolução espacial menor, sendo sobreamostrada no descodificador. O método desenvolvido visa melhorar a vista auxiliar sobreamostrada no descodificador utilizando informação dos detalhes da vista base, ou seja, as componentes de alta frequência. Este processo depende de transformadas Afim para realizar um mapeamento geométrico entre a informação de alta frequência da vista base de resolução completa e a vista auxiliar de menor resolução. Adicionalmente, de modo a manter a continuidade do conteúdo da imagem entre regiões, evitando artefatos de blocos, o mapeamento utiliza uma malha de triangulação da vista auxiliar aplicado à imagem de detalhes obtida a partir da vista base. A técnica proposta é comparada com um método de estimação de disparidade por correspondência de blocos, sendo que os resultados mostram que para algumas sequências a técnica desenvolvida melhora não só a qualidade objetiva (PSNR) até 2.2 dB, mas também a qualidade subjetiva, para a mesma taxa de compressão global.

Practical 3D reconstruction based on photometric stereo

Photometric Stereo is a powerful image based 3D reconstruction technique that has recently been used to obtain very high quality reconstructions. However, in its classic form, Photometric Stereo suffers from two main limitations: Firstly, one needs to obtain images of the 3D scene under multiple different illuminations. As a result the 3D scene needs to remain static during illumination changes, which prohibits the reconstruction of deforming objects. Secondly, the images obtained must be from a single viewpoint. This leads to depth-map based 2.5 reconstructions, instead of full 3D surfaces. The aim of this Chapter is to show how these limitations can be alleviated, leading to the derivation of two practical 3D acquisition systems: The first one, based on the powerful Coloured Light Photometric Stereo method can be used to reconstruct moving objects such as cloth or human faces. The second, permits the complete 3D reconstruction of challenging objects such as porcelain vases. In addition to algorithmic details, the Chapter pays attention to practical issues such as setup calibration, detection and correction of self and cast shadows. We provide several evaluation experiments as well as reconstruction results. © 2010 Springer-Verlag Berlin Heidelberg.

Advanced Applications of 3D Dosimetry and 3D Printing in Radiation Therapy

As complex radiotherapy techniques become more readily-practiced, comprehensive 3D dosimetry is a growing necessity for advanced quality assurance. However, clinical implementation has been impeded by a wide variety of factors, including the expense of dedicated optical dosimeter readout tools, high operational costs, and the overall difficulty of use. To address these issues, a novel dry-tank optical CT scanner was designed for PRESAGE 3D dosimeter readout, relying on 3D printed components and omitting costly parts from preceding optical scanners. This work details the design, prototyping, and basic commissioning of the Duke Integrated-lens Optical Scanner (DIOS).

The convex scanning geometry was designed in ScanSim, an in-house Monte Carlo optical ray-tracing simulation. ScanSim parameters were used to build a 3D rendering of a convex ‘solid tank’ for optical-CT, which is capable of collimating a point light source into telecentric geometry without significant quantities of refractive-index matched fluid. The model was 3D printed, processed, and converted into a negative mold via rubber casting to produce a transparent polyurethane scanning tank. The DIOS was assembled with the solid tank, a 3W red LED light source, a computer-controlled rotation stage, and a 12-bit CCD camera. Initial optical phantom studies show negligible spatial inaccuracies in 2D projection images and 3D tomographic reconstructions. A PRESAGE 3D dose measurement for a 4-field box treatment plan from Eclipse shows 95% of voxels passing gamma analysis at 3%/3mm criteria. Gamma analysis between tomographic images of the same dosimeter in the DIOS and DLOS systems show 93.1% agreement at 5%/1mm criteria. From this initial study, the DIOS has demonstrated promise as an economically-viable optical-CT scanner. However, further improvements will be necessary to fully develop this system into an accurate and reliable tool for advanced QA.

Pre-clinical animal studies are used as a conventional means of translational research, as a midpoint between in-vitro cell studies and clinical implementation. However, modern small animal radiotherapy platforms are primitive in comparison with conventional linear accelerators. This work also investigates a series of 3D printed tools to expand the treatment capabilities of the X-RAD 225Cx orthovoltage irradiator, and applies them to a feasibility study of hippocampal avoidance in rodent whole-brain radiotherapy.

As an alternative material to lead, a novel 3D-printable tungsten-composite ABS plastic, GMASS, was tested to create precisely-shaped blocks. Film studies show virtually all primary radiation at 225 kVp can be attenuated by GMASS blocks of 0.5cm thickness. A state-of-the-art software, BlockGen, was used to create custom hippocampus-shaped blocks from medical image data, for any possible axial treatment field arrangement. A custom 3D printed bite block was developed to immobilize and position a supine rat for optimal hippocampal conformity. An immobilized rat CT with digitally-inserted blocks was imported into the SmART-Plan Monte-Carlo simulation software to determine the optimal beam arrangement. Protocols with 4 and 7 equally-spaced fields were considered as viable treatment options, featuring improved hippocampal conformity and whole-brain coverage when compared to prior lateral-opposed protocols. Custom rodent-morphic PRESAGE dosimeters were developed to accurately reflect these treatment scenarios, and a 3D dosimetry study was performed to confirm the SmART-Plan simulations. Measured doses indicate significant hippocampal sparing and moderate whole-brain coverage.

Towards Mapping of Rock Walls Using a UAV-Mounted 2D Laser Scanner in GPS Denied Environments

In geotechnical engineering, the stability of rock excavations and walls is estimated by using tools that include a map of the orientations of exposed rock faces. However, measuring these orientations by using conventional methods can be time consuming, sometimes dangerous, and is limited to regions of the exposed rock that are reachable by a human. This thesis introduces a 2D, simulated, quadcopter-based rock wall mapping algorithm for GPS denied environments such as underground mines or near high walls on surface. The proposed algorithm employs techniques from the field of robotics known as simultaneous localization and mapping (SLAM) and is a step towards 3D rock wall mapping. Not only are quadcopters agile, but they can hover. This is very useful for confined spaces such as underground or near rock walls. The quadcopter requires sensors to enable self localization and mapping in dark, confined and GPS denied environments. However, these sensors are limited by the quadcopter payload and power restrictions. Because of these restrictions, a light weight 2D laser scanner is proposed. As a first step towards a 3D mapping algorithm, this thesis proposes a simplified scenario in which a simulated 1D laser range finder and 2D IMU are mounted on a quadcopter that is moving on a plane. Because the 1D laser does not provide enough information to map the 2D world from a single measurement, many measurements are combined over the trajectory of the quadcopter. Least Squares Optimization (LSO) is used to optimize the estimated trajectory and rock face for all data collected over the length of a light. Simulation results show that the mapping algorithm developed is a good first step. It shows that by combining measurements over a trajectory, the scanned rock face can be estimated using a lower-dimensional range sensor. A swathing manoeuvre is introduced as a way to promote loop closures within a short time period, thus reducing accumulated error. Some suggestions on how to improve the algorithm are also provided.

Face recognition using a unified 3D morphable model

We address the problem of 3D-assisted 2D face recognition in scenarios when the input image is subject to degradations or exhibits intra-personal variations not captured by the 3D model. The proposed solution involves a novel approach to learn a subspace spanned by perturbations caused by the missing modes of variation and image degradations, using 3D face data reconstructed from 2D images rather than 3D capture. This is accomplished by modelling the difference in the texture map of the 3D aligned input and reference images. A training set of these texture maps then defines a perturbation space which can be represented using PCA bases. Assuming that the image perturbation subspace is orthogonal to the 3D face model space, then these additive components can be recovered from an unseen input image, resulting in an improved fit of the 3D face model. The linearity of the model leads to efficient fitting. Experiments show that our method achieves very competitive face recognition performance on Multi-PIE and AR databases. We also present baseline face recognition results on a new data set exhibiting combined pose and illumination variations as well as occlusion.

Creazione di una costmap 2D per la navigazione di un rover terrestre

L’obiettivo di questa tesi è riuscire ad elaborare una point cloud 3D proveniente dal laser scanner per individuare possibili ostacoli e creare con essa, successivamente, una mappa che permetta la navigazione di un rover.

Paire stéréoscopique Panomorphe pour la reconstruction 3D d’objets d’intérêt dans une scène

Il existe désormais une grande variété de lentilles panoramiques disponibles sur le marché dont certaines présentant des caractéristiques étonnantes. Faisant partie de cette dernière catégorie, les lentilles Panomorphes sont des lentilles panoramiques anamorphiques dont le profil de distorsion est fortement non-uniforme, ce qui cause la présence de zones de grandissement augmenté dans le champ de vue. Dans un contexte de robotique mobile, ces particularités peuvent être exploitées dans des systèmes stéréoscopiques pour la reconstruction 3D d’objets d’intérêt qui permettent à la fois une bonne connaissance de l’environnement, mais également l’accès à des détails plus fins en raison des zones de grandissement augmenté. Cependant, à cause de leur complexité, ces lentilles sont difficiles à calibrer et, à notre connaissance, aucune étude n’a réellement été menée à ce propos. L’objectif principal de cette thèse est la conception, l’élaboration et l’évaluation des performances de systèmes stéréoscopiques Panomorphes. Le calibrage a été effectué à l’aide d’une technique établie utilisant des cibles planes et d’une boîte à outils de calibrage dont l’usage est répandu. De plus, des techniques mathématiques nouvelles visant à rétablir la symétrie de révolution dans l’image (cercle) et à uniformiser la longueur focale (cercle uniforme) ont été développées pour voir s’il était possible d’ainsi faciliter le calibrage. Dans un premier temps, le champ de vue a été divisé en zones à l’intérieur desquelles la longueur focale instantanée varie peu et le calibrage a été effectué pour chacune d’entre elles. Puis, le calibrage général des systèmes a aussi été réalisé pour tout le champ de vue simultanément. Les résultats ont montré que la technique de calibrage par zone ne produit pas de gain significatif quant à la qualité des reconstructions 3D d’objet d’intérêt par rapport au calibrage général. Cependant, l’étude de cette nouvelle approche a permis de réaliser une évaluation des performances des systèmes stéréoscopiques Panomorphes sur tout le champ de vue et de montrer qu’il est possible d’effectuer des reconstructions 3D de qualité dans toutes les zones. De plus, la technique mathématique du cercle a produit des résultats de reconstructions 3D en général équivalents à l’utilisation des coordonnées originales. Puisqu’il existe des outils de calibrage qui, contrairement à celui utilisé dans ce travail, ne disposent que d’un seul degré de liberté sur la longueur focale, cette technique pourrait rendre possible le calibrage de lentilles Panomorphes à l’aide de ceux-ci. Finalement, certaines conclusions ont pu être dégagées quant aux facteurs déterminants influençant la qualité de la reconstruction 3D à l’aide de systèmes stéréoscopiques Panomorphes et aux caractéristiques à privilégier dans le choix des lentilles. La difficulté à calibrer les optiques Panomorphes en laboratoire a mené à l’élaboration d’une technique de calibrage virtuel utilisant un logiciel de conception optique et une boîte à outils de calibrage. Cette approche a permis d’effectuer des simulations en lien avec l’impact des conditions d’opération sur les paramètres de calibrage et avec l’effet des conditions de calibrage sur la qualité de la reconstruction. Des expérimentations de ce type sont pratiquement impossibles à réaliser en laboratoire mais représentent un intérêt certain pour les utilisateurs. Le calibrage virtuel d’une lentille traditionnelle a aussi montré que l’erreur de reprojection moyenne, couramment utilisée comme façon d’évaluer la qualité d’un calibrage, n’est pas nécessairement un indicateur fiable de la qualité de la reconstruction 3D. Il est alors nécessaire de disposer de données supplémentaires pour juger adéquatement de la qualité d’un calibrage.

3D-grafiikan selaintekniikat, erikoistapauksena karttasovellukset

Harava on karttapohjainen kyselypalvelu, jonka avulla voidaan kerätä tietoa erilaisista kyselykohteista. Harava-kyselypalvelussa kyselyihin voidaan vastata muun muassa tekstikentillä, monivalinnoilla ja merkitsemällä alueita ja pisteitä karttapohjaan. Tutkielman tavoitteena oli löytää Harava-kyselypalvelun 2D-karttojen rinnalle 3D-karttavaihtoehto. Aluksi tutkittiin, mitä eri vaihtoehtoja löytyy 3D-komponenttien esittämiseen selaimessa. Tutkituista vaihtoehdoista parhaimmaksi tähän tarkoitukseen osoittautui WebGL-kirjasto. WebGL-kirjaston käyttö suoraan osoittautui vaikeaksi, joten etsittiin rajapintaa, jonka avulla WebGL-kirjaston käyttö helpottuisi. Käsittelyyn otettiin karttapalveluita sekä 3D-mallien esittämiseen tarkoitettuja JavaScript-kirjastoja, jotka käyttävät WebGL-kirjastoa rajapinnan kautta. Näistä sopivimmaksi osoittautui Cesium. Cesium on JavaScript-kirjasto, jonka avulla voidaan toteuttaa 2D-kartta ja 3D-karttapallo sekä upottaa karttapohjaan 3D-elementtejä.

Programación de videojuegos de puzzles en 2D usando Unity: tutorial y ejemplo práctico del juego.

Unity3d es un motor gráfico para la implementación de videojuegos creado por Unity3d Technologies que tiene el honor de ser una de las principales herramientas que propició el auge de los estudios independientes. Pese a estar enfocado al desarrollo de videojuegos en 3D, los desarrolladores lograban crear contenido en 2D a base de manipulaciones como la cámara y de utilizar herramientas no nativas. Afortunadamente, a partir de la versión 4.3 Unity3d integra herramientas nativas que facilitan esta tarea. Este proyecto consiste en la creación de un tutorial sobre la implementación de un videojuego del género puzles en 2D y enfocado a plataformas móviles, empleando para ello Unity3d y concretamente las nuevas herramientas nativas para 2D. La implementación de un videojuego es una tarea muy complicada y extensa, es por ello que en el tutorial se va a tratar la fase de diseño del videojuego y a nivel de implementación centrarse en dichas herramientas para el 2D, mostrando al final una versión jugable del videojuego implementado. Por otra parte, al ser numerosas las herramientas, es imposible tratar cada una de las herramientas aunque se intentará abarcar lo máximo posible y mencionar las que no se incluyan dentro de la implementación del videojuego. El resultado generado de este proyecto puede constituir una referencia para otras personas interesadas en aprender cómo usar Unity3d para programar videojuegos en 2D, lo cual les allanará seguramente el camino y les suavizará la curva de aprendizaje.

Porosity imaged by a vector projection algorithm correlates with fractal dimension measured on 3D models obtained by microCT

International audience

Physical Design Methodologies for Low Power and Reliable 3D ICs

As the semiconductor industry struggles to maintain its momentum down the path following the Moore's Law, three dimensional integrated circuit (3D IC) technology has emerged as a promising solution to achieve higher integration density, better performance, and lower power consumption. However, despite its significant improvement in electrical performance, 3D IC presents several serious physical design challenges. In this dissertation, we investigate physical design methodologies for 3D ICs with primary focus on two areas: low power 3D clock tree design, and reliability degradation modeling and management. Clock trees are essential parts for digital system which dissipate a large amount of power due to high capacitive loads. The majority of existing 3D clock tree designs focus on minimizing the total wire length, which produces sub-optimal results for power optimization. In this dissertation, we formulate a 3D clock tree design flow which directly optimizes for clock power. Besides, we also investigate the design methodology for clock gating a 3D clock tree, which uses shutdown gates to selectively turn off unnecessary clock activities. Different from the common assumption in 2D ICs that shutdown gates are cheap thus can be applied at every clock node, shutdown gates in 3D ICs introduce additional control TSVs, which compete with clock TSVs for placement resources. We explore the design methodologies to produce the optimal allocation and placement for clock and control TSVs so that the clock power is minimized. We show that the proposed synthesis flow saves significant clock power while accounting for available TSV placement area. Vertical integration also brings new reliability challenges including TSV's electromigration (EM) and several other reliability loss mechanisms caused by TSV-induced stress. These reliability loss models involve complex inter-dependencies between electrical and thermal conditions, which have not been investigated in the past. In this dissertation we set up an electrical/thermal/reliability co-simulation framework to capture the transient of reliability loss in 3D ICs. We further derive and validate an analytical reliability objective function that can be integrated into the 3D placement design flow. The reliability aware placement scheme enables co-design and co-optimization of both the electrical and reliability property, thus improves both the circuit's performance and its lifetime. Our electrical/reliability co-design scheme avoids unnecessary design cycles or application of ad-hoc fixes that lead to sub-optimal performance. Vertical integration also enables stacking DRAM on top of CPU, providing high bandwidth and short latency. However, non-uniform voltage fluctuation and local thermal hotspot in CPU layers are coupled into DRAM layers, causing a non-uniform bit-cell leakage (thereby bit flip) distribution. We propose a performance-power-resilience simulation framework to capture DRAM soft error in 3D multi-core CPU systems. In addition, a dynamic resilience management (DRM) scheme is investigated, which adaptively tunes CPU's operating points to adjust DRAM's voltage noise and thermal condition during runtime. The DRM uses dynamic frequency scaling to achieve a resilience borrow-in strategy, which effectively enhances DRAM's resilience without sacrificing performance. The proposed physical design methodologies should act as important building blocks for 3D ICs and push 3D ICs toward mainstream acceptance in the near future.

Reimaginar el mito de las gucamayas ilustraciones 3D: transición de las guacamayas de seres mágicos a mujeres

El siguiente proyecto, es una propuesta gráfica ilustrada en 3D, basada en la mitología cañari. El estudio realizado esboza breves rasgos de la cosmovisión generada a través del mito de la guacamaya. Donde se revela datos importantes que construye la identidad cultural del pueblo cañari. De igual manera se refleja la ilustración como proceso de creación de conocimientos, donde se comprende y exponen los contextos de la imagen dados desde el estudio anatómico y el uso de programas orientados a la generación de imágenes 2d y 3d. Asimismo se refleja la interpretación del mito expuesto desde la mirada del autor, desplegando potencias particulares que redefinen las practicas tradicionales, permitiendo que la ilustración se piense así misma como una producción y no solo como representación