Efficient Model-based 3D Tracking by Using Direct Image Registration

This thesis deals with the problem of efficiently tracking 3D objects in sequences of images. We tackle the efficient 3D tracking problem by using direct image registration. This problem is posed as an iterative optimization procedure that minimizes a brightness error norm. We review the most popular iterative methods for image registration in the literature, turning our attention to those algorithms that use efficient optimization techniques. Two forms of efficient registration algorithms are investigated. The first type comprises the additive registration algorithms: these algorithms incrementally compute the motion parameters by linearly approximating the brightness error function. We centre our attention on Hager and Belhumeur’s factorization-based algorithm for image registration. We propose a fundamental requirement that factorization-based algorithms must satisfy to guarantee good convergence, and introduce a systematic procedure that automatically computes the factorization. Finally, we also bring out two warp functions to register rigid and nonrigid 3D targets that satisfy the requirement. The second type comprises the compositional registration algorithms, where the brightness function error is written by using function composition. We study the current approaches to compositional image alignment, and we emphasize the importance of the Inverse Compositional method, which is known to be the most efficient image registration algorithm. We introduce a new algorithm, the Efficient Forward Compositional image registration: this algorithm avoids the necessity of inverting the warping function, and provides a new interpretation of the working mechanisms of the inverse compositional alignment. By using this information, we propose two fundamental requirements that guarantee the convergence of compositional image registration methods. Finally, we support our claims by using extensive experimental testing with synthetic and real-world data. We propose a distinction between image registration and tracking when using efficient algorithms. We show that, depending whether the fundamental requirements are hold, some efficient algorithms are eligible for image registration but not for tracking.

A parallel implementation of 3D Zernike moment analysis

Zernike polynomials are a well known set of functions that find many applications in image or pattern characterization because they allow to construct shape descriptors that are invariant against translations, rotations or scale changes. The concepts behind them can be extended to higher dimension spaces, making them also fit to describe volumetric data. They have been less used than their properties might suggest due to their high computational cost. We present a parallel implementation of 3D Zernike moments analysis, written in C with CUDA extensions, which makes it practical to employ Zernike descriptors in interactive applications, yielding a performance of several frames per second in voxel datasets about 2003 in size. In our contribution, we describe the challenges of implementing 3D Zernike analysis in a general-purpose GPU. These include how to deal with numerical inaccuracies, due to the high precision demands of the algorithm, or how to deal with the high volume of input data so that it does not become a bottleneck for the system.

IDT-3D: Identification and tracking in controlled environments using a 3D unified user interface

Identification and tracking of objects in specific environments such as harbors or security areas is a matter of great importance nowadays. With this purpose, numerous systems based on different technologies have been developed, resulting in a great amount of gathered data displayed through a variety of interfaces. Such amount of information has to be evaluated by human operators in order to take the correct decisions, sometimes under highly critical situations demanding both speed and accuracy. In order to face this problem we describe IDT-3D, a platform for identification and tracking of vessels in a harbour environment able to represent fused information in real time using a Virtual Reality application. The effectiveness of using IDT-3D as an integrated surveillance system is currently under evaluation. Preliminary results point to a significant decrease in the times of reaction and decision making of operators facing up a critical situation. Although the current application focus of IDT-3D is quite specific, the results of this research could be extended to the identification and tracking of targets in other controlled environments of interest as coastlines, borders or even urban areas.

Image sub-segmentation by PFCM and Artificial Neural Networks to detect pore space in 2D and 3D CT soil images

The image by Computed Tomography is a non-invasive alternative for observing soil structures, mainly pore space. The pore space correspond in soil data to empty or free space in the sense that no material is present there but only fluids, the fluid transport depend of pore spaces in soil, for this reason is important identify the regions that correspond to pore zones. In this paper we present a methodology in order to detect pore space and solid soil based on the synergy of the image processing, pattern recognition and artificial intelligence. The mathematical morphology is an image processing technique used for the purpose of image enhancement. In order to find pixels groups with a similar gray level intensity, or more or less homogeneous groups, a novel image sub-segmentation based on a Possibilistic Fuzzy c-Means (PFCM) clustering algorithm was used. The Artificial Neural Networks (ANNs) are very efficient for demanding large scale and generic pattern recognition applications for this reason finally a classifier based on artificial neural network is applied in order to classify soil images in two classes, pore space and solid soil respectively.

Multi-Resolution Texture Coding for Multi-Resolution 3D Meshes

We present an innovative system to encode and transmit textured multi-resolution 3D meshes in a progressive way, with no need to send several texture images, one for each mesh LOD (Level Of Detail). All texture LODs are created from the finest one (associated to the finest mesh), but can be re- constructed progressively from the coarsest thanks to refinement images calculated in the encoding process, and transmitted only if needed. This allows us to adjust the LOD/quality of both 3D mesh and texture according to the rendering power of the device that will display them, and to the network capacity. Additionally, we achieve big savings in data transmission by avoiding altogether texture coordinates, which are generated automatically thanks to an unwrapping system agreed upon by both encoder and decoder.

Diseño y acondicionamiento acústico y electroacústico de una sala de proyecciones en 3D

Este proyecto está orientado al diseño y el acondicionamiento de una sala de cine siguiendo las normas establecidas por el SMPTE. El primer paso a realizar será el diseño de la sala en el cual habrá que tener en cuenta la distribución de los asientos dentro de la misma, el dimensionado de la pantalla que servirá para establecer la forma y dimensiones del recinto, así como la correcta ubicación del proyector. Posteriormente se realizará el acondicionamiento acústico del cine, con la elección de los diferentes materiales que permitan la obtención de un tiempo de reverberación óptimo. A continuación se procederá a la selección de los equipos electroacústicos más adecuados y a su colocación a lo largo de la sala para posteriormente realizar un estudio de todos los parámetros de esta para garantizar la perfecta escucha dentro de la misma. Se elegirán, al igual que se ha hecho con los elementos electroacústicos, los equipos de video específicos, teniendo en cuenta el sistema de proyección 3D utilizado y se procederá a su instalación dentro de la sala. Se indicará de forma independiente cual será el esquema de conexionado correspondiente a cada una de las partes, tanto de audio como de video. Todos los equipos y parámetros ajustables de la sala, tanto de audio como de video, se realizaran siguiendo las recomendaciones establecidas por el SMPTE para una correcta visión y escucha, así como también el diseño de la sala. Para llevar a cabo todo lo anteriormente descrito se utilizara el programa de simulación EASE 4.3 con él que se ajustaran los parámetros más significativos para verificar que la sala cumple con las condiciones de escucha que determina la norma. Todo esto irá acompañado de un presupuesto detallado de cada uno de los equipos y materiales utilizados, así como de los costes derivados de la mano de obra. Se adjuntarán también los planos de la sala donde se indicarán todas las medidas establecidas a lo largo del proyecto. Para la realización de estos se utilizara el programa de diseño Google SkechUp. Por último se facilitarán las hojas de características de cada uno de los equipos instalados en la sala para conocer sus especificaciones y modo de funcionamiento. Abstract This project is orientated at designing and conditioning a cinema according to standards set by the SMPTE. First of all, the cinema hall needs to be designed, taking into consideration seat distribution and screen dimension, in order to establish the shape and dimensions of the room and the correct location for the projector. Later the acoustic conditioning of the cinema is covered, with the choice of appropriate materials in order to permit an optimum reverberation time. The next step is the selection of the most appropriate electro-acoustic equipment and its positioning throughout the room. A study is then carried out of all the parameters to ensure perfect hearing in the cinema. Then the specific video equipment is chosen, bearing in mind the 3D projection system used and is installed in the theatre. A wiring diagram is indicated for each element used, for both audio and video. All equipment and adjustable parameters of the room, both audio and video, are made according to the recommendations established by the SMPTE for correct viewing and listening, as is the design of the cinema. To carry out the steps described above the EASE 4.3 simulation program is used. This program adjusts all significant parameters to verify that the room complies with the listening conditions determined by the standard. A detailed budget is included for all equipment and materials used, as well as the labour costs. Plans of the room, showing all measurements taken during the project are indicated. This is done using the Google SkechUp program. Finally data sheets are provided for each piece of equipment installed in the room detailing specifications and operating mode.

MEDVIR: 3D visual interface applied to gene profile analisys.

The origins for this work arise in response to the increasing need for biologists and doctors to obtain tools for visual analysis of data. When dealing with multidimensional data, such as medical data, the traditional data mining techniques can be a tedious and complex task, even to some medical experts. Therefore, it is necessary to develop useful visualization techniques that can complement the expert’s criterion, and at the same time visually stimulate and make easier the process of obtaining knowledge from a dataset. Thus, the process of interpretation and understanding of the data can be greatly enriched. Multidimensionality is inherent to any medical data, requiring a time-consuming effort to get a clinical useful outcome. Unfortunately, both clinicians and biologists are not trained in managing more than four dimensions. Specifically, we were aimed to design a 3D visual interface for gene profile analysis easy in order to be used both by medical and biologist experts. In this way, a new analysis method is proposed: MedVir. This is a simple and intuitive analysis mechanism based on the visualization of any multidimensional medical data in a three dimensional space that allows interaction with experts in order to collaborate and enrich this representation. In other words, MedVir makes a powerful reduction in data dimensionality in order to represent the original information into a three dimensional environment. The experts can interact with the data and draw conclusions in a visual and quickly way.

Simplifying data acquisition in plant canopies- Measurements of leaf angles with a cell phone

1. Canopies are complex multilayered structures comprising individual plant crowns exposing a multifaceted surface area to sunlight. Foliage arrangement and properties are the main mediators of canopy functions. The leaves act as light traps whose exposure to sunlight varies with time of the day, date and latitude in a trade-off between photosynthetic light harvesting and excessive or photoinhibitory light avoidance. To date, ecological research based upon leaf sampling has been limited by the available echnology, with which data acquisition becomes labour intensive and time-consuming, given the verwhelming number of leaves involved. 2. In the present study, our goal involved developing a tool capable of easuring a sufficient number of leaves to enable analysis of leaf populations, tree crowns and canopies.We specifically tested whether a cell phone working as a 3Dpointer could yield reliable, repeatable and valid leaf anglemeasurements with a simple gesture. We evaluated the accuracy of this method under controlled conditions, using a 3D digitizer, and we compared performance in the field with the methods commonly used. We presented an equation to estimate the potential proportion of the leaf exposed to direct sunlight (SAL) at any given time and compared the results with those obtained bymeans of a graphicalmethod. 3. We found a strong and highly significant correlation between the graphical methods and the equation presented. The calibration process showed a strong correlation between the results derived from the two methods with amean relative difference below 10%. Themean relative difference in calculation of instantaneous exposure was below 5%. Our device performed equally well in diverse locations, in which we characterized over 700 leaves in a single day. 4. The newmethod, involving the use of a cell phone, ismuchmore effective than the traditionalmethods or digitizers when the goal is to scale up from leaf position to performance of leaf populations, tree crowns or canopies. Our methodology constitutes an affordable and valuable tool within which to frame a wide range of ecological hypotheses and to support canopy modelling approaches.

MedVir: 3D visual interface applied to gene profile analysis

The use of data mining techniques for the gene profile discovery of diseases, such as cancer, is becoming usual in many researches. These techniques do not usually analyze the relationships between genes in depth, depending on the different variety of manifestations of the disease (related to patients). This kind of analysis takes a considerable amount of time and is not always the focus of the research. However, it is crucial in order to generate personalized treatments to fight the disease. Thus, this research focuses on finding a mechanism for gene profile analysis to be used by the medical and biologist experts. Results: In this research, the MedVir framework is proposed. It is an intuitive mechanism based on the visualization of medical data such as gene profiles, patients, clinical data, etc. MedVir, which is based on an Evolutionary Optimization technique, is a Dimensionality Reduction (DR) approach that presents the data in a three dimensional space. Furthermore, thanks to Virtual Reality technology, MedVir allows the expert to interact with the data in order to tailor it to the experience and knowledge of the expert.

Accurate depth-color scene modeling for 3D contents generation with low cost depth cameras

In this paper, we present a depth-color scene modeling strategy for indoors 3D contents generation. It combines depth and visual information provided by a low-cost active depth camera to improve the accuracy of the acquired depth maps considering the different dynamic nature of the scene elements. Accurate depth and color models of the scene background are iteratively built, and used to detect moving elements in the scene. The acquired depth data is continuously processed with an innovative joint-bilateral filter that efficiently combines depth and visual information thanks to the analysis of an edge-uncertainty map and the detected foreground regions. The main advantages of the proposed approach are: removing depth maps spatial noise and temporal random fluctuations; refining depth data at object boundaries, generating iteratively a robust depth and color background model and an accurate moving object silhouette.

3D facial merging for virtual human reconstruction

There is an increasing need of easy and affordable technologies to automatically generate virtual 3D models from their real counterparts. In particular, 3D human reconstruction has driven the creation of many clever techniques, most of them based on the visual hull (VH) concept. Such techniques do not require expensive hardware; however, they tend to yield 3D humanoids with realistic bodies but mediocre faces, since VH cannot handle concavities. On the other hand, structured light projectors allow to capture very accurate depth data, and thus to reconstruct realistic faces, but they are too expensive to use several of them. We have developed a technique to merge a VH-based 3D mesh of a reconstructed humanoid and the depth data of its face, captured by a single structured light projector. By combining the advantages of both systems in a simple setting, we are able to reconstruct realistic 3D human models with believable faces.

Background foreground segmentation with RGB-D Kinect data: An efficient combination of classifiers

Low cost RGB-D cameras such as the Microsoft’s Kinect or the Asus’s Xtion Pro are completely changing the computer vision world, as they are being successfully used in several applications and research areas. Depth data are particularly attractive and suitable for applications based on moving objects detection through foreground/background segmentation approaches; the RGB-D applications proposed in literature employ, in general, state of the art foreground/background segmentation techniques based on the depth information without taking into account the color information. The novel approach that we propose is based on a combination of classifiers that allows improving background subtraction accuracy with respect to state of the art algorithms by jointly considering color and depth data. In particular, the combination of classifiers is based on a weighted average that allows to adaptively modifying the support of each classifier in the ensemble by considering foreground detections in the previous frames and the depth and color edges. In this way, it is possible to reduce false detections due to critical issues that can not be tackled by the individual classifiers such as: shadows and illumination changes, color and depth camouflage, moved background objects and noisy depth measurements. Moreover, we propose, for the best of the author’s knowledge, the first publicly available RGB-D benchmark dataset with hand-labeled ground truth of several challenging scenarios to test background/foreground segmentation algorithms.