Automatic camera control in virtual environments augmented using multiple sparse videos

Automated virtual camera control has been widely used in animation and interactive virtual environments. We have developed a multiple sparse camera based free view video system prototype that allows users to control the position and orientation of a virtual camera, enabling the observation of a real scene in three dimensions (3D) from any desired viewpoint. Automatic camera control can be activated to follow selected objects by the user. Our method combines a simple geometric model of the scene composed of planes (virtual environment), augmented with visual information from the cameras and pre-computed tracking information of moving targets to generate novel perspective corrected 3D views of the virtual camera and moving objects. To achieve real-time rendering performance, view-dependent textured mapped billboards are used to render the moving objects at their correct locations and foreground masks are used to remove the moving objects from the projected video streams. The current prototype runs on a PC with a common graphics card and can generate virtual 2D views from three cameras of resolution 768 x 576 with several moving objects at about 11 fps. (C)2011 Elsevier Ltd. All rights reserved.

A low cost one-camera optical tracking system for indoor wide-area augmented and virtual reality environments

In the last years the number of industrial applications for Augmented Reality (AR) and Virtual Reality (VR) environments has significantly increased. Optical tracking systems are an important component of AR/VR environments. In this work, a low cost optical tracking system with adequate attributes for professional use is proposed. The system works in infrared spectral region to reduce optical noise. A highspeed camera, equipped with daylight blocking filter and infrared flash strobes, transfers uncompressed grayscale images to a regular PC, where image pre-processing software and the PTrack tracking algorithm recognize a set of retro-reflective markers and extract its 3D position and orientation. Included in this work is a comprehensive research on image pre-processing and tracking algorithms. A testbed was built to perform accuracy and precision tests. Results show that the system reaches accuracy and precision levels slightly worse than but still comparable to professional systems. Due to its modularity, the system can be expanded by using several one-camera tracking modules linked by a sensor fusion algorithm, in order to obtain a larger working range. A setup with two modules was built and tested, resulting in performance similar to the stand-alone configuration.

A low-cost 3D reconstruction system using a single-shot projection of a pattern matrix

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

The influence of subpixel measurement on digital camera calibration

The aim of this work is to evaluate the influence of point measurements in images, with subpixel accuracy, and its contribution in the calibration of digital cameras. Also, the effect of subpixel measurements in 3D coordinates of check points in the object space will be evaluated. With this purpose, an algorithm that allows subpixel accuracy was implemented for semi-automatic determination of points of interest, based on Fõrstner operator. Experiments were accomplished with a block of images acquired with the multispectral camera DuncanTech MS3100-CIR. The influence of subpixel measurements in the adjustment by Least Square Method (LSM) was evaluated by the comparison of estimated standard deviation of parameters in both situations, with manual measurement (pixel accuracy) and with subpixel estimation. Additionally, the influence of subpixel measurements in the 3D reconstruction was also analyzed. Based on the obtained results, i.e., on the quantification of the standard deviation reduction in the Inner Orientation Parameters (IOP) and also in the relative error of the 3D reconstruction, it was shown that measurements with subpixel accuracy are relevant for some tasks in Photogrammetry, mainly for those in which the metric quality is of great relevance, as Camera Calibration.

Interação com objetos digitais 3D em estúdios virtuais

Pós-graduação em Televisão Digital: Informação e Conhecimento - FAAC

3D reconstruction from a vascular tree model

[EN] In this paper, we present a vascular tree model made with synthetic materials and which allows us to obtain images to make a 3D reconstruction.We have used PVC tubes of several diameters and lengths that will let us evaluate the accuracy of our 3D reconstruction. In order to calibrate the camera we have used a corner detector. Also we have used Optical Flow techniques to follow the points through the images going and going back. We describe two general techniques to extract a sequence of corresponding points from multiple views of an object. The resulting sequence of points will be used later to reconstruct a set of 3D points representing the object surfaces on the scene. We have made the 3D reconstruction choosing by chance a couple of images and we have calculated the projection error. After several repetitions, we have found the best 3D location for the point.

Model of vessels for the 3D reconstruction

[EN] In this paper, we present a vascular tree model made with synthetic materials and which allows us to obtain images to make a 3D reconstruction. In order to create this model, we have used PVC tubes of several diameters and lengths that will let us evaluate the accuracy of our 3D reconstruction. We have made the 3D reconstruction from a series of images that we have from our model and after we have calibrated the camera. In order to calibrate it we have used a corner detector. Also we have used Optical Flow techniques to follow the points through the images going and going back. Once we have the set of images where we have located a point, we have made the 3D reconstruction choosing by chance a couple of images and we have calculated the projection error. After several repetitions, we have found the best 3D location for the point.

Regularizing a set of unstructured 3D points from a sequence of stereo images

[EN] In this paper we present a method for the regularization of a set of unstructured 3D points obtained from a sequence of stereo images. This method takes into account the information supplied by the disparity maps computed between pairs of images to constraint the regularization of the set of 3D points. We propose a model based on an energy which is composed of two terms: an attachment term that minimizes the distance from 3D points to the projective lines of camera points, and a second term that allows for the regularization of the set of 3D points by preserving discontinuities presented on the disparity maps. We embed this energy in a 2D finite element method. After minimizing, this method results in a large system of equations that can be optimized for fast computations. We derive an efficient implicit numerical scheme which reduces the number of calculations and memory allocations.

Ottimizzazione di metodi numerici e realizzazione di un setup per l'imaging di colture cellulari 3D con tomografia ad impedenza elettrica

Il presente lavoro di tesi presenta la progettazione, realizzazione e applicazione di un setup sperimentale miniaturizzato per la ricostruzione di immagine, con tecnica di Tomografia ad Impedenza Elettrica (EIT). Il lavoro descritto nel presente elaborato costituisce uno studio di fattibilità preliminare per ricostruire la posizione di piccole porzioni di tessuto (ordine di qualche millimetro) o aggregati cellulari dentro uno scaffold in colture tissutali o cellulari 3D. Il setup disegnato incorpora 8 elettrodi verticali disposti alla periferia di una camera di misura circolare del diametro di 10 mm. Il metodo di analisi EIT è stato svolto utilizzando i) elettrodi conduttivi per tutta l’altezza della camera (usati nel modello EIT bidimensionale e quasi-bidimensionale) e ii) elettrodi per deep brain stimulation (conduttivi esclusivamente su un ridotto volume in punta e posti a tre diverse altezze: alto, centro e basso) usati nel modello EIT tridimensionale. Il metodo ad elementi finiti (FEM) è stato utilizzato per la soluzione sia del problema diretto che del problema inverso, con la ricostruzione della mappa di distribuzione della conduttività entro la camera di misura. Gli esperimenti svolti hanno permesso di ricostruire la mappa di distribuzione di conduttività relativa a campioni dell’ordine del millimetro di diametro. Tali dimensioni sono compatibili con quelle dei campioni oggetto di studio in ingegneria tissutale e, anche, con quelle tipiche dei sistemi organ-on-a-chip. Il metodo EIT sviluppato, il prototipo del setup realizzato e la trattazione statistica dei dati sono attualmente in fase di implementazione in collaborazione con il gruppo del Professor David Holder, Dept. Medical Physics and Bioengineering, University College London (UCL), United Kingdom.

Ricostruzione 3D di oggetti ad alto potere diffusivo mediante triangolazione laser

L'obiettivo principale che questa tesi intende perseguire e la progettazione di un sistema di acquisizione 3D a basso costo basato sul metodo di triangolazione a lama laser. A tale scopo si adotterà una telecamera general-purpose, e si provvederà sia alla realizzazione di un prototipo di sistema di attuazione per automatizzare la scansione dell'oggetto, sia all'implementazione di algoritmi per la calibrazione della geometria laser-telecamera, il rilevamento del profilo laser nell'immagine e la sua successiva ricostruzione 3D, tenendo anche conto delle complicazioni che l'uso di oggetti ad alto potere diffusivo apporta a tale problema, a causa dei marcati effetti di subsurface scattering della luce laser. Il sistema di triangolazione sarà validato caratterizzando, mediante il confronto con pezzi di dimensioni note, la precisione e l'accuratezza delle misurazioni, e valutando i tempi di esecuzione degli algoritmi. Inoltre, le prestazioni ed i risultati delle acquisizioni saranno messi a confronto con quelli ottenuti adottando una telecamera high-end dedicata. Lo studio preliminare svolto in questa sede e propedeutico per la futura realizzazione di un sistema per la ricostruzione in camera bianca di protesi d'osso su misura.

Determining bridge deck deterioration through the use of 3D photogrammetry

The bridge inspection industry has yet to utilize a rapidly growing technology that shows promise to help improve the inspection process. This thesis investigates the abilities that 3D photogrammetry is capable of providing to the bridge inspector for a number of deterioration mechanisms. The technology can provide information about the surface condition of some bridge components, primarily focusing on the surface defects of a concrete bridge which include cracking, spalling and scaling. Testing was completed using a Canon EOS 7D camera which then processed photos using AgiSoft PhotoScan to align the photos and develop models. Further processing of the models was done using ArcMap in the ArcGIS 10 program to view the digital elevation models of the concrete surface. Several experiments were completed to determine the ability of the technique for the detection of the different defects. The cracks that were able to be resolved in this study were a 1/8 inch crack at a distance of two feet above the surface. 3D photogrammetry was able to be detect a depression of 1 inch wide with 3/16 inch depth which would be sufficient to measure any scaling or spalling that would be required be the inspector. The percentage scaled or spalled was also able to be calculated from the digital elevation models in ArcMap. Different camera factors including the distance from the defects, number of photos and angle, were also investigated to see how each factor affected the capabilities. 3D photogrammetry showed great promise in the detection of scaling or spalling of the concrete bridge surface.

Automatic Food Intake Assessment Using Camera Phones

Obesity is becoming an epidemic phenomenon in most developed countries. The fundamental cause of obesity and overweight is an energy imbalance between calories consumed and calories expended. It is essential to monitor everyday food intake for obesity prevention and management. Existing dietary assessment methods usually require manually recording and recall of food types and portions. Accuracy of the results largely relies on many uncertain factors such as user's memory, food knowledge, and portion estimations. As a result, the accuracy is often compromised. Accurate and convenient dietary assessment methods are still blank and needed in both population and research societies. In this thesis, an automatic food intake assessment method using cameras, inertial measurement units (IMUs) on smart phones was developed to help people foster a healthy life style. With this method, users use their smart phones before and after a meal to capture images or videos around the meal. The smart phone will recognize food items and calculate the volume of the food consumed and provide the results to users. The technical objective is to explore the feasibility of image based food recognition and image based volume estimation. This thesis comprises five publications that address four specific goals of this work: (1) to develop a prototype system with existing methods to review the literature methods, find their drawbacks and explore the feasibility to develop novel methods; (2) based on the prototype system, to investigate new food classification methods to improve the recognition accuracy to a field application level; (3) to design indexing methods for large-scale image database to facilitate the development of new food image recognition and retrieval algorithms; (4) to develop novel convenient and accurate food volume estimation methods using only smart phones with cameras and IMUs. A prototype system was implemented to review existing methods. Image feature detector and descriptor were developed and a nearest neighbor classifier were implemented to classify food items. A reedit card marker method was introduced for metric scale 3D reconstruction and volume calculation. To increase recognition accuracy, novel multi-view food recognition algorithms were developed to recognize regular shape food items. To further increase the accuracy and make the algorithm applicable to arbitrary food items, new food features, new classifiers were designed. The efficiency of the algorithm was increased by means of developing novel image indexing method in large-scale image database. Finally, the volume calculation was enhanced through reducing the marker and introducing IMUs. Sensor fusion technique to combine measurements from cameras and IMUs were explored to infer the metric scale of the 3D model as well as reduce noises from these sensors.

Lag Camera: A Moving Multi-Camera Array for Scene-Acquisition

Many applications, such as telepresence, virtual reality, and interactive walkthroughs, require a three-dimensional(3D)model of real-world environments. Methods, such as lightfields, geometric reconstruction and computer vision use cameras to acquire visual samples of the environment and construct a model. Unfortunately, obtaining models of real-world locations is a challenging task. In particular, important environments are often actively in use, containing moving objects, such as people entering and leaving the scene. The methods previously listed have difficulty in capturing the color and structure of the environment while in the presence of moving and temporary occluders. We describe a class of cameras called lag cameras. The main concept is to generalize a camera to take samples over space and time. Such a camera, can easily and interactively detect moving objects while continuously moving through the environment. Moreover, since both the lag camera and occluder are moving, the scene behind the occluder is captured by the lag camera even from viewpoints where the occluder lies in between the lag camera and the hidden scene. We demonstrate an implementation of a lag camera, complete with analysis and captured environments.

Interactive Augmentation of Live Images using a HDR Stereo Camera

Adding virtual objects to real environments plays an important role in todays computer graphics: Typical examples are virtual furniture in a real room and virtual characters in real movies. For a believable appearance, consistent lighting of the virtual objects is required. We present an augmented reality system that displays virtual objects with consistent illumination and shadows in the image of a simple webcam. We use two high dynamic range video cameras with fisheye lenses permanently recording the environment illumination. A sampling algorithm selects a few bright parts in one of the wide angle images and the corresponding points in the second camera image. The 3D position can then be calculated using epipolar geometry. Finally, the selected point lights are used in a multi pass algorithm to draw the virtual object with shadows. To validate our approach, we compare the appearance and shadows of the synthetic objects with real objects.