GrabCutSFM : how 3D information improves unsupervised object segmentation

In this paper, we present an unsupervised graph cut based object segmentation method using 3D information provided by Structure from Motion (SFM), called Grab- CutSFM. Rather than focusing on the segmentation problem using a trained model or human intervention, our approach aims to achieve meaningful segmentation autonomously with direct application to vision based robotics. Generally, object (foreground) and background have certain discriminative geometric information in 3D space. By exploring the 3D information from multiple views, our proposed method can segment potential objects correctly and automatically compared to conventional unsupervised segmentation using only 2D visual cues. Experiments with real video data collected from indoor and outdoor environments verify the proposed approach.

Some Efficient Representations of 3D Information based on Octress

The thesis introduced the octree and addressed the complete nature of problems encountered, while building and imaging system based on octrees. An efficient Bottom-up recursive algorithm and its iterative counterpart for the raster to octree conversion of CAT scan slices, to improve the speed of generating the octree from the slices, the possibility of utilizing the inherent parallesism in the conversion programme is explored in this thesis. The octree node, which stores the volume information in cube often stores the average density information could lead to “patchy”distribution of density during the image reconstruction. In an attempt to alleviate this problem and explored the possibility of using VQ to represent the imformation contained within a cube. Considering the ease of accommodating the process of compressing the information during the generation of octrees from CAT scan slices, proposed use of wavelet transforms to generate the compressed information in a cube. The modified algorithm for generating octrees from the slices is shown to accommodate the eavelet compression easily. Rendering the stored information in the form of octree is a complex task, necessarily because of the requirement to display the volumetric information. The reys traced from each cube in the octree, sum up the density en-route, accounting for the opacities and transparencies produced due to variations in density.

3D information visualisation: An historical perspective

The use of 3D visualisation of digital information is a recent phenomenon. It relies on users understanding 3D perspectival spaces. Questions about the universal access of such spaces has been debated since its inception in the European Renaissance. Perspective has since become a strong cultural influence in Western visual communication. Perspective imaging assists the process of experimenting by the sketching or modelling of ideas. In particular, the recent 3D modelling of an essentially non-dimensional Cyber-space raises questions of how we think about information in general. While alternate methods clearly exist they are rarely explored within the 3D paradigm (such as Chinese isometry). This paper seeks to generate further discussion on the historical background of perspective and its role in underpinning this emergent field. © 2005 IEEE.

2D-3D Face Recognition Based on PCA and Feature Modelling

Hybrid face recognition, using image (2D) and structural (3D) information, has explored the fusion of Nearest Neighbour classifiers. This paper examines the effectiveness of feature modelling for each individual modality, 2D and 3D. Furthermore, it is demonstrated that the fusion of feature modelling techniques for the 2D and 3D modalities yields performance improvements over the individual classifiers. By fusing the feature modelling classifiers for each modality with equal weights the average Equal Error Rate improves from 12.60% for the 2D classifier and 12.10% for the 3D classifier to 7.38% for the Hybrid 2D+3D clasiffier.

Joint 2D and 3D cues for image segmentation towards robotic applications

This thesis investigates the fusion of 3D visual information with 2D image cues to provide 3D semantic maps of large-scale environments in which a robot traverses for robotic applications. A major theme of this thesis was to exploit the availability of 3D information acquired from robot sensors to improve upon 2D object classification alone. The proposed methods have been evaluated on several indoor and outdoor datasets collected from mobile robotic platforms including a quadcopter and ground vehicle covering several kilometres of urban roads.

A model based factorization approach for dense 3D recovery from monocular video

Feature track matrix factorization based methods have been attractive solutions to the Structure-front-motion (Sfnl) problem. Group motion of the feature points is analyzed to get the 3D information. It is well known that the factorization formulations give rise to rank deficient system of equations. Even when enough constraints exist, the extracted models are sparse due the unavailability of pixel level tracks. Pixel level tracking of 3D surfaces is a difficult problem, particularly when the surface has very little texture as in a human face. Only sparsely located feature points can be tracked and tracking error arc inevitable along rotating lose texture surfaces. However, the 3D models of an object class lie in a subspace of the set of all possible 3D models. We propose a novel solution to the Structure-from-motion problem which utilizes the high-resolution 3D obtained from range scanner to compute a basis for this desired subspace. Adding subspace constraints during factorization also facilitates removal of tracking noise which causes distortions outside the subspace. We demonstrate the effectiveness of our formulation by extracting dense 3D structure of a human face and comparing it with a well known Structure-front-motion algorithm due to Brand.

Reconstrução de Superfícies usando um Scanner 3D de Luz Estruturada

Ao longo dos últimos anos, os scanners 3D têm tido uma utilização crescente nas mais variadas áreas. Desde a Medicina à Arqueologia, passando pelos vários tipos de indústria, ´e possível identificar aplicações destes sistemas. Essa crescente utilização deve-se, entre vários factores, ao aumento dos recursos computacionais, à simplicidade e `a diversidade das técnicas existentes, e `as vantagens dos scanners 3D comparativamente com outros sistemas. Estas vantagens são evidentes em áreas como a Medicina Forense, onde a fotografia, tradicionalmente utilizada para documentar objectos e provas, reduz a informação adquirida a duas dimensões. Apesar das vantagens associadas aos scanners 3D, um factor negativo é o preço elevado. No âmbito deste trabalho pretendeu-se desenvolver um scanner 3D de luz estruturada económico e eficaz, e um conjunto de algoritmos para o controlo do scanner, para a reconstrução de superfícies de estruturas analisadas, e para a validação dos resultados obtidos. O scanner 3D implementado ´e constituído por uma câmara e por um projector de vídeo ”off-the-shelf”, e por uma plataforma rotativa desenvolvida neste trabalho. A função da plataforma rotativa consiste em automatizar o scanner de modo a diminuir a interação dos utilizadores. Os algoritmos foram desenvolvidos recorrendo a pacotes de software open-source e a ferramentas gratuitas. O scanner 3D foi utilizado para adquirir informação 3D de um crânio, e o algoritmo para reconstrução de superfícies permitiu obter superfícies virtuais do crânio. Através do algoritmo de validação, as superfícies obtidas foram comparadas com uma superfície do mesmo crânio, obtida por tomografia computorizada (TC). O algoritmo de validação forneceu um mapa de distâncias entre regiões correspondentes nas duas superfícies, que permitiu quantificar a qualidade das superfícies obtidas. Com base no trabalho desenvolvido e nos resultados obtidos, é possível afirmar que foi criada uma base funcional para o varrimento de superfícies 3D de estruturas, apta para desenvolvimento futuro, mostrando que é possível obter alternativas aos métodos comerciais usando poucos recursos financeiros.

Online Robust 3D Mapping Using Structure from Motion Cues

This paper presents a complete solution for creating accurate 3D textured models from monocular video sequences. The methods are developed within the framework of sequential structure from motion, where a 3D model of the environment is maintained and updated as new visual information becomes available. The camera position is recovered by directly associating the 3D scene model with local image observations. Compared to standard structure from motion techniques, this approach decreases the error accumulation while increasing the robustness to scene occlusions and feature association failures. The obtained 3D information is used to generate high quality, composite visual maps of the scene (mosaics). The visual maps are used to create texture-mapped, realistic views of the scene

Overview of coded light projection techniques for automatic 3D profiling

Obtaining automatic 3D profile of objects is one of the most important issues in computer vision. With this information, a large number of applications become feasible: from visual inspection of industrial parts to 3D reconstruction of the environment for mobile robots. In order to achieve 3D data, range finders can be used. Coded structured light approach is one of the most widely used techniques to retrieve 3D information of an unknown surface. An overview of the existing techniques as well as a new classification of patterns for structured light sensors is presented. This kind of systems belong to the group of active triangulation method, which are based on projecting a light pattern and imaging the illuminated scene from one or more points of view. Since the patterns are coded, correspondences between points of the image(s) and points of the projected pattern can be easily found. Once correspondences are found, a classical triangulation strategy between camera(s) and projector device leads to the reconstruction of the surface. Advantages and constraints of the different patterns are discussed

Implementation of a robust coded structured light technique for dynamic 3D measurements

This paper presents the implementation details of a coded structured light system for rapid shape acquisition of unknown surfaces. Such techniques are based on the projection of patterns onto a measuring surface and grabbing images of every projection with a camera. Analyzing the pattern deformations that appear in the images, 3D information of the surface can be calculated. The implemented technique projects a unique pattern so that it can be used to measure moving surfaces. The structure of the pattern is a grid where the color of the slits are selected using a De Bruijn sequence. Moreover, since both axis of the pattern are coded, the cross points of the grid have two codewords (which permits to reconstruct them very precisely), while pixels belonging to horizontal and vertical slits have also a codeword. Different sets of colors are used for horizontal and vertical slits, so the resulting pattern is invariant to rotation. Therefore, the alignment constraint between camera and projector considered by a lot of authors is not necessary

An approach to coded structured light to obtain three dimensional information

The human visual ability to perceive depth looks like a puzzle. We perceive three-dimensional spatial information quickly and efficiently by using the binocular stereopsis of our eyes and, what is mote important the learning of the most common objects which we achieved through living. Nowadays, modelling the behaviour of our brain is a fiction, that is why the huge problem of 3D perception and further, interpretation is split into a sequence of easier problems. A lot of research is involved in robot vision in order to obtain 3D information of the surrounded scene. Most of this research is based on modelling the stereopsis of humans by using two cameras as if they were two eyes. This method is known as stereo vision and has been widely studied in the past and is being studied at present, and a lot of work will be surely done in the future. This fact allows us to affirm that this topic is one of the most interesting ones in computer vision. The stereo vision principle is based on obtaining the three dimensional position of an object point from the position of its projective points in both camera image planes. However, before inferring 3D information, the mathematical models of both cameras have to be known. This step is known as camera calibration and is broadly describes in the thesis. Perhaps the most important problem in stereo vision is the determination of the pair of homologue points in the two images, known as the correspondence problem, and it is also one of the most difficult problems to be solved which is currently investigated by a lot of researchers. The epipolar geometry allows us to reduce the correspondence problem. An approach to the epipolar geometry is describes in the thesis. Nevertheless, it does not solve it at all as a lot of considerations have to be taken into account. As an example we have to consider points without correspondence due to a surface occlusion or simply due to a projection out of the camera scope. The interest of the thesis is focused on structured light which has been considered as one of the most frequently used techniques in order to reduce the problems related lo stereo vision. Structured light is based on the relationship between a projected light pattern its projection and an image sensor. The deformations between the pattern projected into the scene and the one captured by the camera, permits to obtain three dimensional information of the illuminated scene. This technique has been widely used in such applications as: 3D object reconstruction, robot navigation, quality control, and so on. Although the projection of regular patterns solve the problem of points without match, it does not solve the problem of multiple matching, which leads us to use hard computing algorithms in order to search the correct matches. In recent years, another structured light technique has increased in importance. This technique is based on the codification of the light projected on the scene in order to be used as a tool to obtain an unique match. Each token of light is imaged by the camera, we have to read the label (decode the pattern) in order to solve the correspondence problem. The advantages and disadvantages of stereo vision against structured light and a survey on coded structured light are related and discussed. The work carried out in the frame of this thesis has permitted to present a new coded structured light pattern which solves the correspondence problem uniquely and robust. Unique, as each token of light is coded by a different word which removes the problem of multiple matching. Robust, since the pattern has been coded using the position of each token of light with respect to both co-ordinate axis. Algorithms and experimental results are included in the thesis. The reader can see examples 3D measurement of static objects, and the more complicated measurement of moving objects. The technique can be used in both cases as the pattern is coded by a single projection shot. Then it can be used in several applications of robot vision. Our interest is focused on the mathematical study of the camera and pattern projector models. We are also interested in how these models can be obtained by calibration, and how they can be used to obtained three dimensional information from two correspondence points. Furthermore, we have studied structured light and coded structured light, and we have presented a new coded structured light pattern. However, in this thesis we started from the assumption that the correspondence points could be well-segmented from the captured image. Computer vision constitutes a huge problem and a lot of work is being done at all levels of human vision modelling, starting from a)image acquisition; b) further image enhancement, filtering and processing, c) image segmentation which involves thresholding, thinning, contour detection, texture and colour analysis, and so on. The interest of this thesis starts in the next step, usually known as depth perception or 3D measurement.

3D digital elevation model generation

This chapter presents techniques used for the generation of 3D digital elevation models (DEMs) from remotely sensed data. Three methods are explored and discussed—optical stereoscopic imagery, Interferometric Synthetic Aperture Radar (InSAR), and LIght Detection and Ranging (LIDAR). For each approach, the state-of-the-art presented in the literature is reviewed. Techniques involved in DEM generation are presented with accuracy evaluation. Results of DEMs reconstructed from remotely sensed data are illustrated. While the processes of DEM generation from satellite stereoscopic imagery represents a good example of passive, multi-view imaging technology, discussed in Chap. 2 of this book, InSAR and LIDAR use different principles to acquire 3D information. With regard to InSAR and LIDAR, detailed discussions are conducted in order to convey the fundamentals of both technologies.

3D-Cryo-Elektronenmikroskopie des 4x6-Hämocyanins der Vogelspinne Eurypelma californicum unter Oxy- und Deoxybedingungen

Arthropodenhämocyanine und Molluskenhämocyanine, die extrazellulären Atmungsproteine der Arthropoden und Mollusken, unterscheiden sich grundsätzlich im Aufbau, besitzen aber ähnliche aktive Zentren, welche in ihrer oxydierten Form für die Blaufärbung der Hämocyanine verantwortlich sind. Sauerstoff wird im Bindungszentrum zwischen zwei, von sechs Histidinen ligandierten, Kupfer(I)Ionen gebunden. Arthropodenhämocyanine bauen sich artspezifisch aus 1, 2, 4, 6, oder 8 Hexameren mit D3-Symmetrie auf. Die Untereinheiten von je ca. 75 kDa falten sich in drei Domänen unterschiedlicher Funktionen. Der komplexe, hierarchische Zusammenbau der Arthropodenhämocyanine hängt von der Heterogenität der Untereinheiten ab. Die 7 verschieden Sequenzen des 4x6-Hämocyanins von Eurypelma californicum (EcHc) sind biochemisch in der Quartärstruktur lokalisiert. Bislang fehlte noch ein unabhängig erstelltes 3D-Modell der geometrischen Gesamtstruktur welche die hexamere und monomere Topographie eindeutig zeigt. Dessen Erstellung war Gegenstand dieser Arbeit, in Verbindung mit der Zielsetzung, die 3D-Rekonstruktion in den beiden extremen physiologischen Zuständen, mit und ohne gebundenen Sauerstoff, zu erzeugen. Dazu wurden in einer eigens entwickelten Atmosphären-Präparationskammer die Proteine in Lösung schockgefrorenen und mittels Cryo-3D-Elektronenmikroskopie gemessen. Aus den daraus gewonnen Projektionsbildern ließen sich mit der ”Single Particle Analyse“ die 3D-Informationen zurückberechnen. Die 3D-Rekonstruktionen wurden mit der publizierten Röntgenkristallstruktur des hexameren Referenz-Hämocyanins der Languste Panulirus interruptus verifiziert. Die Rekonstruktionen erlaubten die eindeutige Messung diverser in der Literatur diskutierter Parameter der Architektur des 4x6-EcHc und darüber hinaus weiterer geometrischer Parameter, welche hier erstmals veröffentlicht werden. SAXS-Daten sagen extreme Translationen und Rotationen von Teilquartärstrukturen zwischen oxy- und deoxy-EcHc voraus, was von den 3D-Rekonstruktionen der beiden Zustände nicht bestätigt werden konnte: Die 16 Å Rekonstruktion der Deoxyform weicht geometrisch nicht von der 21 Å Rekonstruktion der Oxyform ab. Die Einpassung der publizierten Röntgenstruktur der Untereinheit II des Hämocyanin des Pfeilschwanzkrebses Limulus polyphemus in die Rekonstruktionen unterstützt eine auf der hexameren Hierarchieebene lokalisierte Dynamik der Oxygenierung. Mittels Einpassung modellierter molekularer Strukturen der EcHc-Sequenzen konnte eine erste Vermutung zur Lokalisation der beiden zentralen Linker-Untereinheiten b und c des 4x6-Moleküls gemacht werden: Demnach würde Untereinheit b in den exponierten Hexameren des Moleküls liegen. Aussagen über die Quartärstrukturbindungen auf molekularer Ebene aufgrund der Einpassung modellierter molekularer Daten in die Rekonstruktionen sind als spekulativ einzustufen: a) Die Auflösung der Rekonstruktion ist verbesserungswürdig. b) Es gibt keine adäquate Vorlage für eine verlässliche Strukturvorhersage; die verschiedenen EcHc-Sequenzen liegen nur als Modellierung vor. c) Es wäre eine flexible Einpassung notwendig, um Ungenauigkeiten in den modellierten Strukturen durch Sekundärstrukturanpassung zu minimieren.

3D Ultrasound imaging--a useful non-invasive tool to detect AV fistulas in transplanted kidneys

BACKGROUND: A precise, non-invasive, non-toxic, repeatable, convenient and inexpensive follow-up of renal transplants, especially following biopsies, is in the interest of nephrologists. Formerly, the rate of biopsies leading to AV fistulas had been underestimated. Imaging procedures suited to a detailed judgement of these vascular malformations are to be assessed. METHODS: Three-dimensional (3D) reconstruction techniques of ultrasound flow-directed and non-flow-directed energy mode pictures were compared with a standard procedure, gadolinium-enhanced nuclear magnetic resonance imaging angiography (MRA) using the phase contrast technique. RESULTS: Using B-mode and conventional duplex information, AV fistulas were localized in the upper pole of the kidney transplant of the index patient. The 3D reconstruction provided information about the exact localization and orientation of the fistula in relation to other vascular structures, and the flow along the fistula. The MRA provided localization and orientation information, but less functional information. Flow-directed and non-flow-directed energy mode pictures could be reconstructed to provide 3D information about vascular malformations in transplanted kidneys. CONCLUSION: In transplanted kidneys, 3D-ultrasound angiography may be equally as effective as MRA in localizing and identifying AV malformations. Advantages of the ultrasound method are that it is cheaper, non-toxic, non-invasive, more widely availability and that it even provides more functional information. Future prospective studies will be necessary to evaluate the two techniques further.

Simultaneous 3D object tracking and camera parameter estimation by Bayesian methods and transdimensional MCMC sampling

Multi-camera 3D tracking systems with overlapping cameras represent a powerful mean for scene analysis, as they potentially allow greater robustness than monocular systems and provide useful 3D information about object location and movement. However, their performance relies on accurately calibrated camera networks, which is not a realistic assumption in real surveillance environments. Here, we introduce a multi-camera system for tracking the 3D position of a varying number of objects and simultaneously refin-ing the calibration of the network of overlapping cameras. Therefore, we introduce a Bayesian framework that combines Particle Filtering for tracking with recursive Bayesian estimation methods by means of adapted transdimensional MCMC sampling. Addi-tionally, the system has been designed to work on simple motion detection masks, making it suitable for camera networks with low transmission capabilities. Tests show that our approach allows a successful performance even when starting from clearly inaccurate camera calibrations, which would ruin conventional approaches.