Robust and Efficient 3D Recognition by Alignment

Alignment is a prevalent approach for recognizing 3D objects in 2D images. A major problem with current implementations is how to robustly handle errors that propagate from uncertainties in the locations of image features. This thesis gives a technique for bounding these errors. The technique makes use of a new solution to the problem of recovering 3D pose from three matching point pairs under weak-perspective projection. Furthermore, the error bounds are used to demonstrate that using line segments for features instead of points significantly reduces the false positive rate, to the extent that alignment can remain reliable even in cluttered scenes.

Automatic 3d free form shape matching using the graduated assignment algorithm.

Liu, Yonghuai. Automatic 3d free form shape matching using the graduated assignment algorithm. Pattern Recognition, vol. 38, no. 10, pp. 1615-1631, 2005.

Specialized Mappings Architecture with Applications to Vision-based Estimation of Articulated Body Pose

A fundamental task of vision systems is to infer the state of the world given some form of visual observations. From a computational perspective, this often involves facing an ill-posed problem; e.g., information is lost via projection of the 3D world into a 2D image. Solution of an ill-posed problem requires additional information, usually provided as a model of the underlying process. It is important that the model be both computationally feasible as well as theoretically well-founded. In this thesis, a probabilistic, nonlinear supervised computational learning model is proposed: the Specialized Mappings Architecture (SMA). The SMA framework is demonstrated in a computer vision system that can estimate the articulated pose parameters of a human body or human hands, given images obtained via one or more uncalibrated cameras. The SMA consists of several specialized forward mapping functions that are estimated automatically from training data, and a possibly known feedback function. Each specialized function maps certain domains of the input space (e.g., image features) onto the output space (e.g., articulated body parameters). A probabilistic model for the architecture is first formalized. Solutions to key algorithmic problems are then derived: simultaneous learning of the specialized domains along with the mapping functions, as well as performing inference given inputs and a feedback function. The SMA employs a variant of the Expectation-Maximization algorithm and approximate inference. The approach allows the use of alternative conditional independence assumptions for learning and inference, which are derived from a forward model and a feedback model. Experimental validation of the proposed approach is conducted in the task of estimating articulated body pose from image silhouettes. Accuracy and stability of the SMA framework is tested using artificial data sets, as well as synthetic and real video sequences of human bodies and hands.

3D Curvature Contrast - a geometric or brightness illusion?

Simultaneous contrast effects have been found across a wide range of visual dimensions. We describe a simultaneous contrast effect - three-dimensional curvature contrast - in which the apparent curvature of a surface defined by shading and texture information is influenced by the curvature of a surrounding surface. The effect is strong and easily measurable. We asked whether the effect depends upon the presence of contrast at the level of the internal representation of surface curvature or whether it could be better explained in terms of local changes in the apparent brightness of regions within the test patches induced by luminance transition at the borders. The experimental results suggest that, whicle these luminance-contrast-induced effects do contribute to the observed changes in perceived curvature, there are additional influences. In particular changes in perceived curvature induced by a pattern of curved patches were eliminated or considerably weakened when the inducing pattern was transformed into a photographic negative, a procedure which disrupts the apparent three-dimensional structure of the surface patches without changing their brightness contrast. This suggests a component of the illusion involves comparisons at the level of representation of surface curvature. The observation that three-dimensional curvature contrast presists when the inducing surfaces are spatially separate from the test surface suggests that shape perception involves global, as well as local, operations.

Creating replicas of scultural heritage through 3D reconstruction and low cost 3D printer in Education

The process of making replicas of heritage has traditionally been developed by public agencies, corporations and museums and is not commonly used in schools. Currently there are technologies that allow creating cheap replicas. The new 3D reconstruction software, based on photographs and low cost 3D printers allow to make replicas at a cost much lower than traditional. This article describes the process of creating replicas of the sculpture Goslar Warrior of artist Henry Moore, located in Santa Cruz de Tenerife. To make this process, first, a digital model have been created using Autodesk Recap 360, Autodesk 123D Catch and Autodesk Meshmixer MarkerBot MakerWare applications. Physical replication, has been reproduced in polylactic acid (PLA) by MakerBot Replicator 2 3D printer. In addition, a cost analysis using, in one hand, the printer mentioned, and in the other hand, 3D printing services both online and local, is included. Finally, there has been a specific action with 141 students and 12 high school teachers, who filled a questionnary about the use of sculptural replicas in education.

FROM LINE-DRAWINGS TO IMPRESSIONS OF 3D OBJECTS - DEVELOPING A MODEL TO ACCOUNT FOR THE SHAPES THAT PEOPLE SEE

The interpretations people attach to line drawings reflect shape-related processes in human vision. Their divergences from expectations embodied in related machine vision traditions are summarized, and used to suggest how human vision decomposes the task of interpretation. A model called IO implements this idea. It first identifies geometrically regular, local fragments. Initial decisions fix edge orientations, and this information constrains decisions about other properties. Relations between fragments are explored, beginning with weak consistency checks and moving to fuller ones. IO's output captures multiple distinctive characteristics of human performance, and it suggests steady progress towards understanding shape-related visual processes is possible.

2D silhouette and 3D skeletal models for human detection and tracking

In this paper we propose a statistical model for detection and tracking of human silhouette and the corresponding 3D skeletal structure in gait sequences. We follow a point distribution model (PDM) approach using a Principal Component Analysis (PCA). The problem of non-lineal PCA is partially resolved by applying a different PDM depending of pose estimation; frontal, lateral and diagonal, estimated by Fisher's linear discriminant. Additionally, the fitting is carried out by selecting the closest allowable shape from the training set by means of a nearest neighbor classifier. To improve the performance of the model we develop a human gait analysis to take into account temporal dynamic to track the human body. The incorporation of temporal constraints on the model increase reliability and robustness.

Modelling 3D camera movement for vibration characterisation and multiple object identification with application to lighting assessment

Utilising cameras as a means to survey the surrounding environment is becoming increasingly popular in a number of different research areas and applications. Central to using camera sensors as input to a vision system, is the need to be able to manipulate and process the information captured in these images. One such application, is the use of cameras to monitor the quality of airport landing lighting at aerodromes where a camera is placed inside an aircraft and used to record images of the lighting pattern during the landing phase of a flight. The images are processed to determine a performance metric. This requires the development of custom software for the localisation and identification of luminaires within the image data. However, because of the necessity to keep airport operations functioning as efficiently as possible, it is difficult to collect enough image data to develop, test and validate any developed software. In this paper, we present a technique to model a virtual landing lighting pattern. A mathematical model is postulated which represents the glide path of the aircraft including random deviations from the expected path. A morphological method has been developed to localise and track the luminaires under different operating conditions. © 2011 IEEE.

Vision and Virtuality:The Construction of Narrative Space in Film and Computer Games

This paper seeks to explore the construction of narrative space in 3D PC computer games. With reference to Stephen Heath’s theory of filmic narrative space, the paper will examine how computer games, based on the rendition of a continuous 3D, real-time interactive environment, construct a distinct mode of narrativisation. The dynamic imbrication of the manipulation of 3D objects in a virtual world and the (re)presentation of this virtual mise-en-scene constitute an interaction that affects the concept of narrative in computer games. This leads to several questions that the paper seeks to investigate: How does the construction of space in PC games contribute to the meaning-making process or the gamer’s experience of narrative? How then is this experience of narrative game-space different from that of film?

3D Morphable Model Construction for Robust Ear and Face Recognition

Recent work suggests that the human ear varies significantly between different subjects and can be used for identification. In principle, therefore, using ears in addition to the face within a recognition system could improve accuracy and robustness, particularly for non-frontal views. The paper describes work that investigates this hypothesis using an approach based on the construction of a 3D morphable model of the head and ear. One issue with creating a model that includes the ear is that existing training datasets contain noise and partial occlusion. Rather than exclude these regions manually, a classifier has been developed which automates this process. When combined with a robust registration algorithm the resulting system enables full head morphable models to be constructed efficiently using less constrained datasets. The algorithm has been evaluated using registration consistency, model coverage and minimalism metrics, which together demonstrate the accuracy of the approach. To make it easier to build on this work, the source code has been made available online.

Kernelized structural classification for 3D dogs body parts detection

Despite pattern recognition methods for human behavioral analysis has flourished in the last decade, animal behavioral analysis has been almost neglected. Those few approaches are mostly focused on preserving livestock economic value while attention on the welfare of companion animals, like dogs, is now emerging as a social need. In this work, following the analogy with human behavior recognition, we propose a system for recognizing body parts of dogs kept in pens. We decide to adopt both 2D and 3D features in order to obtain a rich description of the dog model. Images are acquired using the Microsoft Kinect to capture the depth map images of the dog. Upon depth maps a Structural Support Vector Machine (SSVM) is employed to identify the body parts using both 3D features and 2D images. The proposal relies on a kernelized discriminative structural classificator specifically tailored for dogs independently from the size and breed. The classification is performed in an online fashion using the LaRank optimization technique to obtaining real time performances. Promising results have emerged during the experimental evaluation carried out at a dog shelter, managed by IZSAM, in Teramo, Italy.

Exploitation de contraintes photométriques et géométriques en vision : application au suivi, au calibrage et à la reconstruction

Cette thése a été réalisée dans le cadre d'une cotutelle avec l'Institut National Polytechnique de Grenoble (France). La recherche a été effectuée au sein des laboratoires de vision 3D (DIRO, UdM) et PERCEPTION-INRIA (Grenoble).

Impacts fonctionnels du traumatisme craniocérébral léger sur la vision et l'équilibre postural chez l'adulte

Le traumatisme craniocérébral léger (TCCL) a des effets complexes sur plusieurs fonctions cérébrales, dont l’évaluation et le suivi peuvent être difficiles. Les problèmes visuels et les troubles de l’équilibre font partie des plaintes fréquemment rencontrées après un TCCL. En outre, ces problèmes peuvent continuer à affecter les personnes ayant eu un TCCL longtemps après la phase aiguë du traumatisme. Cependant, les évaluations cliniques conventionnelles de la vision et de l’équilibre ne permettent pas, la plupart du temps, d’objectiver ces symptômes, surtout lorsqu’ils s’installent durablement. De plus, il n’existe pas, à notre connaissance, d’étude longitudinale ayant étudié les déficits visuels perceptifs, en tant que tels, ni les troubles de l’équilibre secondaires à un TCCL, chez l’adulte. L’objectif de ce projet était donc de déterminer la nature et la durée des effets d’un tel traumatisme sur la perception visuelle et sur la stabilité posturale, en évaluant des adultes TCCL et contrôles sur une période d’un an. Les mêmes sujets, exactement, ont participé aux deux expériences, qui ont été menées les mêmes jours pour chacun des sujets. L’impact du TCCL sur la perception visuelle de réseaux sinusoïdaux définis par des attributs de premier et de second ordre a d’abord été étudié. Quinze adultes diagnostiqués TCCL ont été évalués 15 jours, 3 mois et 12 mois après leur traumatisme. Quinze adultes contrôles appariés ont été évalués à des périodes identiques. Des temps de réaction (TR) de détection de clignotement et de discrimination de direction de mouvement ont été mesurés. Les niveaux de contraste des stimuli de premier et de second ordre ont été ajustés pour qu’ils aient une visibilité comparable, et les moyennes, médianes, écarts-types (ET) et écarts interquartiles (EIQ) des TR correspondant aux bonnes réponses ont été calculés. Le niveau de symptômes a également été évalué pour le comparer aux données de TR. De façon générale, les TR des TCCL étaient plus longs et plus variables (plus grands ET et EIQ) que ceux des contrôles. De plus, les TR des TCCL étaient plus courts pour les stimuli de premier ordre que pour ceux de second ordre, et plus variables pour les stimuli de premier ordre que pour ceux de second ordre, dans la condition de discrimination de mouvement. Ces observations se sont répétées au cours des trois sessions. Le niveau de symptômes des TCCL était supérieur à celui des participants contrôles, et malgré une amélioration, cet écart est resté significatif sur la période d’un an qui a suivi le traumatisme. La seconde expérience, elle, était destinée à évaluer l’impact du TCCL sur le contrôle postural. Pour cela, nous avons mesuré l’amplitude d’oscillation posturale dans l’axe antéropostérieur et l’instabilité posturale (au moyen de la vitesse quadratique moyenne (VQM) des oscillations posturales) en position debout, les pieds joints, sur une surface ferme, dans cinq conditions différentes : les yeux fermés, et dans un tunnel virtuel tridimensionnel soit statique, soit oscillant de façon sinusoïdale dans la direction antéropostérieure à trois vitesses différentes. Des mesures d’équilibre dérivées de tests cliniques, le Bruininks-Oseretsky Test of Motor Proficiency 2nd edition (BOT-2) et le Balance Error Scoring System (BESS) ont également été utilisées. Les participants diagnostiqués TCCL présentaient une plus grande instabilité posturale (une plus grande VQM des oscillations posturales) que les participants contrôles 2 semaines et 3 mois après le traumatisme, toutes conditions confondues. Ces troubles de l’équilibre secondaires au TCCL n’étaient plus présents un an après le traumatisme. Ces résultats suggèrent également que les déficits affectant les processus d’intégration visuelle mis en évidence dans la première expérience ont pu contribuer aux troubles de l’équilibre secondaires au TCCL. L’amplitude d’oscillation posturale dans l’axe antéropostérieur de même que les mesures dérivées des tests cliniques d’évaluation de l’équilibre (BOT-2 et BESS) ne se sont pas révélées être des mesures sensibles pour quantifier le déficit postural chez les sujets TCCL. L’association des mesures de TR à la perception des propriétés spécifiques des stimuli s’est révélée être à la fois une méthode de mesure particulièrement sensible aux anomalies visuomotrices secondaires à un TCCL, et un outil précis d’investigation des mécanismes sous-jacents à ces anomalies qui surviennent lorsque le cerveau est exposé à un traumatisme léger. De la même façon, les mesures d’instabilité posturale se sont révélées suffisamment sensibles pour permettre de mesurer les troubles de l’équilibre secondaires à un TCCL. Ainsi, le développement de tests de dépistage basés sur ces résultats et destinés à l’évaluation du TCCL dès ses premières étapes apparaît particulièrement intéressant. Il semble également primordial d’examiner les relations entre de tels déficits et la réalisation d’activités de la vie quotidienne, telles que les activités scolaires, professionnelles ou sportives, pour déterminer les impacts fonctionnels que peuvent avoir ces troubles des fonctions visuomotrice et du contrôle de l’équilibre.

Region-Based Feature Interpretation for Recognizing 3D Models in 2D Images

In model-based vision, there are a huge number of possible ways to match model features to image features. In addition to model shape constraints, there are important match-independent constraints that can efficiently reduce the search without the combinatorics of matching. I demonstrate two specific modules in the context of a complete recognition system, Reggie. The first is a region-based grouping mechanism to find groups of image features that are likely to come from a single object. The second is an interpretive matching scheme to make explicit hypotheses about occlusion and instabilities in the image features.

View-Based Strategies for 3D Object Recognition

A persistent issue of debate in the area of 3D object recognition concerns the nature of the experientially acquired object models in the primate visual system. One prominent proposal in this regard has expounded the use of object centered models, such as representations of the objects' 3D structures in a coordinate frame independent of the viewing parameters [Marr and Nishihara, 1978]. In contrast to this is another proposal which suggests that the viewing parameters encountered during the learning phase might be inextricably linked to subsequent performance on a recognition task [Tarr and Pinker, 1989; Poggio and Edelman, 1990]. The 'object model', according to this idea, is simply a collection of the sample views encountered during training. Given that object centered recognition strategies have the attractive feature of leading to viewpoint independence, they have garnered much of the research effort in the field of computational vision. Furthermore, since human recognition performance seems remarkably robust in the face of imaging variations [Ellis et al., 1989], it has often been implicitly assumed that the visual system employs an object centered strategy. In the present study we examine this assumption more closely. Our experimental results with a class of novel 3D structures strongly suggest the use of a view-based strategy by the human visual system even when it has the opportunity of constructing and using object-centered models. In fact, for our chosen class of objects, the results seem to support a stronger claim: 3D object recognition is 2D view-based.