Bin picking de objetos polimórficos convexos usando perceção 3D

O bin picking é um processo de grande interesse na indústria, uma vez que permite maior automatização, aumento da capacidade de produção e redução dos custos. Este tem vindo a evoluir bastante ao longo dos anos e essa evolução fez com que sistemas de perceção 3D começassem a ser implementados. Este trabalho tem como principal objetivo desenvolver um sistema de bin picking usando apenas perceção 3D. O sistema deve ser capaz de determinar a posição e orientação de objetos com diferentes formas e tamanhos, posicionados aleatoriamente numa superfície de trabalho. Os objetos utilizados para fazer os testes experimentais, são esferas, cilindros e prismas, uma vez que abrangem as formas geométricas existentes em muitos produtos submetidos a bin picking. Após a identi cação e seleção do objeto a apanhar, o manipulador deve autonomamente posicionar-se para fazer a aproximação e recolha do mesmo. A aquisição de dados é feita através de uma câmara Kinect. Dos dados recebidos apenas são trabalhados os referentes à profundidade, centrando-se assim este trabalho na análise e tratamento de nuvem de pontos. O sistema desenvolvido cumpre com os objetivos estabelecidos. Consegue localizar e apanhar objetos em várias posições e orientações. Além disso apresenta uma velocidade de processamento compatível com a aplicação em causa.

Porosity imaged by a vector projection algorithm correlates with fractal dimension measured on 3D models obtained by microCT

International audience

Contributions to 3D Data Registration and Representation

Nowadays, new computers generation provides a high performance that enables to build computationally expensive computer vision applications applied to mobile robotics. Building a map of the environment is a common task of a robot and is an essential part to allow the robots to move through these environments. Traditionally, mobile robots used a combination of several sensors from different technologies. Lasers, sonars and contact sensors have been typically used in any mobile robotic architecture, however color cameras are an important sensor due to we want the robots to use the same information that humans to sense and move through the different environments. Color cameras are cheap and flexible but a lot of work need to be done to give robots enough visual understanding of the scenes. Computer vision algorithms are computational complex problems but nowadays robots have access to different and powerful architectures that can be used for mobile robotics purposes. The advent of low-cost RGB-D sensors like Microsoft Kinect which provide 3D colored point clouds at high frame rates made the computer vision even more relevant in the mobile robotics field. The combination of visual and 3D data allows the systems to use both computer vision and 3D processing and therefore to be aware of more details of the surrounding environment. The research described in this thesis was motivated by the need of scene mapping. Being aware of the surrounding environment is a key feature in many mobile robotics applications from simple robotic navigation to complex surveillance applications. In addition, the acquisition of a 3D model of the scenes is useful in many areas as video games scene modeling where well-known places are reconstructed and added to game systems or advertising where once you get the 3D model of one room the system can add furniture pieces using augmented reality techniques. In this thesis we perform an experimental study of the state-of-the-art registration methods to find which one fits better to our scene mapping purposes. Different methods are tested and analyzed on different scene distributions of visual and geometry appearance. In addition, this thesis proposes two methods for 3d data compression and representation of 3D maps. Our 3D representation proposal is based on the use of Growing Neural Gas (GNG) method. This Self-Organizing Maps (SOMs) has been successfully used for clustering, pattern recognition and topology representation of various kind of data. Until now, Self-Organizing Maps have been primarily computed offline and their application in 3D data has mainly focused on free noise models without considering time constraints. Self-organising neural models have the ability to provide a good representation of the input space. In particular, the Growing Neural Gas (GNG) is a suitable model because of its flexibility, rapid adaptation and excellent quality of representation. However, this type of learning is time consuming, specially for high-dimensional input data. Since real applications often work under time constraints, it is necessary to adapt the learning process in order to complete it in a predefined time. This thesis proposes a hardware implementation leveraging the computing power of modern GPUs which takes advantage of a new paradigm coined as General-Purpose Computing on Graphics Processing Units (GPGPU). Our proposed geometrical 3D compression method seeks to reduce the 3D information using plane detection as basic structure to compress the data. This is due to our target environments are man-made and therefore there are a lot of points that belong to a plane surface. Our proposed method is able to get good compression results in those man-made scenarios. The detected and compressed planes can be also used in other applications as surface reconstruction or plane-based registration algorithms. Finally, we have also demonstrated the goodness of the GPU technologies getting a high performance implementation of a CAD/CAM common technique called Virtual Digitizing.

Functionalized Carbon Micro/Nanostructures for Biomolecular Detection

Advancements in the micro-and nano-scale fabrication techniques have opened up new avenues for the development of portable, scalable and easier-to-use biosensors. Over the last few years, electrodes made of carbon have been widely used as sensing units in biosensors due to their attractive physiochemical properties. The aim of this research is to investigate different strategies to develop functionalized high surface carbon micro/nano-structures for electrochemical and biosensing devices. High aspect ratio three-dimensional carbon microarrays were fabricated via carbon microelectromechanical systems (C-MEMS) technique, which is based on pyrolyzing pre-patterned organic photoresist polymers. To further increase the surface area of the carbon microstructures, surface porosity was introduced by two strategies, i.e. (i) using F127 as porogen and (ii) oxygen reactive ion etch (RIE) treatment. Electrochemical characterization showed that porous carbon thin film electrodes prepared by using F127 as porogen had an effective surface area (Aeff 185%) compared to the conventional carbon electrode. To achieve enhanced electrochemical sensitivity for C-MEMS based functional devices, graphene was conformally coated onto high aspect ratio three-dimensional (3D) carbon micropillar arrays using electrostatic spray deposition (ESD) technique. The amperometric response of graphene/carbon micropillar electrode arrays exhibited higher electrochemical activity, improved charge transfer and a linear response towards H2O2 detection between 250μM to 5.5mM. Furthermore, carbon structures with dimensions from 50 nano-to micrometer level have been fabricated by pyrolyzing photo-nanoimprint lithography patterned organic resist polymer. Microstructure, elemental composition and resistivity characterization of the carbon nanostructures produced by this process were very similar to conventional photoresist derived carbon. Surface functionalization of the carbon nanostructures was performed using direct amination technique. Considering the need for requisite functional groups to covalently attach bioreceptors on the carbon surface for biomolecule detection, different oxidation techniques were compared to study the types of carbon–oxygen groups formed on the surface and their percentages with respect to different oxidation pretreatment times. Finally, a label-free detection strategy using signaling aptamer/protein binding complex for platelet-derived growth factor oncoprotein detection on functionalized three-dimensional carbon microarrays platform was demonstrated. The sensor showed near linear relationship between the relative fluorescence difference and protein concentration even in the sub-nanomolar range with an excellent detection limit of 5 pmol.

The neuroscience of cognitive enhancement : enhanced attention, working memory and visual information processing speed using 3D-MOT

Des interventions ciblant l’amélioration cognitive sont de plus en plus à l’intérêt dans nombreux domaines, y compris la neuropsychologie. Bien qu'il existe de nombreuses méthodes pour maximiser le potentiel cognitif de quelqu’un, ils sont rarement appuyé par la recherche scientifique. D’abord, ce mémoire examine brièvement l'état des interventions d'amélioration cognitives. Il décrit premièrement les faiblesses observées dans ces pratiques et par conséquent il établit un modèle standard contre lequel on pourrait et devrait évaluer les diverses techniques ciblant l'amélioration cognitive. Une étude de recherche est ensuite présenté qui considère un nouvel outil de l'amélioration cognitive, une tâche d’entrainement perceptivo-cognitive : 3-dimensional multiple object tracking (3D-MOT). Il examine les preuves actuelles pour le 3D-MOT auprès du modèle standard proposé. Les résultats de ce projet démontrent de l’augmentation dans les capacités d’attention, de mémoire de travail visuel et de vitesse de traitement d’information. Cette étude représente la première étape dans la démarche vers l’établissement du 3D-MOT comme un outil d’amélioration cognitive.

Code pattern analysis of object-oriented programming languages

Code patterns, including programming patterns and design patterns, are good references for programming language feature improvement and software re-engineering. However, to our knowledge, no existing research has attempted to detect code patterns based on code clone detection technology. In this study, we build upon the previous work and propose to detect and analyze code patterns from a collection of open source projects using NiPAT technology. Because design patterns are most closely associated with object-oriented languages, we choose Java and Python projects to conduct our study. The tool we use for detecting patterns is NiPAT, a pattern detecting tool originally developed for the TXL programming language based on the NiCad clone detector. We extend NiPAT for the Java and Python programming languages. Then, we try to identify all the patterns from the pattern report and classify them into several different categories. In the end of the study, we analyze all the patterns and compare the differences between Java and Python patterns.

The neuroscience of cognitive enhancement : enhanced attention, working memory and visual information processing speed using 3D-MOT

Des interventions ciblant l’amélioration cognitive sont de plus en plus à l’intérêt dans nombreux domaines, y compris la neuropsychologie. Bien qu'il existe de nombreuses méthodes pour maximiser le potentiel cognitif de quelqu’un, ils sont rarement appuyé par la recherche scientifique. D’abord, ce mémoire examine brièvement l'état des interventions d'amélioration cognitives. Il décrit premièrement les faiblesses observées dans ces pratiques et par conséquent il établit un modèle standard contre lequel on pourrait et devrait évaluer les diverses techniques ciblant l'amélioration cognitive. Une étude de recherche est ensuite présenté qui considère un nouvel outil de l'amélioration cognitive, une tâche d’entrainement perceptivo-cognitive : 3-dimensional multiple object tracking (3D-MOT). Il examine les preuves actuelles pour le 3D-MOT auprès du modèle standard proposé. Les résultats de ce projet démontrent de l’augmentation dans les capacités d’attention, de mémoire de travail visuel et de vitesse de traitement d’information. Cette étude représente la première étape dans la démarche vers l’établissement du 3D-MOT comme un outil d’amélioration cognitive.

Digital 3D reconstruction as a research environment in art and architecture history: uncertainty classification and visualisation

The dissertation addresses the still not solved challenges concerned with the source-based digital 3D reconstruction, visualisation and documentation in the domain of archaeology, art and architecture history. The emerging BIM methodology and the exchange data format IFC are changing the way of collaboration, visualisation and documentation in the planning, construction and facility management process. The introduction and development of the Semantic Web (Web 3.0), spreading the idea of structured, formalised and linked data, offers semantically enriched human- and machine-readable data. In contrast to civil engineering and cultural heritage, academic object-oriented disciplines, like archaeology, art and architecture history, are acting as outside spectators. Since the 1990s, it has been argued that a 3D model is not likely to be considered a scientific reconstruction unless it is grounded on accurate documentation and visualisation. However, these standards are still missing and the validation of the outcomes is not fulfilled. Meanwhile, the digital research data remain ephemeral and continue to fill the growing digital cemeteries. This study focuses, therefore, on the evaluation of the source-based digital 3D reconstructions and, especially, on uncertainty assessment in the case of hypothetical reconstructions of destroyed or never built artefacts according to scientific principles, making the models shareable and reusable by a potentially wide audience. The work initially focuses on terminology and on the definition of a workflow especially related to the classification and visualisation of uncertainty. The workflow is then applied to specific cases of 3D models uploaded to the DFG repository of the AI Mainz. In this way, the available methods of documenting, visualising and communicating uncertainty are analysed. In the end, this process will lead to a validation or a correction of the workflow and the initial assumptions, but also (dealing with different hypotheses) to a better definition of the levels of uncertainty.

Optimized methods to digital survey and 3D modelling of cultural heritage assets

Currently making digital 3D models and replicas of the cultural heritage assets play an important role in the preservation and having a high detail source for future research and intervention. In this dissertation, it is tried to assess different methods for digital surveying and making 3D replicas of cultural heritage assets in different scales of size. The methodologies vary in devices, software, workflow, and the amount of skill that is required. The three phases of the 3D modelling process are data acquisition, modelling, and model presentation. Each of these sections is divided into sub-sections and there are several approaches, methods, devices, and software that may be employed, furthermore, the selection process should be based on the operation's goal, available facilities, the scale and properties of the object or structure to be modeled, as well as the operators' expertise and experience. The most key point to remember is that the 3D modelling operation should be properly accurate, precise, and reliable; therefore, there are so many instructions and pieces of advice on how to perform 3D modelling effectively. It is an attempt to compare and evaluate the various ways of each phase in order to explain and demonstrate their differences, benefits, and drawbacks in order to serve as a simple guide for new and/or inexperienced users.

AGV Safety Areas Obstacle Detection and Interaction with Point Cloud Derived Map

This thesis project aims to the development of an algorithm for the obstacle detection and the interaction between the safety areas of an Automated Guided Vehicles (AGV) and a Point Cloud derived map inside the context of a CAD software. The first part of the project focuses on the implementation of an algorithm for the clipping of general polygons, with which has been possible to: construct the safety areas polygon, derive the sweep of this areas along the navigation path performing a union and detect the intersections with line or polygon representing the obstacles. The second part is about the construction of a map in terms of geometric entities (lines and polygons) starting from a point cloud given by the 3D scan of the environment. The point cloud is processed using: filters, clustering algorithms and concave/convex hull derived algorithms in order to extract line and polygon entities representing obstacles. Finally, the last part aims to use the a priori knowledge of possible obstacle detections on a given segment, to predict the behavior of the AGV and use this prediction to optimize the choice of the vehicle's assigned velocity in that segment, minimizing the travel time.

Fabrication of 2D and 3D microelectrode arrays for amperometric dopamine sensing: towards a metal free approach to bioelectronics

Dopamine is a neurotransmitter which has a role in several psychiatric and neurological disorders. In-vivo detection of its concentration at the microscopic scale would benefit the study of these conditions and help in the development of therapies. The ideal sensor would be biocompatible, able to probe concentrations in microscopic volumes and sensitive to the small physiological concentrations of this molecule (10 nM - 1 μM). The ease of oxidation of dopamine makes it possible to detect it by electrochemical methods. An additional requirement in this kind of experiments when run in water, though, is to have a large potential window inside which no redox reactions with water take place. A promising class of materials which are being explored is the one of pyrolyzed photoresists. Photoresists can be lithographically patterned with micrometric resolution and after pyrolysis leave a glassy carbon material which is conductive, biocompatible and has a large electrochemical water window. In this work I developed a fabrication procedure for microelectrode arrays with three dimensional electrodes, making the whole device using just a negative photoresist called SU8. Making 3D electrodes could be a way to enhance the sensitivity of the electrodes without occupying a bigger footprint on the device. I characterized the electrical, morphological, and electrochemical properties of these electrodes, in particular their sensitivity to dopamine. I also fabricated and tested a two dimensional device for comparison. The three dimensional devices fabricated showed inferior properties to their two dimensional counter parts. I found a possible explanation and suggested some ways in which the fabrication could be improved.

Characterization of in-gap electronic states in two-dimensional single crystal PEA2PbBr4 perovskite for X-ray detection

Hybrid Organic-Inorganic Halide Perovskites (HOIPs) include a large class of materials described with the general formula ABX3, where A is an organic cation, B an inorganic cation and X an halide anion. HOIPs show excellent optoelectronic characteristics such as tunable band gap, high adsorption coefficient and great mobility life-time. A subclass of these materials, the so-called two- dimensional (2D) layered HOIPs, have emerged as potential alternatives to traditional 3D analogs to enhance the stability and increase performance of perovskite devices, with particular regard in the area of ionizing radiation detectors, where these materials have reached truly remarkable milestones. One of the key challenges for future development of efficient and stable 2D perovskite X-ray detector is a complete understanding of the nature of defects that lead to the formation of deep states. Deep states act as non-radiative recombination centers for charge carriers and are one of the factors that most hinder the development of efficient 2D HOIPs-based X-ray detectors. In this work, deep states in PEA2PbBr4 were studied through Photo-Induced Current Transient Spectroscopy (PICTS), a highly sensitive spectroscopic technique capable of detecting the presence of deep states in highly resistive ohmic materials, and characterizing their activation energy, capture cross section and, under stringent conditions, the concentration of these states. The evolution of deep states in PEA 2 PbBr 4 was evaluated after exposure of the material to high doses of ionizing radiation and during aging (one year). The data obtained allowed us to evaluate the contribution of ion migration in PEA2PbBr4. This work represents an important starting point for a better understanding of transport and recombination phenomena in 2D perovskites. To date, the PICTS technique applied to 2D perovskites has not yet been reported in the scientific literature.

Virtualization of Wiring Harness Manipulation Tasks through the PyChrono Simulation Engine

The purpose of this thesis work is the study and creation of a harness modelling system. The model needs to simulate faithfully the physical behaviour of the harness, without any instability or incorrect movements. Since there are various simulation engines that try to model wiring's systems, this thesis work focused on the creation and test of a 3D environment with wiring and other objects through the PyChrono Simulation Engine. Fine-tuning of the simulation parameters were done during the test to achieve the most stable and correct simulation possible, but tests showed the intrinsic limits of the Engine regarding the collisions' detection between the various part of the cables, while collisions between cables and other physical objects such as pavement, walls and others are well managed by the simulator. Finally, the main purpose of the model is to be used to train Artificial Intelligence through Reinforcement Learnings techniques, so we designed, using OpenAI Gym APIs, the general structure of the learning environment, defining its basic functions and an initial framework.

Sviluppo di Algoritmi di Localizzazione 3D Passiva per UAVs

Gli UAV, o meglio conosciuti come ‘droni’, sono aeromobili a pilotaggio remoto il cui utilizzo si estende dal settore militare a quello civile. Quest’ultimi, possono essere attrezzati con numerosi dispositivi accessori, come ad esempio disturbatori di frequenze. La simbiosi UAV-jammer attacca le comunicazioni wireless tramite interferenze a radiofrequenza, per degradare o interrompere il servizio offerto dalle reti. Questo elaborato, si concentra sull’analisi di algoritmi di localizzazione passiva, per stimare la posizione dell’UAV e interrompere l’interferenza. Inizialmente, viene descritto il segnale emesso dall’UAV, che utilizza lo standard di comunicazione 802.11a. A seguire, dato che la localizzazione passiva si basa sulle misure TDOA rilevate da una stazione di monitoraggio a terra, vengono presentati tre algoritmi di stima TDOA, tra i quali fast TDOA, adaptive threshold-based first tap detection e un algoritmo sviluppato per i nuovi sistemi GNSS. Successivamente, vengono esaminati tre algoritmi di localizzazione passiva, che sfruttano il principio dei minimi quadrati (LS), ovvero il CTLS, LCLS e CWLS. Infine, le prestazioni degli algoritmi di localizzazione vengono valutate in un ambiente di simulazione realistico, con canale AWGN o con canale ITU Extended pedestrian A.

The Lea Grating Test In Assessing Detection Grating Acuity In Normal Infants Less Than 4 Months Of Age.

To assess binocular detection grating acuity using the LEA GRATINGS test to establish age-related norms in healthy infants during their first 3 months of life. In this prospective, longitudinal study of healthy infants with clear red reflex at birth, responses to gratings were measured at 1, 2, and 3 months of age using LEA gratings at a distance of 28 cm. The results were recorded as detection grating acuity values, which were arranged in frequency tables and converted to a one-octave scale for statistical analysis. For the repeated measurements, analysis of variance (ANOVA) was used to compare the detection grating acuity results between ages. A total of 133 infants were included. The binocular responses to gratings showed development toward higher mean values and spatial frequencies, ranging from 0.55 ± 0.70 cycles per degree (cpd), or 1.74 ± 0.21 logMAR, in month 1 to 3.11 ± 0.54 cpd, or 0.98 ± 0.16 logMAR, in month 3. Repeated ANOVA indicated differences among grating acuity values in the three age groups. The LEA GRATINGS test allowed assessment of detection grating acuity and its development in a cohort of healthy infants during their first 3 months of life.