Towards a real-time 3D object recognition pipeline on mobile GPGPU computing platforms using low-cost RGB-D sensors

In this project, we propose the implementation of a 3D object recognition system which will be optimized to operate under demanding time constraints. The system must be robust so that objects can be recognized properly in poor light conditions and cluttered scenes with significant levels of occlusion. An important requirement must be met: the system must exhibit a reasonable performance running on a low power consumption mobile GPU computing platform (NVIDIA Jetson TK1) so that it can be integrated in mobile robotics systems, ambient intelligence or ambient assisted living applications. The acquisition system is based on the use of color and depth (RGB-D) data streams provided by low-cost 3D sensors like Microsoft Kinect or PrimeSense Carmine. The range of algorithms and applications to be implemented and integrated will be quite broad, ranging from the acquisition, outlier removal or filtering of the input data and the segmentation or characterization of regions of interest in the scene to the very object recognition and pose estimation. Furthermore, in order to validate the proposed system, we will create a 3D object dataset. It will be composed by a set of 3D models, reconstructed from common household objects, as well as a handful of test scenes in which those objects appear. The scenes will be characterized by different levels of occlusion, diverse distances from the elements to the sensor and variations on the pose of the target objects. The creation of this dataset implies the additional development of 3D data acquisition and 3D object reconstruction applications. The resulting system has many possible applications, ranging from mobile robot navigation and semantic scene labeling to human-computer interaction (HCI) systems based on visual information.

Uso de nubes de puntos 3D para identificación y caracterización de familias de discontinuidades planas en afloramientos rocosos y evaluación de la calidad geomecánica

En este trabajo se estudia el uso de las nubes de puntos en 3D, es decir, un conjunto de puntos en un sistema de referencia cartesiano en R3, para la identificación y caracterización de las discontinuidades que afloran en un macizo rocoso y su aplicación al campo de la Mecánica de Rocas. Las nubes de puntos utilizadas se han adquirido mediante tres técnicas: sintéticas, 3D laser scanner y la técnica de fotogrametría digital Structure From Motion (SfM). El enfoque está orientado a la extracción y caracterización de familias de discontinuidades y su aplicación a la evaluación de la calidad de un talud rocoso mediante la clasificación geomecánica Slope Mass Rating (SMR). El contenido de la misma está dividido en tres bloques, como son: (1) metodología de extracción de discontinuidades y clasificación de la nube de puntos 3D; (2) análisis de espaciados normales en nubes de puntos 3D; y (3) análisis de la evaluación de la calidad geomecánica de taludes rocoso mediante la clasificación geomecánica SMR a partir de nubes de puntos 3D. La primera línea de investigación consiste en el estudio de las nubes de puntos 3D con la finalidad de extraer y caracterizar las discontinuidades planas presentes en la superficie de un macizo rocoso. En primer lugar, se ha recopilado información de las metodologías existentes y la disponibilidad de programas para su estudio. Esto motivó la decisión de investigar y diseñar un proceso de clasificación novedoso, que muestre todos los pasos para su programación e incluso ofreciendo el código programado a la comunidad científica bajo licencia GNU GPL. De esta forma, se ha diseñado una novedosa metodología y se ha programado un software que analiza nubes de puntos 3D de forma semi-automática, permitiendo al usuario interactuar con el proceso de clasificación. Dicho software se llama Discontinuity Set Extractor (DSE). El método se ha validado empleando nubes de puntos sintéticas y adquiridas con 3D laser scanner. En primer lugar, este código analiza la nube de puntos efectuando un test de coplanaridad para cada punto y sus vecinos próximos para, a continuación, calcular el vector normal de la superficie en el punto estudiado. En segundo lugar, se representan los polos de los vectores normales calculados en el paso previo en una falsilla estereográfica. A continuación se calcula la densidad de los polos y los polos con mayor densidad o polos principales. Estos indican las orientaciones de la superficie más representadas, y por tanto las familias de discontinuidades. En tercer lugar, se asigna a cada punto una familia en dependencia del ángulo formado por el vector normal del punto y el de la familia. En este punto el usuario puede visualizar la nube de puntos clasificada con las familias de discontinuidades que ha determinado para validar el resultado intermedio. En cuarto lugar, se realiza un análisis cluster en el que se determina la agrupación de puntos según planos para cada familia (clusters). A continuación, se filtran aquellos que no tengan un número de puntos suficiente y se determina la ecuación de cada plano. Finalmente, se exportan los resultados de la clasificación a un archivo de texto para su análisis y representación en otros programas. La segunda línea de investigación consiste en el estudio del espaciado entre discontinuidades planas que afloran en macizos rocosos a partir de nubes de puntos 3D. Se desarrolló una metodología de cálculo de espaciados a partir de nubes de puntos 3D previamente clasificadas con el fin de determinar las relaciones espaciales entre planos de cada familia y calcular el espaciado normal. El fundamento novedoso del método propuesto es determinar el espaciado normal de familia basándonos en los mismos principios que en campo, pero sin la restricción de las limitaciones espaciales, condiciones de inseguridad y dificultades inherentes al proceso. Se consideraron dos aspectos de las discontinuidades: su persistencia finita o infinita, siendo la primera el aspecto más novedoso de esta publicación. El desarrollo y aplicación del método a varios casos de estudio permitió determinar su ámbito de aplicación. La validación se llevó a cabo con nubes de puntos sintéticas y adquiridas con 3D laser scanner. La tercera línea de investigación consiste en el análisis de la aplicación de la información obtenida con nubes de puntos 3D a la evaluación de la calidad de un talud rocoso mediante la clasificación geomecánica SMR. El análisis se centró en la influencia del uso de orientaciones determinadas con distintas fuentes de información (datos de campo y técnicas de adquisición remota) en la determinación de los factores de ajuste y al valor del índice SMR. Los resultados de este análisis muestran que el uso de fuentes de información y técnicas ampliamente aceptadas pueden ocasionar cambios en la evaluación de la calidad del talud rocoso de hasta una clase geomecánica (es decir, 20 unidades). Asimismo, los análisis realizados han permitido constatar la validez del índice SMR para cartografiar zonas inestables de un talud. Los métodos y programas informáticos desarrollados suponen un importante avance científico para el uso de nubes de puntos 3D para: (1) el estudio y caracterización de las discontinuidades de los macizos rocosos y (2) su aplicación a la evaluación de la calidad de taludes en roca mediante las clasificaciones geomecánicas. Asimismo, las conclusiones obtenidas y los medios y métodos empleados en esta tesis doctoral podrán ser contrastadas y utilizados por otros investigadores, al estar disponibles en la web del autor bajo licencia GNU GPL.

Characterization of rock slopes through slope mass rating using 3D point clouds

Rock mass classification systems are widely used tools for assessing the stability of rock slopes. Their calculation requires the prior quantification of several parameters during conventional fieldwork campaigns, such as the orientation of the discontinuity sets, the main properties of the existing discontinuities and the geo-mechanical characterization of the intact rock mass, which can be time-consuming and an often risky task. Conversely, the use of relatively new remote sensing data for modelling the rock mass surface by means of 3D point clouds is changing the current investigation strategies in different rock slope engineering applications. In this paper, the main practical issues affecting the application of Slope Mass Rating (SMR) for the characterization of rock slopes from 3D point clouds are reviewed, using three case studies from an end-user point of view. To this end, the SMR adjustment factors, which were calculated from different sources of information and processes, using the different softwares, are compared with those calculated using conventional fieldwork data. In the presented analysis, special attention is paid to the differences between the SMR indexes derived from the 3D point cloud and conventional field work approaches, the main factors that determine the quality of the data and some recognized practical issues. Finally, the reliability of Slope Mass Rating for the characterization of rocky slopes is highlighted.

3D models created from ROV videos and their corresponding subtransect dimensions

Underwater video transects have become a common tool for quantitative analysis of the seafloor. However a major difficulty remains in the accurate determination of the area surveyed as underwater navigation can be unreliable and image scaling does not always compensate for distortions due to perspective and topography. Depending on the camera set-up and available instruments, different methods of surface measurement are applied, which make it difficult to compare data obtained by different vehicles. 3-D modelling of the seafloor based on 2-D video data and a reference scale can be used to compute subtransect dimensions. Focussing on the length of the subtransect, the data obtained from 3-D models created with the software PhotoModeler Scanner are compared with those determined from underwater acoustic positioning (ultra short baseline, USBL) and bottom tracking (Doppler velocity log, DVL). 3-D model building and scaling was successfully conducted on all three tested set-ups and the distortion of the reference scales due to substrate roughness was identified as the main source of imprecision. Acoustic positioning was generally inaccurate and bottom tracking unreliable on rough terrain. Subtransect lengths assessed with PhotoModeler were on average 20% longer than those derived from acoustic positioning due to the higher spatial resolution and the inclusion of slope. On a high relief wall bottom tracking and 3-D modelling yielded similar results. At present, 3-D modelling is the most powerful, albeit the most time-consuming, method for accurate determination of video subtransect dimensions.

Point clouds of measurements in the Dechen Cave near Iserlohn, Germany

Modeling natural phenomena from 3D information enhances our understanding of the environment. Dense 3D point clouds are increasingly used as highly detailed input datasets. In addition to the capturing techniques of point clouds with LiDAR, low-cost sensors have been released in the last few years providing access to new research fields and facilitating 3D data acquisition for a broader range of applications. This letter presents an analysis of different speleothem features using 3D point clouds acquired with the gaming device Microsoft® Kinect. We compare the Kinect sensor with terrestrial LiDAR reference measurements using the KinFu pipeline for capturing complete 3D objects (< 4m**3). The results demonstrate the suitability of the Kinect to capture flowstone walls and to derive morphometric parameters of cave features. Although the chosen capturing strategy (KinFu) reveals a high correlation (R2=0.92) of stalagmite morphometry along the vertical object axis, a systematic overestimation (22% for radii and 44% for volume) is found. The comparison of flowstone wall datasets predominantly shows low differences (mean of 1 mm with 7 mm standard deviation) of the order of the Kinect depth precision. For both objects the major differences occur at strongly varying and curved surface structures (e.g. with fine concave parts).

Numerical evaluation of e-fields induced by body motion near high-field MRI scanner

In modern magnetic resonance imaging (MRI), both patients and radiologists are exposed to strong, nonuniform static magnetic fields inside or outside of the scanner, in which the body movement may be able to induce electric currents in tissues which could be possibly harmful. This paper presents theoretical investigations into the spatial distribution of induced E-fields in the human model when moving at various positions around the magnet. The numerical calculations are based on an efficient, quasistatic, finite-difference scheme and an anatomically realistic, full-body, male model. 3D field profiles from an actively-shielded 4 T magnet system are used and the body model projected through the field profile with normalized velocity. The simulation shows that it is possible to induce E-fields/currents near the level of physiological significance under some circumstances and provides insight into the spatial characteristics of the induced fields. The results are easy to extrapolate to very high field strengths for the safety evaluation at a variety of field strengths and motion velocities.

3D navigation with six degrees-of-freedom using a multi-touch display

With the introduction of new input devices, such as multi-touch surface displays, the Nintendo WiiMote, the Microsoft Kinect, and the Leap Motion sensor, among others, the field of Human-Computer Interaction (HCI) finds itself at an important crossroads that requires solving new challenges. Given the amount of three-dimensional (3D) data available today, 3D navigation plays an important role in 3D User Interfaces (3DUI). This dissertation deals with multi-touch, 3D navigation, and how users can explore 3D virtual worlds using a multi-touch, non-stereo, desktop display. ^ The contributions of this dissertation include a feature-extraction algorithm for multi-touch displays (FETOUCH), a multi-touch and gyroscope interaction technique (GyroTouch), a theoretical model for multi-touch interaction using high-level Petri Nets (PeNTa), an algorithm to resolve ambiguities in the multi-touch gesture classification process (Yield), a proposed technique for navigational experiments (FaNS), a proposed gesture (Hold-and-Roll), and an experiment prototype for 3D navigation (3DNav). The verification experiment for 3DNav was conducted with 30 human-subjects of both genders. The experiment used the 3DNav prototype to present a pseudo-universe, where each user was required to find five objects using the multi-touch display and five objects using a game controller (GamePad). For the multi-touch display, 3DNav used a commercial library called GestureWorks in conjunction with Yield to resolve the ambiguity posed by the multiplicity of gestures reported by the initial classification. The experiment compared both devices. The task completion time with multi-touch was slightly shorter, but the difference was not statistically significant. The design of experiment also included an equation that determined the level of video game console expertise of the subjects, which was used to break down users into two groups: casual users and experienced users. The study found that experienced gamers performed significantly faster with the GamePad than casual users. When looking at the groups separately, casual gamers performed significantly better using the multi-touch display, compared to the GamePad. Additional results are found in this dissertation.^

Advanced Applications of 3D Dosimetry and 3D Printing in Radiation Therapy

As complex radiotherapy techniques become more readily-practiced, comprehensive 3D dosimetry is a growing necessity for advanced quality assurance. However, clinical implementation has been impeded by a wide variety of factors, including the expense of dedicated optical dosimeter readout tools, high operational costs, and the overall difficulty of use. To address these issues, a novel dry-tank optical CT scanner was designed for PRESAGE 3D dosimeter readout, relying on 3D printed components and omitting costly parts from preceding optical scanners. This work details the design, prototyping, and basic commissioning of the Duke Integrated-lens Optical Scanner (DIOS).

The convex scanning geometry was designed in ScanSim, an in-house Monte Carlo optical ray-tracing simulation. ScanSim parameters were used to build a 3D rendering of a convex ‘solid tank’ for optical-CT, which is capable of collimating a point light source into telecentric geometry without significant quantities of refractive-index matched fluid. The model was 3D printed, processed, and converted into a negative mold via rubber casting to produce a transparent polyurethane scanning tank. The DIOS was assembled with the solid tank, a 3W red LED light source, a computer-controlled rotation stage, and a 12-bit CCD camera. Initial optical phantom studies show negligible spatial inaccuracies in 2D projection images and 3D tomographic reconstructions. A PRESAGE 3D dose measurement for a 4-field box treatment plan from Eclipse shows 95% of voxels passing gamma analysis at 3%/3mm criteria. Gamma analysis between tomographic images of the same dosimeter in the DIOS and DLOS systems show 93.1% agreement at 5%/1mm criteria. From this initial study, the DIOS has demonstrated promise as an economically-viable optical-CT scanner. However, further improvements will be necessary to fully develop this system into an accurate and reliable tool for advanced QA.

Pre-clinical animal studies are used as a conventional means of translational research, as a midpoint between in-vitro cell studies and clinical implementation. However, modern small animal radiotherapy platforms are primitive in comparison with conventional linear accelerators. This work also investigates a series of 3D printed tools to expand the treatment capabilities of the X-RAD 225Cx orthovoltage irradiator, and applies them to a feasibility study of hippocampal avoidance in rodent whole-brain radiotherapy.

As an alternative material to lead, a novel 3D-printable tungsten-composite ABS plastic, GMASS, was tested to create precisely-shaped blocks. Film studies show virtually all primary radiation at 225 kVp can be attenuated by GMASS blocks of 0.5cm thickness. A state-of-the-art software, BlockGen, was used to create custom hippocampus-shaped blocks from medical image data, for any possible axial treatment field arrangement. A custom 3D printed bite block was developed to immobilize and position a supine rat for optimal hippocampal conformity. An immobilized rat CT with digitally-inserted blocks was imported into the SmART-Plan Monte-Carlo simulation software to determine the optimal beam arrangement. Protocols with 4 and 7 equally-spaced fields were considered as viable treatment options, featuring improved hippocampal conformity and whole-brain coverage when compared to prior lateral-opposed protocols. Custom rodent-morphic PRESAGE dosimeters were developed to accurately reflect these treatment scenarios, and a 3D dosimetry study was performed to confirm the SmART-Plan simulations. Measured doses indicate significant hippocampal sparing and moderate whole-brain coverage.

3D Navigation with Six Degrees-of-Freedom using a Multi-Touch Display

With the introduction of new input devices, such as multi-touch surface displays, the Nintendo WiiMote, the Microsoft Kinect, and the Leap Motion sensor, among others, the field of Human-Computer Interaction (HCI) finds itself at an important crossroads that requires solving new challenges. Given the amount of three-dimensional (3D) data available today, 3D navigation plays an important role in 3D User Interfaces (3DUI). This dissertation deals with multi-touch, 3D navigation, and how users can explore 3D virtual worlds using a multi-touch, non-stereo, desktop display. The contributions of this dissertation include a feature-extraction algorithm for multi-touch displays (FETOUCH), a multi-touch and gyroscope interaction technique (GyroTouch), a theoretical model for multi-touch interaction using high-level Petri Nets (PeNTa), an algorithm to resolve ambiguities in the multi-touch gesture classification process (Yield), a proposed technique for navigational experiments (FaNS), a proposed gesture (Hold-and-Roll), and an experiment prototype for 3D navigation (3DNav). The verification experiment for 3DNav was conducted with 30 human-subjects of both genders. The experiment used the 3DNav prototype to present a pseudo-universe, where each user was required to find five objects using the multi-touch display and five objects using a game controller (GamePad). For the multi-touch display, 3DNav used a commercial library called GestureWorks in conjunction with Yield to resolve the ambiguity posed by the multiplicity of gestures reported by the initial classification. The experiment compared both devices. The task completion time with multi-touch was slightly shorter, but the difference was not statistically significant. The design of experiment also included an equation that determined the level of video game console expertise of the subjects, which was used to break down users into two groups: casual users and experienced users. The study found that experienced gamers performed significantly faster with the GamePad than casual users. When looking at the groups separately, casual gamers performed significantly better using the multi-touch display, compared to the GamePad. Additional results are found in this dissertation.

Towards Mapping of Rock Walls Using a UAV-Mounted 2D Laser Scanner in GPS Denied Environments

In geotechnical engineering, the stability of rock excavations and walls is estimated by using tools that include a map of the orientations of exposed rock faces. However, measuring these orientations by using conventional methods can be time consuming, sometimes dangerous, and is limited to regions of the exposed rock that are reachable by a human. This thesis introduces a 2D, simulated, quadcopter-based rock wall mapping algorithm for GPS denied environments such as underground mines or near high walls on surface. The proposed algorithm employs techniques from the field of robotics known as simultaneous localization and mapping (SLAM) and is a step towards 3D rock wall mapping. Not only are quadcopters agile, but they can hover. This is very useful for confined spaces such as underground or near rock walls. The quadcopter requires sensors to enable self localization and mapping in dark, confined and GPS denied environments. However, these sensors are limited by the quadcopter payload and power restrictions. Because of these restrictions, a light weight 2D laser scanner is proposed. As a first step towards a 3D mapping algorithm, this thesis proposes a simplified scenario in which a simulated 1D laser range finder and 2D IMU are mounted on a quadcopter that is moving on a plane. Because the 1D laser does not provide enough information to map the 2D world from a single measurement, many measurements are combined over the trajectory of the quadcopter. Least Squares Optimization (LSO) is used to optimize the estimated trajectory and rock face for all data collected over the length of a light. Simulation results show that the mapping algorithm developed is a good first step. It shows that by combining measurements over a trajectory, the scanned rock face can be estimated using a lower-dimensional range sensor. A swathing manoeuvre is introduced as a way to promote loop closures within a short time period, thus reducing accumulated error. Some suggestions on how to improve the algorithm are also provided.

Planar Odometry from a Radial Laser Scanner. A Range Flow-based Approach

In this paper we present a fast and precise method to estimate the planar motion of a lidar from consecutive range scans. For every scanned point we formulate the range flow constraint equation in terms of the sensor velocity, and minimize a robust function of the resulting geometric constraints to obtain the motion estimate. Conversely to traditional approaches, this method does not search for correspondences but performs dense scan alignment based on the scan gradients, in the fashion of dense 3D visual odometry. The minimization problem is solved in a coarse-to-fine scheme to cope with large displacements, and a smooth filter based on the covariance of the estimate is employed to handle uncertainty in unconstraint scenarios (e.g. corridors). Simulated and real experiments have been performed to compare our approach with two prominent scan matchers and with wheel odometry. Quantitative and qualitative results demonstrate the superior performance of our approach which, along with its very low computational cost (0.9 milliseconds on a single CPU core), makes it suitable for those robotic applications that require planar odometry. For this purpose, we also provide the code so that the robotics community can benefit from it.

Contributions to 3D Data Registration and Representation

Nowadays, new computers generation provides a high performance that enables to build computationally expensive computer vision applications applied to mobile robotics. Building a map of the environment is a common task of a robot and is an essential part to allow the robots to move through these environments. Traditionally, mobile robots used a combination of several sensors from different technologies. Lasers, sonars and contact sensors have been typically used in any mobile robotic architecture, however color cameras are an important sensor due to we want the robots to use the same information that humans to sense and move through the different environments. Color cameras are cheap and flexible but a lot of work need to be done to give robots enough visual understanding of the scenes. Computer vision algorithms are computational complex problems but nowadays robots have access to different and powerful architectures that can be used for mobile robotics purposes. The advent of low-cost RGB-D sensors like Microsoft Kinect which provide 3D colored point clouds at high frame rates made the computer vision even more relevant in the mobile robotics field. The combination of visual and 3D data allows the systems to use both computer vision and 3D processing and therefore to be aware of more details of the surrounding environment. The research described in this thesis was motivated by the need of scene mapping. Being aware of the surrounding environment is a key feature in many mobile robotics applications from simple robotic navigation to complex surveillance applications. In addition, the acquisition of a 3D model of the scenes is useful in many areas as video games scene modeling where well-known places are reconstructed and added to game systems or advertising where once you get the 3D model of one room the system can add furniture pieces using augmented reality techniques. In this thesis we perform an experimental study of the state-of-the-art registration methods to find which one fits better to our scene mapping purposes. Different methods are tested and analyzed on different scene distributions of visual and geometry appearance. In addition, this thesis proposes two methods for 3d data compression and representation of 3D maps. Our 3D representation proposal is based on the use of Growing Neural Gas (GNG) method. This Self-Organizing Maps (SOMs) has been successfully used for clustering, pattern recognition and topology representation of various kind of data. Until now, Self-Organizing Maps have been primarily computed offline and their application in 3D data has mainly focused on free noise models without considering time constraints. Self-organising neural models have the ability to provide a good representation of the input space. In particular, the Growing Neural Gas (GNG) is a suitable model because of its flexibility, rapid adaptation and excellent quality of representation. However, this type of learning is time consuming, specially for high-dimensional input data. Since real applications often work under time constraints, it is necessary to adapt the learning process in order to complete it in a predefined time. This thesis proposes a hardware implementation leveraging the computing power of modern GPUs which takes advantage of a new paradigm coined as General-Purpose Computing on Graphics Processing Units (GPGPU). Our proposed geometrical 3D compression method seeks to reduce the 3D information using plane detection as basic structure to compress the data. This is due to our target environments are man-made and therefore there are a lot of points that belong to a plane surface. Our proposed method is able to get good compression results in those man-made scenarios. The detected and compressed planes can be also used in other applications as surface reconstruction or plane-based registration algorithms. Finally, we have also demonstrated the goodness of the GPU technologies getting a high performance implementation of a CAD/CAM common technique called Virtual Digitizing.

Estudio comparativo de la fiabilidad y reproducibilidad en la medición de los tamaños dentarios y las medidas de las arcadas dentarias entre registros manuales y digitales 3d obtenidos por escaneado intraoral y extraoral

Tradicionalmente, los ortodoncistas han realizado las mediciones dentales en los modelos de yeso, pero los avances tecnológicos permiten ahora a los ortodoncistas realizar esas mediciones en los modelos digitales. El propósito de este estudio fue comparar la fiabilidad y reproducibilidad de las medidas de los tamaños dentarios y las arcadas dentarias entre el método manual y los métodos digitales 3D obtenidos por un escáner intraoral CEREC Omnicam (Sirona Dental Systems) y dos escáneres extraorales: inEos X5 (Sirona Dental Systems) y Dental Scanner SMART (Open Technologies). Un modelo de yeso, un escaneado intraoral y dos modelos digitales con un escáner extraoral fueron realizadas para cada uno de los 20 sujetos. Las medidas de los tamaños dentarios, la distancia intercanina y la distancia intermolar de los modelos digitales se compararon con los correspondientes modelos de yeso (estándar de oro) Se utilizó el test de ANOVA para establecer la fiabilidad entre los cuatro métodos y el coeficiente de correlación intraclase fue calculado para determinar la reproducibilidad intra- e inter-examinador. Los resultados encontrados fueron que no existieron diferencias estadísticamente significativas entre las medidas hechas directamente en los modelos de yeso y los modelos digitales. Los coeficientes de correlación intraclase tanto intra- e inter-examinador fue alto y considerado bueno para los cuatro métodos de medición. CCI> 0.90. Se concluyó que las mediciones en los modelos digitales obtenidos con un escáner extraoral e intraoral son fiables y reproducibles

Kinect with ROS, interact with oculus: towards dynamic user interfaces for robotic teleoperation

Teleoperation remains an important aspect for robotic systems especially when deployed in unstructured environments. While a range of research strives for robots that are completely autonomous, many robotic applications still require some level of human-in-The-loop control. In any situation where teleoperation is required an effective User Interface (UI) remains a key component within the systems design. Current advancements in Virtual Reality (VR) software and hardware such as the Oculus Rift, HTC Vive and Google Cardboard combined with greater transparency to robotic systems afforded by middleware such as the Robot Operating System (ROS) provides an opportunity to rapidly improve traditional teleoperation interfaces. This paper uses a System of System (SoS) approach to present the concept of a Virtual Reality Dynamic User Interface (VRDUI) for the teleoperation of heterogeneous robots. Different geometric virtual workspaces are discussed and a cylindrical workspace aligned with interactive displays is presented as a virtual control room. A presentation mode within the proposed VRDUI is also detailed, this shows how point cloud information obtained from the Microsoft Kinect can be incorporated within the proposed virtual workspace. This point cloud data is successfully processed into an OctoMap utilizing the octree data structure to create a voxelized representation of the 3D scanned environment. The resulting OctoMap is then displayed to an operator as a 3D point cloud using the Oculus Rift Head Mounted Display (HMD).

Dalla generazione di modelli 3D densi mediante TLS e fotogrammetria alla modellazione BIM

La tesi tratta la ricerca di procedure che permettano di rilevare oggetti utilizzando il maggior numero di informazioni geometriche ottenibili da una nuvola di punti densa generata da un rilievo fotogrammetrico o da TLS realizzando un modello 3D importabile in ambiente FEM. Il primo test si è eseguito su una piccola struttura, 1.2x0.5x0.2m, in modo da definire delle procedure di analisi ripetibili; la prima consente di passare dalla nuvola di punti “Cloud” all’oggetto solido “Solid” al modello agli elementi finiti “Fem” e per questo motivo è stata chiamata “metodo CSF”, mentre la seconda, che prevede di realizzare il modello della struttura con un software BIM è stata chiamata semplicemente “metodo BIM”. Una volta dimostrata la fattibilità della procedura la si è validata adottando come oggetto di studio un monumento storico di grandi dimensioni, l’Arco di Augusto di Rimini, confrontando i risultati ottenuti con quelli di altre tesi sulla medesima struttura, in particolare si è fatto riferimento a modelli FEM 2D e a modelli ottenuti da una nuvola di punti con i metodi CAD e con un software scientifico sviluppato al DICAM Cloud2FEM. Sull’arco sono state eseguite due tipi di analisi, una lineare sotto peso proprio e una modale ottenendo risultati compatibili tra i vari metodi sia dal punto di vista degli spostamenti, 0.1-0.2mm, che delle frequenze naturali ma si osserva che le frequenze naturali del modello BIM sono più simili a quelle dei modelli generati da cloud rispetto al modello CAD. Il quarto modo di vibrare invece presenta differenze maggiori. Il confronto con le frequenze naturali del modello FEM ha restituito differenze percentuali maggiori dovute alla natura 2D del modello e all’assenza della muratura limitrofa. Si sono confrontate le tensioni normali dei modelli CSF e BIM con quelle ottenute dal modello FEM ottenendo differenze inferiori a 1.28 kg/cm2 per le tensioni normali verticali e sull’ordine 10-2 kg/cm2 per quelle orizzontali.