Towards a real-time 3D object recognition pipeline on mobile GPGPU computing platforms using low-cost RGB-D sensors

In this project, we propose the implementation of a 3D object recognition system which will be optimized to operate under demanding time constraints. The system must be robust so that objects can be recognized properly in poor light conditions and cluttered scenes with significant levels of occlusion. An important requirement must be met: the system must exhibit a reasonable performance running on a low power consumption mobile GPU computing platform (NVIDIA Jetson TK1) so that it can be integrated in mobile robotics systems, ambient intelligence or ambient assisted living applications. The acquisition system is based on the use of color and depth (RGB-D) data streams provided by low-cost 3D sensors like Microsoft Kinect or PrimeSense Carmine. The range of algorithms and applications to be implemented and integrated will be quite broad, ranging from the acquisition, outlier removal or filtering of the input data and the segmentation or characterization of regions of interest in the scene to the very object recognition and pose estimation. Furthermore, in order to validate the proposed system, we will create a 3D object dataset. It will be composed by a set of 3D models, reconstructed from common household objects, as well as a handful of test scenes in which those objects appear. The scenes will be characterized by different levels of occlusion, diverse distances from the elements to the sensor and variations on the pose of the target objects. The creation of this dataset implies the additional development of 3D data acquisition and 3D object reconstruction applications. The resulting system has many possible applications, ranging from mobile robot navigation and semantic scene labeling to human-computer interaction (HCI) systems based on visual information.

3D models created from ROV videos and their corresponding subtransect dimensions

Underwater video transects have become a common tool for quantitative analysis of the seafloor. However a major difficulty remains in the accurate determination of the area surveyed as underwater navigation can be unreliable and image scaling does not always compensate for distortions due to perspective and topography. Depending on the camera set-up and available instruments, different methods of surface measurement are applied, which make it difficult to compare data obtained by different vehicles. 3-D modelling of the seafloor based on 2-D video data and a reference scale can be used to compute subtransect dimensions. Focussing on the length of the subtransect, the data obtained from 3-D models created with the software PhotoModeler Scanner are compared with those determined from underwater acoustic positioning (ultra short baseline, USBL) and bottom tracking (Doppler velocity log, DVL). 3-D model building and scaling was successfully conducted on all three tested set-ups and the distortion of the reference scales due to substrate roughness was identified as the main source of imprecision. Acoustic positioning was generally inaccurate and bottom tracking unreliable on rough terrain. Subtransect lengths assessed with PhotoModeler were on average 20% longer than those derived from acoustic positioning due to the higher spatial resolution and the inclusion of slope. On a high relief wall bottom tracking and 3-D modelling yielded similar results. At present, 3-D modelling is the most powerful, albeit the most time-consuming, method for accurate determination of video subtransect dimensions.

Investigation of concentration of economic power; monograph no. 1[-43]

Printed for the use of the Temporary National Economic Committee.

Investigation of concentration of economic power; monograph no. 1[-43]

Printed for the use of the Temporary National Economic Committee.

Investigation of concentration of economic power.

Printed for the use of the Temporary national economic committee.

Determination of lifter design, speed and filling effects in AG mills by 3D DEM

The power required to operate large gyratory mills often exceeds 10 MW. Hence, optimisation of the power consumption will have a significant impact on the overall economic performance and environmental impact of the mineral processing plant. In most of the published models of tumbling mills (e.g. [Morrell, S., 1996. Power draw of wet tumbling mills and its relationship to charge dynamics, Part 2: An empirical approach to modelling of mill power draw. Trans. Inst. Mining Metall. (Section C: Mineral Processing Ext. Metall.) 105, C54-C62. Austin, L.G., 1990. A mill power equation for SAG mills. Miner. Metall. Process. 57-62]), the effect of lifter design and its interaction with mill speed and filling are not incorporated. Recent experience suggests that there is an opportunity for improving grinding efficiency by choosing the appropriate combination of these variables. However, it is difficult to experimentally determine the interactions of these variables in a full scale mill. Although some work has recently been published using DEM simulations, it was basically. limited to 2D. The discrete element code, Particle Flow Code 3D (PFC3D), has been used in this work to model the effects of lifter height (525 cm) and mill speed (50-90% of critical) on the power draw and frequency distribution of specific energy (J/kg) of normal impacts in a 5 m diameter autogenous (AG) mill. It was found that the distribution of the impact energy is affected by the number of lifters, lifter height, mill speed and mill filling. Interactions of lifter design, mill speed and mill filling are demonstrated through three dimensional distinct element methods (3D DEM) modelling. The intensity of the induced stresses (shear and normal) on lifters, and hence the lifter wear, is also simulated. (C) 2004 Elsevier Ltd. All rights reserved.

Parallel 3D Simulation of a Fault Gouge using the Lattice Solid Model

Despite the insight gained from 2-D particle models, and given that the dynamics of crustal faults occur in 3-D space, the question remains, how do the 3-D fault gouge dynamics differ from those in 2-D? Traditionally, 2-D modeling has been preferred over 3-D simulations because of the computational cost of solving 3-D problems. However, modern high performance computing architectures, combined with a parallel implementation of the Lattice Solid Model (LSM), provide the opportunity to explore 3-D fault micro-mechanics and to advance understanding of effective constitutive relations of fault gouge layers. In this paper, macroscopic friction values from 2-D and 3-D LSM simulations, performed on an SGI Altix 3700 super-cluster, are compared. Two rectangular elastic blocks of bonded particles, with a rough fault plane and separated by a region of randomly sized non-bonded gouge particles, are sheared in opposite directions by normally-loaded driving plates. The results demonstrate that the gouge particles in the 3-D models undergo significant out-of-plane motion during shear. The 3-D models also exhibit a higher mean macroscopic friction than the 2-D models for varying values of interparticle friction. 2-D LSM gouge models have previously been shown to exhibit accelerating energy release in simulated earthquake cycles, supporting the Critical Point hypothesis. The 3-D models are shown to also display accelerating energy release, and good fits of power law time-to-failure functions to the cumulative energy release are obtained.

Using Coding to Improve Localization and Backscatter Communication Performance in Low-Power Sensor Networks

Backscatter communication is an emerging wireless technology that recently has gained an increase in attention from both academic and industry circles. The key innovation of the technology is the ability of ultra-low power devices to utilize nearby existing radio signals to communicate. As there is no need to generate their own energetic radio signal, the devices can benefit from a simple design, are very inexpensive and are extremely energy efficient compared with traditional wireless communication. These benefits have made backscatter communication a desirable candidate for distributed wireless sensor network applications with energy constraints.

The backscatter channel presents a unique set of challenges. Unlike a conventional one-way communication (in which the information source is also the energy source), the backscatter channel experiences strong self-interference and spread Doppler clutter that mask the information-bearing (modulated) signal scattered from the device. Both of these sources of interference arise from the scattering of the transmitted signal off of objects, both stationary and moving, in the environment. Additionally, the measurement of the location of the backscatter device is negatively affected by both the clutter and the modulation of the signal return.

This work proposes a channel coding framework for the backscatter channel consisting of a bi-static transmitter/receiver pair and a quasi-cooperative transponder. It proposes to use run-length limited coding to mitigate the background self-interference and spread-Doppler clutter with only a small decrease in communication rate. The proposed method applies to both binary phase-shift keying (BPSK) and quadrature-amplitude modulation (QAM) scheme and provides an increase in rate by up to a factor of two compared with previous methods.

Additionally, this work analyzes the use of frequency modulation and bi-phase waveform coding for the transmitted (interrogating) waveform for high precision range estimation of the transponder location. Compared to previous methods, optimal lower range sidelobes are achieved. Moreover, since both the transmitted (interrogating) waveform coding and transponder communication coding result in instantaneous phase modulation of the signal, cross-interference between localization and communication tasks exists. Phase discriminating algorithm is proposed to make it possible to separate the waveform coding from the communication coding, upon reception, and achieve localization with increased signal energy by up to 3 dB compared with previous reported results.

The joint communication-localization framework also enables a low-complexity receiver design because the same radio is used both for localization and communication.

Simulations comparing the performance of different codes corroborate the theoretical results and offer possible trade-off between information rate and clutter mitigation as well as a trade-off between choice of waveform-channel coding pairs. Experimental results from a brass-board microwave system in an indoor environment are also presented and discussed.

The effects of transients on photospheric and chromospheric power distributions

We have observed a quiet Sun region with the Swedish 1-meter Solar Telescope (SST) equipped with CRISP Imaging SpectroPolarimeter. High-resolution, high-cadence, H-alpha line scanning images were taken to observe different layers of the solar atmosphere from the photosphere to upper chromosphere. We study the distribution of power in different period-bands at different heights. Power maps of the upper photosphere and the lower chromosphere show suppressed power surrounding the magnetic-network elements, known as "magnetic shadows". These also show enhanced power close to the photosphere, traditionally referred to as"power halos". The interaction between acoustic waves and inclined magnetic fields is generally believed to be responsible for these two effects. In this study we explore if small-scale transients can influence the distribution of power at different heights. We show that the presence of transients, like mottles, Rapid Blueshifted Excursions (RBEs) and Rapid Redshifted Excursions (RREs), can strongly influence the power-maps. The short and finite lifetime of these events strongly affects all powermaps, potentially influencing the observed power distribution. We show that Doppler-shifted transients like RBEs and RREs that occur ubiquitously, can have a dominant effect on the formation ofthe power halos in the quiet Sun. For magnetic shadows, transients like mottles do not seem to have a significant effect in the power suppression around 3 minutes and wave interaction may play a key role here. Our high cadence observations reveal that flows, waves and shocks manifest in presence of magnetic fields to form a non-linear magnetohydrodynamic system.

Développement de procédés technologiques pour une intégration 3D monolithique de dispositifs nanoélectroniques sur CMOS

Résumé : Le transistor monoélectronique (SET) est un dispositif nanoélectronique très attractif à cause de son ultra-basse consommation d’énergie et sa forte densité d’intégration, mais il n’a pas les capacités suffisantes pour pouvoir remplacer complètement la technologie CMOS. Cependant, la combinaison de la technologie SET avec celle du CMOS est une voie intéressante puisqu’elle permet de profiter des forces de chacune, afin d’obtenir des circuits avec des fonctionnalités additionnelles et uniques. Cette thèse porte sur l’intégration 3D monolithique de nanodispositifs dans le back-end-of-line (BEOL) d’une puce CMOS. Cette approche permet d’obtenir des circuits hybrides et de donner une valeur ajoutée aux puces CMOS actuelles sans altérer le procédé de fabrication du niveau des transistors MOS. L’étude se base sur le procédé nanodamascène classique développé à l’UdeS qui a permis la fabrication de dispositifs nanoélectroniques sur un substrat de SiO2. Ce document présente les travaux réalisés sur l’optimisation du procédé de fabrication nanodamascène, afin de le rendre compatible avec le BEOL de circuits CMOS. Des procédés de gravure plasma adaptés à la fabrication de nanostructures métalliques et diélectriques sont ainsi développés. Le nouveau procédé nanodamascène inverse a permis de fabriquer des jonctions MIM et des SET métalliques sur une couche de SiO2. Les caractérisations électriques de MIM et de SET formés avec des jonctions TiN/Al2O3 ont permis de démontrer la présence de pièges dans les jonctions et la fonctionnalité d’un SET à basse température (1,5 K). Le transfert de ce procédé sur CMOS et le procédé d’interconnexions verticales sont aussi développés par la suite. Finalement, un circuit 3D composé d’un nanofil de titane connecté verticalement à un transistor MOS est réalisé et caractérisé avec succès. Les résultats obtenus lors de cette thèse permettent de valider la possibilité de co-intégrer verticalement des dispositifs nanoélectroniques avec une technologie CMOS, en utilisant un procédé de fabrication compatible.

A 3D printed electromagnetic nonlinear vibration energy harvester

A 3D printed electromagnetic vibration energy harvester is presented. The motion of the device is in-plane with the excitation vibrations, and this is enabled through the exploitation of a leaf isosceles trapezoidal flexural pivot topology. This topology is ideally suited for systems requiring restricted out-of-plane motion and benefits from being fabricated monolithically. This is achieved by 3D printing the topology with materials having a low flexural modulus. The presented system has a nonlinear softening spring response, as a result of designed magnetic force interactions. A discussion of fatigue performance is presented and it is suggested that whilst fabricating, the raster of the suspension element is printed perpendicular to the flexural direction and that the experienced stress is as low as possible during operation, to ensure longevity. A demonstrated power of ~25 μW at 0.1 g is achieved and 2.9 mW is demonstrated at 1 g. The corresponding bandwidths reach up-to 4.5 Hz. The system's corresponding power density of ~0.48 mW cm−3 and normalised power integral density of 11.9 kg m−3 (at 1 g) are comparable to other in-plane systems found in the literature.

3D Ionospheric Effects on HF Propagation and Heating

The thesis uses a three-dimensional, first-principles model of the ionosphere in combination with High Frequency (HF) raytracing model to address key topics related to the physics of HF propagation and artificial ionospheric heating. In particular: 1. Explores the effect of the ubiquitous electron density gradients caused by Medium Scale Traveling Ionospheric Disturbances (MSTIDs) on high-angle of incidence HF radio wave propagation. Previous studies neglected the all-important presence of horizontal gradients in both the cross- and down-range directions, which refract the HF waves, significantly changing their path through the ionosphere. The physics-based ionosphere model SAMI3/ESF is used to generate a self-consistently evolving MSTID that allows for the examination of the spatio-temporal progression of the HF radio waves in the ionosphere. 2. Tests the potential and determines engineering requirements for ground- based high power HF heaters to trigger and control the evolution of Equatorial Spread F (ESF). Interference from ESF on radio wave propagation through the ionosphere remains a critical issue on HF systems reliability. Artificial HF heating has been shown to create plasma density cavities in the ionosphere similar to those that may trigger ESF bubbles. The work explores whether HF heating may trigger or control ESF bubbles. 3. Uses the combined ionosphere and HF raytracing models to create the first self-consistent HF Heating model. This model is utilized to simulate results from an Arecibo experiment and to provide understanding of the physical mechanism behind observed phenomena. The insights gained provide engineering guidance for new artificial heaters that are being built for use in low to middle latitude regions. In accomplishing the above topics: (i) I generated a model MSTID using the SAMI3/ESF code, and used a raytrace model to examine the effects of the MSTID gradients on radio wave propagation observables; (ii) I implemented a three- dimensional HF heating model in SAMI3/ESF and used the model to determine whether HF heating could artificially generate an ESF bubble; (iii) I created the first self-consistent model for artificial HF heating using the SAMI3/ESF ionosphere model and the MoJo raytrace model and ran a series of simulations that successfully modeled the results of early artificial heating experiments at Arecibo.

Architectural-Physical Co-Design of 3D CPUs with Micro-Fluidic Cooling

The performance, energy efficiency and cost improvements due to traditional technology scaling have begun to slow down and present diminishing returns. Underlying reasons for this trend include fundamental physical limits of transistor scaling, the growing significance of quantum effects as transistors shrink, and a growing mismatch between transistors and interconnects regarding size, speed and power. Continued Moore's Law scaling will not come from technology scaling alone, and must involve improvements to design tools and development of new disruptive technologies such as 3D integration. 3D integration presents potential improvements to interconnect power and delay by translating the routing problem into a third dimension, and facilitates transistor density scaling independent of technology node. Furthermore, 3D IC technology opens up a new architectural design space of heterogeneously-integrated high-bandwidth CPUs. Vertical integration promises to provide the CPU architectures of the future by integrating high performance processors with on-chip high-bandwidth memory systems and highly connected network-on-chip structures. Such techniques can overcome the well-known CPU performance bottlenecks referred to as memory and communication wall. However the promising improvements to performance and energy efficiency offered by 3D CPUs does not come without cost, both in the financial investments to develop the technology, and the increased complexity of design. Two main limitations to 3D IC technology have been heat removal and TSV reliability. Transistor stacking creates increases in power density, current density and thermal resistance in air cooled packages. Furthermore the technology introduces vertical through silicon vias (TSVs) that create new points of failure in the chip and require development of new BEOL technologies. Although these issues can be controlled to some extent using thermal-reliability aware physical and architectural 3D design techniques, high performance embedded cooling schemes, such as micro-fluidic (MF) cooling, are fundamentally necessary to unlock the true potential of 3D ICs. A new paradigm is being put forth which integrates the computational, electrical, physical, thermal and reliability views of a system. The unification of these diverse aspects of integrated circuits is called Co-Design. Independent design and optimization of each aspect leads to sub-optimal designs due to a lack of understanding of cross-domain interactions and their impacts on the feasibility region of the architectural design space. Co-Design enables optimization across layers with a multi-domain view and thus unlocks new high-performance and energy efficient configurations. Although the co-design paradigm is becoming increasingly necessary in all fields of IC design, it is even more critical in 3D ICs where, as we show, the inter-layer coupling and higher degree of connectivity between components exacerbates the interdependence between architectural parameters, physical design parameters and the multitude of metrics of interest to the designer (i.e. power, performance, temperature and reliability). In this dissertation we present a framework for multi-domain co-simulation and co-optimization of 3D CPU architectures with both air and MF cooling solutions. Finally we propose an approach for design space exploration and modeling within the new Co-Design paradigm, and discuss the possible avenues for improvement of this work in the future.

Contributions to 3D Data Registration and Representation

Nowadays, new computers generation provides a high performance that enables to build computationally expensive computer vision applications applied to mobile robotics. Building a map of the environment is a common task of a robot and is an essential part to allow the robots to move through these environments. Traditionally, mobile robots used a combination of several sensors from different technologies. Lasers, sonars and contact sensors have been typically used in any mobile robotic architecture, however color cameras are an important sensor due to we want the robots to use the same information that humans to sense and move through the different environments. Color cameras are cheap and flexible but a lot of work need to be done to give robots enough visual understanding of the scenes. Computer vision algorithms are computational complex problems but nowadays robots have access to different and powerful architectures that can be used for mobile robotics purposes. The advent of low-cost RGB-D sensors like Microsoft Kinect which provide 3D colored point clouds at high frame rates made the computer vision even more relevant in the mobile robotics field. The combination of visual and 3D data allows the systems to use both computer vision and 3D processing and therefore to be aware of more details of the surrounding environment. The research described in this thesis was motivated by the need of scene mapping. Being aware of the surrounding environment is a key feature in many mobile robotics applications from simple robotic navigation to complex surveillance applications. In addition, the acquisition of a 3D model of the scenes is useful in many areas as video games scene modeling where well-known places are reconstructed and added to game systems or advertising where once you get the 3D model of one room the system can add furniture pieces using augmented reality techniques. In this thesis we perform an experimental study of the state-of-the-art registration methods to find which one fits better to our scene mapping purposes. Different methods are tested and analyzed on different scene distributions of visual and geometry appearance. In addition, this thesis proposes two methods for 3d data compression and representation of 3D maps. Our 3D representation proposal is based on the use of Growing Neural Gas (GNG) method. This Self-Organizing Maps (SOMs) has been successfully used for clustering, pattern recognition and topology representation of various kind of data. Until now, Self-Organizing Maps have been primarily computed offline and their application in 3D data has mainly focused on free noise models without considering time constraints. Self-organising neural models have the ability to provide a good representation of the input space. In particular, the Growing Neural Gas (GNG) is a suitable model because of its flexibility, rapid adaptation and excellent quality of representation. However, this type of learning is time consuming, specially for high-dimensional input data. Since real applications often work under time constraints, it is necessary to adapt the learning process in order to complete it in a predefined time. This thesis proposes a hardware implementation leveraging the computing power of modern GPUs which takes advantage of a new paradigm coined as General-Purpose Computing on Graphics Processing Units (GPGPU). Our proposed geometrical 3D compression method seeks to reduce the 3D information using plane detection as basic structure to compress the data. This is due to our target environments are man-made and therefore there are a lot of points that belong to a plane surface. Our proposed method is able to get good compression results in those man-made scenarios. The detected and compressed planes can be also used in other applications as surface reconstruction or plane-based registration algorithms. Finally, we have also demonstrated the goodness of the GPU technologies getting a high performance implementation of a CAD/CAM common technique called Virtual Digitizing.

HPC Accelerators with 3D Memory

After a decade evolving in the High Performance Computing arena, GPU-equipped supercomputers have con- quered the top500 and green500 lists, providing us unprecedented levels of computational power and memory bandwidth. This year, major vendors have introduced new accelerators based on 3D memory, like Xeon Phi Knights Landing by Intel and Pascal architecture by Nvidia. This paper reviews hardware features of those new HPC accelerators and unveils potential performance for scientific applications, with an emphasis on Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM) used by commercial products according to roadmaps already announced.