Exploiting Nested Parallelism on Heterogeneous Processors

Heterogeneous computing systems have become common in modern processor architectures. These systems, such as those released by AMD, Intel, and Nvidia, include both CPU and GPU cores on a single die available with reduced communication overhead compared to their discrete predecessors. Currently, discrete CPU/GPU systems are limited, requiring larger, regular, highly-parallel workloads to overcome the communication costs of the system. Without the traditional communication delay assumed between GPUs and CPUs, we believe non-traditional workloads could be targeted for GPU execution. Specifically, this thesis focuses on the execution model of nested parallel workloads on heterogeneous systems. We have designed a simulation flow which utilizes widely used CPU and GPU simulators to model heterogeneous computing architectures. We then applied this simulator to non-traditional GPU workloads using different execution models. We also have proposed a new execution model for nested parallelism allowing users to exploit these heterogeneous systems to reduce execution time.

Self-Organizing Map Neural Architectures Based on Limit Cycle Attractors

Recent efforts to develop large-scale neural architectures have paid relatively little attention to the use of self-organizing maps (SOMs). Part of the reason is that most conventional SOMs use a static encoding representation: Each input is typically represented by the fixed activation of a single node in the map layer. This not only carries information in an inefficient and unreliable way that impedes building robust multi-SOM neural architectures, but it is also inconsistent with rhythmic oscillations in biological neural networks. Here I develop and study an alternative encoding scheme that instead uses limit cycle attractors of multi-focal activity patterns to represent input patterns/sequences. Such a fundamental change in representation raises several questions: Can this be done effectively and reliably? If so, will map formation still occur? What properties would limit cycle SOMs exhibit? Could multiple such SOMs interact effectively? Could robust architectures based on such SOMs be built for practical applications? The principal results of examining these questions are as follows. First, conditions are established for limit cycle attractors to emerge in a SOM through self-organization when encoding both static and temporal sequence inputs. It is found that under appropriate conditions a set of learned limit cycles are stable, unique, and preserve input relationships. In spite of the continually changing activity in a limit cycle SOM, map formation continues to occur reliably. Next, associations between limit cycles in different SOMs are learned. It is shown that limit cycles in one SOM can be successfully retrieved by another SOM’s limit cycle activity. Control timings can be set quite arbitrarily during both training and activation. Importantly, the learned associations generalize to new inputs that have never been seen during training. Finally, a complete neural architecture based on multiple limit cycle SOMs is presented for robotic arm control. This architecture combines open-loop and closed-loop methods to achieve high accuracy and fast movements through smooth trajectories. The architecture is robust in that disrupting or damaging the system in a variety of ways does not completely destroy the system. I conclude that limit cycle SOMs have great potentials for use in constructing robust neural architectures.

Nano-Engineering of Densely Packed Electrochemical Energy Storage Architectures by Atomic Layer Deposition

Nanostructures are highly attractive for future electrical energy storage devices because they enable large surface area and short ion transport time through thin electrode layers for high power devices. Significant enhancement in power density of batteries has been achieved by nano-engineered structures, particularly anode and cathode nanostructures spatially separated far apart by a porous membrane and/or a defined electrolyte region. A self-aligned nanostructured battery fully confined within a single nanopore presents a powerful platform to determine the rate performance and cyclability limits of nanostructured storage devices. Atomic layer deposition (ALD) has enabled us to create and evaluate such structures, comprised of nanotubular electrodes and electrolyte confined within anodic aluminum oxide (AAO) nanopores. The V2O5- V2O5 symmetric nanopore battery displays exceptional power-energy performance and cyclability when tested as a massively parallel device (~2billion/cm2), each with ~1m3 volume (~1fL). Cycled between 0.2V and 1.8V, this full cell has capacity retention of 95% at 5C rate and 46% at 150C, with more than 1000 charge/discharge cycles. These results demonstrate the promise of ultrasmall, self-aligned/regular, densely packed nanobattery structures as a testbed to study ionics and electrodics at the nanoscale with various geometrical modifications and as a building block for high performance energy storage systems[1, 2]. Further increase of full cell output potential is also demonstrated in asymmetric full cell configurations with various low voltage anode materials. The asymmetric full cell nanopore batteries, comprised of V2O5 as cathode and prelithiated SnO2 or anatase phase TiO2 as anode, with integrated nanotubular metal current collectors underneath each nanotubular storage electrode, also enabled by ALD. By controlling the amount of lithium ion prelithiated into SnO2 anode, we can tune full cell output voltage in the range of 0.3V and 3V. This asymmetric nanopore battery array displays exceptional rate performance and cyclability. When cycled between 1V and 3V, it has capacity retention of approximately 73% at 200C rate compared to 1C, with only 2% capacity loss after more than 500 charge/discharge cycles. With increased full cell output potential, the asymmetric V2O5-SnO2 nanopore battery shows significantly improved energy and power density. This configuration presents a more realistic test - through its asymmetric (vs symmetric) configuration – of performance and cyclability in nanoconfined environment. This dissertation covers (1) Ultra small electrochemical storage platform design and fabrication, (2) Electron and ion transport in nanostructured electrodes inside a half cell configuration, (3) Ion transport between anode and cathode in confined nanochannels in symmetric full cells, (4) Scale up energy and power density with geometry optimization and low voltage anode materials in asymmetric full cell configurations. As a supplement, selective growth of ALD to improve graphene conductance will also be discussed[3]. References: 1. Liu, C., et al., (Invited) A Rational Design for Batteries at Nanoscale by Atomic Layer Deposition. ECS Transactions, 2015. 69(7): p. 23-30. 2. Liu, C.Y., et al., An all-in-one nanopore battery array. Nature Nanotechnology, 2014. 9(12): p. 1031-1039. 3. Liu, C., et al., Improving Graphene Conductivity through Selective Atomic Layer Deposition. ECS Transactions, 2015. 69(7): p. 133-138.