Automatic Data Allocation and Buffer Management for Multi-GPU Machines

Multi-GPU machines are being increasingly used in high-performance computing. Each GPU in such a machine has its own memory and does not share the address space either with the host CPU or other GPUs. Hence, applications utilizing multiple GPUs have to manually allocate and manage data on each GPU. Existing works that propose to automate data allocations for GPUs have limitations and inefficiencies in terms of allocation sizes, exploiting reuse, transfer costs, and scalability. We propose a scalable and fully automatic data allocation and buffer management scheme for affine loop nests on multi-GPU machines. We call it the Bounding-Box-based Memory Manager (BBMM). BBMM can perform at runtime, during standard set operations like union, intersection, and difference, finding subset and superset relations on hyperrectangular regions of array data (bounding boxes). It uses these operations along with some compiler assistance to identify, allocate, and manage data required by applications in terms of disjoint bounding boxes. This allows it to (1) allocate exactly or nearly as much data as is required by computations running on each GPU, (2) efficiently track buffer allocations and hence maximize data reuse across tiles and minimize data transfer overhead, and (3) and as a result, maximize utilization of the combined memory on multi-GPU machines. BBMM can work with any choice of parallelizing transformations, computation placement, and scheduling schemes, whether static or dynamic. Experiments run on a four-GPU machine with various scientific programs showed that BBMM reduces data allocations on each GPU by up to 75% compared to current allocation schemes, yields performance of at least 88% of manually written code, and allows excellent weak scaling.

GreedyDual-Join: Locality-Aware Buffer Management for Approximate Join Processing Over Data Streams

We investigate adaptive buffer management techniques for approximate evaluation of sliding window joins over multiple data streams. In many applications, data stream processing systems have limited memory or have to deal with very high speed data streams. In both cases, computing the exact results of joins between these streams may not be feasible, mainly because the buffers used to compute the joins contain much smaller number of tuples than the tuples contained in the sliding windows. Therefore, a stream buffer management policy is needed in that case. We show that the buffer replacement policy is an important determinant of the quality of the produced results. To that end, we propose GreedyDual-Join (GDJ) an adaptive and locality-aware buffering technique for managing these buffers. GDJ exploits the temporal correlations (at both long and short time scales), which we found to be prevalent in many real data streams. We note that our algorithm is readily applicable to multiple data streams and multiple joins and requires almost no additional system resources. We report results of an experimental study using both synthetic and real-world data sets. Our results demonstrate the superiority and flexibility of our approach when contrasted to other recently proposed techniques.

Novel Radio Link Buffer Management Schemes for End-User Multi-class Traffic in High Speed Packet Access Networks

The requirement to provide multimedia services with QoS support in mobile networks has led to standardization and deployment of high speed data access technologies such as the High Speed Downlink Packet Access (HSDPA) system. HSDPA improves downlink packet data and multimedia services support in WCDMA-based cellular networks. As is the trend in emerging wireless access technologies, HSDPA supports end-user multi-class sessions comprising parallel flows with diverse Quality of Service (QoS) requirements, such as real-time (RT) voice or video streaming concurrent with non real-time (NRT) data service being transmitted to the same user, with differentiated queuing at the radio link interface. Hence, in this paper we present and evaluate novel radio link buffer management schemes for QoS control of multimedia traffic comprising concurrent RT and NRT flows in the same HSDPA end-user session. The new buffer management schemes—Enhanced Time Space Priority (E-TSP) and Dynamic Time Space Priority (D-TSP)—are designed to improve radio link and network resource utilization as well as optimize end-to-end QoS performance of both RT and NRT flows in the end-user session. Both schemes are based on a Time-Space Priority (TSP) queuing system, which provides joint delay and loss differentiation between the flows by queuing (partially) loss tolerant RT flow packets for higher transmission priority but with restricted access to the buffer space, whilst allowing unlimited access to the buffer space for delay-tolerant NRT flow but with queuing for lower transmission priority. Experiments by means of extensive system-level HSDPA simulations demonstrates that with the proposed TSP-based radio link buffer management schemes, significant end-to-end QoS performance gains accrue to end-user traffic with simultaneous RT and NRT flows, in addition to improved resource utilization in the radio access network.

Dynamic Buffer Management for Multimedia Services in Beyond 3G Wireless Networks

This paper investigates a dynamic buffer man-agement scheme for QoS control of multimedia services in be-yond 3G wireless systems. The scheme is studied in the context of the state-of-the-art 3.5G system i.e. the High Speed Downlink Packet Access (HSDPA) which enhances 3G UMTS to support high-speed packet switched services. Unlike earlier systems, UMTS-evolved systems from HSDPA and beyond incorporate mechanisms such as packet scheduling and HARQ in the base station necessitating data buffering at the air interface. This introduces a potential bottleneck to end-to-end communication. Hence, buffer management at the air interface is crucial for end-to-end QoS support of multimedia services with multi-plexed parallel diverse flows such as video and data in the same end-user session. The dynamic buffer management scheme for HSDPA multimedia sessions with aggregated real-time and non real-time flows is investigated via extensive HSDPA simulations. The impact of the scheme on end-to-end traffic performance is evaluated with an example multimedia session comprising a real-time streaming flow concurrent with TCP-based non real-time flow. Results demonstrate that the scheme can guar-antee the end-to-end QoS of the real-time streaming flow, whilst simultaneously protecting the non real-time flow from starva-tion resulting in improved end-to-end throughput performance

Dynamic Buffer Management for Multimedia Services in 3.5G Wireless Networks

This paper presents and investigates a dynamic
buffer management scheme for QoS control of multimedia
services in a 3.5G wireless system i.e. the High Speed Downlink
Packet Access (HSDPA). HSDPA was introduced to enhance
UMTS for high-speed packet switched services. With HSDPA,
packet scheduling and HARQ mechanisms in the base station
require data buffering at the air interface thus introducing a
potential bottleneck to end-to-end communication. Hence, for
multimedia services with multiplexed parallel diverse flows
such as video and data in the same end-user session, buffer
management schemes in the base station are essential to support
end-to-end QoS provision. We propose a dynamic buffer management
scheme for HSDPA multimedia sessions with aggregated real-time and non real-time flows in the paper. The end-to-end performance impact of the scheme is evaluated with an example multimedia session comprising a real-time streaming
flow concurrent with TCP-based non real-time flow via extensive HSDPA simulations. Results demonstrate that the scheme can guarantee the end-to-end QoS of the real-time streaming flow, whilst simultaneously protecting non real-time flow from starvation resulting in improved end-to-end throughput performance

End-to-End QoS Enhancement ofHSDPA End-User Multi-flow Traffic Using RAN Buffer Management

High speed downlink packet access (HSDPA) was introduced to UMTS radio access segment to provide higher capacity for new packet switched services. As a result, packet switched sessions with multiple diverse traffic flows such as concurrent voice and data, or video and data being transmitted to the same user are a likely commonplace cellular packet data scenario. In HSDPA, radio access network (RAN) buffer management schemes are essential to support the end-to-end QoS of such sessions. Hence in this paper we present the end-to-end performance study of a proposed RAN buffer management scheme for multi-flow sessions via dynamic system-level HSDPA simulations. The scheme is an enhancement of a time-space priority (TSP) queuing strategy applied to the node B MAC-hs buffer allocated to an end user with concurrent real-time (RT) and non-real-time (NRT) flows during a multi-flow session. The experimental multi- flow scenario is a packet voice call with concurrent TCP-based file download to the same user. Results show that with the proposed enhancements to the TSP-based RAN buffer management, end-to-end QoS performance gains accrue to the NRT flow without compromising RT flow QoS of the same end user session

A Dynamic Buffer Management Scheme for End-to-End QoS Enhancement of Multi-flow Services in HSDPA

End-user multi-flow services support is a crucial aspect of current and next generation mobile networks. This paper presents a dynamic buffer management strategy for HSDPA end-user multi-flow traffic with aggregated real-time and non-real-time flows. The scheme incorporates dynamic priority switching between the flows for transmission on the HSDPA radio channel. The end-to-end performance of the proposed strategy is investigated with an end-user multi-flow session of simultaneous VoIP and TCP-based downlink traffic using detailed HSDPA system-level simulations. Compared to an equivalent static buffer management scheme, the results show that end-to-end throughput performance gains in the non-real-time flow and better HSDPA channel utilization is attainable without compromising the real-time VoIP flow QoS constraints

Buffer Management for Multimedia QoS control over HSDPA Downlink

HSDPA specifications include support for a flexible framework for QoS management. In this paper, it is shown how buffer management could be incorporated into HSDPA QoS framework for 'multimedia' traffic QoS control in the MAC-hs of the Node-B. A time-space-priority (TSP) scheme is proposed as viable buffer management scheme to this effect. Comparative simulation study with other schemes is presented, demonstrating the effectiveness of the TSP buffer management scheme for 'multimedia' service QoS control in HSDPA Node-B data buffers

Row-buffer reorganization: simultaneously improving performance and reducing energy in DRAMs

In this paper, based on the temporal and spatial locality characteristics of memory accesses in multicores, we propose a re-organization of the existing single large row buffer in a DRAM bank into multiple smaller row-buffers. The proposed configuration helps improve the row hit rates and also brings down the energy required for row-activations. The major contribution of this work is proposing such a reorganization without requiring any significant changes to the existing widely accepted DRAM specifications. Our proposed reorganization improves performance by 35.8%, 14.5% and 21.6% in quad, eight and sixteen core workloads along with a 42%, 28% and 31% reduction in DRAM energy. Additionally, we introduce a Need Based Allocation scheme for buffer management that shows additional performance improvement.