Delay control and parallel admission algorithms for real-time anycast flow

An anycast flow is a flow that can be connected to any one of the members in a group of designated (replicated) servers (called anycast group). In this paper, we derive a set of formulas for calculating the end-to-end delay bound for the anycast flows and present novel admission control algorithms for anycast flows with real-time constraints. Given such an anycast group, our algorithms can effectively select the paths for anycast flows' admission and connection based on the least end-to-end delay bounds evaluated. We also present a parallel admission control algorithm that can effectively calculate the available paths with a short delay bound for different destinations in the anycast group so that a best path with the shortest delay bound can be chosen.

A parallel downloading algorithm for redundant networks

In this paper, we study the downloading mechanism of BitTorrent (or BT), a P2P based popular and convenient parallel downloading software tool, point out some of its limitations, and propose an algorithm to improve its performance. In particular, we address the limitations of BT by using neighbours in P2P networks to resolve the redundant copies problem and to optimise the downloading speed. Our preliminary experiments show that the proposed enhancement algorithm works well.

An enhanced parallel downloading algorithm for BT

BitTorrent (or BT) is a P2P based popular and convenient parallel downloading software tool. In this paper, we study the downloading mechanism of BitTorrent, point out some of its limitations, and propose an algorithm to improve its performance. Two major limitations of BitTorrent are, first its downloading speed is slow at the beginning of a downloading or when there is only a few clients. Second, current algorithms cannot achieve the best
parallel downloading degree as the selection of sub-pieces is random, and a file may not be downloaded when the file provider leaves the network unexpectedly. In this paper we address these problems by using neighbours in P2P networks to resolve the redundant copies and to optimise the download speed. Our preliminary experiments show that the proposed enhancement algorithm works well.

A coarse-grained parallel genetic algorithm employing cluster analysis for multi-modal numerical optimisation

This paper describes a technique for improving the performance of parallel genetic algorithms on multi-modal numerical optimisation problems. It employs a cluster analysis algorithm to identify regions of the search space in which more than one sub-population is sampling. Overlapping clusters are merged in one sub-population whilst a simple derating function is applied to samples in all other sub-populations to discourage them from further sampling in that region. This approach leads to a better distribution of the search effort across multiple subpopulations and helps to prevent premature convergence. On the test problems used, significant performance improvements over the traditional island model implementation are realised.

An efficient mapreduce-based parallel clustering algorithm for distributed traffic subarea division

Traffic subarea division is vital for traffic system management and traffic network analysis in intelligent transportation systems (ITSs). Since existing methods may not be suitable for big traffic data processing, this paper presents a MapReduce-based Parallel Three-Phase K -Means (Par3PKM) algorithm for solving traffic subarea division problem on a widely adopted Hadoop distributed computing platform. Specifically, we first modify the distance metric and initialization strategy of K -Means and then employ a MapReduce paradigm to redesign the optimized K -Means algorithm for parallel clustering of large-scale taxi trajectories. Moreover, we propose a boundary identifying method to connect the borders of clustering results for each cluster. Finally, we divide traffic subarea of Beijing based on real-world trajectory data sets generated by 12,000 taxis in a period of one month using the proposed approach. Experimental evaluation results indicate that when compared with K -Means, Par2PK-Means, and ParCLARA, Par3PKM achieves higher efficiency, more accuracy, and better scalability and can effectively divide traffic subarea with big taxi trajectory data.

A novel parallel algorithm for frequent itemsets mining in massive small files datasets

In big data analysis, frequent itemsets mining plays a key role in mining associations, correlations and causality. Since some traditional frequent itemsets mining algorithms are unable to handle massive small files datasets effectively, such as high memory cost, high I/O overhead, and low computing performance, we propose a novel parallel frequent itemsets mining algorithm based on the FP-Growth algorithm and discuss its applications in this paper. First, we introduce a small files processing strategy for massive small files datasets to compensate defects of low read-write speed and low processing efficiency in Hadoop. Moreover, we use MapReduce to redesign the FP-Growth algorithm for implementing parallel computing, thereby improving the overall performance of frequent itemsets mining. Finally, we apply the proposed algorithm to the association analysis of the data from the national college entrance examination and admission of China. The experimental results show that the proposed algorithm is feasible and valid for a good speedup and a higher mining efficiency, and can meet the actual requirements of frequent itemsets mining for massive small files datasets. © 2014 ISSN 2185-2766.

A parallel DNA fragment assembly algorithm based on eulerian superpath approach

Fragments assembly is among the core problems in the research of Genome. Although many assembly tools based on the "overlap-layout-consensus" paradigm are widely used such as in the Human Genome Project currently, they still can not resolve the "repeats problem" in the DNA sequencing. For the purpose of resolving such problem, Pevzner et al. put forward a new Euler Superpath assembly algorithm. But it needs a big and complex de Bruijin graph which consumes large amounts of memories i.e. becomes the bottleneck of the performance. We present a parallel DNA fragment assembly algorithm based on the Eularian Superpath theory and solve the bottleneck in the current assembly program. The experimental results demonstrate that our approach has a good scalability, and can be used in DNA assembly of middle and large size of eukaryote genome.

Dividing traffic sub-areas based on a parallel K-means algorithm

In order to alleviate the traffic congestion and reduce the complexity of traffic control and management, it is necessary to exploit traffic sub-areas division which should be effective in planing traffic. Some researchers applied the K-Means algorithm to divide traffic sub-areas on the taxi trajectories. However, the traditional K-Means algorithms faced difficulties in processing large-scale Global Position System(GPS) trajectories of taxicabs with the restrictions of memory, I/O, computing performance. This paper proposes a Parallel Traffic Sub-Areas Division(PTSD) method which consists of two stages, on the basis of the Parallel K-Means(PKM) algorithm. During the first stage, we develop a process to cluster traffic sub-areas based on the PKM algorithm. Then, the second stage, we identify boundary of traffic sub-areas on the base of cluster result. According to this method, we divide traffic sub-areas of Beijing on the real-word (GPS) trajectories of taxicabs. The experiment and discussion show that the method is effective in dividing traffic sub-areas.

A parallel algorithm for calculation of determinants and minors using arbitrary precision arithmetic

We present a parallel algorithm for calculating determinants of matrices in arbitrary precision arithmetic on computer clusters. This algorithm limits data movements between the nodes and computes not only the determinant but also all the minors corresponding to a particular row or column at a little extra cost, and also the determinants and minors of all the leading principal submatrices at no extra cost. We implemented the algorithm in arbitrary precision arithmetic, suitable for very ill conditioned matrices, and empirically estimated the loss of precision. In our scenario the cost of computation is bigger than that of data movement. The algorithm was applied to studies of Riemann’s zeta function.

A parallel random forest algorithm for big data in a spark cloud computing environment

With the emergence of the big data age, the issue of how to obtain valuable knowledge from a dataset efficiently and accurately has attracted increasingly attention from both academia and industry. This paper presents a Parallel Random Forest (PRF) algorithm for big data on the Apache Spark platform. The PRF algorithm is optimized based on a hybrid approach combining data-parallel and task-parallel optimization. From the perspective of data-parallel optimization, a vertical data-partitioning method is performed to reduce the data communication cost effectively, and a data-multiplexing method is performed is performed to allow the training dataset to be reused and diminish the volume of data. From the perspective of task-parallel optimization, a dual parallel approach is carried out in the training process of RF, and a task Directed Acyclic Graph (DAG) is created according to the parallel training process of PRF and the dependence of the Resilient Distributed Datasets (RDD) objects. Then, different task schedulers are invoked for the tasks in the DAG. Moreover, to improve the algorithm's accuracy for large, high-dimensional, and noisy data, we perform a dimension-reduction approach in the training process and a weighted voting approach in the prediction process prior to parallelization. Extensive experimental results indicate the superiority and notable advantages of the PRF algorithm over the relevant algorithms implemented by Spark MLlib and other studies in terms of the classification accuracy, performance, and scalability.

A parallel Euler approach for large-scale biological sequence assembly

Biological sequence assembly is an essential step for sequencing the genomes of organisms. Sequence assembly is very computing intensive especially for the large-scale sequence assembly. Parallel computing is an effective way to reduce the computing time and support the assembly for large amount of biological fragments. Euler sequence assembly algorithm is an innovative algorithm proposed recently. The advantage of this algorithm is that its computing complexity is polynomial and it provides a better solution to the notorious “repeat” problem. This paper introduces the parallelization of the Euler sequence assembly algorithm. All the Genome fragments generated by whole genome shotgun (WGS) will be assembled as a whole rather than dividing them into groups which may incurs errors due to the inaccurate group partition. The implemented system can be run on supercomputers, network of workstations or even network of PC computers. The experimental results have demonstrated the performance of our system.

Design of multiple sequence alignment algorithms on parallel, distributed memory supercomputers

The challenge of comparing two or more genomes that have undergone recombination and substantial amounts of segmental loss and gain has recently been addressed for small numbers of genomes. However, datasets of hundreds of genomes are now common and their sizes will only increase in the future. Multiple sequence alignment of hundreds of genomes remains an intractable problem due to quadratic increases in compute time and memory footprint. To date, most alignment algorithms are designed for commodity clusters without parallelism. Hence, we propose the design of a multiple sequence alignment algorithm on massively parallel, distributed memory supercomputers to enable research into comparative genomics on large data sets. Following the methodology of the sequential progressiveMauve algorithm, we design data structures including sequences and sorted k-mer lists on the IBM Blue Gene/P supercomputer (BG/P). Preliminary results show that we can reduce the memory footprint so that we can potentially align over 250 bacterial genomes on a single BG/P compute node. We verify our results on a dataset of E.coli, Shigella and S.pneumoniae genomes. Our implementation returns results matching those of the original algorithm but in 1/2 the time and with 1/4 the memory footprint for scaffold building. In this study, we have laid the basis for multiple sequence alignment of large-scale datasets on a massively parallel, distributed memory supercomputer, thus enabling comparison of hundreds instead of a few genome sequences within reasonable time.

Parallel matrix inversion techniques

In this paper, we present techniques for inverting sparse, symmetric and positive definite matrices on parallel and distributed computers. We propose two algorithms, one for SIMD implementation and the other for MIMD implementation. These algorithms are modified versions of Gaussian elimination and they take into account the sparseness of the matrix. Our algorithms perform better than the general parallel Gaussian elimination algorithm. In order to demonstrate the usefulness of our technique, we implemented the snake problem using our sparse matrix algorithm. Our studies reveal that the proposed sparse matrix inversion algorithm significantly reduces the time taken for obtaining the solution of the snake problem. In this paper, we present the results of our experimental work.