Dynamic load balancing in parallel KD-tree k-means

One among the most influential and popular data mining methods is the k-Means algorithm for cluster analysis. Techniques for improving the efficiency of k-Means have been largely explored in two main directions. The amount of computation can be significantly reduced by adopting geometrical constraints and an efficient data structure, notably a multidimensional binary search tree (KD-Tree). These techniques allow to reduce the number of distance computations the algorithm performs at each iteration. A second direction is parallel processing, where data and computation loads are distributed over many processing nodes. However, little work has been done to provide a parallel formulation of the efficient sequential techniques based on KD-Trees. Such approaches are expected to have an irregular distribution of computation load and can suffer from load imbalance. This issue has so far limited the adoption of these efficient k-Means variants in parallel computing environments. In this work, we provide a parallel formulation of the KD-Tree based k-Means algorithm for distributed memory systems and address its load balancing issue. Three solutions have been developed and tested. Two approaches are based on a static partitioning of the data set and a third solution incorporates a dynamic load balancing policy.

Dynamic load balancing for the distributed mining of molecular structures

In molecular biology, it is often desirable to find common properties in large numbers of drug candidates. One family of methods stems from the data mining community, where algorithms to find frequent graphs have received increasing attention over the past years. However, the computational complexity of the underlying problem and the large amount of data to be explored essentially render sequential algorithms useless. In this paper, we present a distributed approach to the frequent subgraph mining problem to discover interesting patterns in molecular compounds. This problem is characterized by a highly irregular search tree, whereby no reliable workload prediction is available. We describe the three main aspects of the proposed distributed algorithm, namely, a dynamic partitioning of the search space, a distribution process based on a peer-to-peer communication framework, and a novel receiverinitiated load balancing algorithm. The effectiveness of the distributed method has been evaluated on the well-known National Cancer Institute’s HIV-screening data set, where we were able to show close-to linear speedup in a network of workstations. The proposed approach also allows for dynamic resource aggregation in a non dedicated computational environment. These features make it suitable for large-scale, multi-domain, heterogeneous environments, such as computational grids.

Decentralized load balancing for highly irregular search problems

In this paper, we present a distributed computing framework for problems characterized by a highly irregular search tree, whereby no reliable workload prediction is available. The framework is based on a peer-to-peer computing environment and dynamic load balancing. The system allows for dynamic resource aggregation, does not depend on any specific meta-computing middleware and is suitable for large-scale, multi-domain, heterogeneous environments, such as computational Grids. Dynamic load balancing policies based on global statistics are known to provide optimal load balancing performance, while randomized techniques provide high scalability. The proposed method combines both advantages and adopts distributed job-pools and a randomized polling technique. The framework has been successfully adopted in a parallel search algorithm for subgraph mining and evaluated on a molecular compounds dataset. The parallel application has shown good calability and close-to linear speedup in a distributed network of workstations.

Energy saving in intelligent buildings - via energy harvesting in wireless sensor networks

Wireless sensor networks (WSNs) have been widely used in pervasive systems such as intelligent buildings. As a vital factor of product cost, energy consuming in WSN has been focused upon, but only via energy harvesting can the problem be overcome radically. This article presents a new approach to harvesting electromagnetic energy for WSN from useless radio frequency (RF) signals transmitted in WSN, with a quantitative analysis showing its feasibility.

Radio frequency energy harvesting in wireless sensor networks

As a vital factor affecting system cost and lifetime, energy consumption in wireless sensor networks (WSNs) has been paid much attention to. This article presents a new approach to making use of electromagnetic energy from useless radio frequency (RF) signals transmitted in WSNs, with a quantitative analysis showing its feasibility. A mechanism to harvest the energy either passively or actively is proposed.

People or machines? Assessing the impacts of smart meters and load controllers in Italian office spaces

Government initiatives in several developed and developing countries to roll-out smart meters call for research on the sustainability impacts of these devices. In principle smart meters bring about higher control over energy theft and lower consumption, but require a high level of engagement by end-users. An alternative consists of load controllers, which control the load according to pre-set parameters. To date, research has focused on the impacts of these two alternatives separately. This study compares the sustainability impacts of smart meters and load controllers in an occupied office building in Italy. The assessment is carried out on three different floors of the same building. Findings show that demand reductions associated with a smart meter device are 5.2% higher than demand reductions associated with the load controller.

Soft balancing in the Americas: Latin American opposition to U.S. intervention, 1898–1936

In the aftermath of the 2003 U.S.-led invasion of Iraq, scholars of international relations debated how to best characterize the rising tide of global opposition. The concept of “soft balancing” emerged as an influential, though contested, explanation of a new phenomenon in a unipolar world: states seeking to constrain the ability of the United States to deploy military force by using multinational organizations, international law, and coalition building. Soft balancing can also be observed in regional unipolar systems. Multinational archival research reveals how Argentina, Mexico, and other Latin American countries responded to expanding U.S. power and military assertiveness in the early twentieth century through coordinated diplomatic maneuvering that provides a strong example of soft balancing. Examination of this earlier case makes an empirical contribution to the emerging soft-balancing literature and suggests that soft balancing need not lead to hard balancing or open conflict.

Implementing intelligent cores using processor virtualization for fault tolerance

Processor virtualization for process migration in distributed parallel computing systems has formed a significant component of research on load balancing. In contrast, the potential of processor virtualization for fault tolerance has been addressed minimally. The work reported in this paper is motivated towards extending concepts of processor virtualization towards ‘intelligent cores’ as a means to achieve fault tolerance in distributed parallel computing systems. Intelligent cores are an abstraction of the hardware processing cores, with the incorporation of cognitive capabilities, on which parallel tasks can be executed and migrated. When a processing core executing a task is predicted to fail the task being executed is proactively transferred onto another core. A parallel reduction algorithm incorporating concepts of intelligent cores is implemented on a computer cluster using Adaptive MPI and Charm ++. Preliminary results confirm the feasibility of the approach.

Uncertainties in annual riverine phosphorus load estimation: Impact of load estimation methodology, sampling frequency, baseflow index and catchment population density

Models developed to identify the rates and origins of nutrient export from land to stream require an accurate assessment of the nutrient load present in the water body in order to calibrate model parameters and structure. These data are rarely available at a representative scale and in an appropriate chemical form except in research catchments. Observational errors associated with nutrient load estimates based on these data lead to a high degree of uncertainty in modelling and nutrient budgeting studies. Here, daily paired instantaneous P and flow data for 17 UK research catchments covering a total of 39 water years (WY) have been used to explore the nature and extent of the observational error associated with nutrient flux estimates based on partial fractions and infrequent sampling. The daily records were artificially decimated to create 7 stratified sampling records, 7 weekly records, and 30 monthly records from each WY and catchment. These were used to evaluate the impact of sampling frequency on load estimate uncertainty. The analysis underlines the high uncertainty of load estimates based on monthly data and individual P fractions rather than total P. Catchments with a high baseflow index and/or low population density were found to return a lower RMSE on load estimates when sampled infrequently than those with a tow baseflow index and high population density. Catchment size was not shown to be important, though a limitation of this study is that daily records may fail to capture the full range of P export behaviour in smaller catchments with flashy hydrographs, leading to an underestimate of uncertainty in Load estimates for such catchments. Further analysis of sub-daily records is needed to investigate this fully. Here, recommendations are given on load estimation methodologies for different catchment types sampled at different frequencies, and the ways in which this analysis can be used to identify observational error and uncertainty for model calibration and nutrient budgeting studies. (c) 2006 Elsevier B.V. All rights reserved.