Fault tolerant decentralised K-Means clustering for asynchronous large-scale networks

The K-Means algorithm for cluster analysis is one of the most influential and popular data mining methods. Its straightforward parallel formulation is well suited for distributed memory systems with reliable interconnection networks, such as massively parallel processors and clusters of workstations. However, in large-scale geographically distributed systems the straightforward parallel algorithm can be rendered useless by a single communication failure or high latency in communication paths. The lack of scalable and fault tolerant global communication and synchronisation methods in large-scale systems has hindered the adoption of the K-Means algorithm for applications in large networked systems such as wireless sensor networks, peer-to-peer systems and mobile ad hoc networks. This work proposes a fully distributed K-Means algorithm (EpidemicK-Means) which does not require global communication and is intrinsically fault tolerant. The proposed distributed K-Means algorithm provides a clustering solution which can approximate the solution of an ideal centralised algorithm over the aggregated data as closely as desired. A comparative performance analysis is carried out against the state of the art sampling methods and shows that the proposed method overcomes the limitations of the sampling-based approaches for skewed clusters distributions. The experimental analysis confirms that the proposed algorithm is very accurate and fault tolerant under unreliable network conditions (message loss and node failures) and is suitable for asynchronous networks of very large and extreme scale.

A spatio-temporal structure-based approach to drought characterisation

Drought characterisation is an intrinsically spatio-temporal problem. A limitation of previous approaches to characterisation is that they discard much of the spatio-temporal information by reducing events to a lower-order subspace. To address this, an explicit 3-dimensional (longitude, latitude, time) structure-based method is described in which drought events are defined by a spatially and temporarily coherent set of points displaying standardised precipitation below a given threshold. Geometric methods can then be used to measure similarity between individual drought structures. Groupings of these similarities provide an alternative to traditional methods for extracting recurrent space-time signals from geophysical data. The explicit consideration of structure encourages the construction of summary statistics which relate to the event geometry. Example measures considered are the event volume, centroid, and aspect ratio. The utility of a 3-dimensional approach is demonstrated by application to the analysis of European droughts (15 °W to 35°E, and 35 °N to 70°N) for the period 1901–2006. Large-scale structure is found to be abundant with 75 events identified lasting for more than 3 months and spanning at least 0.5 × 106 km2. Near-complete dissimilarity is seen between the individual drought structures, and little or no regularity is found in the time evolution of even the most spatially similar drought events. The spatial distribution of the event centroids and the time evolution of the geographic cross-sectional areas strongly suggest that large area, sustained droughts result from the combination of multiple small area (∼106 km2) short duration (∼3 months) events. The small events are not found to occur independently in space. This leads to the hypothesis that local water feedbacks play an important role in the aggregation process.

Pollinator community responses to the spatial population structure of wild plants: A pan-European approach

Land-use changes can alter the spatial population structure of plant species, which may in turn affect the attractiveness of flower aggregations to different groups of pollinators at different spatial scales. To assess how pollinators respond to spatial heterogeneity of plant distributions and whether honeybees affect visitation by other pollinators we used an extensive data set comprising ten plant species and their flower visitors from five European countries. In particular we tested the hypothesis that the composition of the flower visitor community in terms of visitation frequencies by different pollinator groups were affected by the spatial plant population structure, viz. area and density measures, at a within-population (‘patch’) and among-population (‘population’) scale. We found that patch area and population density were the spatial variables that best explained the variation in visitation frequencies within the pollinator community. Honeybees had higher visitation frequencies in larger patches, while bumblebees and hoverflies had higher visitation frequencies in sparser populations. Solitary bees had higher visitation frequencies in sparser populations and smaller patches. We also tested the hypothesis that honeybees affect the composition of the pollinator community by altering the visitation frequencies of other groups of pollinators. There was a positive relationship between visitation frequencies of honeybees and bumblebees, while the relationship with hoverflies and solitary bees varied (positive, negative and no relationship) depending on the plant species under study. The overall conclusion is that the spatial structure of plant populations affects different groups of pollinators in contrasting ways at both the local (‘patch’) and the larger (‘population’) scales and, that honeybees affect the flower visitation by other pollinator groups in various ways, depending on the plant species under study. These contrasting responses emphasize the need to investigate the entire pollinator community when the effects of landscape change on plant–pollinator interactions are studied.

Scale sensitivity of drivers of environmental change across Europe

The development of effective environmental management plans and policies requires a sound understanding of the driving forces involved in shaping and altering the structure and function of ecosystems. However, driving forces, especially anthropogenic ones, are defined and operate at multiple administrative levels, which do not always match ecological scales. This paper presents an innovative methodology of analysing drivers of change by developing a typology of scale sensitivity of drivers that classifies and describes the way they operate across multiple administrative levels. Scale sensitivity varies considerably among drivers, which can be classified into five broad categories depending on the response of ‘evenness’ and ‘intensity change’ when moving across administrative levels. Indirect drivers tend to show low scale sensitivity, whereas direct drivers show high scale sensitivity, as they operate in a non-linear way across the administrative scale. Thus policies addressing direct drivers of change, in particular, need to take scale into consideration during their formulation. Moreover, such policies must have a strong spatial focus, which can be achieved either by encouraging local–regional policy making or by introducing high flexibility in (inter)national policies to accommodate increased differentiation at lower administrative levels. High quality data is available for several drivers, however, the availability of consistent data at all levels for non-anthropogenic drivers is a major constraint to mapping and assessing their scale sensitivity. This lack of data may hinder effective policy making for environmental management, since it restricts the ability to fully account for scale sensitivity of natural drivers in policy design.

Synoptic scale disturbances of the Indian Summer Monsoon as simulated in a high resolution climate model

Hamburg atmospheric general circulation model ECHAM3 at T106 resolution (1.125' lat.Aon.) has considerable skill in reproducing the observed seasonal reversal of mean sea level pressure, the location of the summer heat low as well as the position of the monsoon trough over the Indian subcontinent. The present-day climate and its seasonal cycle are realistically simulated by the model over this region. The model simulates the structure, intensity, frequency, movement and lifetime of monsoon depressions remarkably well. The number of monsoon depressions/storms simulated by the model in a year ranged from 5 to 12 with an average frequency of 8.4 yr-', not significantly different from the observed climatology. The model also simulates the interannual variability in the formation of depressions over the north Bay of Bengal during the summer monsoon season. In the warmer atmosphere under doubled CO2 conditions, the number of monsoon depressions/cyclonic storms forming in Indian seas in a year ranged from 5 to 11 with an average frequency of 7.6 yr-', not significantly different from those inferred in the control run of the model. However, under doubled CO2 conditions, fewer depressions formed in the month of June. Neither the lowest central pressure nor the maximum wind speed changes appreciably in monsoon depressions identified under simulated enhanced greenhouse conditions. The analysis suggests there will be no significant changes in the number and intensity of monsoon depressions in a warmer atmosphere.

Consistent large-scale temperature responses in warm and cold climates

Climate-model simulations of the large-scale temperature responses to increased radiative forcing include enhanced land-sea contrast, stronger response at higher latitudes than in the tropics, and differential responsesin warm and cool season climates to uniform forcing. Here we show that these patterns are also characteristic of model simulations of past climates. The differences in the responses over land as opposed to over the ocean, between high and low latitudes, and between summer and winter are remarkably consistent (proportional and nearly linear) across simulations of both cold and warm climates. Similar patterns also appear in historical observations and paleoclimatic reconstructions, implying that such responses are characteristic features of the climate system, and not simple model artifacts, thereby increasing our confidence in the ability of climate models to correctly simulate different climatic states.

Cyclones causing wind storms in the Mediterranean: characteristics, trends and links to large-scale patterns

A climatology of cyclones with a focus on their relation to wind storm tracks in the Mediterranean region (MR) is presented. Trends in the frequency of cyclones and wind storms, as well as variations associated with the North Atlantic Oscillation (NAO), the East Atlantic/West Russian (EAWR) and the Scandinavian variability pattern (SCAND) are discussed. The study is based on the ERA40 reanalysis dataset. Wind storm tracks are identified by tracking clusters of adjacent grid boxes characterised by extremely high local wind speeds. The wind track is assigned to a cyclone track independently identified with an objective scheme. Areas with high wind activity – quantified by extreme wind tracks – are typically located south of the Golf of Genoa, south of Cyprus, southeast of Sicily and west of the Iberian Peninsula. About 69% of the wind storms are caused by cyclones located in the Mediterranean region, while the remaining 31% can be attributed to North Atlantic or Northern European cyclones. The North Atlantic Oscillation, the East Atlantic/West Russian pattern and the Scandinavian pattern all influence the amount and spatial distribution of wind inducing cyclones and wind events in the MR. The strongest signals exist for the NAO and the EAWR pattern, which are both associated with an increase in the number of organised strong wind events in the eastern MR during their positive phase. On the other hand, the storm numbers decrease over the western MR for the positive phase of the NAO and over the central MR during the positive phase of the EAWR pattern. The positive phase of the Scandinavian pattern is associated with a decrease in the number of winter wind storms over most of the MR. A third of the trends in the number of wind storms and wind producing cyclones during the winter season of the ERA40 period may be attributed to the variability of the North Atlantic Oscillation.

Synoptic-scale controls of persistent low temperature and icy weather over southern China in January 2008

In January 2008, central and southern China experienced persistent low temperatures, freezing rain, and snow. The large-scale conditions associated with the occurrence and development of these snowstorms are examined in order to identify the key synoptic controls leading to this event. Three main factors are identified: 1) the persistent blocking high over Siberia, which remained quasi-stationary around 65°E for 3 weeks, led to advection of dry and cold Siberian air down to central and southern China; 2) a strong persistent southwesterly flow associated with the western Pacific subtropical high led to enhanced moisture advection from the Bay of Bengal into central and southern China; and 3) the deep inversion layer in the lower troposphere associated with the extended snow cover over most of central and southern China. The combination of these three factors is likely responsible for the unusual severity of the event, and hence a long return period

Rossby waves and two-dimensional turbulence in a large-scale zonal jet

The theory of homogeneous barotropic beta-plane turbulence is here extended to include effects arising from spatial inhomogeneity in the form of a zonal shear flow. Attention is restricted to the geophysically important case of zonal flows that are barotropically stable and are of larger scale than the resulting transient eddy field. Because of the presumed scale separation, the disturbance enstrophy is approximately conserved in a fully nonlinear sense, and the (nonlinear) wave-mean-flow interaction may be characterized as a shear-induced spectral transfer of disturbance enstrophy along lines of constant zonal wavenumber k. In this transfer the disturbance energy is generally not conserved. The nonlinear interactions between different disturbance components are turbulent for scales smaller than the inverse of Rhines's cascade-arrest scale κβ[identical with] (β0/2urms)½ and in this regime their leading-order effect may be characterized as a tendency to spread the enstrophy (and energy) along contours of constant total wavenumber κ [identical with] (k2 + l2)½. Insofar as this process of turbulent isotropization involves spectral transfer of disturbance enstrophy across lines of constant zonal wavenumber k, it can be readily distinguished from the shear-induced transfer which proceeds along them. However, an analysis in terms of total wavenumber K alone, which would be justified if the flow were homogeneous, would tend to mask the differences. The foregoing theoretical ideas are tested by performing direct numerical simulation experiments. It is found that the picture of classical beta-plane turbulence is altered, through the effect of the large-scale zonal flow, in the following ways: (i) while the turbulence is still confined to K Kβ, the disturbance field penetrates to the largest scales of motion; (ii) the larger disturbance scales K < Kβ exhibit a tendency to meridional rather than zonal anisotropy, namely towards v2 > u2 rather than vice versa; (iii) the initial spectral transfer rate away from an isotropic intermediate-scale source is significantly enhanced by the shear-induced transfer associated with straining by the zonal flow. This last effect occurs even when the large-scale shear appears weak to the energy-containing eddies, in the sense that dU/dy [double less-than sign] κ for typical eddy length and velocity scales.

Large-scale two-dimensional turbulence in the atmosphere

Global FGGE data are used to investigate several aspects of large-scale turbulence in the atmosphere. The approach follows that for two-dimensional, nondivergent turbulent flows which are homogeneous and isotropic on the sphere. Spectra of kinetic energy, enstrophy and available potential energy are obtained for both the stationary and transient parts of the flow. Nonlinear interaction terms and fluxes of energy and enstrophy through wavenumber space are calculated and compared with the theory. A possible method of parameterizing the interactions with unresolved scales is considered. Two rather different flow regimes are found in wavenumber space. The high-wavenumber regime is dominated by the transient components of the flow and exhibits, at least approximately, several of the conditions characterizing homogeneous and isotropic turbulence. This region of wavenumber space also displays some of the features of an enstrophy-cascading inertial subrange. The low-wavenumber region, on the other hand, is dominated by the stationary component of the flow, exhibits marked anisotropy and, in contrast to the high-wavenumber regime, displays a marked change between January and July.

Scale sensitivity of drivers of environmental change across Europe

The development of effective environmental management plans and policies requires a sound understanding of the driving forces involved in shaping and altering the structure and function of ecosystems. However, driving forces, especially anthropogenic ones, are defined and operate at multiple administrative levels, which do not always match ecological scales. This paper presents an innovative methodology of analysing drivers of change by developing a typology of scale sensitivity of drivers that classifies and describes the way they operate across multiple administrative levels. Scale sensitivity varies considerably among drivers, which can be classified into five broad categories depending on the response of ‘evenness’ and ‘intensity change’ when moving across administrative levels. Indirect drivers tend to show low scale sensitivity, whereas direct drivers show high scale sensitivity, as they operate in a non-linear way across the administrative scale. Thus policies addressing direct drivers of change, in particular, need to take scale into consideration during their formulation. Moreover, such policies must have a strong spatial focus, which can be achieved either by encouraging local–regional policy making or by introducing high flexibility in (inter)national policies to accommodate increased differentiation at lower administrative levels. High quality data is available for several drivers, however, the availability of consistent data at all levels for non-anthropogenic drivers is a major constraint to mapping and assessing their scale sensitivity. This lack of data may hinder effective policy making for environmental management, since it restricts the ability to fully account for scale sensitivity of natural drivers in policy design.

Convergent structure of multitrophic communities over three continents

Ecological theory predicts that communities using the same resources should have similar structure, but evolutionary constraints on colonization and niche shifts may hamper such convergence. Multitrophic communities of wasps exploiting fig fruits, which first evolved about 75MYA, do not show long-term “inheritance” of taxonomic (lineage) composition or species diversity. However, communities on three continents have converged ecologically in the presence and relative abundance of five insect guilds that we define. Some taxa fill the same niches in each community (phylogenetic niche conservatism). However, we show that overall convergence in ecological community structure depends also on a combination of niche shifts by resident lineages and local colonizations of figs by other insect lineages. Our study explores new ground, and develops new heuristic tools, in combining ecology and phylogeny to address patterns in the complex multitrophic communities of insect on plants, which comprise a large part of terrestrial biodiversity.

Dynamic group communication for large-scale parallel data mining

Exascale systems are the next frontier in high-performance computing and are expected to deliver a performance of the order of 10^18 operations per second using massive multicore processors. Very large- and extreme-scale parallel systems pose critical algorithmic challenges, especially related to concurrency, locality and the need to avoid global communication patterns. This work investigates a novel protocol for dynamic group communication that can be used to remove the global communication requirement and to reduce the communication cost in parallel formulations of iterative data mining algorithms. The protocol is used to provide a communication-efficient parallel formulation of the k-means algorithm for cluster analysis. The approach is based on a collective communication operation for dynamic groups of processes and exploits non-uniform data distributions. Non-uniform data distributions can be either found in real-world distributed applications or induced by means of multidimensional binary search trees. The analysis of the proposed dynamic group communication protocol has shown that it does not introduce significant communication overhead. The parallel clustering algorithm has also been extended to accommodate an approximation error, which allows a further reduction of the communication costs. The effectiveness of the exact and approximate methods has been tested in a parallel computing system with 64 processors and in simulations with 1024 processing elements.

Nationalised large-scale mining, trade unions and community representation: perspectives from Northern Madagascar

This article critically explores the nature and purpose of relationships and inter-dependencies between stakeholders in the context of a parastatal chromite mining company in the Betsiboka Region of Northern Madagascar. An examination of the institutional arrangements at the interface between the mining company and local communities identified power hierarchies and dependencies in the context of a dominant paternalistic environment. The interactions, inter alia, limited social cohesion and intensified the fragility and weakness of community representation, which was further influenced by ethnic hierarchies between the varied community groups; namely, indigenous communities and migrants to the area from different ethnic groups. Moreover, dependencies and nepotism, which may exist at all institutional levels, can create civil society stakeholder representatives who are unrepresentative of the society they are intended to represent. Similarly, a lack of horizontal and vertical trust and reciprocity inherent in Malagasy society engenders a culture of low expectations regarding transparency and accountability, which further catalyses a cycle of nepotism and elite rent-seeking behaviour. On the other hand, leaders retain power with minimal vertical delegation or decentralisation of authority among levels of government and limit opportunities to benefit the elite, perpetuating rent-seeking behaviour within the privileged minority. Within the union movement, pluralism and the associated politicisation of individual unions restricts solidarity, which impacts on the movement’s capacity to act as a cohesive body of opinion and opposition. Nevertheless, the unions’ drive to improve their social capital has increased expectations of transparency and accountability, resulting in demands for greater engagement in decision-making processes.