The Climate-G testbed: towards large scale distributed data management for climate change

Climate-G is a large scale distributed testbed devoted to climate change research. It is an unfunded effort started in 2008 and involving a wide community both in Europe and US. The testbed is an interdisciplinary effort involving partners from several institutions and joining expertise in the field of climate change and computational science. Its main goal is to allow scientists carrying out geographical and cross-institutional data discovery, access, analysis, visualization and sharing of climate data. It represents an attempt to address, in a real environment, challenging data and metadata management issues. This paper presents a complete overview about the Climate-G testbed highlighting the most important results that have been achieved since the beginning of this project.

Fault tolerant decentralised K-Means clustering for asynchronous large-scale networks

The K-Means algorithm for cluster analysis is one of the most influential and popular data mining methods. Its straightforward parallel formulation is well suited for distributed memory systems with reliable interconnection networks, such as massively parallel processors and clusters of workstations. However, in large-scale geographically distributed systems the straightforward parallel algorithm can be rendered useless by a single communication failure or high latency in communication paths. The lack of scalable and fault tolerant global communication and synchronisation methods in large-scale systems has hindered the adoption of the K-Means algorithm for applications in large networked systems such as wireless sensor networks, peer-to-peer systems and mobile ad hoc networks. This work proposes a fully distributed K-Means algorithm (EpidemicK-Means) which does not require global communication and is intrinsically fault tolerant. The proposed distributed K-Means algorithm provides a clustering solution which can approximate the solution of an ideal centralised algorithm over the aggregated data as closely as desired. A comparative performance analysis is carried out against the state of the art sampling methods and shows that the proposed method overcomes the limitations of the sampling-based approaches for skewed clusters distributions. The experimental analysis confirms that the proposed algorithm is very accurate and fault tolerant under unreliable network conditions (message loss and node failures) and is suitable for asynchronous networks of very large and extreme scale.

Formation of vesicles through solvent assisted self-assembly of hydrophobic pentapeptides: encapsulation and pH responsive release of dyes by the vesicles

In the biomimetic design two hydrophobic pentapetides Boc-Ile-Aib-Leu-Phe-Ala-OMe ( I) and Boc-Gly-Ile-Aib-Leu-Phe-OMe (II) (Aib: alpha-aminoisobutyric acid) containing one Aib each are found to undergo solvent assisted self-assembly in methanol/water to form vesicular structures, which can be disrupted by simple addition of acid. The nanovesicles are found to encapsulate dye molecules that can be released by the addition of acid as confirmed by fluorescence microscopy and UV studies. The influence of solvent polarity on the morphology of the materials generated from the peptides has been examined systematically, and shows that fibrillar structures are formed in less polar chloroform/petroleum ether mixture and vesicular structures are formed in more polar methanol/water. Single crystal X-ray diffraction studies reveal that while beta-sheet mediated self-assembly leads to the formation of fibrillar structures, the solvated beta-sheet structure leads to the formation of vesicular structures. The results demonstrate that even hydrophobic peptides can generate vesicular structures from polar solvent which may be employed in model studies of complex biological phenomena.

Epoxide ring opening in a zinc(II) complex in water without any Lewis acid catalyst: formation of only one diastereomer out of 2

The epoxide ring in 5,6-dihydro-5,6-epoxy-1,10-phenanthroline (L) opens up in its reaction with 4-methylaniline and 4-methoxyaniline in water in equimolar proportion at room temperature without any Lewis acid catalyst to give a monohydrate of 6-(4-methyl-phenylamino)-5,6-dihydro-1,10-phenanthrolin-5-ol (L′·H2O) and 6-(4-methoxyphenyl-amino)-5,6-dihydro-1,10-phenanthrolin-5-ol (L″) respectively. Reaction time decreases from 72 to 14 h in boiling water. But the yields become less. Reaction of L with Zn(ClO4)2·6H2O in methanol in 3:1 molar ratio at room temperature affords white [ZnL3](ClO4)2·H2O. The X-ray crystal structure of the acetonitrile solvate [ZnL3](ClO4)2·MeCN has been determined which shows that the metal has a distorted octahedral N6 coordination sphere. [ZnL3](ClO4)2·2H2O reacts with 4-methylaniline and 4-methoxyaniline in boiling water in 1:3 molar proportion in the absence of any Lewis acid catalyst to produce [ZnL′3](ClO4)2·4H2O and [ZnL″3](ClO4)2·H2O, respectively in 1–4 h time in somewhat low yield. In the 1H NMR spectra of [ZnL′3](ClO4)2·4H2O and [ZnL″3](ClO4)2·H2O, only one sharp methyl signal is observed implicating that only one diastereomer out of the 23 possibilities is formed. The same diastereomers are obtained when L′·H2O and L″ are reacted directly with Zn(ClO4)2·6H2O in tetrahydrofuran at room temperature in very good yields. Reactions of L′·H2O and L″ with Ru(phen)2Cl2·2H2O (phen = 1,10-phenanthroline) in equimolar proportion in methanol–water mixture under refluxing condition lead to the isolation of two diastereomers of [Ru(phen)2L′](ClO4)2·2H2O and [Ru(phen)2L″](ClO4)2·2H2O.

A spatio-temporal structure-based approach to drought characterisation

Drought characterisation is an intrinsically spatio-temporal problem. A limitation of previous approaches to characterisation is that they discard much of the spatio-temporal information by reducing events to a lower-order subspace. To address this, an explicit 3-dimensional (longitude, latitude, time) structure-based method is described in which drought events are defined by a spatially and temporarily coherent set of points displaying standardised precipitation below a given threshold. Geometric methods can then be used to measure similarity between individual drought structures. Groupings of these similarities provide an alternative to traditional methods for extracting recurrent space-time signals from geophysical data. The explicit consideration of structure encourages the construction of summary statistics which relate to the event geometry. Example measures considered are the event volume, centroid, and aspect ratio. The utility of a 3-dimensional approach is demonstrated by application to the analysis of European droughts (15 °W to 35°E, and 35 °N to 70°N) for the period 1901–2006. Large-scale structure is found to be abundant with 75 events identified lasting for more than 3 months and spanning at least 0.5 × 106 km2. Near-complete dissimilarity is seen between the individual drought structures, and little or no regularity is found in the time evolution of even the most spatially similar drought events. The spatial distribution of the event centroids and the time evolution of the geographic cross-sectional areas strongly suggest that large area, sustained droughts result from the combination of multiple small area (∼106 km2) short duration (∼3 months) events. The small events are not found to occur independently in space. This leads to the hypothesis that local water feedbacks play an important role in the aggregation process.

Fractional order modeling of large three-dimensional RC networks

An incidence matrix analysis is used to model a three-dimensional network consisting of resistive and capacitive elements distributed across several interconnected layers. A systematic methodology for deriving a descriptor representation of the network with random allocation of the resistors and capacitors is proposed. Using a transformation of the descriptor representation into standard state-space form, amplitude and phase admittance responses of three-dimensional random RC networks are obtained. Such networks display an emergent behavior with a characteristic Jonscher-like response over a wide range of frequencies. A model approximation study of these networks is performed to infer the admittance response using integral and fractional order models. It was found that a fractional order model with only seven parameters can accurately describe the responses of networks composed of more than 70 nodes and 200 branches with 100 resistors and 100 capacitors. The proposed analysis can be used to model charge migration in amorphous materials, which may be associated to specific macroscopic or microscopic scale fractal geometrical structures in composites displaying a viscoelastic electromechanical response, as well as to model the collective responses of processes governed by random events described using statistical mechanics.

Consistent large-scale temperature responses in warm and cold climates

Climate-model simulations of the large-scale temperature responses to increased radiative forcing include enhanced land-sea contrast, stronger response at higher latitudes than in the tropics, and differential responsesin warm and cool season climates to uniform forcing. Here we show that these patterns are also characteristic of model simulations of past climates. The differences in the responses over land as opposed to over the ocean, between high and low latitudes, and between summer and winter are remarkably consistent (proportional and nearly linear) across simulations of both cold and warm climates. Similar patterns also appear in historical observations and paleoclimatic reconstructions, implying that such responses are characteristic features of the climate system, and not simple model artifacts, thereby increasing our confidence in the ability of climate models to correctly simulate different climatic states.

Cyclones causing wind storms in the Mediterranean: characteristics, trends and links to large-scale patterns

A climatology of cyclones with a focus on their relation to wind storm tracks in the Mediterranean region (MR) is presented. Trends in the frequency of cyclones and wind storms, as well as variations associated with the North Atlantic Oscillation (NAO), the East Atlantic/West Russian (EAWR) and the Scandinavian variability pattern (SCAND) are discussed. The study is based on the ERA40 reanalysis dataset. Wind storm tracks are identified by tracking clusters of adjacent grid boxes characterised by extremely high local wind speeds. The wind track is assigned to a cyclone track independently identified with an objective scheme. Areas with high wind activity – quantified by extreme wind tracks – are typically located south of the Golf of Genoa, south of Cyprus, southeast of Sicily and west of the Iberian Peninsula. About 69% of the wind storms are caused by cyclones located in the Mediterranean region, while the remaining 31% can be attributed to North Atlantic or Northern European cyclones. The North Atlantic Oscillation, the East Atlantic/West Russian pattern and the Scandinavian pattern all influence the amount and spatial distribution of wind inducing cyclones and wind events in the MR. The strongest signals exist for the NAO and the EAWR pattern, which are both associated with an increase in the number of organised strong wind events in the eastern MR during their positive phase. On the other hand, the storm numbers decrease over the western MR for the positive phase of the NAO and over the central MR during the positive phase of the EAWR pattern. The positive phase of the Scandinavian pattern is associated with a decrease in the number of winter wind storms over most of the MR. A third of the trends in the number of wind storms and wind producing cyclones during the winter season of the ERA40 period may be attributed to the variability of the North Atlantic Oscillation.

Rossby waves and two-dimensional turbulence in a large-scale zonal jet

The theory of homogeneous barotropic beta-plane turbulence is here extended to include effects arising from spatial inhomogeneity in the form of a zonal shear flow. Attention is restricted to the geophysically important case of zonal flows that are barotropically stable and are of larger scale than the resulting transient eddy field. Because of the presumed scale separation, the disturbance enstrophy is approximately conserved in a fully nonlinear sense, and the (nonlinear) wave-mean-flow interaction may be characterized as a shear-induced spectral transfer of disturbance enstrophy along lines of constant zonal wavenumber k. In this transfer the disturbance energy is generally not conserved. The nonlinear interactions between different disturbance components are turbulent for scales smaller than the inverse of Rhines's cascade-arrest scale κβ[identical with] (β0/2urms)½ and in this regime their leading-order effect may be characterized as a tendency to spread the enstrophy (and energy) along contours of constant total wavenumber κ [identical with] (k2 + l2)½. Insofar as this process of turbulent isotropization involves spectral transfer of disturbance enstrophy across lines of constant zonal wavenumber k, it can be readily distinguished from the shear-induced transfer which proceeds along them. However, an analysis in terms of total wavenumber K alone, which would be justified if the flow were homogeneous, would tend to mask the differences. The foregoing theoretical ideas are tested by performing direct numerical simulation experiments. It is found that the picture of classical beta-plane turbulence is altered, through the effect of the large-scale zonal flow, in the following ways: (i) while the turbulence is still confined to K Kβ, the disturbance field penetrates to the largest scales of motion; (ii) the larger disturbance scales K < Kβ exhibit a tendency to meridional rather than zonal anisotropy, namely towards v2 > u2 rather than vice versa; (iii) the initial spectral transfer rate away from an isotropic intermediate-scale source is significantly enhanced by the shear-induced transfer associated with straining by the zonal flow. This last effect occurs even when the large-scale shear appears weak to the energy-containing eddies, in the sense that dU/dy [double less-than sign] κ for typical eddy length and velocity scales.

Multi-satellite measurements of large diurnal warming events

Diurnal warming events between 5 and 7 K, spatially coherent over large areas (∼1000 km), are observed in independent satellite measurements of ocean surface temperature. The majority of the large events occurred in the extra-tropics. Given sufficient heating (from solar radiation), the location and magnitude of these events appears to be primarily determined by large-scale wind patterns. The amplitude of the measured diurnal heating scales inversely with the spatial resolution of the different sensors used in this study. These results indicate that predictions of peak diurnal warming using wind speeds with a 25 km spatial resolution available from satellite sensors and those with 50–100 km resolution from Numerical Weather Prediction models may have underestimated warming. Thus, the use of these winds in modeling diurnal effects will be limited in accuracy by both the temporal and spatial resolution of the wind fields.

Dynamic group communication for large-scale parallel data mining

Exascale systems are the next frontier in high-performance computing and are expected to deliver a performance of the order of 10^18 operations per second using massive multicore processors. Very large- and extreme-scale parallel systems pose critical algorithmic challenges, especially related to concurrency, locality and the need to avoid global communication patterns. This work investigates a novel protocol for dynamic group communication that can be used to remove the global communication requirement and to reduce the communication cost in parallel formulations of iterative data mining algorithms. The protocol is used to provide a communication-efficient parallel formulation of the k-means algorithm for cluster analysis. The approach is based on a collective communication operation for dynamic groups of processes and exploits non-uniform data distributions. Non-uniform data distributions can be either found in real-world distributed applications or induced by means of multidimensional binary search trees. The analysis of the proposed dynamic group communication protocol has shown that it does not introduce significant communication overhead. The parallel clustering algorithm has also been extended to accommodate an approximation error, which allows a further reduction of the communication costs. The effectiveness of the exact and approximate methods has been tested in a parallel computing system with 64 processors and in simulations with 1024 processing elements.

Nationalised large-scale mining, trade unions and community representation: perspectives from Northern Madagascar

This article critically explores the nature and purpose of relationships and inter-dependencies between stakeholders in the context of a parastatal chromite mining company in the Betsiboka Region of Northern Madagascar. An examination of the institutional arrangements at the interface between the mining company and local communities identified power hierarchies and dependencies in the context of a dominant paternalistic environment. The interactions, inter alia, limited social cohesion and intensified the fragility and weakness of community representation, which was further influenced by ethnic hierarchies between the varied community groups; namely, indigenous communities and migrants to the area from different ethnic groups. Moreover, dependencies and nepotism, which may exist at all institutional levels, can create civil society stakeholder representatives who are unrepresentative of the society they are intended to represent. Similarly, a lack of horizontal and vertical trust and reciprocity inherent in Malagasy society engenders a culture of low expectations regarding transparency and accountability, which further catalyses a cycle of nepotism and elite rent-seeking behaviour. On the other hand, leaders retain power with minimal vertical delegation or decentralisation of authority among levels of government and limit opportunities to benefit the elite, perpetuating rent-seeking behaviour within the privileged minority. Within the union movement, pluralism and the associated politicisation of individual unions restricts solidarity, which impacts on the movement’s capacity to act as a cohesive body of opinion and opposition. Nevertheless, the unions’ drive to improve their social capital has increased expectations of transparency and accountability, resulting in demands for greater engagement in decision-making processes.

The heliospheric magnetic field

The heliospheric magnetic field (HMF) is the extension of the coronal magnetic field carried out into the solar system by the solar wind. It is the means by which the Sun interacts with planetary magnetospheres and channels charged particles propagating through the heliosphere. As the HMF remains rooted at the solar photosphere as the Sun rotates, the large-scale HMF traces out an Archimedean spiral. This pattern is distorted by the interaction of fast and slow solar wind streams, as well as the interplanetary manifestations of transient solar eruptions called coronal mass ejections. On the smaller scale, the HMF exhibits an array of waves, discontinuities, and turbulence, which give hints to the solar wind formation process. This review aims to summarise observations and theory of the small- and large-scale structure of the HMF. Solar-cycle and cycle-to-cycle evolution of the HMF is discussed in terms of recent spacecraft observations and pre-spaceage proxies for the HMF in geomagnetic and galactic cosmic ray records.

The SPARC Data Initiative: Comparison of upper troposphere/lower stratosphere ozone climatologies from limb-viewing instruments and the nadir-viewing Tropospheric Emission Spectrometer

We present the first comprehensive intercomparison of currently available satellite ozone climatologies in the upper troposphere/lower stratosphere (UTLS) (300–70 hPa) as part of the Stratosphere-troposphere Processes and their Role in Climate (SPARC) Data Initiative. The Tropospheric Emission Spectrometer (TES) instrument is the only nadir-viewing instrument in this initiative, as well as the only instrument with a focus on tropospheric composition. We apply the TES observational operator to ozone climatologies from the more highly vertically resolved limb-viewing instruments. This minimizes the impact of differences in vertical resolution among the instruments and allows identification of systematic differences in the large-scale structure and variability of UTLS ozone. We find that the climatologies from most of the limb-viewing instruments show positive differences (ranging from 5 to 75%) with respect to TES in the tropical UTLS, and comparison to a “zonal mean” ozonesonde climatology indicates that these differences likely represent a positive bias for p ≤ 100 hPa. In the extratropics, there is good agreement among the climatologies regarding the timing and magnitude of the ozone seasonal cycle (differences in the peak-to-peak amplitude of <15%) when the TES observational operator is applied, as well as very consistent midlatitude interannual variability. The discrepancies in ozone temporal variability are larger in the tropics, with differences between the data sets of up to 55% in the seasonal cycle amplitude. However, the differences among the climatologies are everywhere much smaller than the range produced by current chemistry-climate models, indicating that the multiple-instrument ensemble is useful for quantitatively evaluating these models.