Dynamic group communication for large-scale parallel data mining

Exascale systems are the next frontier in high-performance computing and are expected to deliver a performance of the order of 10^18 operations per second using massive multicore processors. Very large- and extreme-scale parallel systems pose critical algorithmic challenges, especially related to concurrency, locality and the need to avoid global communication patterns. This work investigates a novel protocol for dynamic group communication that can be used to remove the global communication requirement and to reduce the communication cost in parallel formulations of iterative data mining algorithms. The protocol is used to provide a communication-efficient parallel formulation of the k-means algorithm for cluster analysis. The approach is based on a collective communication operation for dynamic groups of processes and exploits non-uniform data distributions. Non-uniform data distributions can be either found in real-world distributed applications or induced by means of multidimensional binary search trees. The analysis of the proposed dynamic group communication protocol has shown that it does not introduce significant communication overhead. The parallel clustering algorithm has also been extended to accommodate an approximation error, which allows a further reduction of the communication costs. The effectiveness of the exact and approximate methods has been tested in a parallel computing system with 64 processors and in simulations with 1024 processing elements.

Quantifying the importance of galactic cosmic rays in cloud microphysical processes

Galactic Cosmic Rays are one of the major sources of ion production in the troposphere and stratosphere. Recent studies have shown that ions form electrically charged clusters which may grow to become cloud droplets. Aerosol particles charge by the attachment of ions and electrons. The collision efficiency between a particle and a water droplet increases, if the particle is electrically charged, and thus aerosol-cloud interactions can be enhanced. Because these microphysical processes may change radiative properties of cloud and impact Earth's climate it is important to evaluate these processes' quantitative effects. Five different models developed independently have been coupled to investigate this. The first model estimates cloud height from dew point temperature and the temperature profile. The second model simulates the cloud droplet growth from aerosol particles using the cloud parcel concept. In the third model, the scavenging rate of the aerosol particles is calculated using the collision efficiency between charged particles and droplets. The fourth model calculates electric field and charge distribution on water droplets and aerosols within cloud. The fifth model simulates the global electric circuit (GEC), which computes the conductivity and ionic concentration in the atmosphere in altitude range 0–45 km. The first four models are initially coupled to calculate the height of cloud, boundary condition of cloud, followed by growth of droplets, charge distribution calculation on aerosols and cloud droplets and finally scavenging. These models are incorporated with the GEC model. The simulations are verified with experimental data of charged aerosol for various altitudes. Our calculations showed an effect of aerosol charging on the CCN concentration within the cloud, due to charging of aerosols increase the scavenging of particles in the size range 0.1 µm to 1 µm.

Success and failure of grassroots innovations for addressing climate change: the case of the Transition Movement

Grassroots innovations emerge as networks generating innovative solutions for climate change adaptation and mitigation. However, it is unclear if grassroots innovations can be successful in responding to climate change. Little evidence exists on replication, international comparisons are rare, and research tends to overlook discontinued responses in favour of successful ones. We take the Transition Movement as a case study of a rapidly spreading transnational grassroots network, and include both active and non-active local transition initiatives. We investigate the replication of grassroots innovations in different contexts with the aim to uncover general patterns of success and failure, and identify questions for future research. An online survey was carried out in 23 countries (N=276). The data analysis entailed testing the effect of internal and contextual factors of success as drawn from the existing literature, and the identification of clusters of transition initiatives with similar internal and contextual factor configurations. Most transition initiatives consider themselves successful. Success is defined along the lines of social connectivity and empowerment, and external environmental impact. We find that less successful transition initiatives might underestimate the importance of contextual factors and material resources in influencing success. We also find that their diffusion is linked to the combination of local-global learning processes, and that there is an incubation period during which a transition initiative is consolidated. Transition initiatives seem capable of generalising organisational principles derived from unique local experiences that seem to be effective in other local contexts. However, the geographical locations matter with regard to where transition initiatives take root and the extent of their success, and ‘place attachment’ may have a role in the diffusion of successful initatives. We suggest that longitudinal comparative studies can advance our understanding in this regard, as well as inform the changing nature of the definition of success at different stages of grassroots innovation development, and the dynamic nature of local and global linkages.

Automated determination of auroral breakup during the substorm expansion phase using all sky imager data

This technique paper describes a novel method for quantitatively and routinely identifying auroral breakup following substorm onset using the Time History of Events and Macroscale Interactions During Substorms (THEMIS) all-sky imagers (ASIs). Substorm onset is characterised by a brightening of the aurora that is followed by auroral poleward expansion and auroral breakup. This breakup can be identified by a sharp increase in the auroral intensity i(t) and the time derivative of auroral intensity i'(t). Utilising both i(t) and i'(t) we have developed an algorithm for identifying the time interval and spatial location of auroral breakup during the substorm expansion phase within the field of view of ASI data based solely on quantifiable characteristics of the optical auroral emissions. We compare the time interval determined by the algorithm to independently identified auroral onset times from three previously published studies. In each case the time interval determined by the algorithm is within error of the onset independently identified by the prior studies. We further show the utility of the algorithm by comparing the breakup intervals determined using the automated algorithm to an independent list of substorm onset times. We demonstrate that up to 50% of the breakup intervals characterised by the algorithm are within the uncertainty of the times identified in the independent list. The quantitative description and routine identification of an interval of auroral brightening during the substorm expansion phase provides a foundation for unbiased statistical analysis of the aurora to probe the physics of the auroral substorm as a new scientific tool for aiding the identification of the processes leading to auroral substorm onset.

Tropical climates at the Last Glacial Maximum: a new synthesis of terrestrial palaeoclimate data. I. Vegetation, lake-levels and geochemistry.

Palaeodata in synthesis form are needed as benchmarks for the Palaeoclimate Modelling Intercomparison Project (PMIP). Advances since the last synthesis of terrestrial palaeodata from the last glacial maximum (LGM) call for a new evaluation, especially of data from the tropics. Here pollen, plant-macrofossil, lake-level, noble gas (from groundwater) and δ18O (from speleothems) data are compiled for 18±2 ka (14C), 32 °N–33 °S. The reliability of the data was evaluated using explicit criteria and some types of data were re-analysed using consistent methods in order to derive a set of mutually consistent palaeoclimate estimates of mean temperature of the coldest month (MTCO), mean annual temperature (MAT), plant available moisture (PAM) and runoff (P-E). Cold-month temperature (MAT) anomalies from plant data range from −1 to −2 K near sea level in Indonesia and the S Pacific, through −6 to −8 K at many high-elevation sites to −8 to −15 K in S China and the SE USA. MAT anomalies from groundwater or speleothems seem more uniform (−4 to −6 K), but the data are as yet sparse; a clear divergence between MAT and cold-month estimates from the same region is seen only in the SE USA, where cold-air advection is expected to have enhanced cooling in winter. Regression of all cold-month anomalies against site elevation yielded an estimated average cooling of −2.5 to −3 K at modern sea level, increasing to ≈−6 K by 3000 m. However, Neotropical sites showed larger than the average sea-level cooling (−5 to −6 K) and a non-significant elevation effect, whereas W and S Pacific sites showed much less sea-level cooling (−1 K) and a stronger elevation effect. These findings support the inference that tropical sea-surface temperatures (SSTs) were lower than the CLIMAP estimates, but they limit the plausible average tropical sea-surface cooling, and they support the existence of CLIMAP-like geographic patterns in SST anomalies. Trends of PAM and lake levels indicate wet LGM conditions in the W USA, and at the highest elevations, with generally dry conditions elsewhere. These results suggest a colder-than-present ocean surface producing a weaker hydrological cycle, more arid continents, and arguably steeper-than-present terrestrial lapse rates. Such linkages are supported by recent observations on freezing-level height and tropical SSTs; moreover, simulations of “greenhouse” and LGM climates point to several possible feedback processes by which low-level temperature anomalies might be amplified aloft.

Characterizing sampling bias in the trace gas climatologies of the SPARC Data Initiative

Monthly zonal mean climatologies of atmospheric measurements from satellite instruments can have biases due to the nonuniform sampling of the atmosphere by the instruments. We characterize potential sampling biases in stratospheric trace gas climatologies of the Stratospheric Processes and Their Role in Climate (SPARC) Data Initiative using chemical fields from a chemistry climate model simulation and sampling patterns from 16 satellite-borne instruments. The exercise is performed for the long-lived stratospheric trace gases O3 and H2O. Monthly sampling biases for O3 exceed 10% for many instruments in the high-latitude stratosphere and in the upper troposphere/lower stratosphere, while annual mean sampling biases reach values of up to 20% in the same regions for some instruments. Sampling biases for H2O are generally smaller than for O3, although still notable in the upper troposphere/lower stratosphere and Southern Hemisphere high latitudes. The most important mechanism leading to monthly sampling bias is nonuniform temporal sampling, i.e., the fact that for many instruments, monthly means are produced from measurements which span less than the full month in question. Similarly, annual mean sampling biases are well explained by nonuniformity in the month-to-month sampling by different instruments. Nonuniform sampling in latitude and longitude are shown to also lead to nonnegligible sampling biases, which are most relevant for climatologies which are otherwise free of biases due to nonuniform temporal sampling.

The SPARC Data Initiative: Comparison of upper troposphere/lower stratosphere ozone climatologies from limb-viewing instruments and the nadir-viewing Tropospheric Emission Spectrometer

We present the first comprehensive intercomparison of currently available satellite ozone climatologies in the upper troposphere/lower stratosphere (UTLS) (300–70 hPa) as part of the Stratosphere-troposphere Processes and their Role in Climate (SPARC) Data Initiative. The Tropospheric Emission Spectrometer (TES) instrument is the only nadir-viewing instrument in this initiative, as well as the only instrument with a focus on tropospheric composition. We apply the TES observational operator to ozone climatologies from the more highly vertically resolved limb-viewing instruments. This minimizes the impact of differences in vertical resolution among the instruments and allows identification of systematic differences in the large-scale structure and variability of UTLS ozone. We find that the climatologies from most of the limb-viewing instruments show positive differences (ranging from 5 to 75%) with respect to TES in the tropical UTLS, and comparison to a “zonal mean” ozonesonde climatology indicates that these differences likely represent a positive bias for p ≤ 100 hPa. In the extratropics, there is good agreement among the climatologies regarding the timing and magnitude of the ozone seasonal cycle (differences in the peak-to-peak amplitude of <15%) when the TES observational operator is applied, as well as very consistent midlatitude interannual variability. The discrepancies in ozone temporal variability are larger in the tropics, with differences between the data sets of up to 55% in the seasonal cycle amplitude. However, the differences among the climatologies are everywhere much smaller than the range produced by current chemistry-climate models, indicating that the multiple-instrument ensemble is useful for quantitatively evaluating these models.

Investigating hydrological regimes and processes in a set of catchments with temporary waters in Mediterranean Europe

Seven catchments of diverse size in Mediterranean Europe were investigated in order to understand the main aspects of their hydrological functioning. The methods included the analysis of daily and monthly precipitation, monthly potential evapotranspiration rates, flow duration curves, rainfall runoff relationships and catchment internal data for the smaller and more instrumented catchments. The results showed that the catchments were less dry than initially considered. Only one of them was really semi-arid throughout the year. All the remaining catchments showed wet seasons when precipitation exceeded potential evapotrans-piration, allowing aquifer recharge, wet runoff generation mechanisms and relevant baseflow contribution. Nevertheless, local infiltration excess (Hortonian) overland flow was inferred during summer storms in some catchments and urban overland flow in some others. The roles of karstic groundwater, human disturbance and low winter temperatures were identified as having an important impact on the hydrological regime in some of the catchments.

New opportunities for secure communication networks using shaped femtosecond laser pulses inducing filamentation processes in the atmosphere

The current study discusses new opportunities for secure ground to satellite communications using shaped femtosecond pulses that induce spatial hole burning in the atmosphere for efficient communications with data encoded within super-continua generated by femtosecond pulses. Refractive index variation across the different layers in the atmosphere may be modelled using assumptions that the upper strata of the atmosphere and troposphere behaving as layered composite amorphous dielectric networks composed of resistors and capacitors with different time constants across each layer. Input-output expressions of the dynamics of the networks in the frequency domain provide the transmission characteristics of the propagation medium. Femtosecond pulse shaping may be used to optimize the pulse phase-front and spectral composition across the different layers in the atmosphere. A generic procedure based on evolutionary algorithms to perform the pulse shaping is proposed. In contrast to alternative procedures that would require ab initio modelling and calculations of the propagation constant for the pulse through the atmosphere, the proposed approach is adaptive, compensating for refractive index variations along the column of air between the transmitter and receiver.

Sources, sinks, and transatlantic transport of North African dust aerosol: a multimodel analysis and comparison with remote sensing data

This study evaluates model-simulated dust aerosols over North Africa and the North Atlantic from five global models that participated in the Aerosol Comparison between Observations and Models phase II model experiments. The model results are compared with satellite aerosol optical depth (AOD) data from Moderate Resolution Imaging Spectroradiometer (MODIS), Multiangle Imaging Spectroradiometer (MISR), and Sea-viewing Wide Field-of-view Sensor, dust optical depth (DOD) derived from MODIS and MISR, AOD and coarse-mode AOD (as a proxy of DOD) from ground-based Aerosol Robotic Network Sun photometer measurements, and dust vertical distributions/centroid height from Cloud Aerosol Lidar with Orthogonal Polarization and Atmospheric Infrared Sounder satellite AOD retrievals. We examine the following quantities of AOD and DOD: (1) the magnitudes over land and over ocean in our study domain, (2) the longitudinal gradient from the dust source region over North Africa to the western North Atlantic, (3) seasonal variations at different locations, and (4) the dust vertical profile shape and the AOD centroid height (altitude above or below which half of the AOD is located). The different satellite data show consistent features in most of these aspects; however, the models display large diversity in all of them, with significant differences among the models and between models and observations. By examining dust emission, removal, and mass extinction efficiency in the five models, we also find remarkable differences among the models that all contribute to the discrepancies of model-simulated dust amount and distribution. This study highlights the challenges in simulating the dust physical and optical processes, even in the best known dust environment, and stresses the need for observable quantities to constrain the model processes.

The low-latitude boundary layer: Application of ISTP advances to past data

The destruction of the four Cluster craft was a major loss to the planned ISTP effort, of which studies of the magnetopause and low-latitude boundary layer (LLBL) were an important part. While awaiting the re-flight mission, Cluster-II, we have been applying advances in our understanding made using other ISTP craft (like Polar and Wind) and using ground-based facilities (in particular the EISCAT incoherent scatter radars and the SuperDARN HF coherent radars) to measurements of the LLBL made in 1984 and 1985 by the AMPTE-UKS and -IRM spacecraft pair. In particular, one unexplained result of the AMPTE mission was that the electron characteristics could, in nearly all cases, order independent measurements near the magnetopause, such as the magnetic field, ion temperatures and the plasma flow. Studies of the cusp have shown that the precipitation is ordered by the time-elapsed since the field line was opened by reconnection. This insight has allowed us to reanalyse the AMPTE data and show that the ordering by the transition parameter is also due to the variation of time elapsed since reconnection, with the important implication that reconnection usually coats most of the dayside magnetopause with at least some newly-opened field lines. In addition, we can use the electron characteristics to isolate features like RDs, slow-mode shocks and slow-mode expansion fans. The ion characteristics can be used to compute the reconnection rate. We here retrospectively apply these new techniques, developed in the ISTP era, to a much-studied flux transfer event observed by the AMPTE satellites. As a result, we gain new understanding of its cause and structure.

Superthermal ion signatures of auroral acceleration processes

The retarding ion mass spectrometer on the Dynamics Explorer 1 spacecraft has generated a unique data set which documents, among other things, the occurrence of non-Maxwellian superthermal features in the auroral topside ionosphere distribution functions. In this paper, we provide a representative sampling of the observed features and their spatial morphology as observed at altitudes in the range from a few thousand kilometers to a few earth radii. At lower altitudes, these features appear at auroral latitudes separating regions of polar cap and subauroral light ion polar wind. The most common signature is the appearance of an upgoing energetic tail having conical lobes representing significant ion heat and number flux in all species, including O+. Transverse ion heating below the observation point at several thousand kilometers is clearly associated with O+ outflows. In some events observed, transverse acceleration apparently involves nearly the entire thermal plasma, the distribution function becomes highly anisotropic with T⊥ > T∥, and may actually develop a minimum at zero velocity, i.e., become a torus having as its axis the local magnetic field direction. At higher altitudes, the localized dayside source region appears as a field aligned flow which is dispersed tailward across the polar cap according to parallel velocity by antisunward convective flow, so that upflowing low energy O+ ions appear well within the polar cap region. While this flow can appear beamlike in a given location, the energy dispersion observed implies a very broad energy distribution at the source, extending from a few tenths of an eV to in excess of 50 eV. On the nightside, upgoing ion beams are found to be latitudinally bounded by regions of ion conics whose half angles increase with increasing separation from the beam region, indicating low altitude transverse acceleration in immediate proximity to, and below, the parallel acceleration region. These observations reveal a clear distinction between classical polar wind ion outflow and O+ enhanced superthermal flows, and confirm the importance of low altitude transverse acceleration in ionospheric plasma transport, as suggested by previous observations.

Global burned-land estimation in Latin America using MODIS composite data

This paper presents results of the AQL2004 project, which has been develope within the GOFC-GOLD Latin American network of remote sensing and forest fires (RedLatif). The project intended to obtain monthly burned-land maps of the entire region, from Mexico to Patagonia, using MODIS (moderate-resolution imaging spectroradiometer) reflectance data. The project has been organized in three different phases: acquisition and preprocessing of satellite data; discrimination of burned pixels; and validation of results. In the first phase, input data consisting of 32-day composites of MODIS 500-m reflectance data generated by the Global Land Cover Facility (GLCF) of the University of Maryland (College Park, Maryland, U.S.A.) were collected and processed. The discrimination of burned areas was addressed in two steps: searching for "burned core" pixels using postfire spectral indices and multitemporal change detection and mapping of burned scars using contextual techniques. The validation phase was based on visual analysis of Landsat and CBERS (China-Brazil Earth Resources Satellite) images. Validation of the burned-land category showed an agreement ranging from 30% to 60%, depending on the ecosystem and vegetation species present. The total burned area for the entire year was estimated to be 153 215 km2. The most affected countries in relation to their territory were Cuba, Colombia, Bolivia, and Venezuela. Burned areas were found in most land covers; herbaceous vegetation (savannas and grasslands) presented the highest proportions of burned area, while perennial forest had the lowest proportions. The importance of croplands in the total burned area should be taken with reserve, since this cover presented the highest commission errors. The importance of generating systematic products of burned land areas for different ecological processes is emphasized.

Assessing the sensitivities of a distributed snow model to forcing data resolution

Highly heterogeneous mountain snow distributions strongly affect soil moisture patterns; local ecology; and, ultimately, the timing, magnitude, and chemistry of stream runoff. Capturing these vital heterogeneities in a physically based distributed snow model requires appropriately scaled model structures. This work looks at how model scale—particularly the resolutions at which the forcing processes are represented—affects simulated snow distributions and melt. The research area is in the Reynolds Creek Experimental Watershed in southwestern Idaho. In this region, where there is a negative correlation between snow accumulation and melt rates, overall scale degradation pushed simulated melt to earlier in the season. The processes mainly responsible for snow distribution heterogeneity in this region—wind speed, wind-affected snow accumulations, thermal radiation, and solar radiation—were also independently rescaled to test process-specific spatiotemporal sensitivities. It was found that in order to accurately simulate snowmelt in this catchment, the snow cover needed to be resolved to 100 m. Wind and wind-affected precipitation—the primary influence on snow distribution—required similar resolution. Thermal radiation scaled with the vegetation structure (~100 m), while solar radiation was adequately modeled with 100–250-m resolution. Spatiotemporal sensitivities to model scale were found that allowed for further reductions in computational costs through the winter months with limited losses in accuracy. It was also shown that these modeling-based scale breaks could be associated with physiographic and vegetation structures to aid a priori modeling decisions.

Evidence of a topographic signal in surface soil moisture derived from ENVISAT ASAR wide swath data

The susceptibility of a catchment to flooding is affected by its soil moisture prior to an extreme rainfall event. While soil moisture is routinely observed by satellite instruments, results from previous work on the assimilation of remotely sensed soil moisture into hydrologic models have been mixed. This may have been due in part to the low spatial resolution of the observations used. In this study, the remote sensing aspects of a project attempting to improve flow predictions from a distributed hydrologic model by assimilating soil moisture measurements are described. Advanced Synthetic Aperture Radar (ASAR) Wide Swath data were used to measure soil moisture as, unlike low resolution microwave data, they have sufficient resolution to allow soil moisture variations due to local topography to be detected, which may help to take into account the spatial heterogeneity of hydrological processes. Surface soil moisture content (SSMC) was measured over the catchments of the Severn and Avon rivers in the South West UK. To reduce the influence of vegetation, measurements were made only over homogeneous pixels of improved grassland determined from a land cover map. Radar backscatter was corrected for terrain variations and normalized to a common incidence angle. SSMC was calculated using change detection. To search for evidence of a topographic signal, the mean SSMC from improved grassland pixels on low slopes near rivers was compared to that on higher slopes. When the mean SSMC on low slopes was 30–90%, the higher slopes were slightly drier than the low slopes. The effect was reversed for lower SSMC values. It was also more pronounced during a drying event. These findings contribute to the scant information in the literature on the use of high resolution SAR soil moisture measurement to improve hydrologic models.