Parallel random prism: a computationally efficient ensemble learner for classification

Generally classifiers tend to overfit if there is noise in the training data or there are missing values. Ensemble learning methods are often used to improve a classifier's classification accuracy. Most ensemble learning approaches aim to improve the classification accuracy of decision trees. However, alternative classifiers to decision trees exist. The recently developed Random Prism ensemble learner for classification aims to improve an alternative classification rule induction approach, the Prism family of algorithms, which addresses some of the limitations of decision trees. However, Random Prism suffers like any ensemble learner from a high computational overhead due to replication of the data and the induction of multiple base classifiers. Hence even modest sized datasets may impose a computational challenge to ensemble learners such as Random Prism. Parallelism is often used to scale up algorithms to deal with large datasets. This paper investigates parallelisation for Random Prism, implements a prototype and evaluates it empirically using a Hadoop computing cluster.

From ensemble forecasts to predictive distribution functions

The translation of an ensemble of model runs into a probability distribution is a common task in model-based prediction. Common methods for such ensemble interpretations proceed as if verification and ensemble were draws from the same underlying distribution, an assumption not viable for most, if any, real world ensembles. An alternative is to consider an ensemble as merely a source of information rather than the possible scenarios of reality. This approach, which looks for maps between ensembles and probabilistic distributions, is investigated and extended. Common methods are revisited, and an improvement to standard kernel dressing, called ‘affine kernel dressing’ (AKD), is introduced. AKD assumes an affine mapping between ensemble and verification, typically not acting on individual ensemble members but on the entire ensemble as a whole, the parameters of this mapping are determined in parallel with the other dressing parameters, including a weight assigned to the unconditioned (climatological) distribution. These amendments to standard kernel dressing, albeit simple, can improve performance significantly and are shown to be appropriate for both overdispersive and underdispersive ensembles, unlike standard kernel dressing which exacerbates over dispersion. Studies are presented using operational numerical weather predictions for two locations and data from the Lorenz63 system, demonstrating both effectiveness given operational constraints and statistical significance given a large sample.

The European Flood Alert System and the communication, perception, and use of ensemble predictions for operational flood risk management

Although ensemble prediction systems (EPS) are increasingly promoted as the scientific state-of-the-art for operational flood forecasting, the communication, perception, and use of the resulting alerts have received much less attention. Using a variety of qualitative research methods, including direct user feedback at training workshops, participant observation during site visits to 25 forecasting centres across Europe, and in-depth interviews with 69 forecasters, civil protection officials, and policy makers involved in operational flood risk management in 17 European countries, this article discusses the perception, communication, and use of European Flood Alert System (EFAS) alerts in operational flood management. In particular, this article describes how the design of EFAS alerts has evolved in response to user feedback and desires for a hydrographic-like way of visualizing EFAS outputs. It also documents a variety of forecaster perceptions about the value and skill of EFAS forecasts and the best way of using them to inform operational decision making. EFAS flood alerts were generally welcomed by flood forecasters as a sort of ‘pre-alert’ to spur greater internal vigilance. In most cases, however, they did not lead, by themselves, to further preparatory action or to earlier warnings to the public or emergency services. Their hesitancy to act in response to medium-term, probabilistic alerts highlights some wider institutional obstacles to the hopes in the research community that EPS will be readily embraced by operational forecasters and lead to immediate improvements in flood incident management. The EFAS experience offers lessons for other hydrological services seeking to implement EPS operationally for flood forecasting and warning. Copyright © 2012 John Wiley & Sons, Ltd.

Northern winter stratospheric temperature and ozone responses to ENSO inferred from an ensemble of Chemistry Climate Models

The connection between the El Ni˜no Southern Oscillation (ENSO) and the Northern polar stratosphere has been established from observations and atmospheric modeling. Here a systematic inter-comparison of the sensitivity of the modeled stratosphere to ENSO in Chemistry Climate Models (CCMs) is reported. This work uses results from a number of the CCMs included in the 2006 ozone assessment. In the lower stratosphere, the mean of all model simulations reports a warming of the polar vortex during strong ENSO events in February–March, consistent with but smaller than the estimate from satellite observations and ERA40 reanalysis. The anomalous warming is associated with an anomalous dynamical increase of column ozone north of 70� N that is accompanied by coherent column ozone decrease in the Tropics, in agreement with that deduced from the NIWA column ozone database, implying an increased residual circulation in the mean of all model simulations during ENSO. The spread in the model responses is partly due to the large internal stratospheric variability and it is shown that it crucially depends on the representation of the tropospheric ENSO teleconnection in the models.

A low-order model investigation of the analysis of gravity waves in the ensemble Kalman filter.

The behavior of the ensemble Kalman filter (EnKF) is examined in the context of a model that exhibits a nonlinear chaotic (slow) vortical mode coupled to a linear (fast) gravity wave of a given amplitude and frequency. It is shown that accurate recovery of both modes is enhanced when covariances between fast and slow normal-mode variables (which reflect the slaving relations inherent in balanced dynamics) are modeled correctly. More ensemble members are needed to recover the fast, linear gravity wave than the slow, vortical motion. Although the EnKF tends to diverge in the analysis of the gravity wave, the filter divergence is stable and does not lead to a great loss of accuracy. Consequently, provided the ensemble is large enough and observations are made that reflect both time scales, the EnKF is able to recover both time scales more accurately than optimal interpolation (OI), which uses a static error covariance matrix. For OI it is also found to be problematic to observe the state at a frequency that is a subharmonic of the gravity wave frequency, a problem that is in part overcome by the EnKF.However, error in themodeled gravity wave parameters can be detrimental to the performance of the EnKF and remove its implied advantages, suggesting that a modified algorithm or a method for accounting for model error is needed.

Information-based data selection for ensemble data assimilation

Ensemble-based data assimilation is rapidly proving itself as a computationally-efficient and skilful assimilation method for numerical weather prediction, which can provide a viable alternative to more established variational assimilation techniques. However, a fundamental shortcoming of ensemble techniques is that the resulting analysis increments can only span a limited subspace of the state space, whose dimension is less than the ensemble size. This limits the amount of observational information that can effectively constrain the analysis. In this paper, a data selection strategy that aims to assimilate only the observational components that matter most and that can be used with both stochastic and deterministic ensemble filters is presented. This avoids unnecessary computations, reduces round-off errors and minimizes the risk of importing observation bias in the analysis. When an ensemble-based assimilation technique is used to assimilate high-density observations, the data-selection procedure allows the use of larger localization domains that may lead to a more balanced analysis. Results from the use of this data selection technique with a two-dimensional linear and a nonlinear advection model using both in situ and remote sounding observations are discussed.

A Lagrangian model of air-mass photochemistry and mixing using a trajectory ensemble: the Cambridge Tropospheric Trajectory model of Chemistry And Transport (CiTTyCAT) version 4.2

A Lagrangian model of photochemistry and mixing is described (CiTTyCAT, stemming from the Cambridge Tropospheric Trajectory model of Chemistry And Transport), which is suitable for transport and chemistry studies throughout the troposphere. Over the last five years, the model has been developed in parallel at several different institutions and here those developments have been incorporated into one "community" model and documented for the first time. The key photochemical developments include a new scheme for biogenic volatile organic compounds and updated emissions schemes. The key physical development is to evolve composition following an ensemble of trajectories within neighbouring air-masses, including a simple scheme for mixing between them via an evolving "background profile", both within the boundary layer and free troposphere. The model runs along trajectories pre-calculated using winds and temperature from meteorological analyses. In addition, boundary layer height and precipitation rates, output from the analysis model, are interpolated to trajectory points and used as inputs to the mixing and wet deposition schemes. The model is most suitable in regimes when the effects of small-scale turbulent mixing are slow relative to advection by the resolved winds so that coherent air-masses form with distinct composition and strong gradients between them. Such air-masses can persist for many days while stretching, folding and thinning. Lagrangian models offer a useful framework for picking apart the processes of air-mass evolution over inter-continental distances, without being hindered by the numerical diffusion inherent to global Eulerian models. The model, including different box and trajectory modes, is described and some output for each of the modes is presented for evaluation. The model is available for download from a Subversion-controlled repository by contacting the corresponding authors.

Quantification of chemical and physical processes influencing ozone during long-range transport using a trajectory ensemble

During long-range transport, many distinct processes – including photochemistry, deposition, emissions and mixing – contribute to the transformation of air mass composition. Partitioning the effects of different processes can be useful when considering the sensitivity of chemical transformation to, for example, a changing environment or anthropogenic influence. However, transformation is not observed directly, since mixing ratios are measured, and models must be used to relate changes to processes. Here, four cases from the ITCT-Lagrangian 2004 experiment are studied. In each case, aircraft intercepted a distinct air mass several times during transport over the North Atlantic, providing a unique dataset and quantifying the net changes in composition from all processes. A new framework is presented to deconstruct the change in O3 mixing ratio (Δ O3) into its component processes, which were not measured directly, taking into account the uncertainty in measurements, initial air mass variability and its time evolution. The results show that the net chemical processing (Δ O3chem) over the whole simulation is greater than net physical processing (Δ O3phys) in all cases. This is in part explained by cancellation effects associated with mixing. In contrast, each case is in a regime of either net photochemical destruction (lower tropospheric transport) or production (an upper tropospheric biomass burning case). However, physical processes influence O3 indirectly through addition or removal of precursor gases, so that changes to physical parameters in a model can have a larger effect on Δ O3chem than Δ O3phys. Despite its smaller magnitude, the physical processing distinguishes the lower tropospheric export cases, since the net photochemical O3 change is −5 ppbv per day in all three cases. Processing is quantified using a Lagrangian photochemical model with a novel method for simulating mixing through an ensemble of trajectories and a background profile that evolves with them. The model is able to simulate the magnitude and variability of the observations (of O3, CO, NOy and some hydrocarbons) and is consistent with the time-average OH following air-masses inferred from hydrocarbon measurements alone (by Arnold et al., 2007). Therefore, it is a useful new method to simulate air mass evolution and variability, and its sensitivity to process parameters.

Regional changes in wind energy potential over Europe using regional climate model ensemble projections

The impact of climate change on wind power generation potentials over Europe is investigated by considering ensemble projections from two regional climate models (RCMs) driven by a global climate model (GCM). Wind energy density and its interannual variability are estimated based on hourly near-surface wind speeds. Additionally, the possible impact of climatic changes on the energy output of a sample 2.5-MW turbine is discussed. GCM-driven RCM simulations capture the behavior and variability of current wind energy indices, even though some differences exist when compared with reanalysis-driven RCM simulations. Toward the end of the twenty-first century, projections show significant changes of energy density on annual average across Europe that are substantially stronger in seasonal terms. The emergence time of these changes varies from region to region and season to season, but some long-term trends are already statistically significant in the middle of the twenty-first century. Over northern and central Europe, the wind energy potential is projected to increase, particularly in winter and autumn. In contrast, energy potential over southern Europe may experience a decrease in all seasons except for the Aegean Sea. Changes for wind energy output follow the same patterns but are of smaller magnitude. The GCM/RCM model chains project a significant intensification of both interannual and intra-annual variability of energy density over parts of western and central Europe, thus imposing new challenges to a reliable pan-European energy supply in future decades.

Ensemble projections for wine production in the Douro Valley of Portugal

Wine production is largely governed by atmospheric conditions, such as air temperature and precipitation, together with soil management and viticultural/enological practices. Therefore, anthropogenic climate change is likely to have important impacts on the winemaking sector worldwide. An important winemaking region is the Portuguese Douro Valley, which is known by its world-famous Port Wine. The identification of robust relationships between atmospheric factors and wine parameters is of great relevance for the region. A multivariate linear regression analysis of a long wine production series (1932–2010) reveals that high rainfall and cool temperatures during budburst, shoot and inflorescence development (February-March) and warm temperatures during flowering and berry development (May) are generally favourable to high production. The probabilities of occurrence of three production categories (low, normal and high) are also modelled using multinomial logistic regression. Results show that both statistical models are valuable tools for predicting the production in a given year with a lead time of 3–4 months prior to harvest. These statistical models are applied to an ensemble of 16 regional climate model experiments following the SRES A1B scenario to estimate possible future changes. Wine production is projected to increase by about 10 % by the end of the 21st century, while the occurrence of high production years is expected to increase from 25 % to over 60 %. Nevertheless, further model development will be needed to include other aspects that may shape production in the future. In particular, the rising heat stress and/or changes in ripening conditions could limit the projected production increase in future decades.

European storminess and associated circulation weather types: future changes deduced from a multi-model ensemble of GCM simulations

A range of possible changes in the frequency and characteristics of European wind storms under future climate conditions was investigated on the basis of a multi-model ensemble of 9 coupled global climate model (GCM) simulations for the 20th and 21st centuries following the IPCC SRES A1B scenario. A multi-model approach allowed an estimation of the (un)certainties of the climate change signals. General changes in large-scale atmospheric flow were analysed, the occurrence of wind storms was quantified, and atmospheric features associated with wind storm events were considered. Identified storm days were investigated according to atmospheric circulation, associated pressure patterns, cyclone tracks and wind speed patterns. Validation against reanalysis data revealed that the GCMs are in general capable of realistically reproducing characteristics of European circulation weather types (CWTs) and wind storms. Results are given with respect to frequency of occurrence, storm-associated flow conditions, cyclone tracks and specific wind speed patterns. Under anthropogenic climate change conditions (SRES A1B scenario), increased frequency of westerly flow during winter is detected over the central European investigation area. In the ensemble mean, the number of detected wind storm days increases between 19 and 33% for 2 different measures of storminess, only 1 GCM revealed less storm days. The increased number of storm days detected in most models is disproportionately high compared to the related CWT changes. The mean intensity of cyclones associated with storm days in the ensemble mean increases by about 10 (±10)% in the Eastern Atlantic, near the British Isles and in the North Sea. Accordingly, wind speeds associated with storm events increase significantly by about 5 (±5)% over large parts of central Europe, mainly on days with westerly flow. The basic conclusions of this work remain valid if different ensemble contructions are considered, leaving out an outlier model or including multiple runs of one particular model.

Statistical uncertainty of changes in winter storms over the North Atlantic and Europe in an ensemble of transient climate simulations

Winter storms are among the most important natural hazards affecting Europe. We quantify changes in storm frequency and intensity over the North Atlantic and Europe under future climate scenarios in terms of return periods (RPs) considering uncertainties due to both sampling and methodology. RPs of North Atlantic storms' minimum central pressure (CP) and maximum vorticity (VOR) remain unchanged by 2100 for both the A1B and A2 scenarios compared to the present climate. Whereas shortened RPs for VOR of all intensities are detected for the area between British Isles/North-Sea/western Europe as early as 2040. However, the changes in storm VOR RP may be unrealistically large: a present day 50 (20) year event becomes approximately a 9 (5.5) year event in both A1B and A2 scenarios by 2100. The detected shortened RPs of storms implies a higher risk of occurrence of damaging wind events over Europe.

Changing Northern Hemisphere storm tracks in an ensemble of IPCC climate change simulations

Winter storm-track activity over the Northern Hemisphere and its changes in a greenhouse gas scenario (the Special Report on Emission Scenarios A1B forcing) are computed from an ensemble of 23 single runs from 16 coupled global climate models (CGCMs). All models reproduce the general structures of the observed climatological storm-track pattern under present-day forcing conditions. Ensemble mean changes resulting from anthropogenic forcing include an increase of baroclinic wave activity over the eastern North Atlantic, amounting to 5%–8% by the end of the twenty-first century. Enhanced activity is also found over the Asian continent and over the North Pacific near the Aleutian Islands. At high latitudes and over parts of the subtropics, activity is reduced. Variations of the individual models around the ensemble average signal are not small, with a median of the pattern correlation near r = 0.5. There is, however, no evidence for a link between deviations in present-day climatology and deviations with respect to climate change.

Changes in storm track and cyclone activity in three SRES ensemble experiments with the ECHAM5/MPI-OM1 GCM

Synoptic activity over the Northern Hemisphere is evaluated in ensembles of ECHAM5/MPI-OM1 simulations for recent climate conditions (20C) and for three climate scenarios (following SRES A1B, A2, B1). A close agreement is found between the simulations for present day climate and the respective results from reanalysis. Significant changes in the winter mid-tropospheric storm tracks are detected in all three scenario simulations. Ensemble mean climate signals are rather similar, with particularly large activity increases downstream of the Atlantic storm track over Western Europe. The magnitude of this signal is largely dependent on the imposed change in forcing. However, differences between individual ensemble members may be large. With respect to the surface cyclones, the scenario runs produce a reduction in cyclonic track density over the mid-latitudes, even in the areas with increasing mid-tropospheric activity. The largest decrease in track densities occurs at subtropical latitudes, e.g., over the Mediterranean Basin. An increase of cyclone intensities is detected for limited areas (e.g., near Great Britain and Aleutian Isles) for the A1B and A2 experiments. The changes in synoptic activity are associated with alterations of the Northern Hemisphere circulation and background conditions (blocking frequencies, jet stream). The North Atlantic Oscillation index also shows increased values with enhanced forcing. With respect to the effects of changing synoptic activity, the regional change in cyclone intensities is accompanied by alterations of the extreme surface winds, with increasing values over Great Britain, North and Baltic Seas, as well as the areas with vanishing sea ice, and decreases over much of the subtropics.

Changing European storm loss potentials under modified climate conditions according to ensemble simulations of the ECHAM5/MPI-OM1 GMC

A simple storm loss model is applied to an ensemble of ECHAM5/MPI-OM1 GCM simulations in order to estimate changes of insured loss potentials over Europe in the 21st century. Losses are computed based on the daily maximum wind speed for each grid point. The calibration of the loss model is performed using wind data from the ERA40-Reanalysis and German loss data. The obtained annual losses for the present climate conditions (20C, three realisations) reproduce the statistical features of the historical insurance loss data for Germany. The climate change experiments correspond to the SRES-Scenarios A1B and A2, and for each of them three realisations are considered. On average, insured loss potentials increase for all analysed European regions at the end of the 21st century. Changes are largest for Germany and France, and lowest for Portugal/Spain. Additionally, the spread between the single realisations is large, ranging e.g. for Germany from −4% to +43% in terms of mean annual loss. Moreover, almost all simulations show an increasing interannual variability of storm damage. This assessment is even more pronounced if no adaptation of building structure to climate change is considered. The increased loss potentials are linked with enhanced values for the high percentiles of surface wind maxima over Western and Central Europe, which in turn are associated with an enhanced number and increased intensity of extreme cyclones over the British Isles and the North Sea.