HESS Opinions ``Ensembles, uncertainty and flood prediction''

Ensemble predictions are being used more frequently to model the propagation of uncertainty through complex, coupled meteorological, hydrological and coastal models, with the goal of better characterising flood risk. In this paper, we consider the issues that we judge to be important when designing and evaluating ensemble predictions, and make recommendations for the guidance of future research.

Global hydrology modelling and uncertainty: running multiple ensembles with a campus grid.

Uncertainties associated with the representation of various physical processes in global climate models (GCMs) mean that, when projections from GCMs are used in climate change impact studies, the uncertainty propagates through to the impact estimates. A complete treatment of this ‘climate model structural uncertainty’ is necessary so that decision-makers are presented with an uncertainty range around the impact estimates. This uncertainty is often underexplored owing to the human and computer processing time required to perform the numerous simulations. Here, we present a 189-member ensemble of global river runoff and water resource stress simulations that adequately address this uncertainty. Following several adaptations and modifications, the ensemble creation time has been reduced from 750 h on a typical single-processor personal computer to 9 h of high-throughput computing on the University of Reading Campus Grid. Here, we outline the changes that had to be made to the hydrological impacts model and to the Campus Grid, and present the main results. We show that, although there is considerable uncertainty in both the magnitude and the sign of regional runoff changes across different GCMs with climate change, there is much less uncertainty in runoff changes for regions that experience large runoff increases (e.g. the high northern latitudes and Central Asia) and large runoff decreases (e.g. the Mediterranean). Furthermore, there is consensus that the percentage of the global population at risk to water resource stress will increase with climate change.

Probabilistic simulations of crop yield over western India using the DEMETER seasonal hindcast ensembles

Process-based integrated modelling of weather and crop yield over large areas is becoming an important research topic. The production of the DEMETER ensemble hindcasts of weather allows this work to be carried out in a probabilistic framework. In this study, ensembles of crop yield (groundnut, Arachis hypogaea L.) were produced for 10 2.5 degrees x 2.5 degrees grid cells in western India using the DEMETER ensembles and the general large-area model (GLAM) for annual crops. Four key issues are addressed by this study. First, crop model calibration methods for use with weather ensemble data are assessed. Calibration using yield ensembles was more successful than calibration using reanalysis data (the European Centre for Medium-Range Weather Forecasts 40-yr reanalysis, ERA40). Secondly, the potential for probabilistic forecasting of crop failure is examined. The hindcasts show skill in the prediction of crop failure, with more severe failures being more predictable. Thirdly, the use of yield ensemble means to predict interannual variability in crop yield is examined and their skill assessed relative to baseline simulations using ERA40. The accuracy of multi-model yield ensemble means is equal to or greater than the accuracy using ERA40. Fourthly, the impact of two key uncertainties, sowing window and spatial scale, is briefly examined. The impact of uncertainty in the sowing window is greater with ERA40 than with the multi-model yield ensemble mean. Subgrid heterogeneity affects model accuracy: where correlations are low on the grid scale, they may be significantly positive on the subgrid scale. The implications of the results of this study for yield forecasting on seasonal time-scales are as follows. (i) There is the potential for probabilistic forecasting of crop failure (defined by a threshold yield value); forecasting of yield terciles shows less potential. (ii) Any improvement in the skill of climate models has the potential to translate into improved deterministic yield prediction. (iii) Whilst model input uncertainties are important, uncertainty in the sowing window may not require specific modelling. The implications of the results of this study for yield forecasting on multidecadal (climate change) time-scales are as follows. (i) The skill in the ensemble mean suggests that the perturbation, within uncertainty bounds, of crop and climate parameters, could potentially average out some of the errors associated with mean yield prediction. (ii) For a given technology trend, decadal fluctuations in the yield-gap parameter used by GLAM may be relatively small, implying some predictability on those time-scales.

Synthesis, crystal structure and hydrolysis of a dinuclear copper(II) complex constructed by N2O donor Schiff base and 4,4 '-bipyridine: discrete supra-molecular ensembles vs. oligomers

The tridentate Schiff base ligand, 7-amino-4-methyl-5-aza-3-hepten-2-one (HAMAH), prepared by the mono-condensation of 1,2diaminoethane and acetylacetone, reacts with Cu(BF4)(2) center dot 6H(2)O to produce initially a dinuclear Cu(II) complex, [{Cu(AMAH)}(2) (mu-4,4'-bipyJ](BF4)(2) (1) which undergoes hydrolysis in the reaction mixture and finally produces a linear polymeric chain compound, [Cu(acac)(2)(mu-4,4'-bipy)](n) (2). The geometry around the copper atom in compound 1 is distorted square planar while that in compound 2 is essentially an elongated octahedron. On the other hand, the ligand HAMAH reacts with Cu(ClO4)(2) center dot 6H(2)O to yield a polymeric zigzag chain, [{Cu(acac)(CH3OH)(mu-4,4'-bipy)}(ClO4)](n) (3). The geometry of the copper atom in 3 is square pyramidal with the two bipyridine molecules in the cis equatorial positions. All three complexes have been characterized by elemental analysis, IR and UV-Vis spectroscopy and single crystal X-ray diffraction studies. A probable explanation for the different size and shape of the reported polynuclear complexes formed by copper(II) and 4,4'-bipyridine has been put forward by taking into account the denticity and crystal field strength of the blocking ligand as well as the Jahn-Teller effect in copper(II). (c) 2007 Elsevier Ltd. All rights reserved.

Predictability of the western North Pacific summer climate demonstrated by the coupled models of ENSEMBLES

The Asian monsoon system, including the western North Pacific (WNP), East Asian, and Indian monsoons, dominates the climate of the Asia-Indian Ocean-Pacific region, and plays a significant role in the global hydrological and energy cycles. The prediction of monsoons and associated climate features is a major challenge in seasonal time scale climate forecast. In this study, a comprehensive assessment of the interannual predictability of the WNP summer climate has been performed using the 1-month lead retrospective forecasts (hindcasts) of five state-of-the-art coupled models from ENSEMBLES for the period of 1960–2005. Spatial distribution of the temporal correlation coefficients shows that the interannual variation of precipitation is well predicted around the Maritime Continent and east of the Philippines. The high skills for the lower-tropospheric circulation and sea surface temperature (SST) spread over almost the whole WNP. These results indicate that the models in general successfully predict the interannual variation of the WNP summer climate. Two typical indices, the WNP summer precipitation index and the WNP lower-tropospheric circulation index (WNPMI), have been used to quantify the forecast skill. The correlation coefficient between five models’ multi-model ensemble (MME) mean prediction and observations for the WNP summer precipitation index reaches 0.66 during 1979–2005 while it is 0.68 for the WNPMI during 1960–2005. The WNPMI-regressed anomalies of lower-tropospheric winds, SSTs and precipitation are similar between observations and MME. Further analysis suggests that prediction reliability of the WNP summer climate mainly arises from the atmosphere–ocean interaction over the tropical Indian and the tropical Pacific Ocean, implying that continuing improvement in the representation of the air–sea interaction over these regions in CGCMs is a key for long-lead seasonal forecast over the WNP and East Asia. On the other hand, the prediction of the WNP summer climate anomalies exhibits a remarkable spread resulted from uncertainty in initial conditions. The summer anomalies related to the prediction spread, including the lower-tropospheric circulation, SST and precipitation anomalies, show a Pacific-Japan or East Asia-Pacific pattern in the meridional direction over the WNP. Our further investigations suggest that the WNPMI prediction spread arises mainly from the internal dynamics in air–sea interaction over the WNP and Indian Ocean, since the local relationships among the anomalous SST, circulation, and precipitation associated with the spread are similar to those associated with the interannual variation of the WNPMI in both observations and MME. However, the magnitudes of these anomalies related to the spread are weaker, ranging from one third to a half of those anomalies associated with the interannual variation of the WNPMI in MME over the tropical Indian Ocean and subtropical WNP. These results further support that the improvement in the representation of the air–sea interaction over the tropical Indian Ocean and subtropical WNP in CGCMs is a key for reducing the prediction spread and for improving the long-lead seasonal forecast over the WNP and East Asia.

Trends in Austral jet position in ensembles of high- and low-top CMIP5 models

Trends in the position of the DJF Austral jet have been analysed for multi-model ensemble simulations of a subset of high- and low-top models for the periods 1960-2000, 2000-2050, and 2050-2098 under the CMIP5 historical, RCP4.5, and RCP8.5 scenarios. Comparison with ERA-Interim, CFSR and the NCEP/NCAR reanalysis shows that the DJF and annual mean jet positions in CMIP5 models are equatorward of reanalyses for the 1979-2006 mean. Under the RCP8.5 scenario, the mean jet position in the high-top models moves 3 degrees poleward of its 1860-1900 position by 2098, compared to just over 2 degrees for the low-top models. Changes in jet position are linked to changes in the meridional temperature gradient. Compared to low-top models, the high-top models predict greater warming in the tropical upper troposphere due to increased greenhouse gases for all periods considered: up to 0.28 K/decade more in the period 2050-2098 under the RCP8.5 scenario. Larger polar lower-stratospheric cooling is seen in high-top models: -1.64 K/decade compared to -1.40 K/decade in the period 1960-2000, mainly in response to ozone depletion, and -0.41 K/decade compared to -0.12 K/decade in the period 2050-2098, mainly in response to increases in greenhouse gases. Analysis suggests that there may be a linear relationship between the trend in jet position and meridional temperature gradient, even under strong forcing. There were no clear indications of an approach to a geometric limit on the absolute magnitude of the poleward shift by 2100.

Evaluating raw ensembles with the continuous ranked probability score

The continuous ranked probability score (CRPS) is a frequently used scoring rule. In contrast with many other scoring rules, the CRPS evaluates cumulative distribution functions. An ensemble of forecasts can easily be converted into a piecewise constant cumulative distribution function with steps at the ensemble members. This renders the CRPS a convenient scoring rule for the evaluation of ‘raw’ ensembles, obviating the need for sophisticated ensemble model output statistics or dressing methods prior to evaluation. In this article, a relation between the CRPS score and the quantile score is established. The evaluation of ‘raw’ ensembles using the CRPS is discussed in this light. It is shown that latent in this evaluation is an interpretation of the ensemble as quantiles but with non-uniform levels. This needs to be taken into account if the ensemble is evaluated further, for example with rank histograms.

Rank histograms of stratified Monte-Carlo ensembles

The application of forecast ensembles to probabilistic weather prediction has spurred considerable interest in their evaluation. Such ensembles are commonly interpreted as Monte Carlo ensembles meaning that the ensemble members are perceived as random draws from a distribution. Under this interpretation, a reasonable property to ask for is statistical consistency, which demands that the ensemble members and the verification behave like draws from the same distribution. A widely used technique to assess statistical consistency of a historical dataset is the rank histogram, which uses as a criterion the number of times that the verification falls between pairs of members of the ordered ensemble. Ensemble evaluation is rendered more specific by stratification, which means that ensembles that satisfy a certain condition (e.g., a certain meteorological regime) are evaluated separately. Fundamental relationships between Monte Carlo ensembles, their rank histograms, and random sampling from the probability simplex according to the Dirichlet distribution are pointed out. Furthermore, the possible benefits and complications of ensemble stratification are discussed. The main conclusion is that a stratified Monte Carlo ensemble might appear inconsistent with the verification even though the original (unstratified) ensemble is consistent. The apparent inconsistency is merely a result of stratification. Stratified rank histograms are thus not necessarily flat. This result is demonstrated by perfect ensemble simulations and supplemented by mathematical arguments. Possible methods to avoid or remove artifacts that stratification induces in the rank histogram are suggested.

Challenges in communicating and using ensembles in operational flood forecasting.

Following trends in operational weather forecasting, where ensemble prediction systems (EPS) are now increasingly the norm, flood forecasters are beginning to experiment with using similar ensemble methods. Most of the effort to date has focused on the substantial technical challenges of developing coupled rainfall-runoff systems to represent the full cascade of uncertainties involved in predicting future flooding. As a consequence much less attention has been given to the communication and eventual use of EPS flood forecasts. Drawing on interviews and other research with operational flood forecasters from across Europe, this paper highlights a number of challenges to communicating and using ensemble flood forecasts operationally. It is shown that operational flood forecasters understand the skill, operational limitations, and informational value of EPS products in a variety of different and sometimes contradictory ways. Despite the efforts of forecasting agencies to design effective ways to communicate EPS forecasts to non-experts, operational flood forecasters were often skeptical about the ability of forecast recipients to understand or use them appropriately. It is argued that better training and closer contacts between operational flood forecasters and EPS system designers can help ensure the uncertainty represented by EPS forecasts is represented in ways that are most appropriate and meaningful for their intended consumers, but some fundamental political and institutional challenges to using ensembles, such as differing attitudes to false alarms and to responsibility for management of blame in the event of poor or mistaken forecasts are also highlighted. Copyright © 2010 Royal Meteorological Society.

Predictability of the western North Pacific summer climate associated with different ENSO phases by ENSEMBLES multi-model seasonal forecasts

Predictability of the western North Pacific (WNP) summer climate associated with different El Niño–Southern Oscillation (ENSO) phases is investigated in this study based on the 1-month lead retrospective forecasts of five state-of-the-art coupled models from ENSEMBLES. During the period from 1960 to 2005, the models well capture the WNP summer climate anomalies during most of years in different ENSO phases except the La Niña decaying summers. In the El Niño developing, El Niño decaying and La Niña developing summers, the prediction skills are high for the WNP summer monsoon index (WNPMI), with the prediction correlation larger than 0.7. The high prediction skills of the lower-tropospheric circulation during these phases are found mainly over the tropical western Pacific Ocean, South China Sea and subtropical WNP. These good predictions correspond well to their close teleconnection with ENSO and the high prediction skills of tropical SSTs. By contrast, for the La Niña decaying summers, the prediction skills are considerably low with the prediction correlation for the WNPMI near to zero and low prediction skills around the Philippines and subtropical WNP. These poor predictions relate to the weak summer anomalies of the WNPMI during the La Niña decaying years and no significant connections between the WNP lower-tropospheric circulation anomalies and the SSTs over the tropical central and eastern Pacific Ocean in observations. However, the models tend to predict an apparent anomalous cyclone over the WNP during the La Niña decaying years, indicating a linearity of the circulation response over WNP in the models prediction in comparison with that during the El Niño decaying years which differs from observations. In addition, the models show considerable capability in describing the WNP summer anomalies during the ENSO neutral summers. These anomalies are related to the positive feedback between the WNP lower-tropospheric circulation and the local SSTs. The models can capture this positive feedback but with some uncertainties from different ensemble members during the ENSO neutral summers.

A spatial view of ensemble spread in convection permitting ensembles

With movement toward kilometer-scale ensembles, new techniques are needed for their characterization. A new methodology is presented for detailed spatial ensemble characterization using the fractions skill score (FSS). To evaluate spatial forecast differences, the average and standard deviation are taken of the FSS calculated over all ensemble member–member pairs at different scales and lead times. These methods were found to give important information about the ensemble behavior allowing the identification of useful spatial scales, spinup times for the model, and upscale growth of errors and forecast differences. The ensemble spread was found to be highly dependent on the spatial scales considered and the threshold applied to the field. High thresholds picked out localized and intense values that gave large temporal variability in ensemble spread: local processes and undersampling dominate for these thresholds. For lower thresholds the ensemble spread increases with time as differences between the ensemble members upscale. Two convective cases were investigated based on the Met Office United Model run at 2.2-km resolution. Different ensemble types were considered: ensembles produced using the Met Office Global and Regional Ensemble Prediction System (MOGREPS) and an ensemble produced using different model physics configurations. Comparison of the MOGREPS and multiphysics ensembles demonstrated the utility of spatial ensemble evaluation techniques for assessing the impact of different perturbation strategies and the need for assessing spread at different, believable, spatial scales.

Method for exploratory cluster analysis and visualisation of single-trial ERP ensembles

Background: The validity of ensemble averaging on event-related potential (ERP) data has been questioned, due to its assumption that the ERP is identical across trials. Thus, there is a need for preliminary testing for cluster structure in the data. New method: We propose a complete pipeline for the cluster analysis of ERP data. To increase the signalto-noise (SNR) ratio of the raw single-trials, we used a denoising method based on Empirical Mode Decomposition (EMD). Next, we used a bootstrap-based method to determine the number of clusters, through a measure called the Stability Index (SI). We then used a clustering algorithm based on a Genetic Algorithm (GA)to define initial cluster centroids for subsequent k-means clustering. Finally, we visualised the clustering results through a scheme based on Principal Component Analysis (PCA). Results: After validating the pipeline on simulated data, we tested it on data from two experiments – a P300 speller paradigm on a single subject and a language processing study on 25 subjects. Results revealed evidence for the existence of 6 clusters in one experimental condition from the language processing study. Further, a two-way chi-square test revealed an influence of subject on cluster membership.