Reliability, sufficiency, and the decomposition of proper scores

Scoring rules are an important tool for evaluating the performance of probabilistic forecasting schemes. A scoring rule is called strictly proper if its expectation is optimal if and only if the forecast probability represents the true distribution of the target. In the binary case, strictly proper scoring rules allow for a decomposition into terms related to the resolution and the reliability of a forecast. This fact is particularly well known for the Brier Score. In this article, this result is extended to forecasts for finite-valued targets. Both resolution and reliability are shown to have a positive effect on the score. It is demonstrated that resolution and reliability are directly related to forecast attributes that are desirable on grounds independent of the notion of scores. This finding can be considered an epistemological justification of measuring forecast quality by proper scoring rules. A link is provided to the original work of DeGroot and Fienberg, extending their concepts of sufficiency and refinement. The relation to the conjectured sharpness principle of Gneiting, et al., is elucidated.

The concept of exchangeability in ensemble forecasting

A set of random variables is exchangeable if its joint distribution function is invariant under permutation of the arguments. The concept of exchangeability is discussed, with a view towards potential application in evaluating ensemble forecasts. It is argued that the paradigm of ensembles being an independent draw from an underlying distribution function is probably too narrow; allowing ensemble members to be merely exchangeable might be a more versatile model. The question is discussed whether established methods of ensemble evaluation need alteration under this model, with reliability being given particular attention. It turns out that the standard methodology of rank histograms can still be applied. As a first application of the exchangeability concept, it is shown that the method of minimum spanning trees to evaluate the reliability of high dimensional ensembles is mathematically sound.

Ice formation and development in aged, wintertime cumulus over the UK: observations and modelling

In situ high resolution aircraft measurements of cloud microphysical properties were made in coordination with ground based remote sensing observations of a line of small cumulus clouds, using Radar and Lidar, as part of the Aerosol Properties, PRocesses And InfluenceS on the Earth's climate (APPRAISE) project. A narrow but extensive line (~100 km long) of shallow convective clouds over the southern UK was studied. Cloud top temperatures were observed to be higher than −8 °C, but the clouds were seen to consist of supercooled droplets and varying concentrations of ice particles. No ice particles were observed to be falling into the cloud tops from above. Current parameterisations of ice nuclei (IN) numbers predict too few particles will be active as ice nuclei to account for ice particle concentrations at the observed, near cloud top, temperatures (−7.5 °C). The role of mineral dust particles, consistent with concentrations observed near the surface, acting as high temperature IN is considered important in this case. It was found that very high concentrations of ice particles (up to 100 L−1) could be produced by secondary ice particle production providing the observed small amount of primary ice (about 0.01 L−1) was present to initiate it. This emphasises the need to understand primary ice formation in slightly supercooled clouds. It is shown using simple calculations that the Hallett-Mossop process (HM) is the likely source of the secondary ice. Model simulations of the case study were performed with the Aerosol Cloud and Precipitation Interactions Model (ACPIM). These parcel model investigations confirmed the HM process to be a very important mechanism for producing the observed high ice concentrations. A key step in generating the high concentrations was the process of collision and coalescence of rain drops, which once formed fell rapidly through the cloud, collecting ice particles which caused them to freeze and form instant large riming particles. The broadening of the droplet size-distribution by collision-coalescence was, therefore, a vital step in this process as this was required to generate the large number of ice crystals observed in the time available. Simulations were also performed with the WRF (Weather, Research and Forecasting) model. The results showed that while HM does act to increase the mass and number concentration of ice particles in these model simulations it was not found to be critical for the formation of precipitation. However, the WRF simulations produced a cloud top that was too cold and this, combined with the assumption of continual replenishing of ice nuclei removed by ice crystal formation, resulted in too many ice crystals forming by primary nucleation compared to the observations and parcel modelling.

Probabilistic forecasting: why model imperfection is a poison pill

This volume is a serious attempt to open up the subject of European philosophy of science to real thought, and provide the structural basis for the interdisciplinary development of its specialist fields, but also to provoke reflection on the idea of ‘European philosophy of science’. This efforts should foster a contemporaneous reflection on what might be meant by philosophy of science in Europe and European philosophy of science, and how in fact awareness of it could assist philosophers interpret and motivate their research through a stronger collective identity. The overarching aim is to set the background for a collaborative project organising, systematising, and ultimately forging an identity for, European philosophy of science by creating research structures and developing research networks across Europe to promote its development.

Forecasting extinction risk of ectotherms under climate warming: an evolutionary perspective

1. It has been postulated that climate warming may pose the greatest threat species in the tropics, where ectotherms have evolved more thermal specialist physiologies. Although species could rapidly respond to environmental change through adaptation, little is known about the potential for thermal adaptation, especially in tropical species. 2. In the light of the limited empirical evidence available and predictions from mutation-selection theory, we might expect tropical ectotherms to have limited genetic variance to enable adaptation. However, as a consequence of thermodynamic constraints, we might expect this disadvantage to be at least partially offset by a fitness advantage, that is, the ‘hotter-is-better’ hypothesis. 3. Using an established quantitative genetics model and metabolic scaling relationships, we integrate the consequences of the opposing forces of thermal specialization and thermodynamic constraints on adaptive potential by evaluating extinction risk under climate warming. We conclude that the potential advantage of a higher maximal development rate can in theory more than offset the potential disadvantage of lower genetic variance associated with a thermal specialist strategy. 4. Quantitative estimates of extinction risk are fundamentally very sensitive to estimates of generation time and genetic variance. However, our qualitative conclusion that the relative risk of extinction is likely to be lower for tropical species than for temperate species is robust to assumptions regarding the effects of effective population size, mutation rate and birth rate per capita. 5. With a view to improving ecological forecasts, we use this modelling framework to review the sensitivity of our predictions to the model’s underpinning theoretical assumptions and the empirical basis of macroecological patterns that suggest thermal specialization and fitness increase towards the tropics. We conclude by suggesting priority areas for further empirical research.

The 2010-2011 drought in the Horn of Africa in ECMWF reanalysis and seasonal forecast products

This study evaluates the use of European Centre for Medium-Range Weather Forecasts (ECMWF) products in monitoring and forecasting drought conditions during the recent 2010–2011 drought in the Horn of Africa (HoA). The region was affected by a precipitation deficit in both the October–December 2010 and March–May 2011 rainy seasons. These anomalies were captured by the ERA-Interim reanalysis (ERAI), despite its limitations in representing the March–May interannual variability. Soil moisture anomalies of ERAI also identified the onset of the drought condition early in October 2010 with a persistent drought still present in September 2011. This signal was also evident in normalized difference vegetation index (NDVI) remote sensing data. The precipitation deficit in October–December 2010 was associated with a strong La Niña event. The ECMWF seasonal forecasts for the October–December 2010 season predicted the La Niña event from June 2010 onwards. The forecasts also predicted a below-average October–December rainfall, from July 2010 onwards. The subsequent March–May rainfall anomaly was only captured by the new ECWMF seasonal forecast system in the forecasts starting in March 2011. Our analysis shows that a recent (since 1999) drying in the region during the March–May season is captured by the new ECMWF seasonal forecast system and is consistent with recently published results. The HoA region and its population are highly vulnerable to future droughts, thus global monitoring and forecasting of drought, such as that presented here, will become increasingly important in the future. Copyright © 2012 Royal Meteorological Society

Characteristics of high-resolution versions of the Met Office unified model for forecasting convection over the United Kingdom

With many operational centers moving toward order 1-km-gridlength models for routine weather forecasting, this paper presents a systematic investigation of the properties of high-resolution versions of the Met Office Unified Model for short-range forecasting of convective rainfall events. The authors describe a suite of configurations of the Met Office Unified Model running with grid lengths of 12, 4, and 1 km and analyze results from these models for a number of convective cases from the summers of 2003, 2004, and 2005. The analysis includes subjective evaluation of the rainfall fields and comparisons of rainfall amounts, initiation, cell statistics, and a scale-selective verification technique. It is shown that the 4- and 1-km-gridlength models often give more realistic-looking precipitation fields because convection is represented explicitly rather than parameterized. However, the 4-km model representation suffers from large convective cells and delayed initiation because the grid length is too long to correctly reproduce the convection explicitly. These problems are not as evident in the 1-km model, although it does suffer from too numerous small cells in some situations. Both the 4- and 1-km models suffer from poor representation at the start of the forecast in the period when the high-resolution detail is spinning up from the lower-resolution (12 km) starting data used. A scale-selective precipitation verification technique implies that for later times in the forecasts (after the spinup period) the 1-km model performs better than the 12- and 4-km models for lower rainfall thresholds. For higher thresholds the 4-km model scores almost as well as the 1-km model, and both do better than the 12-km model.

Multi-year droughts in Europe: analysis of development and causes

Whilst hydrological systems can show resilience to short-term streamflow deficiencies during within-year droughts, prolonged deficits during multi-year droughts are a significant threat to water resources security in Europe. This study uses a threshold-based objective classification of regional hydrological drought to qualitatively examine the characteristics, spatio-temporal evolution and synoptic climatic drivers of multi-year drought events in 1962–64, 1975–76 and 1995–97, on a European scale but with particular focus on the UK. Whilst all three events are multi-year, pan-European phenomena, their development and causes can be contrasted. The critical factor in explaining the unprecedented severity of the 1975–76 event is the consecutive occurrence of winter and summer drought. In contrast, 1962–64 was a succession of dry winters, mitigated by quiescent summers, whilst 1995–97 lacked spatial coherence and was interrupted by wet interludes. Synoptic climatic conditions vary within and between multi-year droughts, suggesting that regional factors modulate the climate signal in streamflow drought occurrence. Despite being underpinned by qualitatively similar climatic conditions and commonalities in evolution and characteristics, each of the three droughts has a unique spatio-temporal signature. An improved understanding of the spatio-temporal evolution and characteristics of multi-year droughts has much to contribute to monitoring and forecasting capability, and to improved mitigation strategies.

Examining the large-scale spatial coherence of European drought using regional indicators of precipitation and streamflow deficit

Droughts tend to evolve slowly and affect large areas simultaneously, which suggests that improved understanding of spatial coherence of drought would enable better mitigation of drought impacts through enhanced monitoring and forecasting strategies. This study employs an up-to-date dataset of over 500 river flow time series from 11 European countries, along with a gridded precipitation dataset, to examine the spatial coherence of drought in Europe using regional indicators of precipitation and streamflow deficit. The drought indicators were generated for 24 homogeneous regions and, for selected regions, historical drought characteristics were corroborated with previous work. The spatial coherence of drought characteristics was then examined at a European scale. Historical droughts generally have distinctive signatures in their spatio-temporal development, so there was limited scope for using the evolution of historical events to inform forecasting. Rather, relationships were explored in time series of drought indicators between regions. Correlations were generally low, but multivariate analyses revealed broad continental-scale patterns, which appear to be related to large-scale atmospheric circulation indices (in particular, the North Atlantic Oscillation and the East Atlantic West Russia pattern). A novel methodology for forecasting was developed (and demonstrated with reference to the United Kingdom), which predicts drought from drought i.e. uses spatial coherence of drought to facilitate early warning of drought in a target region, from drought which is developing elsewhere in Europe.Whilst the skill of the methodology is relatively modest at present, this approach presents a potential new avenue for forecasting, which offers significant advantages in that it allows prediction for all seasons, and also shows some potential for forecasting the termination of drought conditions.

Commodity futures prices: more evidence on forecast power, risk premia and the theory of storage

In this paper, we examine the temporal stability of the evidence for two commodity futures pricing theories. We investigate whether the forecast power of commodity futures can be attributed to the extent to which they exhibit seasonality and we also consider whether there are time varying parameters or structural breaks in these pricing relationships. Compared to previous studies, we find stronger evidence of seasonality in the basis, which supports the theory of storage. The power of the basis to forecast subsequent price changes is also strengthened, while results on the presence of a risk premium are inconclusive. In addition, we show that the forecasting power of commodity futures cannot be attributed to the extent to which they exhibit seasonality. We find that in most cases where structural breaks occur, only changes in the intercepts and not the slopes are detected, illustrating that the forecast power of the basis is stable over different economic environments.

Forecasting global ENSO-related climate anomalies

Long-range global climate forecasts have been made by use of a model for predicting a tropical Pacific sea surface temperature (SST) in tandem with an atmospheric general circulation model. The SST is predicted first at long lead times into the future. These ocean forecasts are then used to force the atmospheric model and so produce climate forecasts at lead times of the SST forecasts. Prediction of the wintertime 500 mb height, surface air temperature and precipitation for seven large climatic events of the 1970–1990s by this two-tiered technique agree well in general with observations over many regions of the globe. The levels of agreement are high enough in some regions to have practical utility.

A two-tiered approach to long-range climate forecasting

Long-range global climate forecasts were made by use of a model for predicting a tropical Pacific sea-surface temperature (SST) in tandem with an atmospheric general circulation model. The SST is predicted first at long lead times into the future. These ocean forecasts are then used to force the atmospheric model and so produce climate forecasts at lead times of the SST forecasts. Prediction of seven large climatic events of the 1970s to 1990s by this technique are in good agreement with observations over many regions of the globe.

Design of a forecasting system for ECMWF

As laid out in its convention there are 8 different objectives for ECMWF. One of the major objectives will consist of the preparation, on a regular basis, of the data necessary for the preparation of medium-range weather forecasts. The interpretation of this item is that the Centre will make forecasts once a day for a prediction period of up to 10 days. It is also evident that the Centre should not carry out any real weather forecasting but merely disseminate to the member countries the basic forecasting parameters with an appropriate resolution in space and time. It follows from this that the forecasting system at the Centre must from the operational point of view be functionally integrated with the Weather Services of the Member Countries. The operational interface between ECMWF and the Member Countries must be properly specified in order to get a reasonable flexibility for both systems. The problem of making numerical atmospheric predictions for periods beyond 4-5 days differs substantially from 2-3 days forecasting. From the physical point we can define a medium range forecast as a forecast where the initial disturbances have lost their individual structure. However we are still interested to predict the atmosphere in a similar way as in short range forecasting which means that the model must be able to predict the dissipation and decay of the initial phenomena and the creation of new ones. With this definition, medium range forecasting is indeed very difficult and generally regarded as more difficult than extended forecasts, where we usually only predict time and space mean values. The predictability of atmospheric flow has been extensively studied during the last years in theoretical investigations and by numerical experiments. As has been discussed elsewhere in this publication (see pp 338 and 431) a 10-day forecast is apparently on the fringe of predictability.