A verification framework for interannual-to-decadal predictions experiments

Decadal predictions have a high profile in the climate science community and beyond, yet very little is known about their skill. Nor is there any agreed protocol for estimating their skill. This paper proposes a sound and coordinated framework for verification of decadal hindcast experiments. The framework is illustrated for decadal hindcasts tailored to meet the requirements and specifications of CMIP5 (Coupled Model Intercomparison Project phase 5). The chosen metrics address key questions about the information content in initialized decadal hindcasts. These questions are: (1) Do the initial conditions in the hindcasts lead to more accurate predictions of the climate, compared to un-initialized climate change projections? and (2) Is the prediction model’s ensemble spread an appropriate representation of forecast uncertainty on average? The first question is addressed through deterministic metrics that compare the initialized and uninitialized hindcasts. The second question is addressed through a probabilistic metric applied to the initialized hindcasts and comparing different ways to ascribe forecast uncertainty. Verification is advocated at smoothed regional scales that can illuminate broad areas of predictability, as well as at the grid scale, since many users of the decadal prediction experiments who feed the climate data into applications or decision models will use the data at grid scale, or downscale it to even higher resolution. An overall statement on skill of CMIP5 decadal hindcasts is not the aim of this paper. The results presented are only illustrative of the framework, which would enable such studies. However, broad conclusions that are beginning to emerge from the CMIP5 results include (1) Most predictability at the interannual-to-decadal scale, relative to climatological averages, comes from external forcing, particularly for temperature; (2) though moderate, additional skill is added by the initial conditions over what is imparted by external forcing alone; however, the impact of initialization may result in overall worse predictions in some regions than provided by uninitialized climate change projections; (3) limited hindcast records and the dearth of climate-quality observational data impede our ability to quantify expected skill as well as model biases; and (4) as is common to seasonal-to-interannual model predictions, the spread of the ensemble members is not necessarily a good representation of forecast uncertainty. The authors recommend that this framework be adopted to serve as a starting point to compare prediction quality across prediction systems. The framework can provide a baseline against which future improvements can be quantified. The framework also provides guidance on the use of these model predictions, which differ in fundamental ways from the climate change projections that much of the community has become familiar with, including adjustment of mean and conditional biases, and consideration of how to best approach forecast uncertainty.

The proportionality of global warming to cumulative carbon emissions

The global temperature response to increasing atmospheric CO2 is often quantified by metrics such as equilibrium climate sensitivity and transient climate response1. These approaches, however, do not account for carbon cycle feedbacks and therefore do not fully represent the net response of the Earth system to anthropogenic CO2 emissions. Climate–carbon modelling experiments have shown that: (1) the warming per unit CO2 emitted does not depend on the background CO2 concentration2; (2) the total allowable emissions for climate stabilization do not depend on the timing of those emissions3, 4, 5; and (3) the temperature response to a pulse of CO2 is approximately constant on timescales of decades to centuries3, 6, 7, 8. Here we generalize these results and show that the carbon–climate response (CCR), defined as the ratio of temperature change to cumulative carbon emissions, is approximately independent of both the atmospheric CO2 concentration and its rate of change on these timescales. From observational constraints, we estimate CCR to be in the range 1.0–2.1 °C per trillion tonnes of carbon (Tt C) emitted (5th to 95th percentiles), consistent with twenty-first-century CCR values simulated by climate–carbon models. Uncertainty in land-use CO2 emissions and aerosol forcing, however, means that higher observationally constrained values cannot be excluded. The CCR, when evaluated from climate–carbon models under idealized conditions, represents a simple yet robust metric for comparing models, which aggregates both climate feedbacks and carbon cycle feedbacks. CCR is also likely to be a useful concept for climate change mitigation and policy; by combining the uncertainties associated with climate sensitivity, carbon sinks and climate–carbon feedbacks into a single quantity, the CCR allows CO2-induced global mean temperature change to be inferred directly from cumulative carbon emissions.

Induction of modular classification rules: using Jmax-pruning

The Prism family of algorithms induces modular classification rules which, in contrast to decision tree induction algorithms, do not necessarily fit together into a decision tree structure. Classifiers induced by Prism algorithms achieve a comparable accuracy compared with decision trees and in some cases even outperform decision trees. Both kinds of algorithms tend to overfit on large and noisy datasets and this has led to the development of pruning methods. Pruning methods use various metrics to truncate decision trees or to eliminate whole rules or single rule terms from a Prism rule set. For decision trees many pre-pruning and postpruning methods exist, however for Prism algorithms only one pre-pruning method has been developed, J-pruning. Recent work with Prism algorithms examined J-pruning in the context of very large datasets and found that the current method does not use its full potential. This paper revisits the J-pruning method for the Prism family of algorithms and develops a new pruning method Jmax-pruning, discusses it in theoretical terms and evaluates it empirically.

Jmax-pruning: a facility for the information theoretic pruning of modular classification rules

The Prism family of algorithms induces modular classification rules in contrast to the Top Down Induction of Decision Trees (TDIDT) approach which induces classification rules in the intermediate form of a tree structure. Both approaches achieve a comparable classification accuracy. However in some cases Prism outperforms TDIDT. For both approaches pre-pruning facilities have been developed in order to prevent the induced classifiers from overfitting on noisy datasets, by cutting rule terms or whole rules or by truncating decision trees according to certain metrics. There have been many pre-pruning mechanisms developed for the TDIDT approach, but for the Prism family the only existing pre-pruning facility is J-pruning. J-pruning not only works on Prism algorithms but also on TDIDT. Although it has been shown that J-pruning produces good results, this work points out that J-pruning does not use its full potential. The original J-pruning facility is examined and the use of a new pre-pruning facility, called Jmax-pruning, is proposed and evaluated empirically. A possible pre-pruning facility for TDIDT based on Jmax-pruning is also discussed.

Global and regional temperature-change potentials for near-term climate forcers

We examine the climate effects of the emissions of near-term climate forcers (NTCFs) from 4 continental regions (East Asia, Europe, North America and South Asia) using radiative forcing from the task force on hemispheric transport of air pollution source-receptor global chemical transport model simulations. These simulations model the transport of 3 aerosol species (sulphate, particulate organic matter and black carbon) and 4 ozone precursors (methane, nitric oxides (NOx), volatile organic compounds and carbon monoxide). From the equilibrium radiative forcing results we calculate global climate metrics, global warming potentials (GWPs) and global temperature change potentials (GTPs) and show how these depend on emission region, and can vary as functions of time. For the aerosol species, the GWP(100) values are −37±12, −46±20, and 350±200 for SO2, POM and BC respectively for the direct effects only. The corresponding GTP(100) values are −5.2±2.4, −6.5±3.5, and 50±33. This analysis is further extended by examining the temperature-change impacts in 4 latitude bands. This shows that the latitudinal pattern of the temperature response to emissions of the NTCFs does not directly follow the pattern of the diagnosed radiative forcing. For instance temperatures in the Arctic latitudes are particularly sensitive to NTCF emissions in the northern mid-latitudes. At the 100-yr time horizon the ARTPs show NOx emissions can have a warming effect in the northern mid and high latitudes, but cooling in the tropics and Southern Hemisphere. The northern mid-latitude temperature response to northern mid-latitude emissions of most NTCFs is approximately twice as large as would be implied by the global average.

The Asian summer monsoon: an intercomparison of CMIP5 vs. CMIP3 simulations of the late 20th century

The boreal summer Asian monsoon has been evaluated in 25 Coupled Model Intercomparison Project-5 (CMIP5) and 22 CMIP3 GCM simulations of the late 20th Century. Diagnostics and skill metrics have been calculated to assess the time-mean, climatological annual cycle, interannual variability, and intraseasonal variability. Progress has been made in modeling these aspects of the monsoon, though there is no single model that best represents all of these aspects of the monsoon. The CMIP5 multi-model mean (MMM) is more skillful than the CMIP3 MMM for all diagnostics in terms of the skill of simulating pattern correlations with respect to observations. Additionally, for rainfall/convection the MMM outperforms the individual models for the time mean, the interannual variability of the East Asian monsoon, and intraseasonal variability. The pattern correlation of the time (pentad) of monsoon peak and withdrawal is better simulated than that of monsoon onset. The onset of the monsoon over India is typically too late in the models. The extension of the monsoon over eastern China, Korea, and Japan is underestimated, while it is overestimated over the subtropical western/central Pacific Ocean. The anti-correlation between anomalies of all-India rainfall and Niño-3.4 sea surface temperature is overly strong in CMIP3 and typically too weak in CMIP5. For both the ENSO-monsoon teleconnection and the East Asian zonal wind-rainfall teleconnection, the MMM interannual rainfall anomalies are weak compared to observations. Though simulation of intraseasonal variability remains problematic, several models show improved skill at representing the northward propagation of convection and the development of the tilted band of convection that extends from India to the equatorial west Pacific. The MMM also well represents the space-time evolution of intraseasonal outgoing longwave radiation anomalies. Caution is necessary when using GPCP and CMAP rainfall to validate (1) the time-mean rainfall, as there are systematic differences over ocean and land between these two data sets, and (2) the timing of monsoon withdrawal over India, where the smooth southward progression seen in India Meteorological Department data is better realized in CMAP data compared to GPCP data.

The time course of implicit affective picture processing: an eye movement study

Consistent with a negativity bias account, neuroscientific and behavioral evidence demonstrates modulation of even early sensory processes by unpleasant, potentially threat-relevant information. The aim of this research is to assess the extent to which pleasant and unpleasant visual stimuli presented extrafoveally capture attention and impact eye movement control. We report an experiment examining deviations in saccade metrics in the presence of emotional image distractors that are close to a nonemotional target. We additionally manipulate the saccade latency to test when the emotional distractor has its biggest impact on oculomotor control. The results demonstrate that saccade landing position was pulled toward unpleasant distractors, and that this pull was due to the quick saccade responses. Overall, these findings support a negativity bias account of early attentional control and call for the need to consider the time course of motivated attention when affect is implicit

A comparative study of satellite and ground-based phenology

Long time series of ground-based plant phenology, as well as more than two decades of satellite-derived phenological metrics, are currently available to assess the impacts of climate variability and trends on terrestrial vegetation. Traditional plant phenology provides very accurate information on individual plant species, but with limited spatial coverage. Satellite phenology allows monitoring of terrestrial vegetation on a global scale and provides an integrative view at the landscape level. Linking the strengths of both methodologies has high potential value for climate impact studies. We compared a multispecies index from ground-observed spring phases with two types (maximum slope and threshold approach) of satellite-derived start-of-season (SOS) metrics. We focus on Switzerland from 1982 to 2001 and show that temporal and spatial variability of the multispecies index correspond well with the satellite-derived metrics. All phenological metrics correlate with temperature anomalies as expected. The slope approach proved to deviate strongly from the temporal development of the ground observations as well as from the threshold-defined SOS satellite measure. The slope spring indicator is considered to indicate a different stage in vegetation development and is therefore less suited as a SOS parameter for comparative studies in relation to ground-observed phenology. Satellite-derived metrics are, however, very susceptible to snow cover, and it is suggested that this snow cover should be better accounted for by the use of newer satellite sensors.

Infotainment across access devices: the perceptual impact of multimedia QoS

The introduction of multimedia on pervasive and mobile communication devices raises a number of perceptual quality issues. However, limited work has been done examining the 3-way interaction between use of equipment, user perceptual quality and quality of service. Our work measures user perceptual quality with the quality of perception (QoP) metrics which comprises levels of informational transfer (objective) and user satisfaction (subjective) when users are presented with multimedia video clips at three different frame rates, using four different display devices. Finally, our results will show that variation in frame-rate does not impact a user’s level of information assimilation (IA), however, does impact a users’ perception of multimedia video ‘quality’.

ATD: a multiplatform for semiautomatic 3-D detection of kidneys and their pathology in real time

This research presents a novel multi-functional system for medical Imaging-enabled Assistive Diagnosis (IAD). Although the IAD demonstrator has focused on abdominal images and supports the clinical diagnosis of kidneys using CT/MRI imaging, it can be adapted to work on image delineation, annotation and 3D real-size volumetric modelling of other organ structures such as the brain, spine, etc. The IAD provides advanced real-time 3D visualisation and measurements with fully automated functionalities as developed in two stages. In the first stage, via the clinically driven user interface, specialist clinicians use CT/MRI imaging datasets to accurately delineate and annotate the kidneys and their possible abnormalities, thus creating “3D Golden Standard Models”. Based on these models, in the second stage, clinical support staff i.e. medical technicians interactively define model-based rules and parameters for the integrated “Automatic Recognition Framework” to achieve results which are closest to that of the clinicians. These specific rules and parameters are stored in “Templates” and can later be used by any clinician to automatically identify organ structures i.e. kidneys and their possible abnormalities. The system also supports the transmission of these “Templates” to another expert for a second opinion. A 3D model of the body, the organs and their possible pathology with real metrics is also integrated. The automatic functionality was tested on eleven MRI datasets (comprising of 286 images) and the 3D models were validated by comparing them with the metrics from the corresponding “3D Golden Standard Models”. The system provides metrics for the evaluation of the results, in terms of Accuracy, Precision, Sensitivity, Specificity and Dice Similarity Coefficient (DSC) so as to enable benchmarking of its performance. The first IAD prototype has produced promising results as its performance accuracy based on the most widely deployed evaluation metric, DSC, yields 97% for the recognition of kidneys and 96% for their abnormalities; whilst across all the above evaluation metrics its performance ranges between 96% and 100%. Further development of the IAD system is in progress to extend and evaluate its clinical diagnostic support capability through development and integration of additional algorithms to offer fully computer-aided identification of other organs and their abnormalities based on CT/MRI/Ultra-sound Imaging.

Integrating Sustainability into Brownfield Regeneration: Rhetoric or Reality? – An Analysis of the UK Development Industry

In the UK and elsewhere the use of the term ‘sustainable brownfield regeneration’ has resulted from the interweaving of two key policy themes, comprising ‘sustainable development’ and ‘brownfield regeneration’. This paper provides a critical overview of brownfield policy within the context of the emerging sustainable development agenda in the UK, and examines the development industry's role and attitudes towards key aspects of sustainable development and brownfield regeneration. The paper analyses results from a survey of commercial and residential developers carried out in mid‐2004, underpinned by structured interviews with eleven developers in 2004–2005, which form part of a two‐and‐half‐year EPSRC‐funded project. The results suggest that despite the increasing focus on sustainability in government policy, the development industry seems ill at ease with precisely how sustainable development can be implemented in brownfield schemes. These and other findings, relating to sustainability issues (including the impact of climate change on future brownfield development), have important ramifications for brownfield regeneration policy in the UK. In particular, the research highlights the need for better metrics and benchmarks to be developed to measure ‘sustainable brownfield regeneration’. There also needs to be greater awareness and understanding of alternative clean‐up technologies to ‘dig and dump’.

Benchmarking energy use in licensed restaurants and pubs

Commercial kitchens often leave a large carbon footprint. A new dataset of energy performance metrics from a leading industrial partner is presented. Categorising these types of buildings is challenging. Electricity use has been analysed using data from automated meter readings (AMR) for the purpose of benchmarking and discussed in terms of factors such as size and food output. From the analysed results, consumption is found to be almost double previous sector estimates of 6480 million kWh per year. Recommendations are made to further improve the current benchmarks in order to attain robust, reliable and transparent figures, such as the introduction of normalised performance indicators to include kitchen size (m2) and kWh per thousand-pound turnover.

A comprehensive benchmarking system for evaluating global vegetation models.

We present a benchmark system for global vegetation models. This system provides a quantitative evaluation of multiple simulated vegetation properties, including primary production; seasonal net ecosystem production; vegetation cover, composition and 5 height; fire regime; and runoff. The benchmarks are derived from remotely sensed gridded datasets and site-based observations. The datasets allow comparisons of annual average conditions and seasonal and inter-annual variability, and they allow the impact of spatial and temporal biases in means and variability to be assessed separately. Specifically designed metrics quantify model performance for each process, 10 and are compared to scores based on the temporal or spatial mean value of the observations and a “random” model produced by bootstrap resampling of the observations. The benchmark system is applied to three models: a simple light-use efficiency and water-balance model (the Simple Diagnostic Biosphere Model: SDBM), and the Lund-Potsdam-Jena (LPJ) and Land Processes and eXchanges (LPX) dynamic global 15 vegetation models (DGVMs). SDBM reproduces observed CO2 seasonal cycles, but its simulation of independent measurements of net primary production (NPP) is too high. The two DGVMs show little difference for most benchmarks (including the interannual variability in the growth rate and seasonal cycle of atmospheric CO2), but LPX represents burnt fraction demonstrably more accurately. Benchmarking also identified 20 several weaknesses common to both DGVMs. The benchmarking system provides a quantitative approach for evaluating how adequately processes are represented in a model, identifying errors and biases, tracking improvements in performance through model development, and discriminating among models. Adoption of such a system would do much to improve confidence in terrestrial model predictions of climate change 25 impacts and feedbacks.

Effect of drainage and wildfire on peat hydrophysical properties

Consecutive multiple disturbances to northern peatlands can dramatically impact peat hydrophysical properties. We examine the impact of a double disturbance (drainage and wildfire) on the hydrophysical and moisture retention properties of peat, a key regulator of peatland ecohydrological resilience, and compare this with the impact to each individual disturbance (drainage and wildfire). The compound effect of drainage and wildfire resulted in a shift of the surface datum down the peat profile, revealing denser peat. Less-dense near-surface peats that regulate water-table position and near-surface moisture content, both favourable to Sphagnum recolonization, were lost. At a drained peatland that was then subject to wildfire, peat bulk density increased by 14.1%, von Post humification class increased by two categories and water retention increased by 15.6%, compared with an adjacent burned but undrained (single disturbance) portion of the fen. We discuss the key hydrophysical metrics of peatland vulnerability and outline how they are affected by the isolated impacts of drainage and wildfire, as well as their combined effects. We demonstrate that multiple peatland disturbances have likely led to an increase in hydrological limitations to Sphagnum recovery, which may impact peatland ecohydrological resilience.

Spatial and radiometric fidelity of airborne multispectral imagery in the context of land-cover change analyses

The goal was to quantitatively estimate and compare the fidelity of images acquired with a digital imaging system (ADAR 5500) and generated through scanning of color infrared aerial photographs (SCIRAP) using image-based metrics. Images were collected nearly simultaneously in two repetitive flights to generate multi-temporal datasets. Spatial fidelity of ADAR was lower than that of SCIRAP images. Radiometric noise was higher for SCIRAP than for ADAR images, even though noise from misregistration effects was lower. These results suggest that with careful control of film scanning, the overall fidelity of SCIRAP imagery can be comparable to that of digital multispectral camera data. Therefore, SCIRAP images can likely be used in conjunction with digital metric camera imagery in long-term landcover change analyses.