Equipped to deal with uncertainty in climate and impacts predictions: lessons from internal peer review

The quantification of uncertainty is an increasingly popular topic, with clear importance for climate change policy. However, uncertainty assessments are open to a range of interpretations, each of which may lead to a different policy recommendation. In the EQUIP project researchers from the UK climate modelling, statistical modelling, and impacts communities worked together on ‘end-to-end’ uncertainty assessments of climate change and its impacts. Here, we use an experiment in peer review amongst project members to assess variation in the assessment of uncertainties between EQUIP researchers. We find overall agreement on key sources of uncertainty but a large variation in the assessment of the methods used for uncertainty assessment. Results show that communication aimed at specialists makes the methods used harder to assess. There is also evidence of individual bias, which is partially attributable to disciplinary backgrounds. However, varying views on the methods used to quantify uncertainty did not preclude consensus on the consequential results produced using those methods. Based on our analysis, we make recommendations for developing and presenting statements on climate and its impacts. These include the use of a common uncertainty reporting format in order to make assumptions clear; presentation of results in terms of processes and trade-offs rather than only numerical ranges; and reporting multiple assessments of uncertainty in order to elucidate a more complete picture of impacts and their uncertainties. This in turn implies research should be done by teams of people with a range of backgrounds and time for interaction and discussion, with fewer but more comprehensive outputs in which the range of opinions is recorded.

Improved seasonal prediction of the hot summer of 2003 over Europe through better representation of uncertainty in the land surface

Methods to explicitly represent uncertainties in weather and climate models have been developed and refined over the past decade, and have reduced biases and improved forecast skill when implemented in the atmospheric component of models. These methods have not yet been applied to the land surface component of models. Since the land surface is strongly coupled to the atmospheric state at certain times and in certain places (such as the European summer of 2003), improvements in the representation of land surface uncertainty may potentially lead to improvements in atmospheric forecasts for such events. Here we analyse seasonal retrospective forecasts for 1981–2012 performed with the European Centre for Medium-Range Weather Forecasts’ (ECMWF) coupled ensemble forecast model. We consider two methods of incorporating uncertainty into the land surface model (H-TESSEL): stochastic perturbation of tendencies, and static perturbation of key soil parameters. We find that the perturbed parameter approach considerably improves the forecast of extreme air temperature for summer 2003, through better representation of negative soil moisture anomalies and upward sensible heat flux. Averaged across all the reforecasts the perturbed parameter experiment shows relatively little impact on the mean bias, suggesting perturbations of at least this magnitude can be applied to the land surface without any degradation of model climate. There is also little impact on skill averaged across all reforecasts and some evidence of overdispersion for soil moisture. The stochastic tendency experiments show a large overdispersion for the soil temperature fields, indicating that the perturbation here is too strong. There is also some indication that the forecast of the 2003 warm event is improved for the stochastic experiments, however the improvement is not as large as observed for the perturbed parameter experiment.

Improved Arctic sea-ice thickness projections using bias corrected CMIP5 simulations

Projections of Arctic sea ice thickness (SIT) have the potential to inform stakeholders about accessibility to the region, but are currently rather uncertain. The latest suite of CMIP5 Global Climate Models (GCMs) produce a wide range of simulated SIT in the historical period (1979–2014) and exhibit various biases when compared with the Pan-Arctic Ice Ocean Modelling and Assimilation System (PIOMAS) sea ice reanalysis. We present a new method to constrain such GCM simulations of SIT via a statistical bias correction technique. The bias correction successfully constrains the spatial SIT distribution and temporal variability in the CMIP5 projections whilst retaining the climatic fluctuations from individual ensemble members. The bias correction acts to reduce the spread in projections of SIT and reveals the significant contributions of climate internal variability in the first half of the century and of scenario uncertainty from mid-century onwards. The projected date of ice-free conditions in the Arctic under the RCP8.5 high emission scenario occurs in the 2050s, which is a decade earlier than without the bias correction, with potentially significant implications for stakeholders in the Arctic such as the shipping industry. The bias correction methodology developed could be similarly applied to other variables to reduce spread in climate projections more generally.

Can institutional investors bias real estate portfolio appraisals? Evidence from the market downturn

This paper investigates the extent to which institutional investors may have influenced independent real estate appraisals during the financial crisis. A conceptual model of the determinants of client influence on real estate appraisals is proposed. It is suggested that the extent of clients’ ability and willingness to bias appraisal outputs is contingent upon market and regulatory environments (ethical norms and legal and institutional frameworks), the salience of the appraisal(s) to the client, financial incentives for the appraiser to respond to client pressure, organisational culture, the level of moral reasoning of both individual clients and appraisers, client knowledge and the degree of appraisal uncertainty. The potential of client influence to bias ostensibly independent real estate appraisals is examined using the opportunity afforded by the market downturn commencing in 2007 in the UK. During the market turbulence at the end of 2007, the motivations of different types of owners to bias appraisals diverged clearly and temporarily provided a unique opportunity to assess potential appraisal bias. We use appraisal-based performance data for individual real estate assets to test whether there were significant ownership effects on performance during this period. The results support the hypothesis that real estate appraisals in this period reflected the differing needs of clients.

The Importance of Scale for Spatial-confounding Bias and Precision of Spatial Regression Estimators

Increasingly, regression models are used when residuals are spatially correlated. Prominent examples include studies in environmental epidemiology to understand the chronic health effects of pollutants. I consider the effects of residual spatial structure on the bias and precision of regression coefficients, developing a simple framework in which to understand the key issues and derive informative analytic results. When the spatial residual is induced by an unmeasured confounder, regression models with spatial random effects and closely-related models such as kriging and penalized splines are biased, even when the residual variance components are known. Analytic and simulation results show how the bias depends on the spatial scales of the covariate and the residual; bias is reduced only when there is variation in the covariate at a scale smaller than the scale of the unmeasured confounding. I also discuss how the scales of the residual and the covariate affect efficiency and uncertainty estimation when the residuals can be considered independent of the covariate. In an application on the association between black carbon particulate matter air pollution and birth weight, controlling for large-scale spatial variation appears to reduce bias from unmeasured confounders, while increasing uncertainty in the estimated pollution effect.

Quantifying uncertainty in cost effectiveness analyses

This study compared four alternative approaches (Taylor, Fieller, percentile bootstrap, and bias-corrected bootstrap methods) to estimating confidence intervals (CIs) around cost-effectiveness (CE) ratio. The study consisted of two components: (1) Monte Carlo simulation was conducted to identify characteristics of hypothetical cost-effectiveness data sets which might lead one CI estimation technique to outperform another. These results were matched to the characteristics of an (2) extant data set derived from the National AIDS Demonstration Research (NADR) project. The methods were used to calculate (CIs) for data set. These results were then compared. The main performance criterion in the simulation study was the percentage of times the estimated (CIs) contained the “true” CE. A secondary criterion was the average width of the confidence intervals. For the bootstrap methods, bias was estimated. ^ Simulation results for Taylor and Fieller methods indicated that the CIs estimated using the Taylor series method contained the true CE more often than did those obtained using the Fieller method, but the opposite was true when the correlation was positive and the CV of effectiveness was high for each value of CV of costs. Similarly, the CIs obtained by applying the Taylor series method to the NADR data set were wider than those obtained using the Fieller method for positive correlation values and for values for which the CV of effectiveness were not equal to 30% for each value of the CV of costs. ^ The general trend for the bootstrap methods was that the percentage of times the true CE ratio was contained in CIs was higher for the percentile method for higher values of the CV of effectiveness, given the correlation between average costs and effects and the CV of effectiveness. The results for the data set indicated that the bias corrected CIs were wider than the percentile method CIs. This result was in accordance with the prediction derived from the simulation experiment. ^ Generally, the bootstrap methods are more favorable for parameter specifications investigated in this study. However, the Taylor method is preferred for low CV of effect, and the percentile method is more favorable for higher CV of effect. ^

Traffic uncertainty in toll motorway concessions in Spain

This study assessed the inaccuracy of the traffic estimates for toll motorway concessions in Spain. It was found that the estimates conducted by both the government and the concessionaire showed a significant bias towards overestimating traffic. The level of overestimation in Spain is even greater than that reported by other studies based on worldwide data. The notorious levels of overestimation entail severe burdens to the economics of the concessionaires that often prompt renegotiations of the contracts, which are often accepted by the government. These renegotiations usually end up with toll changes or extension of the concession terms, which have to be ultimately borne by future motorway users. It is postulated herein that the bias towards overestimating traffic in toll motorways in Spain is mostly caused by strategic issues rather than by modelling errors.

Motivations for group membership: The role of subjective importance and uncertainty reduction

An experiment was conducted to investigate the idea that an important motive for identifying with social groups is to reduce subjective uncertainty, particularly uncertainty on subjectively important dimensions that have implications for the self-concept (e.g., Hogg, 1996; Hogg & Mullin, 1999). When people are uncertain on a dimension that is subjectively important, they self-categorize in terms of an available social categorization and, thus, exhibit group behaviors. To test this general hypothesis, group membership, task uncertainty, and task importance were manipulated in a 2 x 2 x 2 between-participants design (N = 128), under relatively minimal group conditions. Ingroup identification and desire for consensual validation of specific attitudes were the key dependent measures, but we also measured social awareness. All three predictions were supported. Participants identified with their group (H1), and desired to obtain consensual validation from ingroup members (H2) when they were uncertain about their judgments on important dimensions, indicating that uncertainty reduction motivated participants towards embracing group membership. In addition, identification mediated the interactive effect of the independent variables on consensual validation (H3), and the experimental results were not associated with an increased sense of social awareness and, therefore, were unlikely to represent only behavioral compliance with generic social norms. Some implications of this research in the study of cults and totalist groups and the explication of genocide and group violence are discussed.

Quantifying uncertainty in citizen weather data

The sheer volume of citizen weather data collected and uploaded to online data hubs is immense. However as with any citizen data it is difficult to assess the accuracy of the measurements. Within this project we quantify just how much data is available, where it comes from, the frequency at which it is collected, and the types of automatic weather stations being used. We also list the numerous possible sources of error and uncertainty within citizen weather observations before showing evidence of such effects in real data. A thorough intercomparison field study was conducted, testing popular models of citizen weather stations. From this study we were able to parameterise key sources of bias. Most significantly the project develops a complete quality control system through which citizen air temperature observations can be passed. The structure of this system was heavily informed by the results of the field study. Using a Bayesian framework the system learns and updates its estimates of the calibration and radiation-induced biases inherent to each station. We then show the benefit of correcting for these learnt biases over using the original uncorrected data. The system also attaches an uncertainty estimate to each observation, which would provide real world applications that choose to incorporate such observations with a measure on which they may base their confidence in the data. The system relies on interpolated temperature and radiation observations from neighbouring professional weather stations for which a Bayesian regression model is used. We recognise some of the assumptions and flaws of the developed system and suggest further work that needs to be done to bring it to an operational setting. Such a system will hopefully allow applications to leverage the additional value citizen weather data brings to longstanding professional observing networks.

Designing for User-facing Uncertainty in Everyday Sensing and Prediction

Thesis (Ph.D.)--University of Washington, 2016-07

SAMPLING BIAS IN EVALUATING THE PROBABILITY OF SEISMICALLY INDUCED SOIL LIQUEFACTION WITH SPT & CPT CASE HISTORIES

Several deterministic and probabilistic methods are used to evaluate the probability of seismically induced liquefaction of a soil. The probabilistic models usually possess some uncertainty in that model and uncertainties in the parameters used to develop that model. These model uncertainties vary from one statistical model to another. Most of the model uncertainties are epistemic, and can be addressed through appropriate knowledge of the statistical model. One such epistemic model uncertainty in evaluating liquefaction potential using a probabilistic model such as logistic regression is sampling bias. Sampling bias is the difference between the class distribution in the sample used for developing the statistical model and the true population distribution of liquefaction and non-liquefaction instances. Recent studies have shown that sampling bias can significantly affect the predicted probability using a statistical model. To address this epistemic uncertainty, a new approach was developed for evaluating the probability of seismically-induced soil liquefaction, in which a logistic regression model in combination with Hosmer-Lemeshow statistic was used. This approach was used to estimate the population (true) distribution of liquefaction to non-liquefaction instances of standard penetration test (SPT) and cone penetration test (CPT) based most updated case histories. Apart from this, other model uncertainties such as distribution of explanatory variables and significance of explanatory variables were also addressed using KS test and Wald statistic respectively. Moreover, based on estimated population distribution, logistic regression equations were proposed to calculate the probability of liquefaction for both SPT and CPT based case history. Additionally, the proposed probability curves were compared with existing probability curves based on SPT and CPT case histories.

Experimental studies of di-jet survival and surface emission bias in Au plus Au collisions via angular correlations with respect to back-to-back leading hadrons

We report first results from an analysis based on a new multi-hadron correlation technique, exploring jet-medium interactions and di-jet surface emission bias at the BNL Relativistic Heavy Ion Collider (RHIC). Pairs of back-to-back high-transverse-momentum hadrons are used for triggers to study associated hadron distributions. In contrast with two-and three-particle correlations with a single trigger with similar kinematic selections, the associated hadron distribution of both trigger sides reveals no modification in either relative pseudorapidity Delta eta or relative azimuthal angle Delta phi from d + Au to central Au + Au collisions. We determine associated hadron yields and spectra as well as production rates for such correlated back-to-back triggers to gain additional insights on medium properties.

Low-bias negative differential resistance in graphene nanoribbon superlattices

We theoretically investigate negative differential resistance (NDR) for ballistic transport in semiconducting armchair graphene nanoribbon (aGNR) superlattices (5 to 20 barriers) at low bias voltages V(SD) < 500 mV. We combine the graphene Dirac Hamiltonian with the Landauer-Buttiker formalism to calculate the current I(SD) through the system. We find three distinct transport regimes in which NDR occurs: (i) a ""classical"" regime for wide layers, through which the transport across band gaps is strongly suppressed, leading to alternating regions of nearly unity and zero transmission probabilities as a function of V(SD) due to crossing of band gaps from different layers; (ii) a quantum regime dominated by superlattice miniband conduction, with current suppression arising from the misalignment of miniband states with increasing V(SD); and (iii) a Wannier-Stark ladder regime with current peaks occurring at the crossings of Wannier-Stark rungs from distinct ladders. We observe NDR at voltage biases as low as 10 mV with a high current density, making the aGNR superlattices attractive for device applications.

Universal zero-bias conductance through a quantum wire side-coupled to a quantum dot

A numerical renormalization-group study of the conductance through a quantum wire containing noninteracting electrons side-coupled to a quantum dot is reported. The temperature and the dot-energy dependence of the conductance are examined in the light of a recently derived linear mapping between the temperature-dependent conductance and the universal function describing the conductance for the symmetric Anderson model of a quantum wire with an embedded quantum dot. Two conduction paths, one traversing the wire, the other a bypass through the quantum dot, are identified. A gate potential applied to the quantum wire is shown to control the current through the bypass. When the potential favors transport through the wire, the conductance in the Kondo regime rises from nearly zero at low temperatures to nearly ballistic at high temperatures. When it favors the dot, the pattern is reversed: the conductance decays from nearly ballistic to nearly zero. When comparable currents flow through the two channels, the conductance is nearly temperature independent in the Kondo regime, and Fano antiresonances in the fixed-temperature plots of the conductance as a function of the dot-energy signal interference between them. Throughout the Kondo regime and, at low temperatures, even in the mixed-valence regime, the numerical data are in excellent agreement with the universal mapping.

Universal zero-bias conductance for the single-electron transistor

The thermal dependence of the zero-bias conductance for the single electron transistor is the target of two independent renormalization-group approaches, both based on the spin-degenerate Anderson impurity model. The first approach, an analytical derivation, maps the Kondo-regime conductance onto the universal conductance function for the particle-hole symmetric model. Linear, the mapping is parametrized by the Kondo temperature and the charge in the Kondo cloud. The second approach, a numerical renormalization-group computation of the conductance as a function the temperature and applied gate voltages offers a comprehensive view of zero-bias charge transport through the device. The first approach is exact in the Kondo regime; the second, essentially exact throughout the parametric space of the model. For illustrative purposes, conductance curves resulting from the two approaches are compared.