Reliability, sufficiency, and the decomposition of proper scores

Scoring rules are an important tool for evaluating the performance of probabilistic forecasting schemes. A scoring rule is called strictly proper if its expectation is optimal if and only if the forecast probability represents the true distribution of the target. In the binary case, strictly proper scoring rules allow for a decomposition into terms related to the resolution and the reliability of a forecast. This fact is particularly well known for the Brier Score. In this article, this result is extended to forecasts for finite-valued targets. Both resolution and reliability are shown to have a positive effect on the score. It is demonstrated that resolution and reliability are directly related to forecast attributes that are desirable on grounds independent of the notion of scores. This finding can be considered an epistemological justification of measuring forecast quality by proper scoring rules. A link is provided to the original work of DeGroot and Fienberg, extending their concepts of sufficiency and refinement. The relation to the conjectured sharpness principle of Gneiting, et al., is elucidated.

Estimating reliability and resolution of probability forecasts through decomposition of the empirical score

References (20)Cited By (1)Export CitationAboutAbstract Proper scoring rules provide a useful means to evaluate probabilistic forecasts. Independent from scoring rules, it has been argued that reliability and resolution are desirable forecast attributes. The mathematical expectation value of the score allows for a decomposition into reliability and resolution related terms, demonstrating a relationship between scoring rules and reliability/resolution. A similar decomposition holds for the empirical (i.e. sample average) score over an archive of forecast–observation pairs. This empirical decomposition though provides a too optimistic estimate of the potential score (i.e. the optimum score which could be obtained through recalibration), showing that a forecast assessment based solely on the empirical resolution and reliability terms will be misleading. The differences between the theoretical and empirical decomposition are investigated, and specific recommendations are given how to obtain better estimators of reliability and resolution in the case of the Brier and Ignorance scoring rule.

Can company-fund manager meetings convey informational benefits? Exploring the rationalisation of equity investment decision making by UK fund managers

Conventional economic theory, applied to information released by listed companies, equates ‘useful’ with ‘price-sensitive’. Stock exchange rules accordingly prohibit the selec- tive, private communication of price-sensitive information. Yet, even in the absence of such communication, UK equity fund managers routinely meet privately with the senior execu- tives of the companies in which they invest. Moreover, they consider these brief, formal and formulaic meetings to be their most important sources of investment information. In this paper we ask how that can be. Drawing on interview and observation data with fund managers and CFOs, we find evidence for three, non-mutually exclusive explanations: that the characterisation of information in conventional economic theory is too restricted, that fund managers fail to act with the rationality that conventional economic theory assumes, and/or that the primary value of the meetings for fund managers is not related to their investment decision making but to the claims of superior knowledge made to clients in marketing their active fund management expertise. Our findings suggest a disconnect between economic theory and economic policy based on that theory, as well as a corre- sponding limitation in research studies that test information-usefulness by assuming it to be synonymous with price-sensitivity. We draw implications for further research into the role of tacit knowledge in equity investment decision-making, and also into the effects of the principal–agent relationship between fund managers and their clients.

A review of the costs and benefits of demand response for electricity in the UK

The recent policy discussion in the UK on the economic case for demand response (DR) calls for a reflection on available evidence regarding its costs and benefits. Existing studies tend to consider the size of investments and returns of certain forms of DR in isolation and do not consider economic welfare effects. From review of existing studies, policy documents, and some simple modelling of benefits of DR in providing reserve for unforeseen events, we demonstrate that the economic case for DR in UK electricity markets is positive. Consideration of economic welfare gains is provided.

The benefits of a compute cluster approach to high spatial resolution biodiversity richness modelling: projecting the impact of climate change on Mediterranean flora

High spatial resolution environmental data gives us a better understanding of the environmental factors affecting plant distributions at fine spatial scales. However, large environmental datasets dramatically increase compute times and output species model size stimulating the need for an alternative computing solution. Cluster computing offers such a solution, by allowing both multiple plant species Environmental Niche Models (ENMs) and individual tiles of high spatial resolution models to be computed concurrently on the same compute cluster. We apply our methodology to a case study of 4,209 species of Mediterranean flora (around 17% of species believed present in the biome). We demonstrate a 16 times speed-up of ENM computation time when 16 CPUs were used on the compute cluster. Our custom Java ‘Merge’ and ‘Downsize’ programs reduce ENM output files sizes by 94%. The median 0.98 test AUC score of species ENMs is aided by various species occurrence data filtering techniques. Finally, by calculating the percentage change of individual grid cell values, we map the projected percentages of plant species vulnerable to climate change in the Mediterranean region between 1950–2000 and 2020.

Estimating regional benefits of reducing targeted pollutants: An application to agricultural effects on water quality and the value of recreational fishing

The general focus of this paper is the regional estimation of marginal benefits of targeted water pollution abatement to instream uses. Benefit estimates are derived from actual consumer choices of recreational fishing activities and the implied expenditures for various levels of water quality. The methodology is applied to measuring the benefits accruing to recreational anglers in Indiana from the abatement of pollutants that are by-products of agricultural crop production.

Measuring research benefits in an imperfect market: second comment

This note generalizes a finding about the necessary and sufficient conditions for research to generate greater benefits in the presence of distortions and highlights a significant source of bias in conventional cost-benefit calculations

Lessons from the private finance initiative in the UK: benefits, problems and critical success factors

Purpose – This paper summarises the main research findings from a detailed, qualitative set of structured interviews and case studies of private finance initiative (PFI) schemes in the UK, which involve the construction of built facilities. The research, which was funded by the Foundation for the Built Environment, examines the emergence of PFI in the UK. Benefits and problems in the PFI process are investigated. Best practice, the key critical factors for success, and lessons for the future are also analysed. Design/methodology/approach – The research is based around 11 semi-structured interviews conducted with stakeholders in key PFI projects in the UK. Findings – The research demonstrates that value for money and risk transfer are key success criteria. High procurement and transaction costs are a feature of PFI projects, and the large-scale nature of PFI projects frequently acts as barrier to entry. Research limitations/implications – The research is based on a limited number of in-depth case study interviews. The paper also shows that further research is needed to find better ways to measure these concepts empirically. Practical implications – The paper is important in highlighting four main areas of practical improvement in the PFI process: value for money assessment; establishing end-user needs; developing competitive markets and developing appropriate skills in the public sector. Originality/value – The paper examines the drivers, barriers and critical success factors for PFI in the UK for the first time in detail and will be of value to property investors, financiers, and others involved in the PFI process.

Reliability of regional climate model trends

A necessary condition for a good probabilistic forecast is that the forecast system is shown to be reliable: forecast probabilities should equal observed probabilities verified over a large number of cases. As climate change trends are now emerging from the natural variability, we can apply this concept to climate predictions and compute the reliability of simulated local and regional temperature and precipitation trends (1950–2011) in a recent multi-model ensemble of climate model simulations prepared for the Intergovernmental Panel on Climate Change (IPCC) fifth assessment report (AR5). With only a single verification time, the verification is over the spatial dimension. The local temperature trends appear to be reliable. However, when the global mean climate response is factored out, the ensemble is overconfident: the observed trend is outside the range of modelled trends in many more regions than would be expected by the model estimate of natural variability and model spread. Precipitation trends are overconfident for all trend definitions. This implies that for near-term local climate forecasts the CMIP5 ensemble cannot simply be used as a reliable probabilistic forecast.

Sesquiterpenoids lactones: benefits to plants and people

Sesquiterpenoids, and specifically sesquiterpene lactones from Asteraceae, may play a highly significant role in human health, both as part of a balanced diet and as pharmaceutical agents, due to their potential for the treatment of cardiovascular disease and cancer. This review highlights the role of sesquiterpene lactones endogenously in the plants that produce them, and explores mechanisms by which they interact in animal and human consumers of these plants. Several mechanisms are proposed for the reduction of inflammation and tumorigenesis at potentially achievable levels in humans. Plants can be classified by their specific array of produced sesquiterpene lactones, showing high levels of translational control. Studies of folk medicines implicate sesquiterpene lactones as the active ingredient in many treatments for other ailments such as diarrhea, burns, influenza, and neurodegradation. In addition to the anti-inflammatory response, sesquiterpene lactones have been found to sensitize tumor cells to conventional drug treatments. This review explores the varied ecological roles of sesquiterpenes in the plant producer, depending upon the plant and the compound. These include allelopathy with other plants, insects, and microbes, thereby causing behavioural or developmental modification to these secondary organisms to the benefit of the sesquiterpenoid producer. Some sesquiterpenoid lactones are antimicrobial, disrupting the cell wall of fungi and invasive bacteria, whereas others protect the plant from environmental stresses that would otherwise cause oxidative damage. Many of the compounds are effective due to their bitter flavor, which has obvious implications for human consumers. The implications of sesquiterpenoid lactone qualitiesfor future crop production are discussed.

Mapping the trickle out benefits of social and environmental enterprises: a pilot study of an East African eco business

• Objectives The objective of this paper is to propose a framework for mapping the sustainable development and poverty alleviation impacts of social and environmental enterprises in Africa. This framework is then piloted with reference to an East African Ecobusiness. • Prior Work This paper is based on data collected as part of a wider research project examining social and environmental enterprises across the 19 countries of Southern and Eastern Africa. In total, the sustainable development and poverty alleviation impacts of 20 in-depth case studies in 4 countries are being examined. • Approach Data was collected using in-depth interviews with multiple stakeholders associated with the case study business. Secondary materials were also analysed and a quantitative survey of customers undertaken. • Results In addition to their impacts on the environment, African eco businesses can also have substantial social, economic and wider poverty alleviation impacts. This paper maps the impacts of a case study East African ecobusiness, as part of developing a social and environmental enterprise impact framework for Africa and the wider developing world. In our case study, positive and negative impacts are identified, while questions are raised in relation to tradeoffs between social and environmental objectives and temporal dimensions of impact. The usefulness of existing frameworks for understanding the social, environmental and development impacts of these kinds of organisations are also considered. • Implications This paper outlines the necessity of building an African-centric impact map to capture the multi-level poverty alleviation and sustainable development impacts of social and environmental enterprise activity in developing world environments. The framework proposed also offers guidance to businesses operating in Africa about the factors that might be considered as part of their wider social and environmental responsibilities. • Value Assessing the impact of social and environmental enterprises, especially as a route to development within low income countries, is receiving increasing attention in academia and beyond. This paper presents a useful contribution to the scarce literature on social and environmental enterprises in Africa.

Restricting microbial exposure in early life negates the immune benefits associated with gut colonization in environments of high microbial diversity

Background: Acquisition of the intestinal microbiota in early life corresponds with the development of the mucosal immune system. Recent work on caesarean-delivered infants revealed that early microbial composition is influenced by birthing method and environment. Furthermore, we have confirmed that early-life environment strongly influences both the adult gut microbiota and development of the gut immune system. Here, we address the impact of limiting microbial exposure after initial colonization on the development of adult gut immunity. Methodology/Principal Findings: Piglets were born in indoor or outdoor rearing units, allowing natural colonization in the immediate period after birth, prior to transfer to high-health status isolators. Strikingly, gut closure and morphological development were strongly affected by isolator-rearing, independent of indoor or outdoor origins of piglets. Isolator-reared animals showed extensive vacuolation and disorganization of the gut epithelium, inferring that normal gut closure requires maturation factors present in maternal milk. Although morphological maturation and gut closure were delayed in isolatorreared animals, these hard-wired events occurred later in development. Type I IFN, IL-22, IL-23 and Th17 pathways were increased in indoor-isolator compared to outdoor-isolator animals during early life, indicating greater immune activation in pigs originating from indoor environments reflecting differences in the early microbiota. This difference was less apparent later in development due to enhanced immune activation and convergence of the microbiota in all isolator-reared animals. This correlated with elevation of Type I IFN pathways in both groups, although T cell pathways were still more affected in indoor-reared animals. Conclusions/Significance: Environmental factors, in particular microbial exposure, influence expression of a large number of immune-related genes. However, the homeostatic effects of microbial colonization in outdoor environments require sustained microbial exposure throughout development. Gut development in high-hygiene environments negatively impacts on normal succession of the gut microbiota and promotes innate immune activation which may impair immune homeostasis.

The AVOID programme’s new simulations of the global benefits of stringent climate change mitigation

Quantitative simulations of the global-scale benefits of climate change mitigation are presented, using a harmonised, self-consistent approach based on a single set of climate change scenarios. The approach draws on a synthesis of output from both physically-based and economics-based models, and incorporates uncertainty analyses. Previous studies have projected global and regional climate change and its impacts over the 21st century but have generally focused on analysis of business-as-usual scenarios, with no explicit mitigation policy included. This study finds that both the economics-based and physically-based models indicate that early, stringent mitigation would avoid a large proportion of the impacts of climate change projected for the 2080s. However, it also shows that not all the impacts can now be avoided, so that adaptation would also therefore be needed to avoid some of the potential damage. Delay in mitigation substantially reduces the percentage of impacts that can be avoided, providing strong new quantitative evidence for the need for stringent and prompt global mitigation action on greenhouse gas emissions, combined with effective adaptation, if large, widespread climate change impacts are to be avoided. Energy technology models suggest that such stringent and prompt mitigation action is technologically feasible, although the estimated costs vary depending on the specific modelling approach and assumptions.

The politics of in-work benefits: The case of the ‘active income of solidarity’ in France

In-work benefits have been introduced in a number of Bismarckian welfare regimes in a context of austerity despite being targeted at politically weak constituents and representing a deviation from prevailing welfare institutions. This article addresses this puzzle by looking at the introduction in 2008 of an in-work benefit scheme in France, the Active Income of Solidarity. The analysis reveals that this reform was the result of a cross-cutting alliance between the conservative party and employers, as well as parts of the socialist party and the union movement. The alliance was possible thanks to actors’ multiple interpretations of the reform. The reform was difficult to oppose given its support by experts and public opinion and because it entailed an increase in revenues for low-income workers.