On reliability analysis of multi-categorical forecasts

Reliability analysis of probabilistic forecasts, in particular through the rank histogram or Talagrand diagram, is revisited. Two shortcomings are pointed out: Firstly, a uniform rank histogram is but a necessary condition for reliability. Secondly, if the forecast is assumed to be reliable, an indication is needed how far a histogram is expected to deviate from uniformity merely due to randomness. Concerning the first shortcoming, it is suggested that forecasts be grouped or stratified along suitable criteria, and that reliability is analyzed individually for each forecast stratum. A reliable forecast should have uniform histograms for all individual forecast strata, not only for all forecasts as a whole. As to the second shortcoming, instead of the observed frequencies, the probability of the observed frequency is plotted, providing and indication of the likelihood of the result under the hypothesis that the forecast is reliable. Furthermore, a Goodness-Of-Fit statistic is discussed which is essentially the reliability term of the Ignorance score. The discussed tools are applied to medium range forecasts for 2 m-temperature anomalies at several locations and lead times. The forecasts are stratified along the expected ranked probability score. Those forecasts which feature a high expected score turn out to be particularly unreliable.

Reliability, sufficiency, and the decomposition of proper scores

Scoring rules are an important tool for evaluating the performance of probabilistic forecasting schemes. A scoring rule is called strictly proper if its expectation is optimal if and only if the forecast probability represents the true distribution of the target. In the binary case, strictly proper scoring rules allow for a decomposition into terms related to the resolution and the reliability of a forecast. This fact is particularly well known for the Brier Score. In this article, this result is extended to forecasts for finite-valued targets. Both resolution and reliability are shown to have a positive effect on the score. It is demonstrated that resolution and reliability are directly related to forecast attributes that are desirable on grounds independent of the notion of scores. This finding can be considered an epistemological justification of measuring forecast quality by proper scoring rules. A link is provided to the original work of DeGroot and Fienberg, extending their concepts of sufficiency and refinement. The relation to the conjectured sharpness principle of Gneiting, et al., is elucidated.

Estimating reliability and resolution of probability forecasts through decomposition of the empirical score

References (20)Cited By (1)Export CitationAboutAbstract Proper scoring rules provide a useful means to evaluate probabilistic forecasts. Independent from scoring rules, it has been argued that reliability and resolution are desirable forecast attributes. The mathematical expectation value of the score allows for a decomposition into reliability and resolution related terms, demonstrating a relationship between scoring rules and reliability/resolution. A similar decomposition holds for the empirical (i.e. sample average) score over an archive of forecast–observation pairs. This empirical decomposition though provides a too optimistic estimate of the potential score (i.e. the optimum score which could be obtained through recalibration), showing that a forecast assessment based solely on the empirical resolution and reliability terms will be misleading. The differences between the theoretical and empirical decomposition are investigated, and specific recommendations are given how to obtain better estimators of reliability and resolution in the case of the Brier and Ignorance scoring rule.

Competition and product survival in the UK car market

The paper examines the extent to which inter- and intra-firm competition influenced the survival of cars in the UK market between 1971 and 1998. It is shown that, while competition influenced product survival in all market segments within the UK car market, the nature of that competition differed between them. In the small family and large family car segments, intra-firm competition dominated inter-firm competition. In contrast, in the luxury/sports car segment only inter-firm competition conditions resulted in product survival. Evidence was also found that the luxury/sports car segment has grown more competitive over time and that firms marketing products in the family car segments have become considerably more successful at avoiding the effects of intra-firm competition.

Tax principles, product differentiation and the nature of competition

We analyze the choice between the origin and destination principles of taxation when there is product differentiation and Bertrand competition. If taxes are redistributed to consumers and demand is linear the origin principle dominates the destination principle whatever the degree of product differentiation and extent of economic integration. With nonlinear demand the origin principle dominates if there is sufficient economic integration. When the social value assigned to tax revenue is higher than the private value, the destination principle dominates for intermediate values of product differentiation and economic integration. The same results are also shown to hold with Cournot competition.

The influence of markets and policy on spatial patterns of non-timber forest product extraction

When villagers extract resources, such as fuelwood, fodder, or medicinal plants from forests, their decisions over where and how much to extract are influenced by market conditions, their particular opportunity costs of time, minimum consumption needs, and access to markets. This paper develops an optimization model of villagers’ extraction behavior that clarifies how, and under what conditions, policies that create incentives such as improved returns to extraction in a buffer zone might be used instead of adversarial enforcement efforts to protect a forest’s pristine ‘‘inner core.’’

Reliability of regional climate model trends

A necessary condition for a good probabilistic forecast is that the forecast system is shown to be reliable: forecast probabilities should equal observed probabilities verified over a large number of cases. As climate change trends are now emerging from the natural variability, we can apply this concept to climate predictions and compute the reliability of simulated local and regional temperature and precipitation trends (1950–2011) in a recent multi-model ensemble of climate model simulations prepared for the Intergovernmental Panel on Climate Change (IPCC) fifth assessment report (AR5). With only a single verification time, the verification is over the spatial dimension. The local temperature trends appear to be reliable. However, when the global mean climate response is factored out, the ensemble is overconfident: the observed trend is outside the range of modelled trends in many more regions than would be expected by the model estimate of natural variability and model spread. Precipitation trends are overconfident for all trend definitions. This implies that for near-term local climate forecasts the CMIP5 ensemble cannot simply be used as a reliable probabilistic forecast.

An improved sum-product estimate for general finite fields

An improved sum-product estimate for subsets of a finite field whose order is not prime is provided. It is shown, under certain conditions, that max{∣∣∣A+A∣∣∣,∣∣∣A⋅A∣∣∣}≫∣∣A∣∣12/11(log2∣∣A∣∣)5/11. This new estimate matches, up to a logarithmic factor, the current best known bound obtained over prime fields by Rudnev

Terrestrial ecosystems from space: a review of earth observation products for macroecology applications

Aim Earth observation (EO) products are a valuable alternative to spectral vegetation indices. We discuss the availability of EO products for analysing patterns in macroecology, particularly related to vegetation, on a range of spatial and temporal scales. Location Global. Methods We discuss four groups of EO products: land cover/cover change, vegetation structure and ecosystem productivity, fire detection, and digital elevation models. We address important practical issues arising from their use, such as assumptions underlying product generation, product accuracy and product transferability between spatial scales. We investigate the potential of EO products for analysing terrestrial ecosystems. Results Land cover, productivity and fire products are generated from long-term data using standardized algorithms to improve reliability in detecting change of land surfaces. Their global coverage renders them useful for macroecology. Their spatial resolution (e.g. GLOBCOVER vegetation, 300 m; MODIS vegetation and fire, ≥ 500 m; ASTER digital elevation, 30 m) can be a limiting factor. Canopy structure and productivity products are based on physical approaches and thus are independent of biome-specific calibrations. Active fire locations are provided in near-real time, while burnt area products show actual area burnt by fire. EO products can be assimilated into ecosystem models, and their validation information can be employed to calculate uncertainties during subsequent modelling. Main conclusions Owing to their global coverage and long-term continuity, EO end products can significantly advance the field of macroecology. EO products allow analyses of spatial biodiversity, seasonal dynamics of biomass and productivity, and consequences of disturbances on regional to global scales. Remaining drawbacks include inter-operability between products from different sensors and accuracy issues due to differences between assumptions and models underlying the generation of different EO products. Our review explains the nature of EO products and how they relate to particular ecological variables across scales to encourage their wider use in ecological applications.

The end of the opportunism vs trust debate: bounded reliability as a new envelope concept in research on MNE governance

Modern transaction cost economics (TCE) thinking has developed into a key intellectual foundation of international business (IB) research, but the Williamsonian version has faced substantial criticism for adopting the behavioral assumption of opportunism. In this paper we assess both the opportunism concept and existing alternatives such as trust within the context of IB research, especially work on multinational enterprise (MNE) governance. Case analyses of nine global MNEs illustrate an alternative to the opportunism assumption that captures more fully the mechanisms underlying failed commitments inside the MNE. As a substitute for the often-criticized assumption of opportunism, we propose the envelope concept of bounded reliability (BRel), an assumption that represents more accurately and more completely the reasons for failed commitments, without invalidating the other critical assumption in conventional TCE (and internalization theory) thinking, namely the widely accepted envelope concept of bounded rationality (BRat). Bounded reliability as an envelope concept includes two main components, within the context of global MNE management: opportunism as intentional deceit, and benevolent preference reversal. The implications for IB research of adopting the bounded reliability concept are far reaching, as this concept may increase the legitimacy of comparative institutional analysis in the social sciences.

An experimental validation of hypothetical WTP for a recyclable product

Using a within-subject experiment, we compare hypothetical and real willingness to pay (WTP) for an improvement in the recyclability of a product. Subjects are faced with a real payment scenario after they have responded to a hypothetical question. Contrary to most of the results obtained in similar studies, at apopulation level, there are no significant median differences between actual and hypothetical stated values of WTP. However,within-subject comparisons between hypothetical and actual values indicate that subjects stating a low (high) hypothetical WTP tend to underestimate (overestimate) the value of their actual contributions.

Examining reliability of seasonal to decadal sea surface temperature forecasts: the role of ensemble dispersion

Useful probabilistic climate forecasts on decadal timescales should be reliable (i.e. forecast probabilities match the observed relative frequencies) but this is seldom examined. This paper assesses a necessary condition for reliability, that the ratio of ensemble spread to forecast error being close to one, for seasonal to decadal sea surface temperature retrospective forecasts from the Met Office Decadal Prediction System (DePreSys). Factors which may affect reliability are diagnosed by comparing this spread-error ratio for an initial condition ensemble and two perturbed physics ensembles for initialized and uninitialized predictions. At lead times less than 2 years, the initialized ensembles tend to be under-dispersed, and hence produce overconfident and hence unreliable forecasts. For longer lead times, all three ensembles are predominantly over-dispersed. Such over-dispersion is primarily related to excessive inter-annual variability in the climate model. These findings highlight the need to carefully evaluate simulated variability in seasonal and decadal prediction systems.Useful probabilistic climate forecasts on decadal timescales should be reliable (i.e. forecast probabilities match the observed relative frequencies) but this is seldom examined. This paper assesses a necessary condition for reliability, that the ratio of ensemble spread to forecast error being close to one, for seasonal to decadal sea surface temperature retrospective forecasts from the Met Office Decadal Prediction System (DePreSys). Factors which may affect reliability are diagnosed by comparing this spread-error ratio for an initial condition ensemble and two perturbed physics ensembles for initialized and uninitialized predictions. At lead times less than 2 years, the initialized ensembles tend to be under-dispersed, and hence produce overconfident and hence unreliable forecasts. For longer lead times, all three ensembles are predominantly over-dispersed. Such over-dispersion is primarily related to excessive inter-annual variability in the climate model. These findings highlight the need to carefully evaluate simulated variability in seasonal and decadal prediction systems.

Adaptive behavior by single-product and multiproduct price setting firms in experimental markets

We focus on the learning dynamics in multiproduct price-setting markets, where firms use past strategies and performance to adapt to the corresponding equilibrium.

The STRATADAPT scale: a measure of marketing strategy adaptation to international business markets

Purpose – The development of marketing strategies optimally adjusted to export markets has been a vitally important topic for both managers and academics for about five decades. However, there is no agreement in the literature about which elements integrate marketing strategy and which components of domestic strategies should be adapted to export markets. The purpose of this paper is to develop a new scale – STRATADAPT. Design/methodology/approach – Results from a sample of small and medium-sized industrial exporting firms support a four-dimensional scale – product, promotion, price, and distribution strategies – of 30 items. The scale presents evidence of composite reliability as well as discriminant and nomological validity. Findings – Findings reveal that all four dimensions of marketing strategy adaptation are positively associated with the amount of the firm's financial resources allocated to export activity. Practical implications – The STRATADAPT scale may assist managers in developing better international marketing strategies as well as in planning more accurate and efficient marketing programs across markets. Originality/value – This study develops a new scale, the STRATADAPT scale, which is a broad measure of export marketing strategy adaptation.