Estimating reliability and resolution of probability forecasts through decomposition of the empirical score

References (20)Cited By (1)Export CitationAboutAbstract Proper scoring rules provide a useful means to evaluate probabilistic forecasts. Independent from scoring rules, it has been argued that reliability and resolution are desirable forecast attributes. The mathematical expectation value of the score allows for a decomposition into reliability and resolution related terms, demonstrating a relationship between scoring rules and reliability/resolution. A similar decomposition holds for the empirical (i.e. sample average) score over an archive of forecast–observation pairs. This empirical decomposition though provides a too optimistic estimate of the potential score (i.e. the optimum score which could be obtained through recalibration), showing that a forecast assessment based solely on the empirical resolution and reliability terms will be misleading. The differences between the theoretical and empirical decomposition are investigated, and specific recommendations are given how to obtain better estimators of reliability and resolution in the case of the Brier and Ignorance scoring rule.

Evaluating raw ensembles with the continuous ranked probability score

The continuous ranked probability score (CRPS) is a frequently used scoring rule. In contrast with many other scoring rules, the CRPS evaluates cumulative distribution functions. An ensemble of forecasts can easily be converted into a piecewise constant cumulative distribution function with steps at the ensemble members. This renders the CRPS a convenient scoring rule for the evaluation of ‘raw’ ensembles, obviating the need for sophisticated ensemble model output statistics or dressing methods prior to evaluation. In this article, a relation between the CRPS score and the quantile score is established. The evaluation of ‘raw’ ensembles using the CRPS is discussed in this light. It is shown that latent in this evaluation is an interpretation of the ensemble as quantiles but with non-uniform levels. This needs to be taken into account if the ensemble is evaluated further, for example with rank histograms.

Developing consensus on hospital prescribing indicators of potential harms amenable to decision support

Aim: To develop a list of prescribing indicators specific for the hospital setting that would facilitate the prospective collection of high severity and/or high frequency prescribing errors, which are also amenable to electronic clinical decision support (CDS). Method: A three-stage consensus technique (electronic Delphi) was carried out with 20 expert pharmacists and physicians across England. Participants were asked to score prescribing errors using a 5-point Likert scale for their likelihood of occurrence and the severity of the most likely outcome. These were combined to produce risk scores, from which median scores were calculated for each indicator across the participants in the study. The degree of consensus between the participants was defined as the proportion that gave a risk score in the same category as the median. Indicators were included if a consensus of 80% or more was achieved. Results: A total of 80 prescribing errors were identified by consensus as being high or extreme risk. The most common drug classes named within the indicators were antibiotics (n=13), antidepressants (n=8), nonsteroidal anti-inflammatory drugs (n=6), and opioid analgesics (n=6).The most frequent error type identified as high or extreme risk were those classified as clinical contraindications (n=29/80). Conclusion: 80 high risk prescribing errors in the hospital setting have been identified by an expert panel. These indicators can serve as the basis for a standardised, validated tool for the collection of data in both paperbased and electronic prescribing processes, as well as to assess the impact of electronic decision support implementation or development.

Accounting for the economic risk, caused by variation in disease severity, in fungicide dose decisions: exemplified for Mycosphaerella graminicola on winter wheat

A method is presented to calculate economic optimum fungicide doses accounting for the risk-aversion of growers responding to variability in disease severity between crops. Simple dose-response and disease-yield loss functions are used to estimate net disease-related costs (fungicide cost, plus disease-induced yield loss) as a function of dose and untreated severity. With fairly general assumptions about the shapes of the probability distribution of disease severity and the other functions involved, we show that a choice of fungicide dose which minimises net costs on average across seasons results in occasional large net costs caused by inadequate control in high disease seasons. This may be unacceptable to a grower with limited capital. A risk-averse grower can choose to reduce the size and frequency of such losses by applying a higher dose as insurance. For example, a grower may decide to accept ‘high loss’ years one year in ten or one year in twenty (i.e. specifying a proportion of years in which disease severity and net costs will be above a specified level). Our analysis shows that taking into account disease severity variation and risk-aversion will usually increase the dose applied by an economically rational grower. The analysis is illustrated with data on septoria tritici leaf blotch of wheat caused by Mycosphaerella graminicola. Observations from untreated field plots at sites across England over three years were used to estimate the probability distribution of disease severities at mid-grain filling. In the absence of a fully reliable disease forecasting scheme, reducing the frequency of ‘high loss’ years requires substantially higher doses to be applied to all crops. Disease resistant cultivars reduce both the optimal dose at all levels of risk and the disease-related costs at all doses.

Increases in prefrontal cortex activity when regulating negative emotion predicts symptom severity trajectory over six months in depression

Context: Emotion regulation is critically disrupted in depression and use of paradigms tapping these processes may uncover essential changes in neurobiology during treatment. In addition, as neuroimaging outcome studies of depression commonly utilize solely baseline and endpoint data – which is more prone to week-to week noise in symptomatology – we sought to use all data points over the course of a six month trial. Objective: To examine changes in neurobiology resulting from successful treatment. Design: Double-blind trial examining changes in the neural circuits involved in emotion regulation resulting from one of two antidepressant treatments over a six month trial. Participants were scanned pretreatment, at 2 months and 6 months posttreatment. Setting: University functional magnetic resonance imaging facility. Participants: 21 patients with Major Depressive Disorder and without other Axis I or Axis II diagnoses and 14 healthy controls. Interventions: Venlafaxine XR (doses up to 300mg) or Fluoxetine (doses up to 80mg). Main Outcome Measure: Neural activity, as measured using functional magnetic resonance imaging during performance of an emotion regulation paradigm as well as regular assessments of symptom severity by the Hamilton Rating Scale for Depression. To utilize all data points, slope trajectories were calculated for rate of change in depression severity as well as rate of change of neural engagement. Results: Those depressed individuals showing the steepest decrease in depression severity over the six months were those individuals showing the most rapid increases in BA10 and right DLPFC activity when regulating negative affect over the same time frame. This relationship was more robust than when using solely the baseline and endpoint data. Conclusions: Changes in PFC engagement when regulating negative affect correlate with changes in depression severity over six months. These results are buttressed by calculating these statistics which are more reliable and robust to week-to-week variation than difference scores.

'Score it upon my taille': the use (and abuse) of tallies by the medieval Exchequer

The paper traces the evolution of the tally from a receipt for cash payments into the treasury, to proof of payments made by royal officials outside of the treasury and finally to an assignment of revenue to be paid out by royal officials. Each of these processes is illustrated by examples drawn from the Exchequer records and explains their significance for royal finance and for historians working on the Exchequer records.

Forecast convergence score: a forecaster's approach to analysing hydro-meteorological forecast systems.

In this paper the properties of a hydro-meteorological forecasting system for forecasting river flows have been analysed using a probabilistic forecast convergence score (FCS). The focus on fixed event forecasts provides a forecaster's approach to system behaviour and adds an important perspective to the suite of forecast verification tools commonly used in this field. A low FCS indicates a more consistent forecast. It can be demonstrated that the FCS annual maximum decreases over the last 10 years. With lead time, the FCS of the ensemble forecast decreases whereas the control and high resolution forecast increase. The FCS is influenced by the lead time, threshold and catchment size and location. It indicates that one should use seasonality based decision rules to issue flood warnings.

Relationships between Oak powdery mildew incidence and severity and commensal fungi

Oak (Quercus robur) powdery mildew is a common and damaging fungal disease. In a local survey at Reading, UK, oak powdery mildew was common on trees of all height classes but was most common on trees of 3-9m. A variety of other fungal species were commonly found growing in association with oak powdery mildew colonies. The abundance of such fungi was estimated through stratified sample surveys for 2.5 years. The taxa most commonly associated with oak powdery mildew were Acremonium sp., Trichoderma sp., Ampelomyces/Phoma sp. and Leptosphaerulina australis. Nearly 90% of mildew colonies were associated with L. australis, which is not generally considered as a mycoparasite or antagonist, in contrast with the other three fungi. Abundance varied between June and October surveys. Acremonium sp. abundance was greater in summer samplings whereas L. australis and Trichoderma sp. abundances were greater in autumn samplings. Ampelomyces/Phoma sp. was never observed in the absence of powdery mildew. Relationships between the mildew-associated fungi and oak powdery mildew appeared curved and differed significantly between sampling years. L. australis was positively correlated with the other three associated fungi studied when powdery mildew was also present. The variety and high population densities of the mildew associated fungi suggest that they may be important in determining the final density of oak mildew and the damage caused by it.

Cocoa flavanol intake improves endothelial function and Framingham Risk Score in healthy men and women: a randomised, controlled, double-masked trial: the Flaviola Health Study

Cocoa flavanol (CF) intake improves endothelial function in patients with cardiovascular risk factors and disease. We investigated the effects of CF on surrogate markers of cardiovascular health in low risk, healthy, middle-aged individuals without history, signs or symptoms of CVD. In a 1-month, open-label, one-armed pilot study, bi-daily ingestion of 450 mg of CF led to a time-dependent increase in endothelial function (measured as flow-mediated vasodilation (FMD)) that plateaued after 2 weeks. Subsequently, in a randomised, controlled, double-masked, parallel-group dietary intervention trial (Clinicaltrials.gov: NCT01799005), 100 healthy, middle-aged (35–60 years) men and women consumed either the CF-containing drink (450 mg) or a nutrient-matched CF-free control bi-daily for 1 month. The primary end point was FMD. Secondary end points included plasma lipids and blood pressure, thus enabling the calculation of Framingham Risk Scores and pulse wave velocity. At 1 month, CF increased FMD over control by 1·2 % (95 % CI 1·0, 1·4 %). CF decreased systolic and diastolic blood pressure by 4·4 mmHg (95 % CI 7·9, 0·9 mmHg) and 3·9 mmHg (95 % CI 6·7, 0·9 mmHg), pulse wave velocity by 0·4 m/s (95 % CI 0·8, 0·04 m/s), total cholesterol by 0·20 mmol/l (95 % CI 0·39, 0·01 mmol/l) and LDL-cholesterol by 0·17 mmol/l (95 % CI 0·32, 0·02 mmol/l), whereas HDL-cholesterol increased by 0·10 mmol/l (95 % CI 0·04, 0·17 mmol/l). By applying the Framingham Risk Score, CF predicted a significant lowering of 10-year risk for CHD, myocardial infarction, CVD, death from CHD and CVD. In healthy individuals, regular CF intake improved accredited cardiovascular surrogates of cardiovascular risk, demonstrating that dietary flavanols have the potential to maintain cardiovascular health even in low-risk subjects.

Piriformospora indica reduces Fusarium head blight disease severity and mycotoxin DON contamination in wheat under UK weather conditions

Piriformospora indica (Sebacinaceae) is a cultivable root endophytic fungus. It colonises the roots of a wide range of host plants. In many settings colonisation promotes host growth, increases yield and protects the host from fungal diseases. We evaluated the effect of P. indica on Fusarium head blight (FHB) disease of winter (cv. Battalion) and spring (cv. Paragon, Mulika, Zircon, Granary, KWS Willow and KWS Kilburn) wheat and consequent contamination by the mycotoxin deoxynivalenol (DON) under UK weather conditions. Interactions of P. indica with an arbuscular mycorrhizal fungus (Funneliformis mosseae), fungicide application (Aviator Xpro) and low and high fertiliser levels were considered. P. indica application reduced FHB disease severity and incidence by 70%. It decreased mycotoxin DON concentration of winter and spring wheat samples by 70% and 80% respectively. P. indica also increased above ground biomass, 1000 grain weight and total grain weight. P. indica reduced disease severity and increased yield in both high and low fertiliser levels. The effect of P. indica was compatible with F. mosseae and foliar fungicide application. P. indica did not have any effects on plant tissue nutrients. These results suggest that P. indica might be useful in biological control of Fusarium diseases of wheat.

The iso-Score Curve Graph. A new tool for competitive bidding

The present work describes a new tool that helps bidders improve their competitive bidding strategies. This new tool consists of an easy-to-use graphical tool that allows the use of more complex decision analysis tools in the field of Competitive Bidding. The graphic tool described here tries to move away from previous bidding models which attempt to describe the result of an auction or a tender process by means of studying each possible bidder with probability density functions. As an illustration, the tool is applied to three practical cases. Theoretical and practical conclusions on the great potential breadth of application of the tool are also presented.

Prevalence and severity of dental caries in schoolchildren of Porto, Portugal

Objectives: To measure the prevalence and severity of dental caries in adolescents of the city of Porto, Portugal, and to assess socioeconomic and behavioural covariates of dental caries experience. Methods: A sample of 700 thirteen-year-old schoolchildren was examined. Results from the dental examination were linked to anthropometric information and to data supplied by two structured questionnaires assessing nutritional factors, socio-demographic characteristics and behaviour related to health promotion. Dental caries was measured using the DMFT index, and two dichotomous outcomes, one assessing the prevalence of dental caries (DMFT>0); the other assessing the prevalence of a high level of dental caries (DMFT=4). Results: Consuming soft drinks derived from cola (irrespective of sugared or diet) two or more times per week, attending a public school, being female and having parents with low educational attainment were identified as risk factors both for having dental caries and for having a high level of dental caries. Conclusion: Caries levels were positively associated with frequency of intake of sweetened foods and drinks.

Fatigue and workability in Brazilian textile companies in different corporate social responsibility score groups

The aim of this study was to evaluate working conditions in the textile industry for different stages of Corporate Social Responsibility (CSR) development, and workers` perception of fatigue and workability. A cross-sectional study was undertaken with 126 workers in the production areas of five Brazilian textile plants. The corporate executive officers and managers of each company provided their personal evaluations of CSR. Companies were divided into 2 groups (higher and lower) of CSR scores. Workers completed questionnaires on fatigue, workability and working conditions. Ergonomic job analysis showed similar results for working conditions, independent of their CSR score. Multivariate analysis models were developed for fatigue and workability, indicating that they are both associated to factors related to working conditions and individual workers` characteristics and life styles. Work organization, (what, how, when, where and for how long the work is done), is also an associated factor for fatigue. This study suggests that workers` opinions should be taken into greater consideration when companies develop their CSR programs, in particular for those relating to working conditions. Relevance to industry: This paper underlines the importance of considering working conditions and workers` opinions of them, work organization and individual workers` characteristics and life styles in order to restore or to maintain workability and to reduce fatigue, independently of how developed a company may be in the field of Corporate Social Responsibility. (C) 2010 Elsevier B.V. All rights reserved.

FIRE INTENSITY AND SEVERITY IN BRAZILIAN CAMPOS GRASSLANDS

Brazilian Campos grasslands are rich in species and the maintenance of its diversity and physiognomy is dependent on disturbance (e.g. fire and grazing) Nevertheless, studies about fire intensity and severity are inexistent. The present paper describes fire parameters, using 14 experimental burn plots in southern Brazil (30 degrees 02` to 30 degrees 04`S, and 51 degrees 06` to 51 degrees 09`W. 311masl). Two sites under different fire histories were chosen: frequently burned and excluded since six years. Experimental burning was performed during summer (2006-2007), when most burning takes place in these grasslands. The following parameters were measured: air temperature and moisture, vegetation height, wind speed, fuel (fine, coarse), fuel moisture, fire temperatures (soil level and at 50cm), ash, residuals, flame freight, fire duration: burning efficiency and fire intensity were later calculated. Fuel load varied from 0.39 to 1.44kg.m(-2). and correlated positively with both fire temperature and fire intensity. Fire temperatures ranged 47 to 537.5 degrees C. being higher in the excluded site Fire intensity was low compared to grassland elsewhere (36 5-319.5kW.m(-1)), differing significantly between sties Fine fuel was the variable that best explained fire intensity. The results on fire intensity and severity in Campos grasslands can be considered a pilot study, since plots were very small. However the data provided can help other researchers to get permission for experimentation using larger plots The results provide support for further studies about the effects of fire on grassland vegetation and for studies involving fire models and fire risk prediction