Inferential, non-parametric statistics to assess quality of probabilistic forecast systems

Many statistical forecast systems are available to interested users. In order to be useful for decision-making, these systems must be based on evidence of underlying mechanisms. Once causal connections between the mechanism and their statistical manifestation have been firmly established, the forecasts must also provide some quantitative evidence of `quality’. However, the quality of statistical climate forecast systems (forecast quality) is an ill-defined and frequently misunderstood property. Often, providers and users of such forecast systems are unclear about what ‘quality’ entails and how to measure it, leading to confusion and misinformation. Here we present a generic framework to quantify aspects of forecast quality using an inferential approach to calculate nominal significance levels (p-values) that can be obtained either by directly applying non-parametric statistical tests such as Kruskal-Wallis (KW) or Kolmogorov-Smirnov (KS) or by using Monte-Carlo methods (in the case of forecast skill scores). Once converted to p-values, these forecast quality measures provide a means to objectively evaluate and compare temporal and spatial patterns of forecast quality across datasets and forecast systems. Our analysis demonstrates the importance of providing p-values rather than adopting some arbitrarily chosen significance levels such as p < 0.05 or p < 0.01, which is still common practice. This is illustrated by applying non-parametric tests (such as KW and KS) and skill scoring methods (LEPS and RPSS) to the 5-phase Southern Oscillation Index classification system using historical rainfall data from Australia, The Republic of South Africa and India. The selection of quality measures is solely based on their common use and does not constitute endorsement. We found that non-parametric statistical tests can be adequate proxies for skill measures such as LEPS or RPSS. The framework can be implemented anywhere, regardless of dataset, forecast system or quality measure. Eventually such inferential evidence should be complimented by descriptive statistical methods in order to fully assist in operational risk management.

Queensland shark control program non-target statistics by year

Spreadsheet of non-target species (bycatch) numbers in the Shark Control Program by species, date of capture, location, size and sex from 2001 onwards The shark control program (SCP) relies on nets or drumlines, or a combination of both, to minimise the threat of shark attack on humans in particular locations. Following is information on numbers and locations of sharks that have been caught by the SCP. It is important to reduce the inadvertent impacts of the SCP on other marine animals (bycatch) without compromising human safety. Bycatch levels are carefully monitored and research is focused on minimising impacts on non-target species. This dataset contains details of non-target numbers in the Shark Control program by species, date of capture, and location from 2001

Getting value out of your sensory testing - the difference from control test

Non-parametric difference tests such as triangle and duo-trio tests traditionally are used to establish differences or similarities between products. However they only supply the researcher with partial answers and often further testing is required to establish the nature, size and direction of differences. This paper looks at the advantages of the difference from control (DFC) test (also known as degree of difference test) and discusses appropriate applications of the test. The scope and principle of the test, panel composition and analysis of results are presented with the aid of suitable examples. Two of the major uses of the DFC test are in quality control and shelf-life testing. The role DFC takes in these areas and the use of other tests to complement the testing is discussed. Controls or standards are important in both these areas and the use of standard products, mental and written standards and blind controls are highlighted. The DFC test has applications in products where the duo-trio and triangle tests cannot be used because of the normal heterogeneity of the product. While the DFC test is a simple difference test it can be structured to give the researcher more valuable data and scope to make informed decisions about their product.

Development of near infrared analysis of faeces to estimate non-grass proportions in diets selected by cattle grazing tropical pastures

Grass (monocots) and non-grass (dicots) proportions in ruminant diets are important nutritionally because the non-grasses are usually higher in nutritive value, particularly protein, than the grasses, especially in tropical pastures. For ruminants grazing tropical pastures where the grasses are C-4 species and most non-grasses are C-3 species, the ratio of C-13/C-12 in diet and faeces, measured as delta C-13 parts per thousand, is proportional to dietary non-grass%. This paper describes the development of a faecal near infrared (NIR) spectroscopy calibration equation for predicting faecal delta C-13 from which dietary grass and non-grass proportions can be calculated. Calibration development used cattle faeces derived from diets containing only C-3 non-grass and C-4 grass components, and a series of expansion and validation steps was employed to develop robustness and predictive reliability. The final calibration equation contained 1637 samples and faecal delta C-13 range (parts per thousand) of [12.27]-[27.65]. Calibration statistics were: standard error of calibration (SEC) of 0.78, standard error of cross-validation (SECV) of 0.80, standard deviation (SD) of reference values of 3.11 and R-2 of 0.94. Validation statistics for the final calibration equation applied to 60 samples were: standard error of prediction (SEP) of 0.87, bias of -0.15, R-2 of 0.92 and RPD of 3.16. The calibration equation was also tested on faeces from diets containing C-4 non-grass species or temperate C-3 grass species. Faecal delta C-13 predictions indicated that the spectral basis of the calibration was not related to C-13/C-12 ratios per se but to consistent differences between grasses and non-grasses in chemical composition and that the differences were modified by photosynthetic pathway. Thus, although the calibration equation could not be used to make valid faecal delta C-13 predictions when the diet contained either C-3 grass or C-4 non-grass, it could be used to make useful estimates of dietary non-grass proportions. It could also be ut :sed to make useful estimates of non-grass in mixed C-3 grass/non-grass diets by applying a modified formula to calculate non-grass from predicted faecal delta C-13. The development of a robust faecal-NIR calibration equation for estimating non-grass proportions in the diets of grazing cattle demonstrated a novel and useful application of NIR spectroscopy in agriculture.

Predicting Additive and Non-additive Genetic Effects from Trials Where Traits Are Affected by Interplot Competition

There are two key types of selection in a plant breeding program, namely selection of hybrids for potential commercial use and the selection of parents for use in future breeding. Oakey et al. (in Theoretical and Applied Genetics 113, 809-819, 2006) showed how both of these aims could be achieved using pedigree information in a mixed model analysis in order to partition genetic effects into additive and non-additive effects. Their approach was developed for field trial data subject to spatial variation. In this paper we extend the approach for data from trials subject to interplot competition. We show how the approach may be used to obtain predictions of pure stand additive and non-additive effects. We develop the methodology in the context of a single field trial using an example from an Australian sorghum breeding program.