Inferential, non-parametric statistics to assess quality of probabilistic forecast systems

Many statistical forecast systems are available to interested users. In order to be useful for decision-making, these systems must be based on evidence of underlying mechanisms. Once causal connections between the mechanism and their statistical manifestation have been firmly established, the forecasts must also provide some quantitative evidence of `quality’. However, the quality of statistical climate forecast systems (forecast quality) is an ill-defined and frequently misunderstood property. Often, providers and users of such forecast systems are unclear about what ‘quality’ entails and how to measure it, leading to confusion and misinformation. Here we present a generic framework to quantify aspects of forecast quality using an inferential approach to calculate nominal significance levels (p-values) that can be obtained either by directly applying non-parametric statistical tests such as Kruskal-Wallis (KW) or Kolmogorov-Smirnov (KS) or by using Monte-Carlo methods (in the case of forecast skill scores). Once converted to p-values, these forecast quality measures provide a means to objectively evaluate and compare temporal and spatial patterns of forecast quality across datasets and forecast systems. Our analysis demonstrates the importance of providing p-values rather than adopting some arbitrarily chosen significance levels such as p < 0.05 or p < 0.01, which is still common practice. This is illustrated by applying non-parametric tests (such as KW and KS) and skill scoring methods (LEPS and RPSS) to the 5-phase Southern Oscillation Index classification system using historical rainfall data from Australia, The Republic of South Africa and India. The selection of quality measures is solely based on their common use and does not constitute endorsement. We found that non-parametric statistical tests can be adequate proxies for skill measures such as LEPS or RPSS. The framework can be implemented anywhere, regardless of dataset, forecast system or quality measure. Eventually such inferential evidence should be complimented by descriptive statistical methods in order to fully assist in operational risk management.

Optical peanut yield monitor: Development and testing

An optical peanut yield monitor was developed, fabricated, and field-tested. The overall system includes an optical mass-flow sensor, a GPS receiver, and a data acquisition system. The concept for the mass-flow sensor is based on that of the cotton yield-monitor sensor developed previously by Thomasson and Sui (2000). A modified version of the sensor was designed to be specific to peanut mass-flow measurement. Field testing of the peanut yield monitor was conducted in Australia during the May 2003 harvest. After subsequent minor modifications, the system was more extensively tested in Mississippi in October of 2003 and November of 2004. Test results showed that the output of the peanut mass-flow sensor was very strongly correlated with the harvested load weight, and the system's performance was stable and reliable during the tests.

Use of optical density as a measure of Claviceps africana conidial suspension concentration

Sorghum ergot, caused by Claviceps africana, has remained a major disease problem in Australia since it was first recorded in 1996, and is the focus of a range of biological and integrated management research. Artificial inoculation using conidial suspensions is an important tool in this research. Ergot infection is greatly influenced by environmental factors, so it is important to reduce controllable sources of variation such as inoculum concentration. The use of optical density was tested as a method of quantifying conidial suspensions of C. africana, as an alternative to haemocytometer counts. This method was found to be accurate and time efficient, with possible applications in other disease systems.