Inferential, non-parametric statistics to assess quality of probabilistic forecast systems

Many statistical forecast systems are available to interested users. In order to be useful for decision-making, these systems must be based on evidence of underlying mechanisms. Once causal connections between the mechanism and their statistical manifestation have been firmly established, the forecasts must also provide some quantitative evidence of `quality’. However, the quality of statistical climate forecast systems (forecast quality) is an ill-defined and frequently misunderstood property. Often, providers and users of such forecast systems are unclear about what ‘quality’ entails and how to measure it, leading to confusion and misinformation. Here we present a generic framework to quantify aspects of forecast quality using an inferential approach to calculate nominal significance levels (p-values) that can be obtained either by directly applying non-parametric statistical tests such as Kruskal-Wallis (KW) or Kolmogorov-Smirnov (KS) or by using Monte-Carlo methods (in the case of forecast skill scores). Once converted to p-values, these forecast quality measures provide a means to objectively evaluate and compare temporal and spatial patterns of forecast quality across datasets and forecast systems. Our analysis demonstrates the importance of providing p-values rather than adopting some arbitrarily chosen significance levels such as p < 0.05 or p < 0.01, which is still common practice. This is illustrated by applying non-parametric tests (such as KW and KS) and skill scoring methods (LEPS and RPSS) to the 5-phase Southern Oscillation Index classification system using historical rainfall data from Australia, The Republic of South Africa and India. The selection of quality measures is solely based on their common use and does not constitute endorsement. We found that non-parametric statistical tests can be adequate proxies for skill measures such as LEPS or RPSS. The framework can be implemented anywhere, regardless of dataset, forecast system or quality measure. Eventually such inferential evidence should be complimented by descriptive statistical methods in order to fully assist in operational risk management.

Optimizing sowing management for short duration dry seeded aman rice on the High Ganges River Floodplain of Bangladesh

Dry seeding of aman rice can facilitate timely crop establishment and early harvest and thus help to alleviate the monga (hunger) period in the High Ganges Flood Plain of Bangladesh. Dry seeding also offers many other potential benefits, including reduced cost of crop establishment and improved soil structure for crops grown in rotation with rice. However, the optimum time for seeding in areas where farmers have access to water for supplementary irrigation has not been determined. We hypothesized that earlier sowing is safer, and that increasing seed rate mitigates the adverse effects of significant rain after sowing on establishment and crop performance. To test these hypotheses, we analyzed long term rainfall data, and conducted field experiments on the effects of sowing date (target dates of 25 May, 10 June, 25 June, and 10 July) and seed rate (20, 40, and 60 kg ha−1) on crop establishment, growth, and yield of dry seeded Binadhan-7 (short duration, 110–120 d) during the 2012 and 2013 rainy seasons. Wet soil as a result of untimely rainfall usually prevented sowing on the last two target dates in both years, but not on the first two dates. Rainfall analysis also suggested a high probability of being able to dry seed in late May/early June, and a low probability of being able to dry seed in late June/early July. Delaying sowing from 25 May/10 June to late June/early July usually resulted in 20–25% lower plant density and lower uniformity of the plant stand as a result of rain shortly after sowing. Delaying sowing also reduced crop duration, and tillering or biomass production when using a low seed rate. For the late June/early July sowings, there was a strong positive relationship between plant density and yield, but this was not the case for earlier sowings. Thus, increasing seed rate compensated for the adverse effect of untimely rains after sowing on plant density and the shorter growth duration of the late sown crops. The results indicate that in this region, the optimum date for sowing dry seeded rice is late May to early June with a seed rate of 40 kg ha−1. Planting can be delayed to late June/early July with no yield loss using a seed rate of 60 kg ha−1, but in many years, the soil is simply too wet to be able to dry seed at this time due to rainfall.

Managing for rainfall variability: effect of grazing strategy on cattle production in a dry tropical savanna

Rainfall variability is a challenge to sustainable and pro. table cattle production in northern Australia. Strategies recommended to manage for rainfall variability, like light or variable stocking, are not widely adopted. This is due partly to the perception that sustainability and profitability are incompatible. A large, long-term grazing trial was initiated in 1997 in north Queensland, Australia, to test the effect of different grazing strategies on cattle production. These strategies are: (i) constant light stocking (LSR) at long-term carrying capacity (LTCC); (ii) constant heavy stocking (HSR) at twice LTCC; (iii) rotational wet-season spelling (R/Spell) at 1.5 LTCC; (iv) variable stocking (VAR), with stocking rates adjusted in May based on available pasture; and (v) a Southern Oscillation Index (SOI) variable strategy, with stocking rates adjusted in November, based on available pasture and SOI seasonal forecasts. Animal performance varied markedly over the 10 years for which data is presented, due to pronounced differences in rainfall and pasture availability. Nonetheless, lighter stocking at or about LTCC consistently gave the best individual liveweight gain (LWG), condition score and skeletal growth; mean LWG per annum was thus highest in the LSR (113 kg), intermediate in the R/Spell (104 kg) and lowest in the HSR(86 kg). MeanLWGwas 106 kg in the VAR and 103 kg in the SOI but, in all years, the relative performance of these strategies was dependent upon the stocking rate applied. After 2 years on the trial, steers from lightly stocked strategies were 60-100 kg heavier and received appreciable carcass price premiums at the meatworks compared to those under heavy stocking. In contrast, LWG per unit area was greatest at stocking rates of about twice LTCC; mean LWG/ha was thus greatest in the HSR (21 kg/ha), but this strategy required drought feeding in four of the 10 years and was unsustainable. Although LWG/ha was lower in the LSR (mean 14 kg/ha), or in strategies that reduced stocking rates in dry years like the VAR(mean 18 kg/ha) and SOI (mean 17 kg/ha), these strategies did not require drought feeding and appeared sustainable. The R/Spell strategy (mean 16 kg/ha) was compromised by an ill-timed fire, but also performed satisfactorily. The present results provide important evidence challenging the assumption that sustainable management in a variable environment is unprofitable. Further research is required to fully quantify the long-term effects of these strategies on land condition and profitability and to extrapolate the results to breeder performance at the property level.

Spatial and temporal effects of grazing management and rainfall on the vertebrate fauna of a tropical savanna

Grazing by domestic livestock is one of the most widespread uses of the rangelands of Australia. There is limited information on the effects of grazing by domestic livestock on the vertebrate fauna of Australia and the establishment of a long-term grazing experiment in north-eastern Queensland at Wambiana provided an opportunity to attempt an examination of the changes in vertebrate fauna as a consequence of the manipulation of stocking rates. The aim was to identify what the relative effects of vegetation type, stocking rate and other landscape-scale environmental factors were on the patterns recorded. Sixteen 1-ha sites were established within three replicated treatments (moderate, heavy and variable stocking rates). The sites were sampled in the wet and dry seasons in 1999-2000 (T-0) and again in 2003-04 (T-1). All paddocks of the treatments were burnt in 1999. Average annual rainfall declined markedly between the two sampling periods, which made interpretation of the data difficult. A total of 127 species of vertebrate fauna comprising five amphibian, 83 bird, 27 reptile and 12 mammal species were recorded. There was strong separation in faunal composition from T-0 to T-1 although changes in mean compositional dissimilarity between the grazing stocking rate treatments were less well defined. There was a relative change in abundance of 24 bird, four mammal and five reptile species from T-0 to T-1. The generalised linear modelling identified that, in the T-1 data, there was significant variation in the abundance of 16 species explained by the grazing and vegetation factors. This study demonstrated that vertebrate fauna assemblage did change and that these changes were attributable to the interplay between the stocking rates, the vegetation types on the sites surveyed, the burning of the experimental paddocks and the decrease in rainfall over the course of the two surveys. It is recommended that the experiment is sampled again but that the focus should be on a rapid survey of abundant taxa (i.e. birds and reptiles) to allow an increase in the frequency of sampling and replication of the data. This would help to articulate more clearly the trajectory of vertebrate change due to the relative effects of stocking rates compared with wider landscape environmental changes. Given the increasing focus on pastoral development in northern Australia, any opportunity to incorporate the collection of data on biodiversity into grazing manipulation experiments should be taken for the assessment of the effects of land management on faunal species.