Inferential, non-parametric statistics to assess quality of probabilistic forecast systems

Many statistical forecast systems are available to interested users. In order to be useful for decision-making, these systems must be based on evidence of underlying mechanisms. Once causal connections between the mechanism and their statistical manifestation have been firmly established, the forecasts must also provide some quantitative evidence of `quality’. However, the quality of statistical climate forecast systems (forecast quality) is an ill-defined and frequently misunderstood property. Often, providers and users of such forecast systems are unclear about what ‘quality’ entails and how to measure it, leading to confusion and misinformation. Here we present a generic framework to quantify aspects of forecast quality using an inferential approach to calculate nominal significance levels (p-values) that can be obtained either by directly applying non-parametric statistical tests such as Kruskal-Wallis (KW) or Kolmogorov-Smirnov (KS) or by using Monte-Carlo methods (in the case of forecast skill scores). Once converted to p-values, these forecast quality measures provide a means to objectively evaluate and compare temporal and spatial patterns of forecast quality across datasets and forecast systems. Our analysis demonstrates the importance of providing p-values rather than adopting some arbitrarily chosen significance levels such as p < 0.05 or p < 0.01, which is still common practice. This is illustrated by applying non-parametric tests (such as KW and KS) and skill scoring methods (LEPS and RPSS) to the 5-phase Southern Oscillation Index classification system using historical rainfall data from Australia, The Republic of South Africa and India. The selection of quality measures is solely based on their common use and does not constitute endorsement. We found that non-parametric statistical tests can be adequate proxies for skill measures such as LEPS or RPSS. The framework can be implemented anywhere, regardless of dataset, forecast system or quality measure. Eventually such inferential evidence should be complimented by descriptive statistical methods in order to fully assist in operational risk management.

Integrated Cotton Farming Systems for Central Queensland

This project has delivered outcomes that address major agronomic and crop protection issues closely linked to the profitability and sustainability of cotton production enterprises in CQ. From an agronomic perspective, the CQ environment was always though to support economically viable cotton production in a wide sowing window from the middle of September to early January prior to this research. The ideal positioning of Bollgard II varieties in the CQ planting window was, therefore, critical to the future of the local cotton industry because growers needed baseline information to determine how best to take advantage of the higher yield potential offered by the Bt cotton technology, optimise irrigation water use and fibre characteristics. The project’s outputs include a number of key agronomic findings. Over three growing seasons, Bollgard II crop planted in the traditional sowing window from the middle of September to the end of October consistently produced the highest yields. The project delivers a clear and quantitative assessment of the impacts of planting outside the traditional cropping window - a yield penalty of between 1-4 bales/ha for November and December planted cotton. Whilst yield penalties associated with December-planted crops are clearly linked to declining heat units in the second half of the crop and a cool finish, those associated with November-planted cotton are not consistent with the theoretical yield potential for this sowing date. Further research to understand and minimize the physiological constraints on November-planted cotton would give CQ cotton growers far greater flexibility to develop mixed/double/rotation cropping farming systems that are relevant to the rapidly evolving nature of Agricultural production in Australia. The equivalence of cultivar types with clearly distinguishable, genetically based growth habits, demonstrated in this project, gives growers important information for making varietal choices. The entomological outcomes of this project represent strategic and tactical tools that are highly relevant to the viability and profitability of the cotton industry in Australia. The future of the cotton industry is inextricably linked to the survival and efficacy of GM cotton. Research done in the Callide irrigation area demonstrates the unquestionable potential for development of alternative and highly effective resistance management strategies for Bollgard II using novel technologies and strategies based on products such as Magnet®. Magnet® and similar technologies will be increasingly important in strategies to preserve the shelf life and efficacy of current and future generations of GM technology. However, more research will be required to address logistical and operational issues related to these new technologies before they can be fully exploited in commercial production systems. From an economic perspective, SLW is the sleeping giant in terms of insect nemeses of cotton, particularly from the standpoint of climate change and an increasingly warmer production environment. An effective sampling and management strategy for SLW which has been delivered by this project will go a long way towards minimising production costs in an environment characterised by rapidly rising input costs. SLW has the potential to permanently debilitate the national cotton industry by influencing market sentiment and quality perceptions. Field validation of the SLW population sampling models and management options in the Dawson irrigation area cotton and southern Queensland during 2006-07 documents the robustness of the entomological research outcomes achieved through this project.

The use of genetic correlations to evaluate associations between SNP markers and quantitative traits

Open-pollinated progeny of Corymbia citriodora established in replicated field trials were assessed for stem diameter, wood density, and pulp yield prior to genotyping single nucleotide polymorphisms (SNP) and testing the significance of associations between markers and assessment traits. Multiple individuals within each family were genotyped and phenotyped, which facilitated a comparison of standard association testing methods and an alternative method developed to relate markers to additive genetic effects. Narrow-sense heritability estimates indicated there was significant additive genetic variance within this population for assessment traits ( h ˆ 2 =0.28to0.44 ) and genetic correlations between the three traits were negligible to moderate (r G = 0.08 to 0.50). The significance of association tests (p values) were compared for four different analyses based on two different approaches: (1) two software packages were used to fit standard univariate mixed models that include SNP-fixed effects, (2) bivariate and multivariate mixed models including each SNP as an additional selection trait were used. Within either the univariate or multivariate approach, correlations between the tests of significance approached +1; however, correspondence between the two approaches was less strong, although between-approach correlations remained significantly positive. Similar SNP markers would be selected using multivariate analyses and standard marker-trait association methods, where the former facilitates integration into the existing genetic analysis systems of applied breeding programs and may be used with either single markers or indices of markers created with genomic selection processes.

Assessing the sustainability of wheat-based cropping systems using simulation modelling: sustainability

Concepts of agricultural sustainability and possible roles of simulation modelling for characterising sustainability were explored by conducting, and reflecting on, a sustainability assessment of rain-fed wheat-based systems in the Middle East and North Africa region. We designed a goal-oriented, model-based framework using the cropping systems model Agricultural Production Systems sIMulator (APSIM). For the assessment, valid (rather than true or false) sustainability goals and indicators were identified for the target system. System-specific vagueness was depicted in sustainability polygons-a system property derived from highly quantitative data-and denoted using descriptive quantifiers. Diagnostic evaluations of alternative tillage practices demonstrated the utility of the framework to quantify key bio-physical and chemical constraints to sustainability. Here, we argue that sustainability is a vague, emergent system property of often wicked complexity that arises out of more fundamental elements and processes. A 'wicked concept of sustainability' acknowledges the breadth of the human experience of sustainability, which cannot be internalised in a model. To achieve socially desirable sustainability goals, our model-based approach can inform reflective evaluation processes that connect with the needs and values of agricultural decision-makers. Hence, it can help to frame meaningful discussions, from which actions might emerge.