Inferential, non-parametric statistics to assess quality of probabilistic forecast systems

Many statistical forecast systems are available to interested users. In order to be useful for decision-making, these systems must be based on evidence of underlying mechanisms. Once causal connections between the mechanism and their statistical manifestation have been firmly established, the forecasts must also provide some quantitative evidence of `quality’. However, the quality of statistical climate forecast systems (forecast quality) is an ill-defined and frequently misunderstood property. Often, providers and users of such forecast systems are unclear about what ‘quality’ entails and how to measure it, leading to confusion and misinformation. Here we present a generic framework to quantify aspects of forecast quality using an inferential approach to calculate nominal significance levels (p-values) that can be obtained either by directly applying non-parametric statistical tests such as Kruskal-Wallis (KW) or Kolmogorov-Smirnov (KS) or by using Monte-Carlo methods (in the case of forecast skill scores). Once converted to p-values, these forecast quality measures provide a means to objectively evaluate and compare temporal and spatial patterns of forecast quality across datasets and forecast systems. Our analysis demonstrates the importance of providing p-values rather than adopting some arbitrarily chosen significance levels such as p < 0.05 or p < 0.01, which is still common practice. This is illustrated by applying non-parametric tests (such as KW and KS) and skill scoring methods (LEPS and RPSS) to the 5-phase Southern Oscillation Index classification system using historical rainfall data from Australia, The Republic of South Africa and India. The selection of quality measures is solely based on their common use and does not constitute endorsement. We found that non-parametric statistical tests can be adequate proxies for skill measures such as LEPS or RPSS. The framework can be implemented anywhere, regardless of dataset, forecast system or quality measure. Eventually such inferential evidence should be complimented by descriptive statistical methods in order to fully assist in operational risk management.

Runoff, soil loss, and nutrient transport from cropping systems on Red Ferrosols in tropical northern Australia.

Runoff, soil loss, and nutrient loss were assessed on a Red Ferrosol in tropical Australia over 3 years. The experiment was conducted using bounded, 100-m(2) field plots cropped to peanuts, maize, or grass. A bare plot, without cover or crop, was also instigated as an extreme treatment. Results showed the importance of cover in reducing runoff, soil loss, and nutrient loss from these soils. Runoff ranged from 13% of incident rainfall for the conventional cultivation to 29% under bare conditions during the highest rainfall year, and was well correlated with event rainfall and rainfall energy. Soil loss ranged from 30 t/ha. year under bare conditions to <6 t/ha. year under cropping. Nutrient losses of 35 kg N and 35 kg P/ha. year under bare conditions and 17 kg N and 11 kg P/ha. year under cropping were measured. Soil carbon analyses showed a relationship with treatment runoff, suggesting that soil properties influenced the rainfall runoff response. The cropping systems model PERFECT was calibrated using runoff, soil loss, and soil water data. Runoff and soil loss showed good agreement with observed data in the calibration, and soil water and yield had reasonable agreement. Longterm runs using historical weather data showed the episodic nature of runoff and soil loss events in this region and emphasise the need to manage land using protective measures such as conservation cropping practices. Farmers involved in related, action-learning activities wished to incorporate conservation cropping findings into their systems but also needed clear production benefits to hasten practice change.

Maximising profitability with limited water in cotton farming systems

Diminishing water supply, changing weather patterns and pressure to enhance environmental flows are making it imperative to optimise water use efficiency (WUE) on cotton/grain farming systems. Growers are looking for better strategies to make the best use of limited water, but it is still not clear how to best use the available water at farm and field scale. This research project investigated the impact of management strategies to deal with limited water supplies on the yield and quality of irrigated cotton and wheat. The objectives were: (1) to develop irrigation management guidelines for the main irrigated crops on the Darling Downs for full- and deficitirrigation scenarios, taking into account the critical factors that affect irrigation decisions at the local level, (2) to quantify the evapotranspiration (ET) of Bollgard II cotton and wheat and its relationship to yield and quality under full- and deficit-irrigation scenarios, and (3) to increase industry awareness and education of farming systems practises for optimised economic water use efficiency.Objective (1) was addressed by (A) collaborating with ASPRU to develop the APSFarm model within APSIM to be able to perform multi-paddock simulations. APSFarm was then tested by conducting a case study at a farm near Dalby, and (B) conducting semi-structured interviews with individual farmers and crop consultants on the Darling Downs to document the strategies they are using to deal with limited water. Objective (2) was addressed by (A) building and installing 12 large (1 m x 1m x 1.5 m) weighing lysimeters to measure crop evapotranspiration. The lysimeters were installed at the Agri-Science Queensland research station at Kingsthorpe in November 2008, (B) conducting field experiments to measure crop evapotranspiration and crop development under four irrigation treatments, including dryland, deficit-irrigation, and full irrigation. Field experiments were conducted with cotton in 2007-08 and 2008-09, and with wheat in 2008 and 2009, and (C) collaborating with USQ on a PhD thesis to quantify the impact of crop stress on crop evapotranspiration and canopy temperature. Glasshouse experiments were conducted with wheat in 2008 and with cotton in 2008-09. Objective (3) was addressed by (A) conducting a field day at Kingsthorpe in 2009, which was attended by 80 participants, (B) presenting information in conferences in Australia and overseas, (D) presenting information at farmers meeting, (E) making presentations to crop consultants, and (F) preparing extension publications.As part of this project we contributed to the development of APSfarm, which has been successfully applied to evaluate the feasibility of practices at the whole-farm scale. From growers and crop consultants interviews we learned that there is a great variety of strategies, at different scales, that they are using to deal with limited water situation. These strategies will be summarised in the "e;Limited Water Guidelines for the Darling Downs"e; that we are currently preparing. As a result of this project, we now have a state-of-the-art lysimeter research facility (23 large weighing lysimeters) to be able to conduct replicated experiments to investigate daily water use of a variety of crops under different irrigation regimes and under different environments. Under this project, a series of field and glasshouse experiments were conducted with cotton and wheat, investigating aspects like: (A) quantification of daily and seasonal crop water use under nonstressed and stressed conditions, (B) impact of row configuration on crop water use, (C) impact of water stress on yield, evapotranspiration, crop vegetative and reproductive development, soil water extraction pattern, yield and yield quality. The information obtained from this project is now being used to develop web-based tools to help growers make planning and day-to-day irrigation decisions.

Legumes or nitrification inhibitors to reduce N2O emissions from subtropical cereal cropping systems in Oxisols?

The DAYCENT biogeochemical model was used to investigate how the use of fertilizers coated with nitrification inhibitors and the introduction of legumes in the crop rotation can affect subtropical cereal production and N2O emissions. The model was validated using comprehensive multi-seasonal, high-frequency dataset from two field investigations conducted on an Oxisol, which is the most common soil type in subtropical regions. Different N fertilizer rates were tested for each N management strategy and simulated under varying weather conditions. DAYCENT was able to reliably predict soil N dynamics, seasonal N2O emissions and crop production, although some discrepancies were observed in the treatments with low or no added N inputs and in the simulation of daily N2O fluxes. Simulations highlighted that the high clay content and the relatively low C levels of the Oxisol analyzed in this study limit the chances for significant amounts of N to be lost via deep leaching or denitrification. The application of urea coated with a nitrification inhibitor was the most effective strategy to minimize N2O emissions. This strategy however did not increase yields since the nitrification inhibitor did not substantially decrease overall N losses compared to conventional urea. Simulations indicated that replacing part of crop N requirements with N mineralized by legume residues is the most effective strategy to reduce N2O emissions and support cereal productivity. The results of this study show that legumes have significant potential to enhance the sustainable and profitable intensification of subtropical cereal cropping systems in Oxisols.