Inferential, non-parametric statistics to assess quality of probabilistic forecast systems

Many statistical forecast systems are available to interested users. In order to be useful for decision-making, these systems must be based on evidence of underlying mechanisms. Once causal connections between the mechanism and their statistical manifestation have been firmly established, the forecasts must also provide some quantitative evidence of `quality’. However, the quality of statistical climate forecast systems (forecast quality) is an ill-defined and frequently misunderstood property. Often, providers and users of such forecast systems are unclear about what ‘quality’ entails and how to measure it, leading to confusion and misinformation. Here we present a generic framework to quantify aspects of forecast quality using an inferential approach to calculate nominal significance levels (p-values) that can be obtained either by directly applying non-parametric statistical tests such as Kruskal-Wallis (KW) or Kolmogorov-Smirnov (KS) or by using Monte-Carlo methods (in the case of forecast skill scores). Once converted to p-values, these forecast quality measures provide a means to objectively evaluate and compare temporal and spatial patterns of forecast quality across datasets and forecast systems. Our analysis demonstrates the importance of providing p-values rather than adopting some arbitrarily chosen significance levels such as p < 0.05 or p < 0.01, which is still common practice. This is illustrated by applying non-parametric tests (such as KW and KS) and skill scoring methods (LEPS and RPSS) to the 5-phase Southern Oscillation Index classification system using historical rainfall data from Australia, The Republic of South Africa and India. The selection of quality measures is solely based on their common use and does not constitute endorsement. We found that non-parametric statistical tests can be adequate proxies for skill measures such as LEPS or RPSS. The framework can be implemented anywhere, regardless of dataset, forecast system or quality measure. Eventually such inferential evidence should be complimented by descriptive statistical methods in order to fully assist in operational risk management.

A Comparison of Methods for Describing Irregular Animal Growth and Testing for Treatment Effects

In many designed experiments with animals liveweight is recorded several times during the trial. Such data are commonly referred to as repeated measures data. An aim of such experiments is generally to compare the growth patterns for the applied treatments. This paper discusses some of the methods of analysing repeated measures data and illustrates the use of cubic smoothing splines to describe irregular cattle growth data. Animal production for a consuming world : proceedings of 9th Congress of the Asian-Australasian Association of Animal Production Societies [AAAP] and 23rd Biennial Conference of the Australian Society of Animal Production [ASAP] and 17th Annual Symposium of the University of Sydney, Dairy Research Foundation, [DRF]. 2-7 July 2000, Sydney, Australia.

Using simulation techniques to investigate methods to determine resistance of helminths to anthelmintic treatment

The widespread and increasing resistance of internal parasites to anthelmintic control is a serious problem for the Australian sheep and wool industry. As part of control programmes, laboratories use the Faecal Egg Count Reduction Test (FECRT) to determine resistance to anthelmintics. It is important to have confidence in the measure of resistance, not only for the producer planning a drenching programme but also for companies investigating the efficacy of their products. The determination of resistance and corresponding confidence limits as given in anthelmintic efficacy guidelines of the Standing Committee on Agriculture (SCA) is based on a number of assumptions. This study evaluated the appropriateness of these assumptions for typical data and compared the effectiveness of the standard FECRT procedure with the effectiveness of alternative procedures. Several sets of historical experimental data from sheep and goats were analysed to determine that a negative binomial distribution was a more appropriate distribution to describe pre-treatment helminth egg counts in faeces than a normal distribution. Simulated egg counts for control animals were generated stochastically from negative binomial distributions and those for treated animals from negative binomial and binomial distributions. Three methods for determining resistance when percent reduction is based on arithmetic means were applied. The first was that advocated in the SCA guidelines, the second similar to the first but basing the variance estimates on negative binomial distributions, and the third using Wadley’s method with the distribution of the response variate assumed negative binomial and a logit link transformation. These were also compared with a fourth method recommended by the International Co-operation on Harmonisation of Technical Requirements for Registration of Veterinary Medicinal Products (VICH) programme, in which percent reduction is based on the geometric means. A wide selection of parameters was investigated and for each set 1000 simulations run. Percent reduction and confidence limits were then calculated for the methods, together with the number of times in each set of 1000 simulations the theoretical percent reduction fell within the estimated confidence limits and the number of times resistance would have been said to occur. These simulations provide the basis for setting conditions under which the methods could be recommended. The authors show that given the distribution of helminth egg counts found in Queensland flocks, the method based on arithmetic not geometric means should be used and suggest that resistance be redefined as occurring when the upper level of percent reduction is less than 95%. At least ten animals per group are required in most circumstances, though even 20 may be insufficient where effectiveness of the product is close to the cut off point for defining resistance.

Investigation and application of methods for enumerating heterotrophs and Escherichia coli in the air within piggery sheds

Aims: To investigate methods for the recovery of airborne bacteria within pig sheds and to then use the appropriate methods to determine the levels of heterotrophs and Escherichia coli in the air within sheds. Methods and Results: AGI-30 impingers and a six-stage Andersen multi-stage sampler (AMS) were used for the collection of aerosols. Betaine and catalase were added to impinger collection fluid and the agar plates used in the AMS. Suitable media for enumerating E. coli with the Andersen sampler were also evaluated. The addition of betaine and catalase gave no marked increase in the recovery of heterotrophs or E. coli. No marked differences were found in the media used for enumeration of E. coli. The levels of heterotrophs and E. coli in three piggeries, during normal pig activities, were 2Æ2 · 105 and 21 CFU m)3 respectively. Conclusions: The failure of the additives to improve the recovery of either heterotrophs or E. coli suggests that these organisms are not stressed in the piggery environment. The levels of heterotrophs in the air inside the three Queensland piggeries investigated are consistent with those previously reported in other studies. Flushing with ponded effluent had no marked or consistent effect on the heterotroph or E. coli levels. Significance and Impact of the Study: Our work suggests that levels of airborne heterotrophs and E. coli inside pig sheds have no strong link with effluent flushing. It would seem unlikely that any single management activity within a pig shed has a dominant influence on levels of airborne heterotrophs and E. coli

Climate information needs of Gascoyne-Murchison pastoralists: a representative study of the Western Australian grazing industry

The Gascoyne-Murchison region of Western Australia experiences an arid to semi-arid climate with a highly variable temporal and spatial rainfall distribution. The region has around 39.2 million hectares available for pastoral lease and supports predominantly catle and sheep grazing leases. In recent years a number of climate forecasting systems have been available offering rainfall probabilities with different lead times and a forecast period; however, the extent to which these systems are capable of fulfilling the requirements of the local pastoralists is still ambiguous. Issues can range from ensuring forecasts are issued with sufficient lead time to enable key planning or decisions to be revoked or altered, to ensuring forecast language is simple and clear, to negate possible misunderstandings in interpretation. A climate research project sought to provide an objective method to determine which available forecasting systems had the greatest forecasting skill at times of the year relevant to local property management. To aid this climate research project, the study reported here was undertaken with an overall objective of exploring local pastoralists' climate information needs. We also explored how well they understand common climate forecast terms such as 'mean', median' and 'probability', and how they interpret and apply forecast information to decisions. A stratified, proportional random sampling was used for the purpose of deriving the representative sample based on rainfall-enterprise combinations. In order to provide more time for decision-making than existing operational forecasts that are issued with zero lead time, pastoralists requested that forecasts be issued for May-July and January-March with lead times counting down from 4 to 0 months. We found forecasts of between 20 and 50 mm break-of-season or follow-up rainfall were likely to influence decisions. Eighty percent of pastoralists demonstrated in a test question that they had a poor technical understanding of how to interpret the standard wording of a probabilistic median rainfall forecast. this is worthy of further research to investigate whether inappropriate management decisions are being made because the forecasts are being misunderstood. We found more than half the respondents regularly access and use weather and climate forecasts or outlook information from a range of sources and almost three-quarters considered climate information or tools useful, with preferred methods for accessing this information by email, faxback service, internet and the Department of Agriculture Western Australia's Pastoral Memo. Despite differences in enterprise types and rainfall seasonality across the region we found seasonal climate forecasting needs were relatively consistent. It became clear that providing basic training and working with pastoralists to help them understand regional climatic drivers, climate terminology and jargon, and the best ways to apply the forecasts to enhance decision-making are important to improve their use of information. Consideration could also be given to engaging a range of producers to write the climate forecasts themselves in the language they use and understand, in consultation with the scientists who prepare the forecasts.

Formative evaluation to benchmark and improve climate-based decision support for graziers in Western Queelsland

Researchers developing climate-based forecasts, workshops, software tools and information to aid grazier decisions undertook an evaluation study to enhance planning and benchmark impact. One hundred graziers in Western Queensland were randomly selected from 7 shires and surveyed by mail and telephone (43 respondents) to explore levels of knowledge and use of climate information, practices and information needs. We found 36% of respondents apply the Southern Oscillation Index to property decisions but 92% were unaware El Niño Southern Oscillation’s predictive signal in the region is greater for pasture growth than rainfall, suggesting they may not recognise the potential of pasture growth forecasts. Almost 75% of graziers consider they are conservative or risk averse in their attitude to managing their enterprise. Mail respondents (n= 20) if given a 68%, on average, probability of exceeding median rainfall forecast may change a decision; almost two-thirds vary stocking rate based on forage available, last year’s pasture growth or the Southern Oscillation Index; the balance maintain a constant stocking rate strategy; 90% have access to a computer; 75% to the internet and 95% have a fax. This paper presents findings of the study and draws comparisons with a similar study of 174 irrigators in the Northern Murray-Darling Basin (Aust. J. Exp. Ag. 44, 247-257). New insights and information gained are helping the team better understand client needs and plan, design and extend tools and information tailored to grazier knowledge, practice, information needs and preferences. Results have also provided a benchmark against which to measure project impact and have influenced the team to make important changes to their project planning, activities and methods for transferring technology tailored to grazier preferences.

Comparison of sampling sites and detection methods for Haemophilus parasuis

Objective To improve the isolation rate and identification procedures for Haemophilus parasuis from pig tissues. Design Thirteen sampling sites and up to three methods were used to confirm the presence of H. parasuis in pigs after experimental challenge. Procedure Colostrum-deprived, naturally farrowed pigs were challenged intratracheally with H parasuis serovar 12 or 4. Samples taken during necropsy were either inoculated onto culture plates, processed directly for PCR or enriched prior to being processed for PCR. The recovery of H parasuis from different sampling sites and using different sampling methods was compared for each serovar. Results H parasuis was recovered from several sample sites for all serovar 12 challenged pigs, while the trachea was the only positive site for all pigs following serovar 4 challenge. The method of solid medium culture of swabs, and confirmation of the identity of cultured bacteria by PCR, resulted in 38% and 14% more positive results on a site basis for serovars 12 and 4, retrospectively, than direct PCR on the swabs. This difference was significant in the serovar 12 challenge. Conclusion Conventional culture proved to be more effective in detecting H parasuis than direct PCR or PCR on enrichment broths. For subacute (serovar 4) infections, the most successful sites for culture or direct PCR were pleural fluid, peritoneal fibrin and fluid, lung and pericardial fluid. For acute (serovar 12) infections, the best sites were lung, heart blood, affected joints and brain. The methodologies and key sampling sites identified in this study will enable improved isolation of H parasuis and aid the diagnosis of Glässer's disease.

A comparison of mark-recapture distance-sampling methods applied to aerial surveys of eastern grey kangaroos

Aerial surveys of kangaroos (Macropus spp.) in Queensland are used to make economically important judgements on the levels of viable commercial harvest. Previous analysis methods for aerial kangaroo surveys have used both mark-recapture methodologies and conventional distance-sampling analyses. Conventional distance sampling has the disadvantage that detection is assumed to be perfect on the transect line, while mark-recapture methods are notoriously sensitive to problems with unmodelled heterogeneity in capture probabilities. We introduce three methodologies for combining together mark-recapture and distance-sampling data, aimed at exploiting the strengths of both methodologies and overcoming the weaknesses. Of these methods, two are based on the assumption of full independence between observers in the mark-recapture component, and this appears to introduce more bias in density estimation than it resolves through allowing uncertain trackline detection. Both of these methods give lower density estimates than conventional distance sampling, indicating a clear failure of the independence assumption. The third method, termed point independence, appears to perform very well, giving credible density estimates and good properties in terms of goodness-of-fit and percentage coefficient of variation. Estimated densities of eastern grey kangaroos range from 21 to 36 individuals km-2, with estimated coefficients of variation between 11% and 14% and estimated trackline detection probabilities primarily between 0.7 and 0.9.

Comparison of nitrogen fertilization methods and rates for subsurface drip irrigated corn in the semi-arid Great Plains

In semi-arid areas such as western Nebraska, interest in subsurface drip irrigation (SDI) for corn is increasing due to restricted irrigation allocations. However, crop response quantification to nitrogen (N) applications with SDI and the environmental benefits of multiple in-season (IS) SDI N applications instead of a single early-season (ES) surface application are lacking. The study was conducted in 2004, 2005, and 2006 at the University of Nebraska-Lincoln West Central Research and Extension Center in North Platte, Nebraska, comparing two N application methods (IS and ES) and three N rates (128, 186, and 278 kg N ha(-1)) using a randomized complete block design with four replications. No grain yield or biomass response was observed in 2004. In 2005 and 2006, corn grain yield and biomass production increased with increasing N rates, and the IS treatment increased grain yield, total N uptake, and gross return after N application costs (GRN) compared to the ES treatment. Chlorophyll meter readings taken at the R3 corn growth stage in 2006 showed that less N was supplied to the plant with ES compared to the IS treatment. At the end of the study, soil NO3-N masses in the 0.9 to 1.8 m depth were greater under the IS treatment compared to the ES treatment. Results suggested that greater losses of NO3-N below the root zone under the ES treatment may have had a negative effect on corn production. Under SDI systems, fertigating a recommended N rate at various corn growth stages can increase yields, GRN, and reduce NO3-N leaching in soils compared to concentrated early-season applications.

Annual forecasting of the Australian macadamia crop - integrating tree census data with statistical climate-adjustment models

To facilitate marketing and export, the Australian macadamia industry requires accurate crop forecasts. Each year, two levels of crop predictions are produced for this industry. The first is an overall longer-term forecast based on tree census data of growers in the Australian Macadamia Society (AMS). This data set currently accounts for around 70% of total production, and is supplemented by our best estimates of non-AMS orchards. Given these total tree numbers, average yields per tree are needed to complete the long-term forecasts. Yields from regional variety trials were initially used, but were found to be consistently higher than the average yields that growers were obtaining. Hence, a statistical model was developed using growers' historical yields, also taken from the AMS database. This model accounted for the effects of tree age, variety, year, region and tree spacing, and explained 65% of the total variation in the yield per tree data. The second level of crop prediction is an annual climate adjustment of these overall long-term estimates, taking into account the expected effects on production of the previous year's climate. This adjustment is based on relative historical yields, measured as the percentage deviance between expected and actual production. The dominant climatic variables are observed temperature, evaporation, solar radiation and modelled water stress. Initially, a number of alternate statistical models showed good agreement within the historical data, with jack-knife cross-validation R2 values of 96% or better. However, forecasts varied quite widely between these alternate models. Exploratory multivariate analyses and nearest-neighbour methods were used to investigate these differences. For 2001-2003, the overall forecasts were in the right direction (when compared with the long-term expected values), but were over-estimates. In 2004 the forecast was well under the observed production, and in 2005 the revised models produced a forecast within 5.1% of the actual production. Over the first five years of forecasting, the absolute deviance for the climate-adjustment models averaged 10.1%, just outside the targeted objective of 10%.

Hardness Methods for Testing Maize Kernels

Maize is a highly important crop to many countries around the world, through the sale of the maize crop to domestic processors and subsequent production of maize products and also provides a staple food to subsistance farms in undeveloped countries. In many countries, there have been long-term research efforts to develop a suitable hardness method that could assist the maize industry in improving efficiency in processing as well as possibly providing a quality specification for maize growers, which could attract a premium. This paper focuses specifically on hardness and reviews a number of methodologies as well as important biochemical aspects of maize that contribute to maize hardness used internationally. Numerous foods are produced from maize, and hardness has been described as having an impact on food quality. However, the basis of hardness and measurement of hardness are very general and would apply to any use of maize from any country. From the published literature, it would appear that one of the simpler methods used to measure hardness is a grinding step followed by a sieving step, using multiple sieve sizes. This would allow the range in hardness within a sample as well as average particle size and/or coarse/fine ratio to be calculated. Any of these parameters could easily be used as reference values for the development of near-infrared (NIR) spectroscopy calibrations. The development of precise NIR calibrations will provide an excellent tool for breeders, handlers, and processors to deliver specific cultivars in the case of growers and bulk loads in the case of handlers, thereby ensuring the most efficient use of maize by domestic and international processors. This paper also considers previous research describing the biochemical aspects of maize that have been related to maize hardness. Both starch and protein affect hardness, with most research focusing on the storage proteins (zeins). Both the content and composition of the zein fractions affect hardness. Genotypes and growing environment influence the final protein and starch content and. to a lesser extent, composition. However, hardness is a highly heritable trait and, hence, when a desirable level of hardness is finally agreed upon, the breeders will quickly be able to produce material with the hardness levels required by the industry.

An improved technique for estimating short-term survival of released line-caught fish, and an application comparing barotrauma-relief methods in red emperor (Lutjanus sebae Cuvier 1816).

Promotion of better procedures for releasing undersize fish, advocacy of catch-and-release angling, and changing minimum legal sizes are increasingly being used as tools for sustainable management of fish stocks. However without knowing the proportion of released fish that survive, the conservation value of any of these measures is uncertain. We developed a floating vertical enclosure to estimate short-term survival of released line-caught tropical and subtropical reef-associated species, and used it to compare the effectiveness of two barotrauma-relief procedures (venting and shotline releasing) on red emperor (Lutjanus sebae). Barotrauma signs varied with capture depth, but not with the size of the fish. Fish from the greatest depths (40-52 m) exhibited extreme signs less frequently than did those from intermediate depths (30-40 m), possibly as a result of swim bladder gas being vented externally through a rupture in the body wall. All but two fish survived the experiment, and as neither release technique significantly improved short-term survival of the red emperor over non-treatment we see little benefit in promoting either venting or shotline releasing for this comparatively resilient species. Floating vertical enclosures can improve short-term post-release mortality estimates as they overcome many problems encountered when constraining fish in submerged cages.

Controlled pollination methods for creating Corymbia hybrids.

Inter-specific Corymbia hybrids are of increasing interest to plantation forestry, yet there is little knowledge of the most suitable controlled pollination methods for this genus. Inter-specific crosses were made between C. torelliana [CT(maternal parent)] and C. citriodora subsp. variegata (CCV), C. henryi (CH) and C. citriodora subsp. citriodora (CCC) using conventional pollination, one-stop pollination (OSP) and artificially-induced protogyny on yellow buds (AIP Y) pollination methods. Additional treatments included AIP on green buds (AIP G) and the use of exclusion bags for the OSP and AIP methods. Inter-specific hybrids (CT x CCV, CT x CH and CT x CCC) were successfully created using all three pollination methods. The AIP Y treatment provided the highest seed yields and achieved time savings of >41% over the conventional and OSP methods, resulting in up to five-fold increases in operator productivity. However, the AIP Y treatment also had the highest C. torelliana contamination levels (9.3–13.2%). The use of exclusion bags with the AIP method had minimal effect on contamination rates, indicating a high proportion of selfpollen contamination. Contamination rates varied between maternal parents, suggesting variation in selfcompatibility for C. torelliana individuals. AIP using semi-ripe green buds was not effective at reducing selfing and had low operator productivity. The AIP method is suitable for use in a large-scale hybrid breeding program for C. torelliana. When self-pollination effects are managed, it could greatly reduce the costs associated with the production of seed of elite family crosses for commercial forestry deployment.

Driving better vegetable irrigation through profitable practice change

The availability and quality of irrigation water has become an issue limiting productivity in many Australian vegetable regions. Production is also under competitive pressure from supply chain forces. Producers look to new technologies, including changing irrigation infrastructure, exploring new water sources, and more complex irrigation management, to survive these stresses. Often there is little objective information investigating which improvements could improve outcomes for vegetable producers, and external communities (e.g. meeting NRM targets). This has led to investment in inappropriate technologies, and costly repetition of errors, as business independently discover the worth of technologies by personal experience. In our project, we investigated technology improvements for vegetable irrigation. Through engagement with industry and other researchers, we identified technologies most applicable to growers, particularly those that addressed priority issues. We developed analytical tools for ‘what if’ scenario testing of technologies. We conducted nine detailed experiments in the Lockyer Valley and Riverina vegetable growing districts, as well as case studies on grower properties in southern Queensland. We investigated root zone monitoring tools (FullStop™ wetting front detectors and Soil Solution Extraction Tubes - SSET), drip system layout, fertigation equipment, and altering planting arrangements. Our project team developed and validated models for broccoli, sweet corn, green beans and lettuce, and spreadsheets for evaluating economic risks associated with new technologies. We presented project outcomes at over 100 extension events, including irrigation showcases, conferences, field days, farm walks and workshops. The FullStops™ were excellent for monitoring root zone conditions (EC, nitrate levels), and managing irrigation with poor quality water. They were easier to interpret than the SSET. The SSET were simpler to install, but required wet soil to be reliable. SSET were an option for monitoring deeper soil zones, unsuitable for FullStop™ installations. Because these root zone tools require expertise, and are labour intensive, we recommend they be used to address specific problems, or as a periodic auditing strategy, not for routine monitoring. In our research, we routinely found high residual N in horticultural soils, with subsequently little crop yield response to additional nitrogen fertiliser. With improved irrigation efficiency (and less leaching), it may be timely to re-examine nitrogen budgets and recommendations for vegetable crops. Where the drip irrigation tube was located close to the crop row (i.e. within 5-8 cm), management of irrigation was easier. It improved nitrogen uptake, water use efficiency, and reduced the risk of poor crop performance through moisture stress, particularly in the early crop establishment phases. Close proximity of the drip tube to the crop row gives the producer more options for managing salty water, and more flexibility in taking risks with forecast rain. In many vegetable crops, proximate drip systems may not be cost-effective. The next best alternative is to push crop rows closer to the drip tube (leading to an asymmetric row structure). The vegetable crop models are good at predicting crop phenology (development stages, time to harvest), input use (water, fertiliser), environmental impacts (nutrient, salt movement) and total yields. The two immediate applications for the models are understanding/predicting/manipulating harvest dates and nitrogen movements in vegetable cropping systems. From the economic tools, the major influences on accumulated profit are price and yield. In doing ‘what if’ analyses, it is very important to be as accurate as possible in ascertaining what the assumed yield and price ranges are. In most vegetable production systems, lowering the required inputs (e.g. irrigation requirement, fertiliser requirement) is unlikely to have a major influence on accumulated profit. However, if a resource is constraining (e.g. available irrigation water), it is usually most profitable to maximise return per unit of that resource.

Current and emerging screening methods to identify post-head-emergence frost adaptation in wheat and barley

Cereal crops can suffer substantial damage if frosts occur at heading. Identification of post-head-emergence frost (PHEF) resistance in cereals poses a number of unique and difficult challenges. Many decades of research have failed to identify genotypes with PHEF resistance that could offer economically significant benefit to growers. Research and breeding gains have been limited by the available screening systems. Using traditional frost screening systems, genotypes that escape frost injury in trials due to spatial temperature differences and/or small differences in phenology can be misidentified as resistant. We believe that by improving techniques to minimize frost escapes, such ofalse-positive' results can be confidently identified and eliminated. Artificial freezing chambers or manipulated natural frost treatments offer many potential advantages but are not yet at the stage where they can be reliably used for frost screening in breeding programmes. Here we describe the development of a novel photoperiod gradient method (PGM) that facilitates screening of genotypes of different phenology under natural field frosts at matched developmental stages. By identifying frost escapes and increasing the efficiency of field screening, the PGM ensures that research effort can be focused on finding genotypes with improved PHEF resistance. To maximize the likelihood of identifying PHEF resistance, we propose that the PGM form part of an integrated strategy to (i) source germplasm;(ii) facilitate high throughput screening; and (iii) permit detailed validation. PGM may also be useful in other studies where either a range of developmental stages and/or synchronized development are desired.