Inferential, non-parametric statistics to assess quality of probabilistic forecast systems

Many statistical forecast systems are available to interested users. In order to be useful for decision-making, these systems must be based on evidence of underlying mechanisms. Once causal connections between the mechanism and their statistical manifestation have been firmly established, the forecasts must also provide some quantitative evidence of `quality’. However, the quality of statistical climate forecast systems (forecast quality) is an ill-defined and frequently misunderstood property. Often, providers and users of such forecast systems are unclear about what ‘quality’ entails and how to measure it, leading to confusion and misinformation. Here we present a generic framework to quantify aspects of forecast quality using an inferential approach to calculate nominal significance levels (p-values) that can be obtained either by directly applying non-parametric statistical tests such as Kruskal-Wallis (KW) or Kolmogorov-Smirnov (KS) or by using Monte-Carlo methods (in the case of forecast skill scores). Once converted to p-values, these forecast quality measures provide a means to objectively evaluate and compare temporal and spatial patterns of forecast quality across datasets and forecast systems. Our analysis demonstrates the importance of providing p-values rather than adopting some arbitrarily chosen significance levels such as p < 0.05 or p < 0.01, which is still common practice. This is illustrated by applying non-parametric tests (such as KW and KS) and skill scoring methods (LEPS and RPSS) to the 5-phase Southern Oscillation Index classification system using historical rainfall data from Australia, The Republic of South Africa and India. The selection of quality measures is solely based on their common use and does not constitute endorsement. We found that non-parametric statistical tests can be adequate proxies for skill measures such as LEPS or RPSS. The framework can be implemented anywhere, regardless of dataset, forecast system or quality measure. Eventually such inferential evidence should be complimented by descriptive statistical methods in order to fully assist in operational risk management.

From inferential statistics to climate knowledge

Climate variability and change are risk factors for climate sensitive activities such as agriculture. Managing these risks requires "climate knowledge", i.e. a sound understanding of causes and consequences of climate variability and knowledge of potential management options that are suitable in light of the climatic risks posed. Often such information about prognostic variables (e.g. yield, rainfall, run-off) is provided in probabilistic terms (e.g. via cumulative distribution functions, CDF), whereby the quantitative assessments of these alternative management options is based on such CDFs. Sound statistical approaches are needed in order to assess whether difference between such CDFs are intrinsic features of systems dynamics or chance events (i.e. quantifying evidences against an appropriate null hypothesis). Statistical procedures that rely on such a hypothesis testing framework are referred to as "inferential statistics" in contrast to descriptive statistics (e.g. mean, median, variance of population samples, skill scores). Here we report on the extension of some of the existing inferential techniques that provides more relevant and adequate information for decision making under uncertainty.

Stock assessment of the Queensland-New South Wales tailor fishery (Pomatomus saltatrix)

Data on catch sizes, catch rates, length-frequency and age composition from the Australian east coast tailor fishery are analysed by three different population dynamic models: a surplus production model, an age-structured model, and a model in which the population is structured by both age and length. The population is found to be very heavily exploited, with its ability to reproduce dependent on the fishery’s incomplete selectivity of one-year-old fish. Estimates of recent harvest rates (proportion of fish available to the fishery that are actually caught in a single year) are over 80%. It is estimated that only 30–50% of one-year-old fish are available to the fishery. Results from the age-length-structured model indicate that both exploitable biomass (total mass of fish selected by the fishery) and egg production have fallen to about half the levels that prevailed in the 1970s, and about 40% of virgin levels. Two-year-old fish appear to have become smaller over the history of the fishery. This is assumed to be due to increased fishing pressure combined with non-selectivity of small one-year-old fish, whereby the one-year-old fish that survive fishing are small and grow into small two-year-old fish the following year. An alternative hypothesis is that the stock has undergone a genetic change towards smaller fish; the true explanation is unknown. The instantaneous natural mortality rate of tailor is hypothesised to be higher than previously thought, with values between 0.8 and 1.3 yr–1 consistent with the models. These values apply only to tailor up to about three years of age, and it is possible that a lower value applies to fish older than three. The analysis finds no evidence that fishing pressure has yet affected recruitment. If a recruitment downturn were to occur, however, under current management and fishing pressure there is a strong chance that the fishery would need a complete closure for several years to recover, and even then recovery would be uncertain. Therefore it is highly desirable to better protect the spawning stock. The major recommendations are • An increase in the minimum size limit from 30cm to 40cm in order to allow most one-year-old fish to spawn, and • An experiment on discard mortality to gauge the proportion of fish between 30cm and 40cm that are likely to survive being caught and released by recreational line fishers (the dominant component of the fishery, currently harvesting roughly 1000t p.a. versus about 200t p.a. from the commercial fishery).

Stakeholders' assessment of welfare indicators for sheep and cattle exported by sea from Australia

An adaptive conjoint analysis was use to evaluate stakeholders' opinion of welfare indicators for ship-transported sheep and cattle, both onboard and in pre-export depots. In consultations with two nominees of each identified stakeholder group (government officials, animal welfare representatives, animal scientists, stockpersons, producers/pre-export depot operators, exporters/ship owners and veterinarians), 18 potential indicators were identified Three levels were assigned to each using industry statistics and expert opinion, representing those observed on the best and worst 5% of voyages and an intermediate value. A computer-based questionnaire was completed by 135 stakeholders (48% of those invited). All indicators were ranked by respondents in the assigned order, except fodder intake, in which case providing the amount necessary to maintain bodyweight was rated better than over or underfeeding, and time in the pre-export assembly depot, in which case 5 days was rated better than 0 or 10 days. The respective Importance Values (a relative rating given by the respondent) for each indicator were, in order of declining importance: mortality (8.6%), clinical disease incidence (8.2%), respiration rate (6.8%), space allowance (6.2%), ammonia levels (6.1%), weight change (6.0%), wet bulb temperature (6.0%), time in assembly depot (5.4%), percentage of animals in hospital pen (5.4%), fodder intake (5.2%), stress-related metabolites (5.0%), percentage of feeding trough utilised (5.0%), injuries (4.8%), percentage of animals able to access food troughs at any one time (4.8%), percentage of animals lying down (4.7%), cortisol concentration (4.5Y.), noise (3.9y.), and photoperiod (3.4%). The different stakeholder groups were relatively consistent in their ranking of the indicators, with all groups nominating the some top two and at least five of the top seven indicators. Some of the top indicators, in particular mortality, disease incidence and temperature, are already recorded in the Australian industry, but the study identified potential new welfare indicators for exported livestock, such as space allowance and ammonia concentration, which could be used to improve welfare standards if validated by scientific data. The top indicators would also be useful worldwide for countries engaging in long distance sea transport of livestock.

Estimating crop area using seasonal time series of enhanced vegetation index from MODIS satellite imagery

Cereal grain is one of the main export commodities of Australian agriculture. Over the past decade, crop yield forecasts for wheat and sorghum have shown appreciable utility for industry planning at shire, state, and national scales. There is now an increasing drive from industry for more accurate and cost-effective crop production forecasts. In order to generate production estimates, accurate crop area estimates are needed by the end of the cropping season. Multivariate methods for analysing remotely sensed Enhanced Vegetation Index (EVI) from 16-day Moderate Resolution Imaging Spectroradiometer (MODIS) satellite imagery within the cropping period (i.e. April-November) were investigated to estimate crop area for wheat, barley, chickpea, and total winter cropped area for a case study region in NE Australia. Each pixel classification method was trained on ground truth data collected from the study region. Three approaches to pixel classification were examined: (i) cluster analysis of trajectories of EVI values from consecutive multi-date imagery during the crop growth period; (ii) harmonic analysis of the time series (HANTS) of the EVI values; and (iii) principal component analysis (PCA) of the time series of EVI values. Images classified using these three approaches were compared with each other, and with a classification based on the single MODIS image taken at peak EVI. Imagery for the 2003 and 2004 seasons was used to assess the ability of the methods to determine wheat, barley, chickpea, and total cropped area estimates. The accuracy at pixel scale was determined by the percent correct classification metric by contrasting all pixel scale samples with independent pixel observations. At a shire level, aggregated total crop area estimates were compared with surveyed estimates. All multi-temporal methods showed significant overall capability to estimate total winter crop area. There was high accuracy at pixel scale (>98% correct classification) for identifying overall winter cropping. However, discrimination among crops was less accurate. Although the use of single-date EVI data produced high accuracy for estimates of wheat area at shire scale, the result contradicted the poor pixel-scale accuracy associated with this approach, due to fortuitous compensating errors. Further studies are needed to extrapolate the multi-temporal approaches to other geographical areas and to improve the lead time for deriving cropped-area estimates before harvest.

Parameter estimation and sensitivity analysis of fat deposition models in beef steers using acslXtreme.

The Davis Growth Model (a dynamic steer growth model encompassing 4 fat deposition models) is currently being used by the phenotypic prediction program of the Cooperative Research Centre (CRC) for Beef Genetic Technologies to predict P8 fat (mm) in beef cattle to assist beef producers meet market specifications. The concepts of cellular hyperplasia and hypertrophy are integral components of the Davis Growth Model. The net synthesis of total body fat (kg) is calculated from the net energy available after accounting tor energy needs for maintenance and protein synthesis. Total body fat (kg) is then partitioned into 4 fat depots (intermuscular, intramuscular, subcutaneous, and visceral). This paper reports on the parameter estimation and sensitivity analysis of the DNA (deoxyribonucleic acid) logistic growth equations and the fat deposition first-order differential equations in the Davis Growth Model using acslXtreme (Hunstville, AL, USA, Xcellon). The DNA and fat deposition parameter coefficients were found to be important determinants of model function; the DNA parameter coefficients with days on feed >100 days and the fat deposition parameter coefficients for all days on feed. The generalized NL2SOL optimization algorithm had the fastest processing time and the minimum number of objective function evaluations when estimating the 4 fat deposition parameter coefficients with 2 observed values (initial and final fat). The subcutaneous fat parameter coefficient did indicate a metabolic difference for frame sizes. The results look promising and the prototype Davis Growth Model has the potential to assist the beef industry meet market specifications.

Using harvest statistics to monitor temporal variation in kangaroo density and harvest rate

Management of the commercial harvest of kangaroos relies on quotas set annually as a proportion of regular estimates of population size. Surveys to generate these estimates are expensive and, in the larger states, logistically difficult; a cheaper alternative is desirable. Rainfall is a disappointingly poor predictor of kangaroo rate of increase in many areas, but harvest statistics (sex ratio, carcass weight, skin size and animals shot per unit time) potentially offer cost-effective indirect monitoring of population abundance (and therefore trend) and status (i.e. under-or overharvest). Furthermore, because harvest data are collected continuously and throughout the harvested areas, they offer the promise of more intensive and more representative coverage of harvest areas than aerial surveys do. To be useful, harvest statistics would need to have a close and known relationship with either population size or harvest rate. We assessed this using longterm (11-22 years) data for three kangaroo species (Macropus rufus, M. giganteus and M. fuliginosus) and common wallaroos (M. robustus) across South Australia, New South Wales and Queensland. Regional variation in kangaroo body size, population composition, shooter efficiency and selectivity required separate analyses in different regions. Two approaches were taken. First, monthly harvest statistics were modelled as a function of a number of explanatory variables, including kangaroo density, harvest rate and rainfall. Second, density and harvest rate were modelled as a function of harvest statistics. Both approaches incorporated a correlated error structure. Many but not all regions had relationships with sufficient precision to be useful for indirect monitoring. However, there was no single relationship that could be applied across an entire state or across species. Combined with rainfall-driven population models and applied at a regional level, these relationships could be used to reduce the frequency of aerial surveys without compromising decisions about harvest management.

205-14 A Monograph of the plant-infecting members of the Order Clavicipitales in

Taxonomic revision of ergots and related fungi in Australia.

Predicting Additive and Non-additive Genetic Effects from Trials Where Traits Are Affected by Interplot Competition

There are two key types of selection in a plant breeding program, namely selection of hybrids for potential commercial use and the selection of parents for use in future breeding. Oakey et al. (in Theoretical and Applied Genetics 113, 809-819, 2006) showed how both of these aims could be achieved using pedigree information in a mixed model analysis in order to partition genetic effects into additive and non-additive effects. Their approach was developed for field trial data subject to spatial variation. In this paper we extend the approach for data from trials subject to interplot competition. We show how the approach may be used to obtain predictions of pure stand additive and non-additive effects. We develop the methodology in the context of a single field trial using an example from an Australian sorghum breeding program.

An overview on the role of Hexanchiformes in marine ecosystems: biology, ecology and conservation status of a primitive order of modern sharks

The large size, high trophic level and wide distribution of Hexanchiformes (cow and frilled sharks) should position this order as important apex predators in coastal and deep-water ecosystems. This review synthesizes available information on Hexanchiformes, including information not yet published, with the purpose of evaluating their conservation status and assessing their ecological roles in the dynamics of marine ecosystems. Comprising six species, this group has a wide global distribution, with members occurring from shallow coastal areas to depths of c. 2500 m. The limited information available on their reproductive biology suggests that they could be vulnerable to overexploitation (e.g. small litter sizes for most species and suspected long gestation periods). Most of the fishing pressure exerted on Hexanchiformes is in the form of commercial by-catch or recreational fishing. Comprehensive stock and impact assessments are unavailable for most species in most regions due to limited information on life history and catch and abundance time series. When hexanchiform species have been commercially harvested, however, they have been unable to sustain targeted fisheries for long periods. The potentially high vulnerability to intense fishing pressure warrants a conservative exploitation of this order until thorough quantitative assessments are conducted. At least some species have been shown to be significant apex predators in the systems they inhabit. Should Hexanchiformes be removed from coastal and deep-water systems, the lack of sympatric shark species that share the same resources suggests no other species would be capable of fulfilling their apex predator role in the short term. This has potential ecosystem consequences such as meso-predator release or trophic cascades. This review proposes some hypotheses on the ecology of Hexanchiformes and their role in ecosystem dynamics, highlighting the areas where critical information is required to stimulate research directions.

High-throughput prediction of eucalypt lignin syringyl/guaiacyl content using multivariate analysis: a comparison between mid-infrared, near-infrared, and Raman spectroscopies for model development

BACKGROUND: In order to rapidly and efficiently screen potential biofuel feedstock candidates for quintessential traits, robust high-throughput analytical techniques must be developed and honed. The traditional methods of measuring lignin syringyl/guaiacyl (S/G) ratio can be laborious, involve hazardous reagents, and/or be destructive. Vibrational spectroscopy can furnish high-throughput instrumentation without the limitations of the traditional techniques. Spectral data from mid-infrared, near-infrared, and Raman spectroscopies was combined with S/G ratios, obtained using pyrolysis molecular beam mass spectrometry, from 245 different eucalypt and Acacia trees across 17 species. Iterations of spectral processing allowed the assembly of robust predictive models using partial least squares (PLS). RESULTS: The PLS models were rigorously evaluated using three different randomly generated calibration and validation sets for each spectral processing approach. Root mean standard errors of prediction for validation sets were lowest for models comprised of Raman (0.13 to 0.16) and mid-infrared (0.13 to 0.15) spectral data, while near-infrared spectroscopy led to more erroneous predictions (0.18 to 0.21). Correlation coefficients (r) for the validation sets followed a similar pattern: Raman (0.89 to 0.91), mid-infrared (0.87 to 0.91), and near-infrared (0.79 to 0.82). These statistics signify that Raman and mid-infrared spectroscopy led to the most accurate predictions of S/G ratio in a diverse consortium of feedstocks. CONCLUSION: Eucalypts present an attractive option for biofuel and biochemical production. Given the assortment of over 900 different species of Eucalyptus and Corymbia, in addition to various species of Acacia, it is necessary to isolate those possessing ideal biofuel traits. This research has demonstrated the validity of vibrational spectroscopy to efficiently partition different potential biofuel feedstocks according to lignin S/G ratio, significantly reducing experiment and analysis time and expense while providing non-destructive, accurate, global, predictive models encompassing a diverse array of feedstocks.

Evaluating and validating abundance monitoring methods in the absence of populations of known size: review and application to a passive tracking index

Rarely is it possible to obtain absolute numbers in free-ranging populations and although various direct and indirect methods are used to estimate abundance, few are validated against populations of known size. In this paper, we apply grounding, calibration and verification methods, used to validate mathematical models, to methods of estimating relative abundance. To illustrate how this might be done, we consider and evaluate the widely applied passive tracking index (PTI) methodology. Using published data, we examine the rationality of PTI methodology, how conceptually animal activity and abundance are related and how alternative methods are subject to similar biases or produce similar abundance estimates and trends. We then attune the method against populations representing a range of densities likely to be encountered in the field. Finally, we compare PTI trends against a prediction that adjacent populations of the same species will have similar abundance values and trends in activity. We show that while PTI abundance estimates are subject to environmental and behavioural stochasticity peculiar to each species, the PTI method and associated variance estimate showed high probability of detection, high precision of abundance values and, generally, low variability between surveys, and suggest that the PTI method applied using this procedure and for these species provides a sensitive and credible index of abundance. This same or similar validation approach can and should be applied to alternative relative abundance methods in order to demonstrate their credibility and justify their use.

Queensland shark control program non-target statistics by year

Spreadsheet of non-target species (bycatch) numbers in the Shark Control Program by species, date of capture, location, size and sex from 2001 onwards The shark control program (SCP) relies on nets or drumlines, or a combination of both, to minimise the threat of shark attack on humans in particular locations. Following is information on numbers and locations of sharks that have been caught by the SCP. It is important to reduce the inadvertent impacts of the SCP on other marine animals (bycatch) without compromising human safety. Bycatch levels are carefully monitored and research is focused on minimising impacts on non-target species. This dataset contains details of non-target numbers in the Shark Control program by species, date of capture, and location from 2001