Inferential, non-parametric statistics to assess quality of probabilistic forecast systems

Many statistical forecast systems are available to interested users. In order to be useful for decision-making, these systems must be based on evidence of underlying mechanisms. Once causal connections between the mechanism and their statistical manifestation have been firmly established, the forecasts must also provide some quantitative evidence of `quality’. However, the quality of statistical climate forecast systems (forecast quality) is an ill-defined and frequently misunderstood property. Often, providers and users of such forecast systems are unclear about what ‘quality’ entails and how to measure it, leading to confusion and misinformation. Here we present a generic framework to quantify aspects of forecast quality using an inferential approach to calculate nominal significance levels (p-values) that can be obtained either by directly applying non-parametric statistical tests such as Kruskal-Wallis (KW) or Kolmogorov-Smirnov (KS) or by using Monte-Carlo methods (in the case of forecast skill scores). Once converted to p-values, these forecast quality measures provide a means to objectively evaluate and compare temporal and spatial patterns of forecast quality across datasets and forecast systems. Our analysis demonstrates the importance of providing p-values rather than adopting some arbitrarily chosen significance levels such as p < 0.05 or p < 0.01, which is still common practice. This is illustrated by applying non-parametric tests (such as KW and KS) and skill scoring methods (LEPS and RPSS) to the 5-phase Southern Oscillation Index classification system using historical rainfall data from Australia, The Republic of South Africa and India. The selection of quality measures is solely based on their common use and does not constitute endorsement. We found that non-parametric statistical tests can be adequate proxies for skill measures such as LEPS or RPSS. The framework can be implemented anywhere, regardless of dataset, forecast system or quality measure. Eventually such inferential evidence should be complimented by descriptive statistical methods in order to fully assist in operational risk management.

Stakeholders' assessment of welfare indicators for sheep and cattle exported by sea from Australia

An adaptive conjoint analysis was use to evaluate stakeholders' opinion of welfare indicators for ship-transported sheep and cattle, both onboard and in pre-export depots. In consultations with two nominees of each identified stakeholder group (government officials, animal welfare representatives, animal scientists, stockpersons, producers/pre-export depot operators, exporters/ship owners and veterinarians), 18 potential indicators were identified Three levels were assigned to each using industry statistics and expert opinion, representing those observed on the best and worst 5% of voyages and an intermediate value. A computer-based questionnaire was completed by 135 stakeholders (48% of those invited). All indicators were ranked by respondents in the assigned order, except fodder intake, in which case providing the amount necessary to maintain bodyweight was rated better than over or underfeeding, and time in the pre-export assembly depot, in which case 5 days was rated better than 0 or 10 days. The respective Importance Values (a relative rating given by the respondent) for each indicator were, in order of declining importance: mortality (8.6%), clinical disease incidence (8.2%), respiration rate (6.8%), space allowance (6.2%), ammonia levels (6.1%), weight change (6.0%), wet bulb temperature (6.0%), time in assembly depot (5.4%), percentage of animals in hospital pen (5.4%), fodder intake (5.2%), stress-related metabolites (5.0%), percentage of feeding trough utilised (5.0%), injuries (4.8%), percentage of animals able to access food troughs at any one time (4.8%), percentage of animals lying down (4.7%), cortisol concentration (4.5Y.), noise (3.9y.), and photoperiod (3.4%). The different stakeholder groups were relatively consistent in their ranking of the indicators, with all groups nominating the some top two and at least five of the top seven indicators. Some of the top indicators, in particular mortality, disease incidence and temperature, are already recorded in the Australian industry, but the study identified potential new welfare indicators for exported livestock, such as space allowance and ammonia concentration, which could be used to improve welfare standards if validated by scientific data. The top indicators would also be useful worldwide for countries engaging in long distance sea transport of livestock.

From inferential statistics to climate knowledge

Climate variability and change are risk factors for climate sensitive activities such as agriculture. Managing these risks requires "climate knowledge", i.e. a sound understanding of causes and consequences of climate variability and knowledge of potential management options that are suitable in light of the climatic risks posed. Often such information about prognostic variables (e.g. yield, rainfall, run-off) is provided in probabilistic terms (e.g. via cumulative distribution functions, CDF), whereby the quantitative assessments of these alternative management options is based on such CDFs. Sound statistical approaches are needed in order to assess whether difference between such CDFs are intrinsic features of systems dynamics or chance events (i.e. quantifying evidences against an appropriate null hypothesis). Statistical procedures that rely on such a hypothesis testing framework are referred to as "inferential statistics" in contrast to descriptive statistics (e.g. mean, median, variance of population samples, skill scores). Here we report on the extension of some of the existing inferential techniques that provides more relevant and adequate information for decision making under uncertainty.

Using harvest statistics to monitor temporal variation in kangaroo density and harvest rate

Management of the commercial harvest of kangaroos relies on quotas set annually as a proportion of regular estimates of population size. Surveys to generate these estimates are expensive and, in the larger states, logistically difficult; a cheaper alternative is desirable. Rainfall is a disappointingly poor predictor of kangaroo rate of increase in many areas, but harvest statistics (sex ratio, carcass weight, skin size and animals shot per unit time) potentially offer cost-effective indirect monitoring of population abundance (and therefore trend) and status (i.e. under-or overharvest). Furthermore, because harvest data are collected continuously and throughout the harvested areas, they offer the promise of more intensive and more representative coverage of harvest areas than aerial surveys do. To be useful, harvest statistics would need to have a close and known relationship with either population size or harvest rate. We assessed this using longterm (11-22 years) data for three kangaroo species (Macropus rufus, M. giganteus and M. fuliginosus) and common wallaroos (M. robustus) across South Australia, New South Wales and Queensland. Regional variation in kangaroo body size, population composition, shooter efficiency and selectivity required separate analyses in different regions. Two approaches were taken. First, monthly harvest statistics were modelled as a function of a number of explanatory variables, including kangaroo density, harvest rate and rainfall. Second, density and harvest rate were modelled as a function of harvest statistics. Both approaches incorporated a correlated error structure. Many but not all regions had relationships with sufficient precision to be useful for indirect monitoring. However, there was no single relationship that could be applied across an entire state or across species. Combined with rainfall-driven population models and applied at a regional level, these relationships could be used to reduce the frequency of aerial surveys without compromising decisions about harvest management.

A Practitioner-Oriented Decision Support Process for Forestry Pest Management

The article discusses a new decision support process for forestry pest management. Over the past few years, DSS have been introduced for forestry pest management, providing forest growers with advice in areas such as selecting the most suitable pesticide and relevant treatment. Most of the initiatives process knowledge from various domains for providing support for specific decision making problems. However, very few studies have identified the requirements of developing a combined process model in which all relevant practitioners can contribute and share knowledge for effective decision making; such an approach would need to include the decision makers’ perspective along with other relevant attributes such as the problem context and relevant policies. We outline a decision support process for forestry pest management, based on the design science research paradigm, in which a focus group technique has application to acquire both expert and practical knowledge in order to construct the DSS solution.

Fishing power increases continue in Queensland's east coast trawl fishery, Australia

The Queensland east coast trawl fishery is by far the largest prawn and scallop otter trawl fleet in Australia in terms of number of vessels, with 504 vessels licensed to fish for species including tiger prawns, endeavour prawns, red spot king prawns, eastern king prawns and saucer scallops by the end of 2004. The vessel fleet has gradually upgraded characteristics such as engine power and use of propeller nozzles, quad nets, global positioning systems (GPS) and computer mapping software. These changes, together with the ever-changing profile of the fleet, were analysed by linear mixed models to quantify annual efficiency increases of an average vessel at catching prawns or scallops. The analyses included vessel characteristics (treated as fixed effects) and vessel identifier codes (treated as random effects). For the period from 1989 to 2004 the models estimated overall fishing power increases of 6% in the northern tiger, 6% in the northern endeavour, 12% in the southern tiger, 18% in the red spot king, 46% in the eastern king prawn and 15% in the saucer scallop sector. The results illustrate the importance of ongoing monitoring of vessel and fleet characteristics and the need to use this information to standardise catch rate indices used in stock assessment and management.

Queensland shark control program non-target statistics by year

Spreadsheet of non-target species (bycatch) numbers in the Shark Control Program by species, date of capture, location, size and sex from 2001 onwards The shark control program (SCP) relies on nets or drumlines, or a combination of both, to minimise the threat of shark attack on humans in particular locations. Following is information on numbers and locations of sharks that have been caught by the SCP. It is important to reduce the inadvertent impacts of the SCP on other marine animals (bycatch) without compromising human safety. Bycatch levels are carefully monitored and research is focused on minimising impacts on non-target species. This dataset contains details of non-target numbers in the Shark Control program by species, date of capture, and location from 2001