Inferential, non-parametric statistics to assess quality of probabilistic forecast systems

Many statistical forecast systems are available to interested users. In order to be useful for decision-making, these systems must be based on evidence of underlying mechanisms. Once causal connections between the mechanism and their statistical manifestation have been firmly established, the forecasts must also provide some quantitative evidence of `quality’. However, the quality of statistical climate forecast systems (forecast quality) is an ill-defined and frequently misunderstood property. Often, providers and users of such forecast systems are unclear about what ‘quality’ entails and how to measure it, leading to confusion and misinformation. Here we present a generic framework to quantify aspects of forecast quality using an inferential approach to calculate nominal significance levels (p-values) that can be obtained either by directly applying non-parametric statistical tests such as Kruskal-Wallis (KW) or Kolmogorov-Smirnov (KS) or by using Monte-Carlo methods (in the case of forecast skill scores). Once converted to p-values, these forecast quality measures provide a means to objectively evaluate and compare temporal and spatial patterns of forecast quality across datasets and forecast systems. Our analysis demonstrates the importance of providing p-values rather than adopting some arbitrarily chosen significance levels such as p < 0.05 or p < 0.01, which is still common practice. This is illustrated by applying non-parametric tests (such as KW and KS) and skill scoring methods (LEPS and RPSS) to the 5-phase Southern Oscillation Index classification system using historical rainfall data from Australia, The Republic of South Africa and India. The selection of quality measures is solely based on their common use and does not constitute endorsement. We found that non-parametric statistical tests can be adequate proxies for skill measures such as LEPS or RPSS. The framework can be implemented anywhere, regardless of dataset, forecast system or quality measure. Eventually such inferential evidence should be complimented by descriptive statistical methods in order to fully assist in operational risk management.

From inferential statistics to climate knowledge

Climate variability and change are risk factors for climate sensitive activities such as agriculture. Managing these risks requires "climate knowledge", i.e. a sound understanding of causes and consequences of climate variability and knowledge of potential management options that are suitable in light of the climatic risks posed. Often such information about prognostic variables (e.g. yield, rainfall, run-off) is provided in probabilistic terms (e.g. via cumulative distribution functions, CDF), whereby the quantitative assessments of these alternative management options is based on such CDFs. Sound statistical approaches are needed in order to assess whether difference between such CDFs are intrinsic features of systems dynamics or chance events (i.e. quantifying evidences against an appropriate null hypothesis). Statistical procedures that rely on such a hypothesis testing framework are referred to as "inferential statistics" in contrast to descriptive statistics (e.g. mean, median, variance of population samples, skill scores). Here we report on the extension of some of the existing inferential techniques that provides more relevant and adequate information for decision making under uncertainty.

Using harvest statistics to monitor temporal variation in kangaroo density and harvest rate

Management of the commercial harvest of kangaroos relies on quotas set annually as a proportion of regular estimates of population size. Surveys to generate these estimates are expensive and, in the larger states, logistically difficult; a cheaper alternative is desirable. Rainfall is a disappointingly poor predictor of kangaroo rate of increase in many areas, but harvest statistics (sex ratio, carcass weight, skin size and animals shot per unit time) potentially offer cost-effective indirect monitoring of population abundance (and therefore trend) and status (i.e. under-or overharvest). Furthermore, because harvest data are collected continuously and throughout the harvested areas, they offer the promise of more intensive and more representative coverage of harvest areas than aerial surveys do. To be useful, harvest statistics would need to have a close and known relationship with either population size or harvest rate. We assessed this using longterm (11-22 years) data for three kangaroo species (Macropus rufus, M. giganteus and M. fuliginosus) and common wallaroos (M. robustus) across South Australia, New South Wales and Queensland. Regional variation in kangaroo body size, population composition, shooter efficiency and selectivity required separate analyses in different regions. Two approaches were taken. First, monthly harvest statistics were modelled as a function of a number of explanatory variables, including kangaroo density, harvest rate and rainfall. Second, density and harvest rate were modelled as a function of harvest statistics. Both approaches incorporated a correlated error structure. Many but not all regions had relationships with sufficient precision to be useful for indirect monitoring. However, there was no single relationship that could be applied across an entire state or across species. Combined with rainfall-driven population models and applied at a regional level, these relationships could be used to reduce the frequency of aerial surveys without compromising decisions about harvest management.

Technical highlights: Invasive plant and animal research 2009-10

Technical highlights 2009–10, with detailed progress reports on the latest invasive plant and animal research undertaken by Biosecurity Queensland, a service unit of the Department of Employment, Economic Development and Innovation.

Technical highlights: Invasive plant and animal research 2008-09

Technical highlights 2008–09, with detailed progress reports on the latest invasive plant and animal research undertaken by Biosecurity Queensland, a service unit of the Department of Employment, Economic Development and Innovation.

Technical highlights: Invasive plant and animal research 2007-08

Technical highlights 2007–08, with detailed progress reports on the latest invasive plant and animal research undertaken by Biosecurity Queensland, a service unit of the Department of Primary Industries and Fisheries.

MOE and MOR assessment technologies for improving graded recovery of exotic pines in Australia.

This project was designed to provide the structural softwood processing industry with the basis for improved green and dry grading to allow maximise MGP grade yields, consistent product performance and reduced processing costs. To achieve this, advanced statistical techniques were used in conjunction with state-of-the-art property measurement systems. Specifically, the project aimed to make two significant steps forward for the Australian structural softwood industry: • assessment of technologies, both existing and novel, that may lead to selection of a consistent, reliable and accurate device for the log yard and green mill. The purpose is to more accurately identify and reject material that will not make a minimum grade of MGP10 downstream; • improved correlation of grading MOE and MOR parameters in the dry mill using new analytical methods and a combination of devices. The three populations tested were stiffness-limited radiata pine, strength-limited radiata pine and Caribbean pine. Resonance tests were conducted on logs prior to sawmilling, and on boards. Raw data from existing in-line systems were captured for the green and dry boards. The dataset was analysed using classical and advanced statistical tools to provide correlations between data sets and to develop efficient strength and stiffness prediction equations. Stiffness and strength prediction algorithms were developed from raw and combined parameters. Parameters were analysed for comparison of prediction capabilities using in-line parameters, off-line parameters and a combination of in-line and off-line parameters. The results show that acoustic resonance techniques have potential for log assessment, to sort for low stiffness and/or low strength, depending on the resource. From the log measurements, a strong correlation was found between the average static MOE of the dried boards within a log and the predicted value. These results have application in segregating logs into structural and non-structural uses. Some commercial technologies are already available for this application such as Hitman LG640. For green boards it was found that in-line and laboratory acoustic devices can provide a good prediction of dry static MOE and moderate prediction for MOR.There is high potential for segregating boards at this stage of processing. Grading after the log breakdown can improve significantly the effectiveness of the mill. Subsequently, reductions in non-structural volumes can be achieved. Depending on the resource it can be expected that a 5 to 8 % reduction in non structural boards won’t be dried with an associated saving of $70 to 85/m3. For dry boards, vibration and a standard Metriguard CLT/HCLT provided a similar level of prediction on stiffness limited resource. However, Metriguard provides a better strength prediction in strength limited resources (due to this equipment’s ability to measure local characteristics). The combination of grading equipment specifically for stiffness related predictors (Metriguard or vibration) with defect detection systems (optical or X-ray scanner) provides a higher level of prediction, especially for MOR. Several commercial technologies are already available for acoustic grading on board such those from Microtec, Luxscan, Falcon engineering or Dynalyse AB for example. Differing combinations of equipment, and their strategic location within the processing chain, can dramatically improve the efficiency of the mill, the level of which will vary depending of the resource. For example, an initial acoustic sorting on green boards combined with an optical scanner associated with an acoustic system for grading dry board can result in a large reduction of the proportion of low value low non-structural produced. The application of classical MLR on several predictors proved to be effective, in particular for MOR predictions. However, the usage of a modern statistics approach(chemometrics tools) such as PLS proved to be more efficient for improving the level of prediction. Compared to existing technologies, the results of the project indicate a good improvement potential for grading in the green mill, ahead of kiln drying and subsequent cost-adding processes. The next stage is the development and refinement of systems for this purpose.

Wood properties and processing outcomes for plantation grown African mahogany (Khaya senegalensis) trees from Clare, Queensland (18 and 20 year-old trees) and Katherine, Northern Territory (14 year-old trees).

Khaya senegalensis, African mahogany, a high-value hardwood, was introduced in the Northern Territory (NT) in the 1950s; included in various trials there and at Weipa, Q in the 1960s-1970s; planted on ex mine sites at Weipa (160 ha) until 1985; revived in farm plantings in Queensland and in trials in the NT in the 1990s; adopted for large-scale, annual planting in the Douglas-Daly region, NT from 2006 and is to have the planted area in the NT extended to at least 20,000 ha. The recent serious interest from plantation growers, including Forest Enterprises Australia Ltd (FEA), has seen the establishment of some large scale commercial plantations. FEA initiated the current study to process relatively young plantation stands from both Northern Territory and Queensland plantations to investigate the sawn wood and veneer recovery and quality from trees ranging from 14 years (NT – 36 trees) to 18-20 years (North Queensland – 31 trees). Field measures of tree size and straightness were complemented with log end splitting assessment and cross-sectional disc sample collection for laboratory wood properties measurements including colour and shrinkage. End-splitting scores assessed on sawn logs were relatively low compared to fast grown plantation eucalypts and did not impact processing negatively. Heartwood proportion in individual trees ranged from 50% up to 92 % of butt cross-sectional disc area for the visually-assessed dark coloured central heartwood and lighter coloured transition wood combined. Dark central heartwood proportion was positively related to tree size (R2 = 0.57). Chemical tests failed to assist in determining heartwood – sapwood boundary. Mean basic density of whole disc samples was 658 kg/m3 and ranged among trees from 603 to 712 kg/m3. When freshly sawn, the heartwood of African mahogany was orange-red to red. Transition wood appeared to be pinkish and the sapwood was a pale yellow colour. Once air dried the heartwood colour generally darkens to pinkish-brown or orange-brown and the effect of prolonged time and sun exposure is to darken and change the heartwood to a red-brown colour. A portable colour measurement spectrophotometer was used to objectively assess colour variation in CIE L*, a* and b* values over time with drying and exposure to sunlight. Capacity to predict standard colour values accurately after varying periods of direct sunlight exposure using results obtained on initial air-dried surfaces decreased with increasing time to sun exposure. The predictions are more accurate for L* values which represent brightness than for variation in the a* values (red spectrum). Selection of superior breeding trees for colour is likely to be based on dried samples exposed to sunlight to reliably highlight wood colour differences. A generally low ratio between tangential and radial shrinkages was found, which was reflected in a low incidence of board distortion (particularly cupping) during drying. A preliminary experiment was carried out to investigate the quality of NIR models to predict shrinkage and density. NIR spectra correlated reasonably well with radial shrinkage and air dried density. When calibration models were applied to their validation sets, radial shrinkage was predicted to an accuracy of 76% with Standard Error of Prediction of 0.21%. There was also a strong predictive power for wood density. These are encouraging results suggesting that NIR spectroscopy has good potential to be used as a non-destructive method to predict shrinkage and wood density using 12mm diameter increment core samples. Average green off saw recovery was 49.5% (range 40 to 69%) for Burdekin Agricultural College (BAC) logs and 41.9% (range 20 to 61%) for Katherine (NT) logs. These figures are about 10% higher than compared to 30-year-old Khaya study by Armstrong et al. (2007) however they are inflated as the green boards were not docked to remove wane prior to being tallied. Of the recovered sawn, dried and dressed volume from the BAC logs, based on the cambial face of boards, 27% could potentially be used for select grade, 40% for medium feature grade and 26% for high feature grades. The heart faces had a slightly higher recovery of select (30%) and medium feature (43%) grade boards with a reduction in the volume of high feature (22%) and reject (6%) grade boards. Distribution of board grades for the NT site aged 14 years followed very similar trends to those of the BAC site boards with an average (between facial and cambial face) 27% could potentially be used for select grade, 42% for medium feature grade, 26% for high feature grade and 5% reject. Relatively to some other subtropical eucalypts, there was a low incidence of borer attack. The major grade limiting defects for both medium and high feature grade boards recovered from the BAC site were knots and wane. The presence of large knots may reflect both management practices and the nature of the genetic material at the site. This stand was not managed for timber production with a very late pruning implemented at about age 12 years. The large amount of wane affected boards is indicative of logs with a large taper and the presence of significant sweep. Wane, knots and skip were the major grade limiting defects for the NT site reflecting considerable amounts of sweep with large taper as might be expected in younger trees. The green veneer recovered from billets of seven Khaya trees rotary peeled on a spindleless lathe produced a recovery of 83% of green billet volume. Dried veneer recovery ranged from 40 to 74 % per billet with an average of 64%. All of the recovered grades were suitable for use in structural ply in accordance to AS/NZ 2269: 2008. The majority of veneer sheets recovered from all billets was C grade (27%) with 20% making D grade and 13% B grade. Total dry sliced veneer recovery from the logs of the two largest logs from each location was estimated to be 41.1%. Very positive results have been recorded in this small scale study. The amount of colour development observed and the very reasonable recoveries of both sawn and veneer products, with a good representation of higher grades in the product distribution, is encouraging. The prospects for significant improvement in these results from well managed and productive stands grown for high quality timber should be high. Additionally, the study has shown the utility of non-destructive evaluation techniques for use in tree improvement programs to improve the quality of future plantations. A few trees combined several of the traits desired of individuals for a first breeding population. Fortunately, the two most promising trees (32, 19) had already been selected for breeding on external traits, and grafts of them are established in the seed orchard.

Sustainable Grazing in the Channel Country Floodplains (phase 2) : A technical report on findings between March 2003 and June 2006

‘Sustainable Grazing in the Channel Country Floodplains’ was initiated by industry to redress the lack of objective information for sustainable management in the floodplains of Cooper Creek and the Diamantina and Georgina Rivers. The project has maintained links with the grazing community and has extensively drawn upon expert local experience and knowledge. The project has provided tools for managers to better anticipate the size of beneficial flooding arising from rains in the upper catchment and to more objectively assess the value of the pasture resulting from flooding. The latest information from the project has enabled customisation of the EDGENetwork™ Grazing Land Management training package for the Channel Country. In combination, these tools will assist in making earlier cattle stocking decisions, including when cattle may need to be mustered out of floodplain paddocks, how many additional cattle will be required to take advantage of the flood–grown pasture, and the timing of cattle turnoff. These will reduce costs by providing a greater lead time to plan cattle movements and purchases, and may enhance the sustainability of the resource base by better matching cattle numbers with the feed on offer.

Value chain technical development

Short-term research projects on insect pest management in processed food.

Technical Advice Tully Cane Demo Farm Sites

Provision of technical advice for Tully cane demo farm sites as part of the Paddock-scale monitoring in the Wet Tropics Natural Resource Management region.

Technical support improved Best Management Practices in irrigated grains farming systems in the Fiztroy Basin Association priority catchments

Broadscale irrigation is a major land use in many of the priority neighbourhood catchments (45,218 hectares in Central Highlands and Dawson) and there is a requirement to provide technical support to sub-regional group field officers and landholders in these priority catchments. This technical support will assist field staff and land managers to identify and implement appropriate, sustainable technologies and management practices.

Queensland shark control program non-target statistics by year

Spreadsheet of non-target species (bycatch) numbers in the Shark Control Program by species, date of capture, location, size and sex from 2001 onwards The shark control program (SCP) relies on nets or drumlines, or a combination of both, to minimise the threat of shark attack on humans in particular locations. Following is information on numbers and locations of sharks that have been caught by the SCP. It is important to reduce the inadvertent impacts of the SCP on other marine animals (bycatch) without compromising human safety. Bycatch levels are carefully monitored and research is focused on minimising impacts on non-target species. This dataset contains details of non-target numbers in the Shark Control program by species, date of capture, and location from 2001