Clinical indicators of quality for Australian residential aged care facilities : establishing reliability, validity, and quality thresholds

Background: In response to the need for more comprehensive quality assessment within Australian residential aged care facilities, the Clinical Care Indicator (CCI) Tool was developed to collect outcome data as a means of making inferences about quality. A national trial of its effectiveness and a Brisbane-based trial of its use within the quality improvement context determined the CCI Tool represented a potentially valuable addition to the Australian aged care system. This document describes the next phase in the CCI Tool.s development; the aims of which were to establish validity and reliability of the CCI Tool, and to develop quality indicator thresholds (benchmarks) for use in Australia. The CCI Tool is now known as the ResCareQA (Residential Care Quality Assessment). Methods: The study aims were achieved through a combination of quantitative data analysis, and expert panel consultations using modified Delphi process. The expert panel consisted of experienced aged care clinicians, managers, and academics; they were initially consulted to determine face and content validity of the ResCareQA, and later to develop thresholds of quality. To analyse its psychometric properties, ResCareQA forms were completed for all residents (N=498) of nine aged care facilities throughout Queensland. Kappa statistics were used to assess inter-rater and test-retest reliability, and Cronbach.s alpha coefficient calculated to determine internal consistency. For concurrent validity, equivalent items on the ResCareQA and the Resident Classification Scales (RCS) were compared using Spearman.s rank order correlations, while discriminative validity was assessed using known-groups technique, comparing ResCareQA results between groups with differing care needs, as well as between male and female residents. Rank-ordered facility results for each clinical care indicator (CCI) were circulated to the panel; upper and lower thresholds for each CCI were nominated by panel members and refined through a Delphi process. These thresholds indicate excellent care at one extreme and questionable care at the other. Results: Minor modifications were made to the assessment, and it was renamed the ResCareQA. Agreement on its content was reached after two Delphi rounds; the final version contains 24 questions across four domains, enabling generation of 36 CCIs. Both test-retest and inter-rater reliability were sound with median kappa values of 0.74 (test-retest) and 0.91 (inter-rater); internal consistency was not as strong, with a Chronbach.s alpha of 0.46. Because the ResCareQA does not provide a single combined score, comparisons for concurrent validity were made with the RCS on an item by item basis, with most resultant correlations being quite low. Discriminative validity analyses, however, revealed highly significant differences in total number of CCIs between high care and low care groups (t199=10.77, p=0.000), while the differences between male and female residents were not significant (t414=0.56, p=0.58). Clinical outcomes varied both within and between facilities; agreed upper and lower thresholds were finalised after three Delphi rounds. Conclusions: The ResCareQA provides a comprehensive, easily administered means of monitoring quality in residential aged care facilities that can be reliably used on multiple occasions. The relatively modest internal consistency score was likely due to the multi-factorial nature of quality, and the absence of an aggregate result for the assessment. Measurement of concurrent validity proved difficult in the absence of a gold standard, but the sound discriminative validity results suggest that the ResCareQA has acceptable validity and could be confidently used as an indication of care quality within Australian residential aged care facilities. The thresholds, while preliminary due to small sample size, enable users to make judgements about quality within and between facilities. Thus it is recommended the ResCareQA be adopted for wider use.

Bayesian network for risk of diarrhea associated with the use of recycled water

Estimating potential health risks associated with recycled (reused) water is highly complex given the multiple factors affecting water quality. We take a conceptual model, which represents the factors and pathways by which recycled water may pose a risk of contracting gastroenteritis, convert the conceptual model to a Bayesian net, and quantify the model using one expert’s opinion. This allows us to make various predictions as to the risks posed under various scenarios. Bayesian nets provide an additional way of modeling the determinants of recycled water quality and elucidating their relative influence on a given disease outcome. The important contribution to Bayesian net methodology is that all model predictions, whether risk or relative risk estimates, are expressed as credible intervals.

Improved design method for biosecurity surveillance and early detection of non-indigenous rats

A recent advance in biosecurity surveillance design aims to benefit island conservation through early and improved detection of incursions by non-indigenous species. The novel aspects of the design are that it achieves a specified power of detection in a cost-managed system, while acknowledging heterogeneity of risk in the study area and stratifying the area to target surveillance deployment. The design also utilises a variety of surveillance system components, such as formal scientific surveys, trapping methods, and incidental sightings by non-biologist observers. These advances in design were applied to black rats (Rattus rattus) representing the group of invasive rats including R. norvegicus, and R. exulans, which are potential threats to Barrow Island, Australia, a high value conservation nature reserve where a proposed liquefied natural gas development is a potential source of incursions. Rats are important to consider as they are prevalent invaders worldwide, difficult to detect early when present in low numbers, and able to spread and establish relatively quickly after arrival. The ‘exemplar’ design for the black rat is then applied in a manner that enables the detection of a range of non-indigenous species of rat that could potentially be introduced. Many of the design decisions were based on expert opinion as data gaps exist in empirical data. The surveillance system was able to take into account factors such as collateral effects on native species, the availability of limited resources on an offshore island, financial costs, demands on expertise and other logistical constraints. We demonstrate the flexibility and robustness of the surveillance system and discuss how it could be updated as empirical data are collected to supplement expert opinion and provide a basis for adaptive management. Overall, the surveillance system promotes an efficient use of resources while providing defined power to detect early rat incursions, translating to reduced environmental, resourcing and financial costs.

Hierarchical Bayesian models for estimating the extent of plant pest invasions

Plant biosecurity requires statistical tools to interpret field surveillance data in order to manage pest incursions that threaten crop production and trade. Ultimately, management decisions need to be based on the probability that an area is infested or free of a pest. Current informal approaches to delimiting pest extent rely upon expert ecological interpretation of presence / absence data over space and time. Hierarchical Bayesian models provide a cohesive statistical framework that can formally integrate the available information on both pest ecology and data. The overarching method involves constructing an observation model for the surveillance data, conditional on the hidden extent of the pest and uncertain detection sensitivity. The extent of the pest is then modelled as a dynamic invasion process that includes uncertainty in ecological parameters. Modelling approaches to assimilate this information are explored through case studies on spiralling whitefly, Aleurodicus dispersus and red banded mango caterpillar, Deanolis sublimbalis. Markov chain Monte Carlo simulation is used to estimate the probable extent of pests, given the observation and process model conditioned by surveillance data. Statistical methods, based on time-to-event models, are developed to apply hierarchical Bayesian models to early detection programs and to demonstrate area freedom from pests. The value of early detection surveillance programs is demonstrated through an application to interpret surveillance data for exotic plant pests with uncertain spread rates. The model suggests that typical early detection programs provide a moderate reduction in the probability of an area being infested but a dramatic reduction in the expected area of incursions at a given time. Estimates of spiralling whitefly extent are examined at local, district and state-wide scales. The local model estimates the rate of natural spread and the influence of host architecture, host suitability and inspector efficiency. These parameter estimates can support the development of robust surveillance programs. Hierarchical Bayesian models for the human-mediated spread of spiralling whitefly are developed for the colonisation of discrete cells connected by a modified gravity model. By estimating dispersal parameters, the model can be used to predict the extent of the pest over time. An extended model predicts the climate restricted distribution of the pest in Queensland. These novel human-mediated movement models are well suited to demonstrating area freedom at coarse spatio-temporal scales. At finer scales, and in the presence of ecological complexity, exploratory models are developed to investigate the capacity for surveillance information to estimate the extent of red banded mango caterpillar. It is apparent that excessive uncertainty about observation and ecological parameters can impose limits on inference at the scales required for effective management of response programs. The thesis contributes novel statistical approaches to estimating the extent of pests and develops applications to assist decision-making across a range of plant biosecurity surveillance activities. Hierarchical Bayesian modelling is demonstrated as both a useful analytical tool for estimating pest extent and a natural investigative paradigm for developing and focussing biosecurity programs.

The role of ecological constraints on expertise development

The role of ecological constraints on the acquisition of sport expertise is gaining attention in sport science, although more research is needed. In this position paper we provide an ecological explanation for expertise acquisition, as alluding to qualitative data that support the idea that unconventional, even aversive, environmental constraints may play an important role in the development of world-class athletes. We exemplify this argument by profiling the role of unconventional practice environments using association football in Brazilian society as a task vehicle. Contrary to the traditional idea that only deliberate training and development programmes can lead to the evolution of expertise, we propose how expert performance might be gained through highly unstructured activities in Brazilian football, that represent a powerful and little understood implicit environmental constraint that can lead to expertise development in sport.

An assessment system for evaluation of driving competencies

Automobiles have deeply impacted the way in which we travel but they have also contributed to many deaths and injury due to crashes. A number of reasons for these crashes have been pointed out by researchers. Inexperience has been identified as a contributing factor to road crashes. Driver’s driving abilities also play a vital role in judging the road environment and reacting in-time to avoid any possible collision. Therefore driver’s perceptual and motor skills remain the key factors impacting on road safety. Our failure to understand what is really important for learners, in terms of competent driving, is one of the many challenges for building better training programs. Driver training is one of the interventions aimed at decreasing the number of crashes that involve young drivers. Currently, there is a need to develop comprehensive driver evaluation system that benefits from the advances in Driver Assistance Systems. A multidisciplinary approach is necessary to explain how driving abilities evolves with on-road driving experience. To our knowledge, driver assistance systems have never been comprehensively used in a driver training context to assess the safety aspect of driving. The aim and novelty of this thesis is to develop and evaluate an Intelligent Driver Training System (IDTS) as an automated assessment tool that will help drivers and their trainers to comprehensively view complex driving manoeuvres and potentially provide effective feedback by post processing the data recorded during driving. This system is designed to help driver trainers to accurately evaluate driver performance and has the potential to provide valuable feedback to the drivers. Since driving is dependent on fuzzy inputs from the driver (i.e. approximate distance calculation from the other vehicles, approximate assumption of the other vehicle speed), it is necessary that the evaluation system is based on criteria and rules that handles uncertain and fuzzy characteristics of the driving tasks. Therefore, the proposed IDTS utilizes fuzzy set theory for the assessment of driver performance. The proposed research program focuses on integrating the multi-sensory information acquired from the vehicle, driver and environment to assess driving competencies. After information acquisition, the current research focuses on automated segmentation of the selected manoeuvres from the driving scenario. This leads to the creation of a model that determines a “competency” criterion through the driving performance protocol used by driver trainers (i.e. expert knowledge) to assess drivers. This is achieved by comprehensively evaluating and assessing the data stream acquired from multiple in-vehicle sensors using fuzzy rules and classifying the driving manoeuvres (i.e. overtake, lane change, T-crossing and turn) between low and high competency. The fuzzy rules use parameters such as following distance, gaze depth and scan area, distance with respect to lanes and excessive acceleration or braking during the manoeuvres to assess competency. These rules that identify driving competency were initially designed with the help of expert’s knowledge (i.e. driver trainers). In-order to fine tune these rules and the parameters that define these rules, a driving experiment was conducted to identify the empirical differences between novice and experienced drivers. The results from the driving experiment indicated that significant differences existed between novice and experienced driver, in terms of their gaze pattern and duration, speed, stop time at the T-crossing, lane keeping and the time spent in lanes while performing the selected manoeuvres. These differences were used to refine the fuzzy membership functions and rules that govern the assessments of the driving tasks. Next, this research focused on providing an integrated visual assessment interface to both driver trainers and their trainees. By providing a rich set of interactive graphical interfaces, displaying information about the driving tasks, Intelligent Driver Training System (IDTS) visualisation module has the potential to give empirical feedback to its users. Lastly, the validation of the IDTS system’s assessment was conducted by comparing IDTS objective assessments, for the driving experiment, with the subjective assessments of the driver trainers for particular manoeuvres. Results show that not only IDTS was able to match the subjective assessments made by driver trainers during the driving experiment but also identified some additional driving manoeuvres performed in low competency that were not identified by the driver trainers due to increased mental workload of trainers when assessing multiple variables that constitute driving. The validation of IDTS emphasized the need for an automated assessment tool that can segment the manoeuvres from the driving scenario, further investigate the variables within that manoeuvre to determine the manoeuvre’s competency and provide integrated visualisation regarding the manoeuvre to its users (i.e. trainers and trainees). Through analysis and validation it was shown that IDTS is a useful assistance tool for driver trainers to empirically assess and potentially provide feedback regarding the manoeuvres undertaken by the drivers.

High resolution tiled display walls

Scalable high-resolution tiled display walls are becoming increasingly important to decision makers and researchers because high pixel counts in combination with large screen areas facilitate content rich, simultaneous display of computer-generated visualization information and high-definition video data from multiple sources. This tutorial is designed to cater for new users as well as researchers who are currently operating tiled display walls or 'OptiPortals'. We will discuss the current and future applications of display wall technology and explore opportunities for participants to collaborate and contribute in a growing community. Multiple tutorial streams will cover both hands-on practical development, as well as policy and method design for embedding these technologies into the research process. Attendees will be able to gain an understanding of how to get started with developing similar systems themselves, in addition to becoming familiar with typical applications and large-scale visualisation techniques. Presentations in this tutorial will describe current implementations of tiled display walls that highlight the effective usage of screen real-estate with various visualization datasets, including collaborative applications such as visualcasting, classroom learning and video conferencing. A feature presentation for this tutorial will be given by Jurgen Schulze from Calit2 at the University of California, San Diego. Jurgen is an expert in scientific visualization in virtual environments, human-computer interaction, real-time volume rendering, and graphics algorithms on programmable graphics hardware.

Ecological aspects of biosecurity surveillance design for the detection of multiple invasive animal species

Complex surveillance problems are common in biosecurity, such as prioritizing detection among multiple invasive species, specifying risk over a heterogeneous landscape, combining multiple sources of surveillance data, designing for specified power to detect, resource management, and collateral effects on the environment. Moreover, when designing for multiple target species, inherent biological differences among species result in different ecological models underpinning the individual surveillance systems for each. Species are likely to have different habitat requirements, different introduction mechanisms and locations, require different methods of detection, have different levels of detectability, and vary in rates of movement and spread. Often there is a further challenge of a lack of knowledge, literature, or data, for any number of the above problems. Even so, governments and industry need to proceed with surveillance programs which aim to detect incursions in order to meet environmental, social and political requirements. We present an approach taken to meet these challenges in one comprehensive and statistically powerful surveillance design for non-indigenous terrestrial vertebrates on Barrow Island, a high conservation nature reserve off the Western Australian coast. Here, the possibility of incursions is increased due to construction and expanding industry on the island. The design, which includes mammals, amphibians and reptiles, provides a complete surveillance program for most potential terrestrial vertebrate invaders. Individual surveillance systems were developed for various potential invaders, and then integrated into an overall surveillance system which meets the above challenges using a statistical model and expert elicitation. We discuss the ecological basis for the design, the flexibility of the surveillance scheme, how it meets the above challenges, design limitations, and how it can be updated as data are collected as a basis for adaptive management.

The relative impact of work-related stress, life stress and driving environment stress on driving outcomes

Previous research has shown the association between stress and crash involvement. The impact of stress on road safety may also be mediated by behaviours including cognitive lapses, errors, and intentional traffic violations. This study aimed to provide a further understanding of the impact that stress from different sources may have upon driving behaviour and road safety. It is asserted that both stress extraneous to the driving environment and stress directly elicited by driving must be considered part of a dynamic system that may have a negative impact on driving behaviours. Two hundred and forty-seven public sector employees from Queensland, Australia, completed self-report measures examining demographics, subjective work-related stress, daily hassles, and aspects of general mental health. Additionally, the Driver Behaviour Questionnaire (DBQ) and the Driver Stress Inventory (DSI) were administered. All participants drove for work purposes regularly, however the study did not specifically focus on full-time professional drivers. Confirmatory factor analysis of the predictor variables revealed three factors: DSI negative affect; DSI risk taking; and extraneous influences (daily hassles, work-related stress, and general mental health). Moderate intercorrelations were found between each of these factors confirming the ‘spillover’ effect. That is, driver stress is reciprocally related to stress in other domains including work and domestic life. Structural equation modelling (SEM) showed that the DSI negative affect factor influenced both lapses and errors, whereas the DSI risk-taking factor was the strongest influence on violations. The SEMs also confirmed that daily hassles extraneous to the driving environment may influence DBQ lapses and violations independently. Accordingly, interventions may be developed to increase driver awareness of the dangers of excessive emotional responses to both driving events and daily hassles (e.g. driving fast to ‘blow off steam’ after an argument). They may also train more effective strategies for self-regulation of emotion and coping when encountering stressful situations on the road.

Mapping the building and construction product system : preliminary report

This Preliminary Report has been prepared by researchers at The Australian Expert Group in Industry Studies (AEGIS) for the Commonwealth Department of Industry, Science and Resources. It is intended to provide a preliminary 'product system map' of the building and construction industries which defines the system, identifies the major segments, describes key industry players and institutions and provides the basis for exploring relationships, innovation and information flows within the industries. This Preliminary Report is the first of a series of five which will explore the building and construction product system in some depth. This first report does not present original research, although it does include some new interview data and analysis of a variety of written sources. This report is rather a reformulation of existing statistical and analytical material from a product system-based perspective. It is intended to provide the basis for subsequent studies by putting what is already known into an alternative framework and allowing us to see it through a new lens.

The dynamics and performance of the building and construction product system

During 1999 the Department of Industry, Science and Resources (ISR) published 4 research reports it had commissioned from the Australian Expert Group in Industry Studies (AEGIS), a research centre of the University of Western Sydney, Macarthur. ISR will shortly publish the fifth and final report in this series. The five reports were commissioned by the Department, as part of the Building and Construction Action Agenda process, to investigate the dynamics and performance of the sector, particularly in relation its innovative capacity. Professor Jane Marceau, PVCR at the University of Western Sydney and Director of AEGIS, led the research team. Dr Karen Manley was the researcher and joint author on three of the five reports. This paper outlines the approach and key findings of each of the five reports. The reports examined 5 key elements of the ‘building and construction product system’. The term ‘product system’ reflects the very broad range of industries and players we consider to contribute to the performance of the building and construction industries. The term ‘product system’ also highlights our focus on the systemic qualities of the building and construction industries. We were most interested in the inter-relationships between key segments and players and how these impacted on the innovation potential of the product system. The ‘building and construction product system’ is hereafter referred to as ‘the industry’ for ease of presentation. All the reports are based, at least in part, on an interviewing or survey research phase which involved gathering data from public and private sector players nationally. The first report ‘maps’ the industry to identify and describe its key elements and the inter-relationships between them. The second report focuses specifically on the linkages between public-sector research organisations and firms in the industry. The third report examines the conditions surrounding the emergence of new businesses in the industry. The fourth report examines how manufacturing businesses are responding to customer demands for ‘total solutions’ to their building and construction needs, by providing various services to clients. The fifth report investigates the capacity of the industry to encourage and undertake energy efficient building design and construction.

Improving accuracy and intelligibility of decisions

Intelligible and accurate risk-based decision-making requires a complex balance of information from different sources, appropriate statistical analysis of this information and consequent intelligent inference and decisions made on the basis of these analyses. Importantly, this requires an explicit acknowledgement of uncertainty in the inputs and outputs of the statistical model. The aim of this paper is to progress a discussion of these issues in the context of several motivating problems related to the wider scope of agricultural production. These problems include biosecurity surveillance design, pest incursion, environmental monitoring and import risk assessment. The information to be integrated includes observational and experimental data, remotely sensed data and expert information. We describe our efforts in addressing these problems using Bayesian models and Bayesian networks. These approaches provide a coherent and transparent framework for modelling complex systems, combining the different information sources, and allowing for uncertainty in inputs and outputs. While the theory underlying Bayesian modelling has a long and well established history, its application is only now becoming more possible for complex problems, due to increased availability of methodological and computational tools. Of course, there are still hurdles and constraints, which we also address through sharing our endeavours and experiences.

GameFlow : a model for evaluating player enjoyment in games

Although player enjoyment is central to computer games, there is currently no accepted model of player enjoyment in games. There are many heuristics in the literature, based on elements such as the game interface, mechanics, gameplay, and narrative. However, there is a need to integrate these heuristics into a validated model that can be used to design, evaluate, and understand enjoyment in games. We have drawn together the various heuristics into a concise model of enjoyment in games that is structured by flow. Flow, a widely accepted model of enjoyment, includes eight elements that, we found, encompass the various heuristics from the literature. Our new model, GameFlow, consists of eight elements -- concentration, challenge, skills, control, clear goals, feedback, immersion, and social interaction. Each element includes a set of criteria for achieving enjoyment in games. An initial investigation and validation of the GameFlow model was carried out by conducting expert reviews of two real-time strategy games, one high-rating and one low-rating, using the GameFlow criteria. The result was a deeper understanding of enjoyment in real-time strategy games and the identification of the strengths and weaknesses of the GameFlow model as an evaluation tool. The GameFlow criteria were able to successfully distinguish between the high-rated and low-rated games and identify why one succeeded and the other failed. We concluded that the GameFlow model can be used in its current form to review games; further work will provide tools for designing and evaluating enjoyment in games.

Addressing sustainable living using a collaborative approach and multi-disciplinary techniques

In order to achieve meaningful reductions in individual ecological footprints, individuals must dramatically alter their day to day behaviours. Effective interventions will need to be evidence based and there is a necessity for the rapid transfer or communication of information from the point of research, into policy and practice. A number of health disciplines, including psychology and public health, share a common mission to promote health and well-being and it is becoming clear that the most practical pathway to achieving this mission is through interdisciplinary collaboration. This paper argues that an interdisciplinary collaborative approach will facilitate research that results in the rapid transfer of findings into policy and practice. The application of this approach is described in relation to the Green Living project which explored the psycho-social predictors of environmentally friendly behaviour. Following a qualitative pilot study, and in consultation with an expert panel comprising academics, industry professionals and government representatives, a self-administered mail survey was distributed to a random sample of 3000 residents of Brisbane and Moreton Bay (Queensland, Australia). The Green Living survey explored specific beliefs which included attitudes, norms, perceived control, intention and behaviour, as well as a number of other constructs such as environmental concern and altruism. This research has two beneficial outcomes. First, it will inform a practical model for predicting sustainable living behaviours and a number of local councils have already expressed an interest in making use of the results as part of their ongoing community engagement programs. Second, it provides an example of how a collaborative interdisciplinary project can provide a more comprehensive approach to research than can be accomplished by a single disciplinary project.

A new and efficient intelligent collaboration scheme for fashion design

Technology-mediated collaboration process has been extensively studied for over a decade. Most applications with collaboration concepts reported in the literature focus on enhancing efficiency and effectiveness of the decision-making processes in objective and well-structured workflows. However, relatively few previous studies have investigated the applications of collaboration schemes to problems with subjective and unstructured nature. In this paper, we explore a new intelligent collaboration scheme for fashion design which, by nature, relies heavily on human judgment and creativity. Techniques such as multicriteria decision making, fuzzy logic, and artificial neural network (ANN) models are employed. Industrial data sets are used for the analysis. Our experimental results suggest that the proposed scheme exhibits significant improvement over the traditional method in terms of the time–cost effectiveness, and a company interview with design professionals has confirmed its effectiveness and significance.