Improving national hospice/palliative care service symptom outcomes systematically through point-of-care data collection, structured feedback and benchmarking

PURPOSE Every health care sector including hospice/palliative care needs to systematically improve services using patient-defined outcomes. Data from the national Australian Palliative Care Outcomes Collaboration aims to define whether hospice/palliative care patients' outcomes and the consistency of these outcomes have improved in the last 3 years. METHODS Data were analysed by clinical phase (stable, unstable, deteriorating, terminal). Patient-level data included the Symptom Assessment Scale and the Palliative Care Problem Severity Score. Nationally collected point-of-care data were anchored for the period July-December 2008 and subsequently compared to this baseline in six 6-month reporting cycles for all services that submitted data in every time period (n = 30) using individual longitudinal multi-level random coefficient models. RESULTS Data were analysed for 19,747 patients (46 % female; 85 % cancer; 27,928 episodes of care; 65,463 phases). There were significant improvements across all domains (symptom control, family care, psychological and spiritual care) except pain. Simultaneously, the interquartile ranges decreased, jointly indicating that better and more consistent patient outcomes were being achieved. CONCLUSION These are the first national hospice/palliative care symptom control performance data to demonstrate improvements in clinical outcomes at a service level as a result of routine data collection and systematic feedback.

Outpatient management of abnormal smears

Objectives: To review the results of the first 403 women treated at the Abnormal Smear and Colposcopy Unit with special reference to the utility, efficacy, acceptability and economy of in-office treatment of cervical lesions by large loop or Fischer cone excision. Design: Retrospective chart review of consecutive patients treated following, referral with an abnormal smear or abnormal cervical morphology, between 1 September 1996 and I August 2001. Setting: Inner city private practice. Sample: A total of 403 consecutive General Practitioner referred women. Methods: Details of referral smear result, colposcopically directed biopsy result, subsequent treatment type and histological result including assessability number of specimens submitted, complications and follow-up assessment were extracted at chart review. Costs of public hospital inpatient and outpatient care, supplied by the Casemix and Clinical Benchmarking Service, Mater Miseraecordae Public Hospitals (with permission to publish), were compared with Medicare rebates. Main outcome measures: A total of 187 women were treated by large loop excision of the transformation zone, and 216 by Fischer cone excision. The number of women who were treated as outpatients under local anaesthetic were 395, while eight patients were treated under general anaesthesia as inpatients. There was poor correlation between referring smear, biopsy and subsequent treatment results. Eight patients had abnormal cytology at follow-up, of whom two have been retreated. Three patients had primary or secondary bleeding requiring treatment and two developed cervical stenosis. Outpatient private practice treatment of women with abnormal smears allows significant savings to the public purse over public or private hospital care. Conclusions: Outpatient treatment of women with abnormal smears, using the Fischer cone technique, is safe, wen accepted, effective and the most cost efficient solution to this public health problem.

Benchmarking clinical indicators of quality for Australian residential aged care facilities

To undertake exploratory benchmarking of a set of clinical indicators of quality care in residential care in Australia, data were collected from 107 residents within four medium-sized facilities (40–80 beds) in Brisbane, Australia. The proportion of residents in each sample facility with a particular clinical problem was compared with US Minimum Data Set quality indicator thresholds. Results demonstrated variability within and between clinical indicators, suggesting breadth of assessment using various clinical indicators of quality is an important factor when monitoring quality of care. More comprehensive and objective measures of quality of care would be of great assistance in determining and monitoring the effectiveness of residential aged care provision in Australia, particularly as demands for accountability by consumers and their families increase. What is known about the topic? The key to quality improvement is effective quality assessment, and one means of evaluating quality of care is through clinical outcomes. The Minimum Data Set quality indicators have been credited with improving quality in United States nursing homes. What does this paper add? The Clinical Care Indicators Tool was used to collect data on clinical outcomes, enabling comparison of data from a small Australian sample with American quality benchmarks to illustrate the utility of providing guidelines for interpretation. What are the implications for practitioners? Collecting and comparing clinical outcome data would enable practitioners to better understand the quality of care being provided and whether practices required review. The Clinical Care Indicator Tool could provide a comprehensive and systematic means of doing this, thus filling a gap in quality monitoring within Australian residential aged care.

BENCHMARKING AND IMPLEMENTATION OF A NEW INDEPENDENT MONTE CARLO DOSE CALCULATION QUALITY ASSURANCE AUDIT TOOL FOR CLINICAL TRIALS

Introduction Commercial treatment planning systems employ a variety of dose calculation algorithms to plan and predict the dose distributions a patient receives during external beam radiation therapy. Traditionally, the Radiological Physics Center has relied on measurements to assure that institutions participating in the National Cancer Institute sponsored clinical trials administer radiation in doses that are clinically comparable to those of other participating institutions. To complement the effort of the RPC, an independent dose calculation tool needs to be developed that will enable a generic method to determine patient dose distributions in three dimensions and to perform retrospective analysis of radiation delivered to patients who enrolled in past clinical trials. Methods A multi-source model representing output for Varian 6 MV and 10 MV photon beams was developed and evaluated. The Monte Carlo algorithm, know as the Dose Planning Method (DPM), was used to perform the dose calculations. The dose calculations were compared to measurements made in a water phantom and in anthropomorphic phantoms. Intensity modulated radiation therapy and stereotactic body radiation therapy techniques were used with the anthropomorphic phantoms. Finally, past patient treatment plans were selected and recalculated using DPM and contrasted against a commercial dose calculation algorithm. Results The multi-source model was validated for the Varian 6 MV and 10 MV photon beams. The benchmark evaluations demonstrated the ability of the model to accurately calculate dose for the Varian 6 MV and the Varian 10 MV source models. The patient calculations proved that the model was reproducible in determining dose under similar conditions described by the benchmark tests. Conclusions The dose calculation tool that relied on a multi-source model approach and used the DPM code to calculate dose was developed, validated, and benchmarked for the Varian 6 MV and 10 MV photon beams. Several patient dose distributions were contrasted against a commercial algorithm to provide a proof of principal to use as an application in monitoring clinical trial activity.

Development of a Systematic Approach to Assessing Quality within Australian Residential Aged Care Facilities: The Clinical Care Indicators Tool

Recent years have seen the introduction of formalised accreditation processes in both community and residential aged care, but these only partially address quality assessment within this sector. Residential aged care in Australia does not yet have a standardised system of resident assessment related to clinical, rather than administrative, outcomes. This paper describes the development of a quality assessment tool aimed at addressing this gap. Utilising previous research and the results of nominal groups with experts in the field, the 21-item Clinical Care Indicators (CCI) Tool for residential aged care was developed and trialled nationally. The CCI Tool was found to be simple to use and an effective means of collecting data on the state of resident health and care, with potential benefits for resident care planning and continuous quality improvement within facilities and organisations. The CCI Tool was further refined through a small intervention study to assess its utility as a quality improvement instrument and to investigate its relationship with resident quality of life. The current version covers 23 clinical indicators, takes about 30 minutes to complete and is viewed favourably by nursing staff who use it. Current work focuses on psychometric analysis and benchmarking, which should enable the CCI Tool to make a positive contribution to the measurement of quality in aged care in Australia.

Clinical outcomes in residential care : setting benchmarks for quality

Aim Australian residential aged care does not have a system of quality assessment related to clinical outcomes, or comprehensive quality benchmarking. The Residential Care Quality Assessment was developed to fill this gap; and this paper discusses the process by which preliminary benchmarks representing high and low quality were developed for it. Methods Data were collected from all residents (n = 498) of nine facilities. Numerator–denominator analysis of clinical outcomes occurred at a facility-level, with rank-ordered results circulated to an expert panel. The panel identified threshold scores to indicate excellent and questionable care quality, and refined these through Delphi process. Results Clinical outcomes varied both within and between facilities; agreed thresholds for excellent and poor outcomes were finalised after three Delphi rounds. Conclusion Use of the Residential Care Quality Assessment provides a concrete means of monitoring care quality and allows benchmarking across facilities; its regular use could contribute to improved care outcomes within residential aged care in Australia.

Clinical indicators of quality for Australian residential aged care facilities : establishing reliability, validity, and quality thresholds

Background: In response to the need for more comprehensive quality assessment within Australian residential aged care facilities, the Clinical Care Indicator (CCI) Tool was developed to collect outcome data as a means of making inferences about quality. A national trial of its effectiveness and a Brisbane-based trial of its use within the quality improvement context determined the CCI Tool represented a potentially valuable addition to the Australian aged care system. This document describes the next phase in the CCI Tool.s development; the aims of which were to establish validity and reliability of the CCI Tool, and to develop quality indicator thresholds (benchmarks) for use in Australia. The CCI Tool is now known as the ResCareQA (Residential Care Quality Assessment). Methods: The study aims were achieved through a combination of quantitative data analysis, and expert panel consultations using modified Delphi process. The expert panel consisted of experienced aged care clinicians, managers, and academics; they were initially consulted to determine face and content validity of the ResCareQA, and later to develop thresholds of quality. To analyse its psychometric properties, ResCareQA forms were completed for all residents (N=498) of nine aged care facilities throughout Queensland. Kappa statistics were used to assess inter-rater and test-retest reliability, and Cronbach.s alpha coefficient calculated to determine internal consistency. For concurrent validity, equivalent items on the ResCareQA and the Resident Classification Scales (RCS) were compared using Spearman.s rank order correlations, while discriminative validity was assessed using known-groups technique, comparing ResCareQA results between groups with differing care needs, as well as between male and female residents. Rank-ordered facility results for each clinical care indicator (CCI) were circulated to the panel; upper and lower thresholds for each CCI were nominated by panel members and refined through a Delphi process. These thresholds indicate excellent care at one extreme and questionable care at the other. Results: Minor modifications were made to the assessment, and it was renamed the ResCareQA. Agreement on its content was reached after two Delphi rounds; the final version contains 24 questions across four domains, enabling generation of 36 CCIs. Both test-retest and inter-rater reliability were sound with median kappa values of 0.74 (test-retest) and 0.91 (inter-rater); internal consistency was not as strong, with a Chronbach.s alpha of 0.46. Because the ResCareQA does not provide a single combined score, comparisons for concurrent validity were made with the RCS on an item by item basis, with most resultant correlations being quite low. Discriminative validity analyses, however, revealed highly significant differences in total number of CCIs between high care and low care groups (t199=10.77, p=0.000), while the differences between male and female residents were not significant (t414=0.56, p=0.58). Clinical outcomes varied both within and between facilities; agreed upper and lower thresholds were finalised after three Delphi rounds. Conclusions: The ResCareQA provides a comprehensive, easily administered means of monitoring quality in residential aged care facilities that can be reliably used on multiple occasions. The relatively modest internal consistency score was likely due to the multi-factorial nature of quality, and the absence of an aggregate result for the assessment. Measurement of concurrent validity proved difficult in the absence of a gold standard, but the sound discriminative validity results suggest that the ResCareQA has acceptable validity and could be confidently used as an indication of care quality within Australian residential aged care facilities. The thresholds, while preliminary due to small sample size, enable users to make judgements about quality within and between facilities. Thus it is recommended the ResCareQA be adopted for wider use.

Trainee-therapists are not all equal: Examination of therapeutic efficiency, effectiveness and early client dropout after 12 months of clinical training

Objective Contemporary research demonstrates the feasibility of assessing therapeutic performance of trainee-therapists through the use of objective measures of client treatment outcome. Further, significant variation between individual therapists based on their client treatment outcomes has been demonstrated. This study sets out to determine whether a reliable composite measure of therapeutic efficiency, effectiveness and early dropout can be developed and used to objectively compare trainee-therapists against each other. Design and methods Treatment outcomes of 611 clients receiving treatment from 58 trainee-therapists enrolled in a professional training programme were tracked with the OQ-45.2 over a 6-year period to assess therapeutic efficiency, therapeutic effectiveness and early client dropout. Results Significant variation between trainee-therapists was observed for each index. Findings of a moderately strong correlation between therapeutic efficiency and effectiveness enabled the ranking of trainee-therapists based upon a composite measure of these indexes. A non-significant correlation was found between early client dropout and measures of therapeutic effectiveness and efficiency. Conclusions The findings stress the importance of utilizing objective measures to track the treatment outcomes. Despite all trainee-therapists being enrolled in the same training programme, significant variation between trainee-therapists' therapeutic efficiency and effectiveness was found to exist. Practitioner points Developing of potential benchmarking tools that enable trainee-therapists, supervisors and educational institutions to quickly assess therapeutic performance can become part of a holistic assessment of a trainee-therapist's clinical development. Despite an inherent optimistic belief that therapists do not cause harm, there appears to be a small and significant proportion of trainee-therapists who consistently evidence little therapeutic change. Considerable variability in trainee-therapists' therapeutic efficiency and effectiveness can exist in the one training programme. Early client dropout may not be associated with therapists' therapeutic effectiveness and efficiency.

Differences in identifying healthcare associated infections using clinical vignettes and the influence of respondent characteristics: A cross-sectional survey of Australian infection prevention staff

Background Australia has commenced public reporting and benchmarking of healthcare associated infections (HAIs), despite not having a standardised national HAI surveillance program. Annual hospital Staphylococcus aureus bloodstream (SAB) infection rates are released online, with other HAIs likely to be reported in the future. Although there are known differences between hospitals in Australian HAI surveillance programs, the effect of these differences on reported HAI rates is not known. Objective To measure the agreement in HAI identification, classification, and calculation of HAI rates, and investigate the influence of differences amongst those undertaking surveillance on these outcomes. Methods A cross-sectional online survey exploring HAI surveillance practices was administered to infection prevention nurses who undertake HAI surveillance. Seven clinical vignettes describing HAI scenarios were included to measure agreement in HAI identification, classification, and calculation of HAI rates. Data on characteristics of respondents was also collected. Three of the vignettes were related to surgical site infection and four to bloodstream infection. Agreement levels for each of the vignettes were calculated. Using the Australian SAB definition, and the National Health and Safety Network definitions for other HAIs, we looked for an association between the proportion of correct answers and the respondents’ characteristics. Results Ninety-two infection prevention nurses responded to the vignettes. One vignette demonstrated 100 % agreement from responders, whilst agreement for the other vignettes varied from 53 to 75 %. Working in a hospital with more than 400 beds, working in a team, and State or Territory was associated with a correct response for two of the vignettes. Those trained in surveillance were more commonly associated with a correct response, whilst those working part-time were less likely to respond correctly. Conclusion These findings reveal the need for further HAI surveillance support for those working part-time and in smaller facilities. It also confirms the need to improve uniformity of HAI surveillance across Australian hospitals, and raises questions on the validity of the current comparing of national HAI SAB rates.

A point prevalence survey of antibiotic prescriptions: benchmarking and patterns of use

center dot Inappropriate antimicrobial use has been associated with increased morbidity and hospital costs.

Explaining variation in cancer survival between 11 jurisdictions in the International Cancer Benchmarking Partnership: a primary care vignette survey

OBJECTIVES: The International Cancer Benchmarking Partnership (ICBP) is a collaboration between 6 countries and 12 jurisdictions with similar primary care-led health services. This study investigates primary care physician (PCP) behaviour and systems that may contribute to the timeliness of investigating for cancer and subsequently, international survival differences.

DESIGN: A validated survey administered to PCPs via the internet set out in two parts: direct questions on primary care structure and practice relating to cancer diagnosis, and clinical vignettes, assessing management of scenarios relating to the diagnosis of lung, colorectal or ovarian cancer.

PARTICIPANTS: 2795 PCPs in 11 jurisdictions: New South Wales and Victoria (Australia), British Columbia, Manitoba, Ontario (Canada), England, Northern Ireland, Wales (UK), Denmark, Norway and Sweden.

PRIMARY AND SECONDARY OUTCOME MEASURES: Analysis compared the cumulative proportion of PCPs in each jurisdiction opting to investigate or refer at each phase for each vignette with 1-year survival, and conditional 5-year survival rates for the relevant cancer and jurisdiction. Logistic regression was used to explore whether PCP characteristics or system differences in each jurisdiction affected the readiness to investigate.

RESULTS: 4 of 5 vignettes showed a statistically significant correlation (p<0.05 or better) between readiness to investigate or refer to secondary care at the first phase of each vignette and cancer survival rates for that jurisdiction. No consistent associations were found between readiness to investigate and selected PCP demographics, practice or health system variables.

CONCLUSIONS: We demonstrate a correlation between the readiness of PCPs to investigate symptoms indicative of cancer and cancer survival rates, one of the first possible explanations for the variation in cancer survival between ICBP countries. No specific health system features consistently explained these findings. Some jurisdictions may consider lowering thresholds for PCPs to investigate for cancer-either directly, or by specialist referral, to improve outcomes.

A modified method for measuring antibiotic use in healthcare settings: implications for antibiotic stewardship and benchmarking

Objectives: To determine whether adjusting the denominator of the common hospital antibiotic use measurement unit (defined daily doses/100 bed-days) by including age-adjusted comorbidity score (100 bed-days/age-adjusted comorbidity score) would result in more accurate and meaningful assessment of hospital antibiotic use. 

Methods: The association between the monthly sum of age-adjusted comorbidity and monthly antibiotic use was measured using time-series analysis (January 2008 to June 2012). For the purposes of conducting internal benchmarking, two antibiotic usage datasets were constructed, i.e. 2004-07 (first study period) and 2008-11 (second study period). Monthly antibiotic use was normalized per 100 bed-days and per 100 bed-days/age-adjusted comorbidity score. 

Results: Results showed that antibiotic use had significant positive relationships with the sum of age-adjusted comorbidity score (P = 0.0004). The results also showed that there was a negative relationship between antibiotic use and (i) alcohol-based hand rub use (P = 0.0370) and (ii) clinical pharmacist activity (P = 0.0031). Normalizing antibiotic use per 100 bed-days contributed to a comparative usage rate of 1.31, i.e. the average antibiotic use during the second period was 31% higher than during the first period. However, normalizing antibiotic use per 100 bed-days per age-adjusted comorbidity score resulted in a comparative usage rate of 0.98, i.e. the average antibiotic use was 2% lower in the second study period. Importantly, the latter comparative usage rate is independent of differences in patient density and case mix characteristics between the two studied populations. 

Conclusions: The proposed modified antibiotic measure provides an innovative approach to compare variations in antibiotic prescribing while taking account of patient case mix effects.