Are diagnosis specific outcome indicators based on administrative data useful in assessing quality of hospital care?

Background: Hospital performance reports based on administrative data should distinguish differences in quality of care between hospitals from case mix related variation and random error effects. A study was undertaken to determine which of 12 diagnosis-outcome indicators measured across all hospitals in one state had significant risk adjusted systematic ( or special cause) variation (SV) suggesting differences in quality of care. For those that did, we determined whether SV persists within hospital peer groups, whether indicator results correlate at the individual hospital level, and how many adverse outcomes would be avoided if all hospitals achieved indicator values equal to the best performing 20% of hospitals. Methods: All patients admitted during a 12 month period to 180 acute care hospitals in Queensland, Australia with heart failure (n = 5745), acute myocardial infarction ( AMI) ( n = 3427), or stroke ( n = 2955) were entered into the study. Outcomes comprised in-hospital deaths, long hospital stays, and 30 day readmissions. Regression models produced standardised, risk adjusted diagnosis specific outcome event ratios for each hospital. Systematic and random variation in ratio distributions for each indicator were then apportioned using hierarchical statistical models. Results: Only five of 12 (42%) diagnosis-outcome indicators showed significant SV across all hospitals ( long stays and same diagnosis readmissions for heart failure; in-hospital deaths and same diagnosis readmissions for AMI; and in-hospital deaths for stroke). Significant SV was only seen for two indicators within hospital peer groups ( same diagnosis readmissions for heart failure in tertiary hospitals and inhospital mortality for AMI in community hospitals). Only two pairs of indicators showed significant correlation. If all hospitals emulated the best performers, at least 20% of AMI and stroke deaths, heart failure long stays, and heart failure and AMI readmissions could be avoided. Conclusions: Diagnosis-outcome indicators based on administrative data require validation as markers of significant risk adjusted SV. Validated indicators allow quantification of realisable outcome benefits if all hospitals achieved best performer levels. The overall level of quality of care within single institutions cannot be inferred from the results of one or a few indicators.

Public reporting of hospital outcomes based on administrative data: risks and opportunities

In the wake of findings from the Bundaberg Hospital and Forster inquiries in Queensland, periodic public release of hospital performance reports has been recommended. A process for developing and releasing such reports is being established by Queensland Health, overseen by an independent expert panel. This recommendation presupposes that public reports based on routinely collected administrative data are accurate; that the public can access, correctly interpret and act upon report contents; that reports motivate hospital clinicians and managers to improve quality of care; and that there are no unintended adverse effects of public reporting. Available research suggests that primary data sources are often inaccurate and incomplete, that reports have low predictive value in detecting outlier hospitals, and that users experience difficulty in accessing and interpreting reports and tend to distrust their findings.

Better queue management in a busy public hospital of a developing country without appointment system::an application using data envelopment analysis

Queuing is a key efficiency criterion in any service industry, including Healthcare. Almost all queue management studies are dedicated to improving an existing Appointment System. In developing countries such as Pakistan, there are no Appointment Systems for outpatients, resulting in excessive wait times. Additionally, excessive overloading, limited resources and cumbersome procedures lead to over-whelming queues. Despite numerous Healthcare applications, Data Envelopment Analysis (DEA) has not been applied for queue assessment. The current study aims to extend DEA modelling and demonstrate its usefulness by evaluating the queue system of a busy public hospital in a developing country, Pakistan, where all outpatients are walk-in; along with construction of a dynamic framework dedicated towards the implementation of the model. The inadequate allocation of doctors/personnel was observed as the most critical issue for long queues. Hence, the Queuing-DEA model has been developed such that it determines the ‘required’ number of doctors/personnel. The results indicated that given extensive wait times or length of queue, or both, led to high target values for doctors/personnel. Hence, this crucial information allows the administrators to ensure optimal staff utilization and controlling the queue pre-emptively, minimizing wait times. The dynamic framework constructed, specifically targets practical implementation of the Queuing-DEA model in resource-poor public hospitals of developing countries such as Pakistan; to continuously monitor rapidly changing queue situation and display latest required personnel. Consequently, the wait times of subsequent patients can be minimized, along with dynamic staff scheduling in the absence of appointments. This dynamic framework has been designed in Excel, requiring minimal training and work for users and automatic update features, with complex technical aspects running in the background. The proposed model and the dynamic framework has the potential to be applied in similar public hospitals, even in other developing countries, where appointment systems for outpatients are non-existent.

Correcting for bias when estimating the cost of hospital-acquired infection : an analysis of lower respiratory tract infections in non-surgical patients

Hospital acquired infections (HAI) are costly but many are avoidable. Evaluating prevention programmes requires data on their costs and benefits. Estimating the actual costs of HAI (a measure of the cost savings due to prevention) is difficult as HAI changes cost by extending patient length of stay, yet, length of stay is a major risk factor for HAI. This endogeneity bias can confound attempts to measure accurately the cost of HAI. We propose a two-stage instrumental variables estimation strategy that explicitly controls for the endogeneity between risk of HAI and length of stay. We find that a 10% reduction in ex ante risk of HAI results in an expected savings of £693 ($US 984).

Determinants of pre-hospital care non-usage for patients with emergency care needs [including commentary by Snooks H.]

Objective-To establish the demographic, health status and insurance determinants of pre-hospital ambulance non-usage for patients with emergency medical needs. Methods-Triage category, date of birth, sex, marital status, country of origin, method and time of arrival, ambulance insurance status, diagnosis, and disposal were collected for all patients who presented over a four month period (n=10 229) to the emergency department of a major provincial hospital. Data for patients with urgent (n=678) or critical care needs (n=332) who did not use pre-hospital care were analysed using Poisson regression. Results-Only a small percentage (6.6%) of the total sample were triaged as having urgent medical needs or critical care needs (3.2%). Predictors of usage for those with urgent care needs included age greater than 65 years (prevalence ratio (PR)=0.54; 95% confidence interval (CI)= 0.35 to 0.83), being admitted to intensive care or transferred to another hospital (PR=0.62; 95% CI=0.44 to 0.89) or ward (PR=0.72; 95% CI=0.56 to 0.93) and ambulance insurance status (PR=0.67; 95% CI=052 to 0.86). Sex, marital status, time of day and country of origin were not predictive of usage and non-usage. Predictors of usage for those with critical care needs included age 65 years or greater (PR=0.45; 95% CI=0.25 to 0.81) and a diagnosis of trauma (PR=0.49; 95% CI=0.26 to 0.92). A non-English speaking background was predictive of non-usage (PR=1.98; 95% CI=1.06 to 3.70). Sex, marital status, time of day, triage and ambulance insurance status were not predictive of non-usage. Conclusions-Socioeconomic and medical factors variously influence ambulance usage depending on the severity or urgency of the medical condition. Ambulance insurance status was less of an influence as severity of condition increased suggesting that, at a critical level of urgency, patients without insurance are willing to pay for a pre-hospital ambulance service.

The reliability of information on work-related injuries available from hospitalisation data in Australia

Objective: To examine the reliability of work-related activity coding for injury-related hospitalisations in Australia. Method: A random sample of 4373 injury-related hospital separations from 1 July 2002 to 30 June 2004 were obtained from a stratified random sample of 50 hospitals across 4 states in Australia. From this sample, cases were identified as work-related if they contained an ICD-10-AM work-related activity code (U73) allocated by either: (i) the original coder; (ii) an independent auditor, blinded to the original code; or (iii) a research assistant, blinded to both the original and auditor codes, who reviewed narrative text extracted from the medical record. The concordance of activity coding and number of cases identified as work-related using each method were compared. Results: Of the 4373 cases sampled, 318 cases were identified as being work-related using any of the three methods for identification. The original coder identified 217 and the auditor identified 266 work-related cases (68.2% and 83.6% of the total cases identified, respectively). Around 10% of cases were only identified through the text description review. The original coder and auditor agreed on the assignment of work-relatedness for 68.9% of cases. Conclusions and Implications: The current best estimates of the frequency of hospital admissions for occupational injury underestimate the burden by around 32%. This is a substantial underestimate that has major implications for public policy, and highlights the need for further work on improving the quality and completeness of routine, administrative data sources for a more complete identification of work-related injuries.

Using Trauma Injury Severity Score (TRISS) variables to predict length of hospital stay following trauma in New Zealand

Aim – To develop and assess the predictive capabilities of a statistical model that relates routinely collected Trauma Injury Severity Score (TRISS) variables to length of hospital stay (LOS) in survivors of traumatic injury. Method – Retrospective cohort study of adults who sustained a serious traumatic injury, and who survived until discharge from Auckland City, Middlemore, Waikato, or North Shore Hospitals between 2002 and 2006. Cubic-root transformed LOS was analysed using two-level mixed-effects regression models. Results – 1498 eligible patients were identified, 1446 (97%) injured from a blunt mechanism and 52 (3%) from a penetrating mechanism. For blunt mechanism trauma, 1096 (76%) were male, average age was 37 years (range: 15-94 years), and LOS and TRISS score information was available for 1362 patients. Spearman’s correlation and the median absolute prediction error between LOS and the original TRISS model was ρ=0.31 and 10.8 days, respectively, and between LOS and the final multivariable two-level mixed-effects regression model was ρ=0.38 and 6.0 days, respectively. Insufficient data were available for the analysis of penetrating mechanism models. Conclusions – Neither the original TRISS model nor the refined model has sufficient ability to accurately or reliably predict LOS. Additional predictor variables for LOS and other indicators for morbidity need to be considered.

Safeguarding Rural Australia : Data Report No. 2 - Intentional Violence

This report focuses on our examination of extant data which have been sourced with respect to intentional violence perpetrated or experienced by males in regional and remote Australia. The nature of intentional violent acts can be physical, sexual or psychological or involve deprivation or neglect. We have presented under the headings of: self-harm including suicide; homicide; assault, sexual assault and the threat of assault; child abuse; other family and intimate partner violence; harassment, stalking and bullying; alcohol related social violence; and animal abuse. State variations in interpersonal violence are also presented. Additional commentary resulting from exploration, examination and analyses of secondary data is published online in complementary reports in this series.

Cost-effectiveness of an intervention to reduce emergency re-admissions to hospital among older patients

Background. The objective is to estimate the cost-effectiveness of an intervention that reduces hospital readmission among older people at high risk. A cost-effectiveness model to estimate the costs and health benefits of the intervention was implemented. Methodology/Principal Findings. The model used data from a randomised controlled trial conducted in an Australian tertiary metropolitan hospital. Participants were acute medical admissions aged >65 years with at least one risk factor for readmission: multiple comorbidities, impaired functionality, aged >75 years, 30 recent multiple admissions, poor social support, history of depression. The intervention was a comprehensive nursing and physiotherapy assessment and an individually tailored program of exercise strategies and nurse home visits with telephone follow-up; commencing in hospital and continuing following discharge for 24 weeks. The change to cost outcomes, including the costs of implementing the intervention and all subsequent use of health care services, and, the change to health benefits, represented by quality adjusted life years, were estimated for the intervention as compared to existing practice. The mean change to total costs and quality 38 adjusted life years for an average individual over 24 weeks participating in the intervention were: cost savings of $333 (95% Bayesian credible interval $-1,932:1,282) and 0.118 extra quality adjusted life years (95% Bayesian credible interval 0.1:0.136). The mean net41 monetary-benefit per individual for the intervention group compared to the usual care condition was $7,907 (95% Bayesian credible interval $5,959:$9,995) for the 24 week period. Conclusions/Significance. The estimation model that describes this intervention predicts cost savings and improved health outcomes. A decision to remain with existing practices causes unnecessary costs and reduced health. Decision makers should consider adopting this 46 program for elderly hospitalised patients.

Temperature variation and emergency hospital admissions for stroke in Brisbane, Australia, 1996-2005

Stroke is a leading cause of disability and death. This study evaluated the association between temperature variation and emergency admissions for stroke in Brisbane, Australia. Daily emergency admissions for stroke, meteorologic and air pollution data were obtained for the period of January 1996 to December 2005. The relative risk of emergency admissions for stroke was estimated with a generalized estimating equations (GEE) model. For primary intracerebral hemorrhage (PIH) emergency admissions, the average daily PIH for the group aged < 65 increased by 15% (95% Confidence Interval (CI): 5, 26%) and 12% (95% CI: 2, 22%) for a 1°C increase in daily maximum temperature and minimum temperature in summer, respectively, after controlling for potential confounding effects of humidity and air pollutants. For ischemic stroke (IS) emergency admissions, the average daily IS for the group aged ≥ 65 decreased by 3% (95% CI: -6, 0%) for a 1°C increase in daily maximum temperature in winter after adjustment for confounding factors. Temperature variation was significantly associated with emergency admissions for stroke, and its impact varied with different type of stroke. Health authorities should pay greater attention to possible increasing emergency care for strokes when temperature changes, in both summer and winter.

Setting hospital infection control policy : a decision-making framework incorporating health economics and healthcare epidemiology

Background: Reducing rates of healthcare acquired infection has been identified by the Australian Commission on Safety and Quality in Health Care as a national priority. One of the goals is the prevention of central venous catheter-related bloodstream infection (CR-BSI). At least 3,500 cases of CR-BSI occur annually in Australian hospitals, resulting in unnecessary deaths and costs to the healthcare system between $25.7 and $95.3 million. Two approaches to preventing these infections have been proposed: use of antimicrobial catheters (A-CVCs); or a catheter care and management ‘bundle’. Given finite healthcare budgets, decisions about the optimal infection control policy require consideration of the effectiveness and value for money of each approach. Objectives: The aim of this research is to use a rational economic framework to inform efficient infection control policy relating to the prevention of CR-BSI in the intensive care unit. It addresses three questions relating to decision-making in this area: 1. Is additional investment in activities aimed at preventing CR-BSI an efficient use of healthcare resources? 2. What is the optimal infection control strategy from amongst the two major approaches that have been proposed to prevent CR-BSI? 3. What uncertainty is there in this decision and can a research agenda to improve decision-making in this area be identified? Methods: A decision analytic model-based economic evaluation was undertaken to identify an efficient approach to preventing CR-BSI in Queensland Health intensive care units. A Markov model was developed in conjunction with a panel of clinical experts which described the epidemiology and prognosis of CR-BSI. The model was parameterised using data systematically identified from the published literature and extracted from routine databases. The quality of data used in the model and its validity to clinical experts and sensitivity to modelling assumptions was assessed. Two separate economic evaluations were conducted. The first evaluation compared all commercially available A-CVCs alongside uncoated catheters to identify which was cost-effective for routine use. The uncertainty in this decision was estimated along with the value of collecting further information to inform the decision. The second evaluation compared the use of A-CVCs to a catheter care bundle. We were unable to estimate the cost of the bundle because it is unclear what the full resource requirements are for its implementation, and what the value of these would be in an Australian context. As such we undertook a threshold analysis to identify the cost and effectiveness thresholds at which a hypothetical bundle would dominate the use of A-CVCs under various clinical scenarios. Results: In the first evaluation of A-CVCs, the findings from the baseline analysis, in which uncertainty is not considered, show that the use of any of the four A-CVCs will result in health gains accompanied by cost-savings. The MR catheters dominate the baseline analysis generating 1.64 QALYs and cost-savings of $130,289 per 1.000 catheters. With uncertainty, and based on current information, the MR catheters remain the optimal decision and return the highest average net monetary benefits ($948 per catheter) relative to all other catheter types. This conclusion was robust to all scenarios tested, however, the probability of error in this conclusion is high, 62% in the baseline scenario. Using a value of $40,000 per QALY, the expected value of perfect information associated with this decision is $7.3 million. An analysis of the expected value of perfect information for individual parameters suggests that it may be worthwhile for future research to focus on providing better estimates of the mortality attributable to CR-BSI and the effectiveness of both SPC and CH/SSD (int/ext) catheters. In the second evaluation of the catheter care bundle relative to A-CVCs, the results which do not consider uncertainty indicate that a bundle must achieve a relative risk of CR-BSI of at least 0.45 to be cost-effective relative to MR catheters. If the bundle can reduce rates of infection from 2.5% to effectively zero, it is cost-effective relative to MR catheters if national implementation costs are less than $2.6 million ($56,610 per ICU). If the bundle can achieve a relative risk of 0.34 (comparable to that reported in the literature) it is cost-effective, relative to MR catheters, if costs over an 18 month period are below $613,795 nationally ($13,343 per ICU). Once uncertainty in the decision is considered, the cost threshold for the bundle increases to $2.2 million. Therefore, if each of the 46 Level III ICUs could implement an 18 month catheter care bundle for less than $47,826 each, this approach would be cost effective relative to A-CVCs. However, the uncertainty is substantial and the probability of error in concluding that the bundle is the cost-effective approach at a cost of $2.2 million is 89%. Conclusions: This work highlights that infection control to prevent CR-BSI is an efficient use of healthcare resources in the Australian context. If there is no further investment in infection control, an opportunity cost is incurred, which is the potential for a more efficient healthcare system. Minocycline/rifampicin catheters are the optimal choice of antimicrobial catheter for routine use in Australian Level III ICUs, however, if a catheter care bundle implemented in Australia was as effective as those used in the large studies in the United States it would be preferred over the catheters if it was able to be implemented for less than $47,826 per Level III ICU. Uncertainty is very high in this decision and arises from multiple sources. There are likely greater costs to this uncertainty for A-CVCs, which may carry hidden costs, than there are for a catheter care bundle, which is more likely to provide indirect benefits to clinical practice and patient safety. Research into the mortality attributable to CR-BSI, the effectiveness of SPC and CH/SSD (int/ext) catheters and the cost and effectiveness of a catheter care bundle in Australia should be prioritised to reduce uncertainty in this decision. This thesis provides the economic evidence to inform one area of infection control, but there are many other infection control decisions for which information about the cost-effectiveness of competing interventions does not exist. This work highlights some of the challenges and benefits to generating and using economic evidence for infection control decision-making and provides support for commissioning more research into the cost-effectiveness of infection control.