How valuable are multiple treatment comparison methods in evidence-based health-care evaluation?

Objectives To compare the use of pair-wise meta-analysis methods to multiple treatment comparison (MTC) methods for evidence-based health-care evaluation to estimate the effectiveness and cost-effectiveness of alternative health-care interventions based on the available evidence. Methods Pair-wise meta-analysis and more complex evidence syntheses, incorporating an MTC component, are applied to three examples: 1) clinical effectiveness of interventions for preventing strokes in people with atrial fibrillation; 2) clinical and cost-effectiveness of using drug-eluting stents in percutaneous coronary intervention in patients with coronary artery disease; and 3) clinical and cost-effectiveness of using neuraminidase inhibitors in the treatment of influenza. We compare the two synthesis approaches with respect to the assumptions made, empirical estimates produced, and conclusions drawn. Results The difference between point estimates of effectiveness produced by the pair-wise and MTC approaches was generally unpredictable—sometimes agreeing closely whereas in other instances differing considerably. In all three examples, the MTC approach allowed the inclusion of randomized controlled trial evidence ignored in the pair-wise meta-analysis approach. This generally increased the precision of the effectiveness estimates from the MTC model. Conclusions The MTC approach to synthesis allows the evidence base on clinical effectiveness to be treated as a coherent whole, include more data, and sometimes relax the assumptions made in the pair-wise approaches. However, MTC models are necessarily more complex than those developed for pair-wise meta-analysis and thus could be seen as less transparent. Therefore, it is important that model details and the assumptions made are carefully reported alongside the results.

The role of economic evaluation in mental health care

Objective: A consequence of the integration of psychiatry into acute and public health medicine is that psychiatrists are being asked to evaluate their services. There is pressure on mental health-care systems because it is recognized that funds should be directed where they can provide the best health outcomes, and also because there are resource constraints which limit our capacity to meet all demands for health care. This pressure can be responded to by evaluation which demonstrates the effectiveness and efficiency of psychiatric treatment. This paper seeks to remind psychiatrists of the fundamental principles of economic evaluation in the hope that these will enable psychiatrists to understand the methods used in evaluation and to work comfortably with evaluators. Method: The paper reviews the basic principles behind economic evaluation, illustrating these with reference to case studies. It describes: (i) the cost of the burden of illness and treatment, and how these costs are measured; (ii) the measurement of treatment outcomes, both as changes in health status and as resources saved; and (iii) the various types of economic evaluation, including cost-minimization, cost-effectiveness, cost-utility and cost-benefit analysis. Results: The advice in the paper provides psychiatrists with the necessary background to work closely with evaluators. A checklist of the critical questions to be addressed is provided as a guide for those undertaking economic evaluations. Conclusions: If psychiatrists are willing to learn the basic principles of economic evaluation and to apply these, they can respond to the challenges of evaluation.

An evaluation approach for a new paradigm - health care integration

This paper explores an approach to the implementation and evaluation of integrated health service delivery. It identifies the key issues involved in integration evaluation, provides a framework for assessment and identifies areas for the development of new tools and measures. A proactive role for evaluators in responding to health service reform is advocated.

An evaluation of tobacco brief intervention training in three Indigenous health care settings in north Queensland

Objective: To evaluate the pilot phase of a tobacco brief intervention program in three Indigenous health care settings in rural and remote north Queensland. Methods: A combination of in-depth interviews with health staff and managers and focus groups with health staff and consumers. Results: The tobacco brief intervention initiative resulted in changes in clinical practice among health care workers in all three sites. Although health workers had reported routinely raising the issue of smoking in a variety of settings prior to the intervention, the training provided them with an additional opportunity to become more aware of new approaches to smoking cessation. Indigenous health workers in particular reported that their own attempts to give up smoking following the training had given them confidence and empathy in offering smoking cessation advice. However, the study found no evidence that anybody had actually given up smoking at six months following the intervention. Integration of brief intervention into routine clinical practice was constrained by organisational, interpersonal and other factors in the broader socio-environmental context. Conclusions/implications: While modest health gains may be possible through brief intervention, the potential effectiveness in Indigenous settings will be limited in the absence of broader strategies aimed at tackling community-identified health priorities such as alcohol misuse, violence, employment and education. Tobacco and other forms of lifestyle brief. intervention need to be part of multi-level health strategies. Training in tobacco brief intervention should address both the Indigenous context and the needs of Indigenous health care workers.

Model for the evaluation of drug-dispensing services in primary health care

OBJECTIVE To develop a model for evaluating the efficacy of drug-dispensing service in primary health care. METHODS An efficacy criterion was adopted to determine the level of achievement of the service objectives. The evaluation model was developed on the basis of a literature search and discussions with experts. The applicability test of the model was conducted in 15 primary health care units in the city of Florianópolis, state of Santa Catarina, in 2010, and data were recorded in structured and pretested questionnaires. RESULTS The model developed was evaluated using five dimensions of analysis for analysis. The model was suitable for evaluating service efficacy and helped to identify the critical points of each service dimension. CONCLUSIONS Adaptations to the data collection technique may be required to adjust for the reality and needs of each situation. The evaluation of the drug-dispensing service should promote adequate access to medications supplied through the public health system.

Validity and reliability of a health care service evaluation instrument for tuberculosis

OBJECTIVE To evaluate the validity and reliability of an instrument that evaluates the structure of primary health care units for the treatment of tuberculosis.METHODS This cross-sectional study used simple random sampling and evaluated 1,037 health care professionals from five Brazilian municipalities (Natal, state of Rio Grande do Norte; Cabedelo, state of Paraíba; Foz do Iguaçu, state of Parana; Sao José do Rio Preto, state of Sao Paulo, and Uberaba, state of Minas Gerais) in 2011. Structural indicators were identified and validated, considering different methods of organization of the health care system in the municipalities of different population sizes. Each structure represented the organization of health care services and contained the resources available for the execution of health care services: physical resources (equipment, consumables, and facilities); human resources (number and qualification); and resources for maintenance of the existing infrastructure and technology (deemed as the organization of health care services). The statistical analyses used in the validation process included reliability analysis, exploratory factor analysis, and confirmatory factor analysis.RESULTS The validation process indicated the retention of five factors, with 85.9% of the total variance explained, internal consistency between 0.6460 and 0.7802, and quality of fit of the confirmatory factor analysis of 0.995 using the goodness-of-fit index. The retained factors comprised five structural indicators: professionals involved in the care of tuberculosis patients, training, access to recording instruments, availability of supplies, and coordination of health care services with other levels of care. Availability of supplies had the best performance and the lowest coefficient of variation among the services evaluated. The indicators of assessment of human resources and coordination with other levels of care had satisfactory performance, but the latter showed the highest coefficient of variation. The performance of the indicators “training” and “access to recording instruments” was inferior to that of other indicators.CONCLUSIONS The instrument showed feasibility of application and potential to assess the structure of primary health care units for the treatment of tuberculosis.

Cost evaluation of cardiovascular magnetic resonance versus coronary angiography for the diagnostic work-up of coronary artery disease: Application of the European Cardiovascular Magnetic Resonance registry data to the German, United Kingdom, Swiss, and United States health care systems.

ABSTRACT: BACKGROUND: Cardiovascular magnetic resonance (CMR) has favorable characteristics for diagnostic evaluation and risk stratification of patients with known or suspected CAD. CMR utilization in CAD detection is growing fast. However, data on its cost-effectiveness are scarce. The goal of this study is to compare the costs of two strategies for detection of significant coronary artery stenoses in patients with suspected coronary artery disease (CAD): 1) Performing CMR first to assess myocardial ischemia and/or infarct scar before referring positive patients (defined as presence of ischemia and/or infarct scar to coronary angiography (CXA) versus 2) a hypothetical CXA performed in all patients as a single test to detect CAD. METHODS: A subgroup of the European CMR pilot registry was used including 2,717 consecutive patients who underwent stress-CMR. From these patients, 21% were positive for CAD (ischemia and/or infarct scar), 73% negative, and 6% uncertain and underwent additional testing. The diagnostic costs were evaluated using invoicing costs of each test performed. Costs analysis was performed from a health care payer perspective in German, United Kingdom, Swiss, and United States health care settings. RESULTS: In the public sectors of the German, United Kingdom, and Swiss health care systems, cost savings from the CMR-driven strategy were 50%, 25% and 23%, respectively, versus outpatient CXA. If CXA was carried out as an inpatient procedure, cost savings were 46%, 50% and 48%, respectively. In the United States context, cost savings were 51% when compared with inpatient CXA, but higher for CMR by 8% versus outpatient CXA. CONCLUSION: This analysis suggests that from an economic perspective, the use of CMR should be encouraged as a management option for patients with suspected CAD.

Evaluation of the Irish Pilot Programme for the Education of Health Care Assistants

Click here to download PDF

Can vouchers deliver? An evaluation of subsidies for maternal health care in Cambodia

OBJECTIVE: To evaluate the effect of vouchers for maternity care in public health-care facilities on the utilization of maternal health-care services in Cambodia. METHODS: The study involved data from the 2010 Cambodian Demographic and Health Survey, which covered births between 2005 and 2010. The effect of voucher schemes, first implemented in 2007, on the utilization of maternal health-care services was quantified using a difference-in-differences method that compared changes in utilization in districts with voucher schemes with changes in districts without them. FINDINGS: Overall, voucher schemes were associated with an increase of 10.1 percentage points (pp) in the probability of delivery in a public health-care facility; among women from the poorest 40% of households, the increase was 15.6 pp. Vouchers were responsible for about one fifth of the increase observed in institutional deliveries in districts with schemes. Universal voucher schemes had a larger effect on the probability of delivery in a public facility than schemes targeting the poorest women. Both types of schemes increased the probability of receiving postnatal care, but the increase was significant only for non-poor women. Universal, but not targeted, voucher schemes significantly increased the probability of receiving antenatal care. CONCLUSION: Voucher schemes increased deliveries in health centres and, to a lesser extent, improved antenatal and postnatal care. However, schemes that targeted poorer women did not appear to be efficient since these women were more likely than less poor women to be encouraged to give birth in a public health-care facility, even with universal voucher schemes.

An evaluation of Primary health care system in Kerala

The thesis entitled An Evaluation of Primary Health Care System in Kerala. The present study is intended to examine the working of primary health care system and its impact on the health status of people. The hypothesis tested in the thesis includes, a. The changes in the health profile require reallocation of resources of primary health care system, b. Rate of utilization depends on the quality of services provided by primary health centers, and c. There is a significant decline in the operational efficiency of the primary health care system. The major elements of primary health care stated in the report of AlmaAta International Conference on Primary Health Care (WHO, 1994)” is studied on the basis of the classification of the elements in to three: Preventive, Promotive, and Curative measures. Preventive measures include Maternal and Child Health Care including family Planning. Provision of water and sanitation is reviewed under promotive measures. Curative measures are studied using the disease profile of the study area. Collection of primary data was done through a sample survey, using pre-tested interview schedule of households of the study area. Multi stage random sampling design was used for selecting the sample. The design of the present study is both descriptive and analytical in nature. As far as the analytical tools are concerned, growth index, percentages, ratios, rates, time series analysis, analysis of variance, chi square test, Z test were used for analyzing the data. Present study revealed that no one in these areas was covered under any type of health insurance. Conclusion states that considering the present changes in the health profile, traditional pattern of resource allocation should be altered to meet the urgent health care needs of the people. Preventive and promotive measures like health education for giving awareness among people to change health habits, diet pattern, life style etc. are to be developed. Proper diagnosis and treatment of the disease at the beginning of the stage itself may help to cure majority of disease. For that, Public health policy must ensure the primary health care as enunciated at Alma- Ata international Conference. At the same time Public health is not to be treated as the sole responsibility of the government. Active community participation is an essential means to attain the goals.

Economic evaluation of latent tuberculosis infection screening among health care workers in Houston

Since interferon-gamma release assays (IGRAs) were introduced in the 2000's, tuberculin skin testing (TST) and IGRAs have been used in various latent tuberculosis infection (LTBI) screening settings. IGRAs are laboratory-based tests and are considered not to be affected by previous Bacille de Calmette et Guérin (BCG) vaccination; however, they are more costly when compared directly with TST, which does not require specimen processing in a laboratory. This study aimed to examine TST and two types of IGRAs, QuantiFERON-TB Gold in Tube (QFT-GIT) and T-SPOT. TB (TSPOT), from an economic viewpoint. Firstly, a systematic literature review was conducted to identify cost related analyses of LTBI screening. Secondly, specific cost information detailing each test's items and labor was collected from an LTBI screening program of health care workers in Houston, and the cost of each test was computed. Thirdly, using the computed cost estimate of each test, cost-effectiveness analyses were conducted to compare TST and IGRAs.^ A literature search showed that a limited number of studies have been conducted, but the IGRA's economic advantages were common among studies. Cost analyses showed that IGRAs were much more costly than TST. The results were consistent with previous studies. In cost-effectiveness analyses, where test cost and consequential TB-related cost were considered, IGRAs showed variable advantages over TST depending on the targeted population. When only non BCG-vaccinated people were considered, TST was the least costly option among the three tests. On the other hand, when only BCG-vaccinated people were considered, IGRAs were less costly options. These results were mostly consistent even with varying assumption parameters.^ IGRAs can be more costly than TST, but their economic disadvantages are alleviated when the target population was BCG-vaccinated. Based on current knowledge, IGRAs may be recommended in a population where the BCG history is mixed. Additional studies are needed to better understand IGRA's reliability among low-incidence and low-risk populations in which background TB prevalence is low.^

Why do health economists promote technology adoption rather than the search for efficiency? A proposal for a change in our approach to economic evaluation in health care

Acknowledgments Financial Support: HERU and HSRU receive a core grant from the Chief Scientist’s Office of the Scottish Government Health and Social Care Directorates, and the Centre for Clinical epidemiology & Evaluation is funded by Vancouver Coastal Health Authority. The model used for the illustrative case study in this paper was developed as part of a NHS Technology Assessment Review, funded by the National Institute for Health Research (NIHR) Health Technology Assessment Program (project number 09/146/01). The views and opinions expressed in this paper are those of the authors and do not necessarily reflect those of the Scottish Government, NHS, Vancouver Coastal Health, NIHR HTA Program or the Department of Health. The authors wish to thank Kathleen Boyd and members of the audience at the UK Health Economists Study Group, for comments received on an earlier version of this paper. We also wish to thank Cynthia Fraser (University of Aberdeen) for literature searches undertaken to inform the manuscript, and Mohsen Sadatsafavi (University of British Columbia) for comments on an earlier draft

How do primary health care teams learn to integrate intimate partner violence (IPV) management? A realist evaluation protocol

Background: Despite the existence of ample literature dealing, on the one hand, with the integration of innovations within health systems and team learning, and, on the other hand, with different aspects of the detection and management of intimate partner violence (IPV) within healthcare facilities, research that explores how health innovations that go beyond biomedical issues—such as IPV management—get integrated into health systems, and that focuses on healthcare teams’ learning processes is, to the best of our knowledge, very scarce if not absent. This realist evaluation protocol aims to ascertain: why, how, and under what circumstances primary healthcare teams engage (if at all) in a learning process to integrate IPV management in their practices; and why, how, and under what circumstances team learning processes lead to the development of organizational culture and values regarding IPV management, and the delivery of IPV management services. Methods: This study will be conducted in Spain using a multiple-case study design. Data will be collected from selected cases (primary healthcare teams) through different methods: individual and group interviews, routinely collected statistical data, documentary review, and observation. Cases will be purposively selected in order to enable testing the initial middle-range theory (MRT). After in-depth exploration of a limited number of cases, additional cases will be chosen for their ability to contribute to refining the emerging MRT to explain how primary healthcare learn to integrate intimate partner violence management. Discussion: Evaluations of health sector responses to IPV are scarce, and even fewer focus on why, how, and when the healthcare services integrate IPV management. There is a consensus that healthcare professionals and healthcare teams play a key role in this integration, and that training is important in order to realize changes. However, little is known about team learning of IPV management, both in terms of how to trigger such learning and how team learning is connected with changes in organizational culture and values, and in service delivery. This realist evaluation protocol aims to contribute to this knowledge by conducting this project in a country, Spain, where great endeavours have been made towards the integration of IPV management within the health system.