How valuable are multiple treatment comparison methods in evidence-based health-care evaluation?

Objectives To compare the use of pair-wise meta-analysis methods to multiple treatment comparison (MTC) methods for evidence-based health-care evaluation to estimate the effectiveness and cost-effectiveness of alternative health-care interventions based on the available evidence. Methods Pair-wise meta-analysis and more complex evidence syntheses, incorporating an MTC component, are applied to three examples: 1) clinical effectiveness of interventions for preventing strokes in people with atrial fibrillation; 2) clinical and cost-effectiveness of using drug-eluting stents in percutaneous coronary intervention in patients with coronary artery disease; and 3) clinical and cost-effectiveness of using neuraminidase inhibitors in the treatment of influenza. We compare the two synthesis approaches with respect to the assumptions made, empirical estimates produced, and conclusions drawn. Results The difference between point estimates of effectiveness produced by the pair-wise and MTC approaches was generally unpredictable—sometimes agreeing closely whereas in other instances differing considerably. In all three examples, the MTC approach allowed the inclusion of randomized controlled trial evidence ignored in the pair-wise meta-analysis approach. This generally increased the precision of the effectiveness estimates from the MTC model. Conclusions The MTC approach to synthesis allows the evidence base on clinical effectiveness to be treated as a coherent whole, include more data, and sometimes relax the assumptions made in the pair-wise approaches. However, MTC models are necessarily more complex than those developed for pair-wise meta-analysis and thus could be seen as less transparent. Therefore, it is important that model details and the assumptions made are carefully reported alongside the results.

The role of affect in consumer evaluation of health care services

Health care services are typically consumed out of necessity, typically to recover from illness. While the consumption of health care services can be emotional given that consumers experience fear, hope, relief, and joy, surprisingly, there is little research on the role of consumer affect in health care consumption. We propose that consumer affect is a heuristic cue that drives evaluation of health care services. Drawing from cognitive appraisal theory and affect-as-information theory, this article tests a research model (N = 492) that investigates consumer affect resulting from service performance on subsequent service outcomes.

Evaluation of a national evidence-based health care course via teleconference in a developing country

Rationale, aims and objectives Continuing health education is essential but challenged. in 2006, the Brazilian Cochrane Center, in collaboration with the Ministry of Health, launched a mass teaching initiative in evidence-based health care (EBH) for public-sector professionals via teleconferencing. This 152-hour, interactive EBH course has enrolled over 4500 professionals. This study aimed to assess the acquisition EBH knowledge and skills, as well as the attitudes and perceptions of a sample of students enrolled in the 2009 course via teleconferencing.Methods This prospective cohort study analyzed three aspects of this 152-hour EBH course that recruited 1040 volunteer participants, all public health sector employees working in 131 different hospitals or health agencies. Pre- and post-course tests using a modified version of the Berlin questionnaire with 20 multiple-choice questions were used to examine knowledge acquisition in a sample of 297 students. Tests were completed upon registration and at course completion. the research projects submitted by 872 participants were evaluated to assess skill acquisition. Answers to an anonymous survey assessed the attitudes and perceptions of 914 participants.Results There was a significant increase in knowledge from baseline to course completion (mean scores 8.2 +/- 3.3 versus 13.7 +/- 3.0, P < 0.001). Over 90% of the research projects were judged to be of adequate quality (appropriate rationale for the study, well-formulated research question and feasible execution); over 95% of the participants were satisfied with the course.Conclusion the Brazilian EBH course via teleconference improved the knowledge and skills of public-sector health professionals and was approved by the vast majority of students.

A framework for developing evaluation capacity in health care settings.

The purpose of this paper is to provide a framework for developing an effective evaluation practice within health care settings. Three features are reviewed; capacity building, the application of evaluation to program activities and the utilization of evaluation recommendations. First, the organizational elements required to establish effective evaluation practice are reviewed emphasizing that an organization's capacity for evaluation develops over time and in stages. Second, a comprehensive evaluation framework is presented which demonstrates how evaluation practice can be applied to all aspects of a program's life cycle, thus promoting the scope of evidence-based decision making within an organization. Finally, factors which influence the adoption of evaluation recommendations by decision makers are reviewed accompanied by strategies to promote the utilization of evaluation recommendations in organization decision making.

An evaluation of Primary health care system in Kerala

The thesis entitled An Evaluation of Primary Health Care System in Kerala. The present study is intended to examine the working of primary health care system and its impact on the health status of people. The hypothesis tested in the thesis includes, a. The changes in the health profile require reallocation of resources of primary health care system, b. Rate of utilization depends on the quality of services provided by primary health centers, and c. There is a significant decline in the operational efficiency of the primary health care system. The major elements of primary health care stated in the report of AlmaAta International Conference on Primary Health Care (WHO, 1994)” is studied on the basis of the classification of the elements in to three: Preventive, Promotive, and Curative measures. Preventive measures include Maternal and Child Health Care including family Planning. Provision of water and sanitation is reviewed under promotive measures. Curative measures are studied using the disease profile of the study area. Collection of primary data was done through a sample survey, using pre-tested interview schedule of households of the study area. Multi stage random sampling design was used for selecting the sample. The design of the present study is both descriptive and analytical in nature. As far as the analytical tools are concerned, growth index, percentages, ratios, rates, time series analysis, analysis of variance, chi square test, Z test were used for analyzing the data. Present study revealed that no one in these areas was covered under any type of health insurance. Conclusion states that considering the present changes in the health profile, traditional pattern of resource allocation should be altered to meet the urgent health care needs of the people. Preventive and promotive measures like health education for giving awareness among people to change health habits, diet pattern, life style etc. are to be developed. Proper diagnosis and treatment of the disease at the beginning of the stage itself may help to cure majority of disease. For that, Public health policy must ensure the primary health care as enunciated at Alma- Ata international Conference. At the same time Public health is not to be treated as the sole responsibility of the government. Active community participation is an essential means to attain the goals.

Using project evaluation to build capacity for integrated health care at local levels

Capacity-building evaluation featuring multidisciplinary cross-agency workshops fostered continuous quality improvement, while focusing on skills required and systemic barriers to health care integration between GPs and a regional hospital.

A cost-based equity weight for use in the economic evaluation of primary health care interventions: case study of the Australian Indigenous population

Background : Efficiency and equity are both important policy objectives in resource allocation. The discipline of health economics has traditionally focused on maximising efficiency, however addressing inequities in health also requires consideration. Methods to incorporate equity within economic evaluation techniques range from qualitative judgements to quantitative outcomes-based equity weights. Yet, due to definitional uncertainties and other inherent limitations, no method has been universally adopted to date. This paper proposes an alternative cost-based equity weight for use in the economic evaluation of interventions delivered from primary health care services.

Methods : Equity is defined in terms of 'access' to health services, with the vertical equity objective to achieve 'equitable access for unequal need'. Using the Australian Indigenous population as an illustrative case study, the magnitude of the equity weight is constructed using the ratio of the costs of providing specific interventions via Indigenous primary health care services compared with the costs of the same interventions delivered via mainstream services. Applying this weight to the costs of subsequent interventions deflates the costs of provision via Indigenous health services, and thus makes comparisons with mainstream more equitable when applied during economic evaluation.

Results : Based on achieving 'equitable access', existing measures of health inequity are suitable for establishing 'need', however the magnitude of health inequity is not necessarily proportional to the magnitude of resources required to redress it. Rather, equitable access may be better measured using appropriate methods of health service delivery for the target group. 'Equity of access' also suggests a focus on the processes of providing equitable health care rather than on outcomes, and therefore supports application of equity weights to the cost side rather than the outcomes side of the economic equation.

Conclusion : Cost-based weights have the potential to provide a pragmatic method of equity weight construction which is both understandable to policy makers and sensitive to the needs of target groups. It could improve the evidence base for resource allocation decisions, and be generalised to other disadvantaged groups who share similar concepts of equity. Development of this decision-making tool represents a potentially important avenue for further health economics research.

Development, implementation and evaluation of a clinical research engagement and leadership capacity building program in a large Australian health care service

BACKGROUND: Health professionals need to be integrated more effectively in clinical research to ensure that research addresses clinical needs and provides practical solutions at the coal face of care. In light of limited evidence on how best to achieve this, evaluation of strategies to introduce, adapt and sustain evidence-based practices across different populations and settings is required. This project aims to address this gap through the co-design, development, implementation, evaluation, refinement and ultimately scale-up of a clinical research engagement and leadership capacity building program in a clinical setting with little to no co-ordinated approach to clinical research engagement and education.

METHODS/DESIGN: The protocol is based on principles of research capacity building and on a six-step framework, which have previously led to successful implementation and long-term sustainability. A mixed methods study design will be used. Methods will include: (1) a review of the literature about strategies that engage health professionals in research through capacity building and/or education in research methods; (2) a review of existing local research education and support elements; (3) a needs assessment in the local clinical setting, including an online cross-sectional survey and semi-structured interviews; (4) co-design and development of an educational and support program; (5) implementation of the program in the clinical environment; and (6) pre- and post-implementation evaluation and ultimately program scale-up. The evaluation focuses on research activity and knowledge, attitudes and preferences about clinical research, evidence-based practice and leadership and post implementation, about their satisfaction with the program. The investigators will evaluate the feasibility and effect of the program according to capacity building measures and will revise where appropriate prior to scale-up.

DISCUSSION: It is anticipated that this clinical research engagement and leadership capacity building program will enable and enhance clinically relevant research to be led and conducted by health professionals in the health setting. This approach will also encourage identification of areas of clinical uncertainty and need that can be addressed through clinical research within the health setting.

Economic evaluation of latent tuberculosis infection screening among health care workers in Houston

Since interferon-gamma release assays (IGRAs) were introduced in the 2000's, tuberculin skin testing (TST) and IGRAs have been used in various latent tuberculosis infection (LTBI) screening settings. IGRAs are laboratory-based tests and are considered not to be affected by previous Bacille de Calmette et Guérin (BCG) vaccination; however, they are more costly when compared directly with TST, which does not require specimen processing in a laboratory. This study aimed to examine TST and two types of IGRAs, QuantiFERON-TB Gold in Tube (QFT-GIT) and T-SPOT. TB (TSPOT), from an economic viewpoint. Firstly, a systematic literature review was conducted to identify cost related analyses of LTBI screening. Secondly, specific cost information detailing each test's items and labor was collected from an LTBI screening program of health care workers in Houston, and the cost of each test was computed. Thirdly, using the computed cost estimate of each test, cost-effectiveness analyses were conducted to compare TST and IGRAs.^ A literature search showed that a limited number of studies have been conducted, but the IGRA's economic advantages were common among studies. Cost analyses showed that IGRAs were much more costly than TST. The results were consistent with previous studies. In cost-effectiveness analyses, where test cost and consequential TB-related cost were considered, IGRAs showed variable advantages over TST depending on the targeted population. When only non BCG-vaccinated people were considered, TST was the least costly option among the three tests. On the other hand, when only BCG-vaccinated people were considered, IGRAs were less costly options. These results were mostly consistent even with varying assumption parameters.^ IGRAs can be more costly than TST, but their economic disadvantages are alleviated when the target population was BCG-vaccinated. Based on current knowledge, IGRAs may be recommended in a population where the BCG history is mixed. Additional studies are needed to better understand IGRA's reliability among low-incidence and low-risk populations in which background TB prevalence is low.^

Why do health economists promote technology adoption rather than the search for efficiency? A proposal for a change in our approach to economic evaluation in health care

Acknowledgments Financial Support: HERU and HSRU receive a core grant from the Chief Scientist’s Office of the Scottish Government Health and Social Care Directorates, and the Centre for Clinical epidemiology & Evaluation is funded by Vancouver Coastal Health Authority. The model used for the illustrative case study in this paper was developed as part of a NHS Technology Assessment Review, funded by the National Institute for Health Research (NIHR) Health Technology Assessment Program (project number 09/146/01). The views and opinions expressed in this paper are those of the authors and do not necessarily reflect those of the Scottish Government, NHS, Vancouver Coastal Health, NIHR HTA Program or the Department of Health. The authors wish to thank Kathleen Boyd and members of the audience at the UK Health Economists Study Group, for comments received on an earlier version of this paper. We also wish to thank Cynthia Fraser (University of Aberdeen) for literature searches undertaken to inform the manuscript, and Mohsen Sadatsafavi (University of British Columbia) for comments on an earlier draft

How do primary health care teams learn to integrate intimate partner violence (IPV) management? A realist evaluation protocol

Background: Despite the existence of ample literature dealing, on the one hand, with the integration of innovations within health systems and team learning, and, on the other hand, with different aspects of the detection and management of intimate partner violence (IPV) within healthcare facilities, research that explores how health innovations that go beyond biomedical issues—such as IPV management—get integrated into health systems, and that focuses on healthcare teams’ learning processes is, to the best of our knowledge, very scarce if not absent. This realist evaluation protocol aims to ascertain: why, how, and under what circumstances primary healthcare teams engage (if at all) in a learning process to integrate IPV management in their practices; and why, how, and under what circumstances team learning processes lead to the development of organizational culture and values regarding IPV management, and the delivery of IPV management services. Methods: This study will be conducted in Spain using a multiple-case study design. Data will be collected from selected cases (primary healthcare teams) through different methods: individual and group interviews, routinely collected statistical data, documentary review, and observation. Cases will be purposively selected in order to enable testing the initial middle-range theory (MRT). After in-depth exploration of a limited number of cases, additional cases will be chosen for their ability to contribute to refining the emerging MRT to explain how primary healthcare learn to integrate intimate partner violence management. Discussion: Evaluations of health sector responses to IPV are scarce, and even fewer focus on why, how, and when the healthcare services integrate IPV management. There is a consensus that healthcare professionals and healthcare teams play a key role in this integration, and that training is important in order to realize changes. However, little is known about team learning of IPV management, both in terms of how to trigger such learning and how team learning is connected with changes in organizational culture and values, and in service delivery. This realist evaluation protocol aims to contribute to this knowledge by conducting this project in a country, Spain, where great endeavours have been made towards the integration of IPV management within the health system.

Mechanisms that Trigger a Good Health-Care Response to Intimate Partner Violence in Spain. Combining Realist Evaluation and Qualitative Comparative Analysis Approaches

Background. Health care professionals, especially those working in primary health-care services, can play a key role in preventing and responding to intimate partner violence. However, there are huge variations in the way health care professionals and primary health care teams respond to intimate partner violence. In this study we tested a previously developed programme theory on 15 primary health care center teams located in four different Spanish regions: Murcia, C Valenciana, Castilla-León and Cantabria. The aim was to identify the key combinations of contextual factors and mechanisms that trigger a good primary health care center team response to intimate partner violence. Methods. A multiple case-study design was used. Qualitative and quantitative information was collected from each of the 15 centers (cases). In order to handle the large amount of information without losing familiarity with each case, qualitative comparative analysis was undertaken. Conditions (context and mechanisms) and outcomes, were identified and assessed for each of the 15 cases, and solution formulae were calculated using qualitative comparative analysis software. Results. The emerging programme theory highlighted the importance of the combination of each team’s self-efficacy, perceived preparation and women-centredness in generating a good team response to intimate partner violence. The use of the protocol and accumulated experience in primary health care were the most relevant contextual/intervention conditions to trigger a good response. However in order to achieve this, they must be combined with other conditions, such as an enabling team climate, having a champion social worker and having staff with training in intimate partner violence. Conclusions. Interventions to improve primary health care teams’ response to intimate partner violence should focus on strengthening team’s self-efficacy, perceived preparation and the implementation of a woman-centred approach. The use of the protocol combined with a large working experience in primary health care, and other factors such as training, a good team climate, and having a champion social worker on the team, also played a key role. Measures to sustain such interventions and promote these contextual factors should be encouraged.