Viewing systematic reviews and meta-analysis in social research through different lenses

Systematic reviews and meta-analyses are used to combine results across studies to determine an overall effect. Meta-analysis is especially useful for combining evidence to inform social policy, but meta-analyses of applied social science research may encounter practical issues arising from the nature of the research domain. The current paper identifies potential resolutions to four issues that may be encountered in systematic reviews and meta-analyses in social research. The four issues are: scoping and targeting research questions appropriate for meta-analysis; selecting eligibility criteria where primary studies vary in research design and choice of outcome measures; dealing with inconsistent reporting in primary studies; and identifying sources of heterogeneity with multiple confounded moderators. The paper presents an overview of each issue with a review of potential resolutions, identified from similar issues encountered in meta-analysis in medical and biological sciences. The discussion aims to share and improve methodology in systematic reviews and meta-analysis by promoting cross-disciplinary communication, that is, to encourage 'viewing through different lenses'.

Dasatinib, nilotinib and standard-dose imatinib for the first line treatment of chronic myeloid leukaemia: Systematic reviews and economic analyses

- Background Nilotinib and dasatinib are now being considered as alternative treatments to imatinib as a first-line treatment of chronic myeloid leukaemia (CML). - Objective This technology assessment reviews the available evidence for the clinical effectiveness and cost-effectiveness of dasatinib, nilotinib and standard-dose imatinib for the first-line treatment of Philadelphia chromosome-positive CML. - Data sources Databases [including MEDLINE (Ovid), EMBASE, Current Controlled Trials, ClinicalTrials.gov, the US Food and Drug Administration website and the European Medicines Agency website] were searched from search end date of the last technology appraisal report on this topic in October 2002 to September 2011. - Review methods A systematic review of clinical effectiveness and cost-effectiveness studies; a review of surrogate relationships with survival; a review and critique of manufacturer submissions; and a model-based economic analysis. - Results Two clinical trials (dasatinib vs imatinib and nilotinib vs imatinib) were included in the effectiveness review. Survival was not significantly different for dasatinib or nilotinib compared with imatinib with the 24-month follow-up data available. The rates of complete cytogenetic response (CCyR) and major molecular response (MMR) were higher for patients receiving dasatinib than for those with imatinib for 12 months' follow-up (CCyR 83% vs 72%, p < 0.001; MMR 46% vs 28%, p < 0.0001). The rates of CCyR and MMR were higher for patients receiving nilotinib than for those receiving imatinib for 12 months' follow-up (CCyR 80% vs 65%, p < 0.001; MMR 44% vs 22%, p < 0.0001). An indirect comparison analysis showed no difference between dasatinib and nilotinib for CCyR or MMR rates for 12 months' follow-up (CCyR, odds ratio 1.09, 95% CI 0.61 to 1.92; MMR, odds ratio 1.28, 95% CI 0.77 to 2.16). There is observational association evidence from imatinib studies supporting the use of CCyR and MMR at 12 months as surrogates for overall all-cause survival and progression-free survival in patients with CML in chronic phase. In the cost-effectiveness modelling scenario, analyses were provided to reflect the extensive structural uncertainty and different approaches to estimating OS. First-line dasatinib is predicted to provide very poor value for money compared with first-line imatinib, with deterministic incremental cost-effectiveness ratios (ICERs) of between £256,000 and £450,000 per quality-adjusted life-year (QALY). Conversely, first-line nilotinib provided favourable ICERs at the willingness-to-pay threshold of £20,000-30,000 per QALY. - Limitations Immaturity of empirical trial data relative to life expectancy, forcing either reliance on surrogate relationships or cumulative survival/treatment duration assumptions. - Conclusions From the two trials available, dasatinib and nilotinib have a statistically significant advantage compared with imatinib as measured by MMR or CCyR. Taking into account the treatment pathways for patients with CML, i.e. assuming the use of second-line nilotinib, first-line nilotinib appears to be more cost-effective than first-line imatinib. Dasatinib was not cost-effective if decision thresholds of £20,000 per QALY or £30,000 per QALY were used, compared with imatinib and nilotinib. Uncertainty in the cost-effectiveness analysis would be substantially reduced with better and more UK-specific data on the incidence and cost of stem cell transplantation in patients with chronic CML. - Funding The Health Technology Assessment Programme of the National Institute for Health Research.

Researching 'what works' in early intervention and prevention of homelessness : From systematic reviews to collaborative inquiry

Mechanisms of intervention and the contexts they are used in interact in complex ways. This helps explain why we can’t overgeneralize about what works in respect of models of service designed to prevent or respond to homelessness. This said there are some key messages from the totality of evidence that has been accumulated to date. First homelessness would be a lot easier to prevent for first or subsequent episodes if adequate and appropriate (developmentally/ culturally) housing was available. Second (and often dependent on the first) timely support of a particular character ‘works’ both in a preventive sense and in periods when people experience ongoing challenges which may render them vulnerable to further homelessness. This paper reflects on some of the critical features of how we can generate and use evidence, and how these complement each other in important ways.

The benefits and challenges of conducting an overview of systematic reviews in public health: a focus on physical activity

Introduction The ultimate aim of Cochrane systematic reviews is to inform policy and practice decisions for better health outcomes. However, due to the increasing numbers of scientific publications, wading through the available evidence of both individual studies and systematic reviews can be challenging and overwhelming even for avid authors and readers. This paper briefly describes the first overview (a systematic review of reviews) of the Cochrane Public Health Group (CPHG) in development and proposes a way forward for the methodologies under consideration.

Public health interventions for increasing physical activity in children, adolescents and adults : an overview of systematic reviews

This is the protocol for a review and there is no abstract. The objectives are as follows: This overview intends to: a) summarise the existing evidence on interventions that aim to increase PA; b) explore whether any effects of the intervention are different within and between populations, and whether these differences form an equity gradient such as an effect that differs according the advantage/disadvantage (e.g. low income and ethnic minorities); c) highlight gaps in the present evidence base that may warrant a Cochrane systematic review to be completed; and c) identify 'up to date' Cochrane reviews. .

Not all systematic reviews are systematic : a meta-review of the quality of current systematic reviews and meta-analyses for remote monitoring in heart failure

Background/aims: Remote monitoring for heart failure has not only been evaluated in a large number of randomised controlled trials, but also in many systematic reviews and meta-analyses. The aim of this meta-review was to identify, appraise and synthesise existing systematic reviews that have evaluated the effects of remote monitoring in heart failure. Methods: Using a Cochrane methodology, we electronically searched all relevant online databases and search engines, performed a forward citation search as well as hand-searched bibliographies. Only fully published systematic reviews of invasive and/or non-invasive remote monitoring interventions were included. Two reviewers independently extracted data. Results: Sixty-five publications from 3333 citations were identified. Seventeen fulfilled the inclusion and exclusion criteria. Quality varied with A Measurement Tool to Assess Systematic Reviews (AMSTAR scores) ranging from 2 to 11 (mean 5.88). Seven reviews (41%) pooled results from individual studies for meta-analysis. Eight (47%) considered all non-invasive remote monitoring strategies. Four (24%) focused specifically on telemonitoring. Four (24%) included studies investigating both non-invasive and invasive technologies. Population characteristics of the included studies were not reported consistently. Mortality and hospitalisations were the most frequently reported outcomes 12 (70%). Only five reviews (29%) reported healthcare costs and compliance. A high degree of heterogeneity was reported in many of the meta-analyses. Conclusions: These results should be considered in context of two negative RCTs of remote monitoring for heart failure that have been published since the meta-analyses (TIM-HF and Tele-HF). However, high quality reviews demonstrated improved mortality, quality of life, reduction in hospitalisations and healthcare costs.

Queensland's budget austerity and its impact on social welfare : is the cure worse than the disease?

While considerable attention has been paid to the austerity experiments in Europe, much less attention has been paid to austerity case studies from other parts of the world. This paper examines the case of Queensland, Australia, where the government has pursued austerity measures, while making dire warnings that unless public debt was slashed and the public service sector downsized,Queensland risked becoming the Spain of Australia. The comparison is incomprehensible, given the very different economic situation in Queensland compared with Spain. This comparison constructed a sense of crisis that helped to mask standard neoliberal economic reform. While pursuing neoliberal economic policies,the Queensland Government has also been introducing draconian laws that limit civil liberties and political freedoms for ordinary citizens. This mix of authoritarianism and austerity has met considerable resistance, and this dynamic is discussed in the paper, along with the predictable and unequal impact that austerity measures have had on the general population and social services.

Dementia and the population health approach: Promise, pitfalls and progress. An Australian perspective

The increasing prevalence of dementia in Australia (and worldwide) over the next few decades poses enormous social, health and economic challenges. In the absence of a cure, strategies to prevent, delay the onset of, or reduce the impact of dementia are required to contain a growing disease burden, and health and care costs. A population health approach has the potential to substantially reduce the impact of dementia. Internationally, many countries have started to adopt population health strategies that incorporate elements of dementia prevention. The authors examine some of the elements of such an approach and barriers to its implementation. International dementia frameworks and strategies were reviewed to identify options utilized for a population health approach to dementia. Internationally and nationally, dementia frameworks are being developed that include population health approaches. Most of the frameworks identified included early diagnosis and intervention, and increasing community awareness as key objectives, while several included promotion of the links between a healthy lifestyle and reduced risk for dementia. A poor evidence base (especially for illness prevention), diagnostic and technical limitations, and policy and implementation issues are significant barriers in maximizing the promise of population health approaches in this area. The review and analysis of the population health approach to dementia will inform national and jurisdictional policy development.

Potential uses of Bayesian networks as tools for synthesis of systematic reviews of complex interventions

Bayesian networks (BNs) are tools for representing expert knowledge or evidence. They are especially useful for synthesising evidence or belief concerning a complex intervention, assessing the sensitivity of outcomes to different situations or contextual frameworks and framing decision problems that involve alternative types of intervention. Bayesian networks are useful extensions to logic maps when initiating a review or to facilitate synthesis and bridge the gap between evidence acquisition and decision-making. Formal elicitation techniques allow development of BNs on the basis of expert opinion. Such applications are useful alternatives to ‘empty’ reviews, which identify knowledge gaps but fail to support decision-making. Where review evidence exists, it can inform the development of a BN. We illustrate the construction of a BN using a motivating example that demonstrates how BNs can ensure coherence, transparently structure the problem addressed by a complex intervention and assess sensitivity to context, all of which are critical components of robust reviews of complex interventions. We suggest that BNs should be utilised to routinely synthesise reviews of complex interventions or empty reviews where decisions must be made despite poor evidence.

Population health planning and health promotion: Opportunities and challenges to improve social determinants

Complex social factors and health issues challenge equitable health outcomes for many people, in particular those living in marginalised communities. Primary health care promises solutions through population health and health promotion approaches to improve social conditions (determinants) affecting health with emphasis on change at systems levels. Yet short-term efficiency focus policy decisions without long-term planning can undermine the effectiveness of primary health care. The workshop goal is to explore opportunities and share ideas about population health planning in Primary Health Networks and other community health care settings, so as to draw out opportunities, challenges and forward thinking health planning and health promotion strategies.

Re-examining the evidence in radiation dermatitis management literature : an overview and a critical appraisal of systematic reviews

Purpose: To provide an overview and a critical appraisal of systematic reviews (SRs) of published interventions for the prevention/management of radiation dermatitis. Methods and Materials: We searched Medline, CINAHL, Embase, and the Cochrane Library. We also manually searched through individual reference lists of potentially eligible articles and a number of key journals in the topic area. Two authors screened all potential articles and included eligible SRs. Two authors critically appraised and extracted key findings from the included reviews using AMSTAR (the measurement tool for “assessment of multiple systematic reviews”). Results: Of 1837 potential titles, 6 SRs were included. A number of interventions have been reported to be potentially beneficial for managing radiation dermatitis. Interventions evaluated in these reviews included skin care advice, steroidal/nonsteroidal topical agents, systemic therapies, modes of radiation delivery, and dressings. However, all the included SRs reported that there is insufficient evidence supporting any single effective intervention. The methodological quality of the included studies varied, and methodological shortfalls in these reviews might create biases to the overall results or recommendations for clinical practice. Conclusions: An up-to-date high-quality SR in the prevention/management of radiation dermatitis is needed to guide practice and direction for future research. We recommend that clinicians or guideline developers critically evaluate the information of SRs in their decision making.

Two decades of exceptional achievements : does the evidence support nurses to favour Cochrane systematic reviews over other systematic reviews?

Systematic reviews (SRs) are increasingly recognised as the standard approach in summarising health research and influence clinical nursing practice and health care decisions (Coster and Norman, 2009, Grimshaw and Russell, 1993 and Griffiths and Norman, 2005). High quality SRs should have a clearly stated set of objectives with pre-defined eligibility criteria for studies; an explicit reproducible methodology; a systematic search that attempts to identify all studies that would meet the eligibility criteria; an assessment of the validity of the findings of the included studies; the assessment of risk of bias; and a systematic presentation and synthesis of the characteristics of findings of the included study (Higgins and Green, 2011). Although SRs are highly regarded and are expected to be rigorous, just as other research, their quality may vary (Choi et al., 2001 and Hoving et al., 2001)...

Spatial modelling of population health screening : the role of accessibility in predicting expected patient flows

This paper describes a generalised linear mixed model (GLMM) approach for understanding spatial patterns of participation in population health screening, in the presence of multiple screening facilities. The models presented have dual focus, namely the prediction of expected patient flows from regions to services and relative rates of participation by region- service combination, with both outputs having meaningful implications for the monitoring of current service uptake and provision. The novelty of this paper lies with the former focus, and an approach for distributing expected participation by region based on proximity to services is proposed. The modelling of relative rates of participation is achieved through the combination of different random effects, as a means of assigning excess participation to different sources. The methodology is applied to participation data collected from a government-funded mammography program in Brisbane, Australia.