Viewing systematic reviews and meta-analysis in social research through different lenses

Systematic reviews and meta-analyses are used to combine results across studies to determine an overall effect. Meta-analysis is especially useful for combining evidence to inform social policy, but meta-analyses of applied social science research may encounter practical issues arising from the nature of the research domain. The current paper identifies potential resolutions to four issues that may be encountered in systematic reviews and meta-analyses in social research. The four issues are: scoping and targeting research questions appropriate for meta-analysis; selecting eligibility criteria where primary studies vary in research design and choice of outcome measures; dealing with inconsistent reporting in primary studies; and identifying sources of heterogeneity with multiple confounded moderators. The paper presents an overview of each issue with a review of potential resolutions, identified from similar issues encountered in meta-analysis in medical and biological sciences. The discussion aims to share and improve methodology in systematic reviews and meta-analysis by promoting cross-disciplinary communication, that is, to encourage 'viewing through different lenses'.

Effectiveness and cost-effectiveness of traditional and new partner notification technologies for curable sexually transmitted infections: observational study, systematic reviews and mathematical modelling

BACKGROUND Partner notification is essential to the comprehensive case management of sexually transmitted infections. Systematic reviews and mathematical modelling can be used to synthesise information about the effects of new interventions to enhance the outcomes of partner notification. OBJECTIVE To study the effectiveness and cost-effectiveness of traditional and new partner notification technologies for curable sexually transmitted infections (STIs). DESIGN Secondary data analysis of clinical audit data; systematic reviews of randomised controlled trials (MEDLINE, EMBASE and Cochrane Central Register of Controlled Trials) published from 1 January 1966 to 31 August 2012 and of studies of health-related quality of life (HRQL) [MEDLINE, EMBASE, ISI Web of Knowledge, NHS Economic Evaluation Database (NHS EED), Database of Abstracts of Reviews of Effects (DARE) and Health Technology Assessment (HTA)] published from 1 January 1980 to 31 December 2011; static models of clinical effectiveness and cost-effectiveness; and dynamic modelling studies to improve parameter estimation and examine effectiveness. SETTING General population and genitourinary medicine clinic attenders. PARTICIPANTS Heterosexual women and men. INTERVENTIONS Traditional partner notification by patient or provider referral, and new partner notification by expedited partner therapy (EPT) or its UK equivalent, accelerated partner therapy (APT). MAIN OUTCOME MEASURES Population prevalence; index case reinfection; and partners treated per index case. RESULTS Enhanced partner therapy reduced reinfection in index cases with curable STIs more than simple patient referral [risk ratio (RR) 0.71; 95% confidence interval (CI) 0.56 to 0.89]. There are no randomised trials of APT. The median number of partners treated for chlamydia per index case in UK clinics was 0.60. The number of partners needed to treat to interrupt transmission of chlamydia was lower for casual than for regular partners. In dynamic model simulations, > 10% of partners are chlamydia positive with look-back periods of up to 18 months. In the presence of a chlamydia screening programme that reduces population prevalence, treatment of current partners achieves most of the additional reduction in prevalence attributable to partner notification. Dynamic model simulations show that cotesting and treatment for chlamydia and gonorrhoea reduce the prevalence of both STIs. APT has a limited additional effect on prevalence but reduces the rate of index case reinfection. Published quality-adjusted life-year (QALY) weights were of insufficient quality to be used in a cost-effectiveness study of partner notification in this project. Using an intermediate outcome of cost per infection diagnosed, doubling the efficacy of partner notification from 0.4 to 0.8 partners treated per index case was more cost-effective than increasing chlamydia screening coverage. CONCLUSIONS There is evidence to support the improved clinical effectiveness of EPT in reducing index case reinfection. In a general heterosexual population, partner notification identifies new infected cases but the impact on chlamydia prevalence is limited. Partner notification to notify casual partners might have a greater impact than for regular partners in genitourinary clinic populations. Recommendations for future research are (1) to conduct randomised controlled trials using biological outcomes of the effectiveness of APT and of methods to increase testing for human immunodeficiency virus (HIV) and STIs after APT; (2) collection of HRQL data should be a priority to determine QALYs associated with the sequelae of curable STIs; and (3) standardised parameter sets for curable STIs should be developed for mathematical models of STI transmission that are used for policy-making. FUNDING The National Institute for Health Research Health Technology Assessment programme.

Reporting and handling missing outcome data in mental health: a systematic review of Cochrane systematic reviews and meta-analyses

OBJECTIVES The purpose of the study was to provide empirical evidence about the reporting of methodology to address missing outcome data and the acknowledgement of their impact in Cochrane systematic reviews in the mental health field. METHODS Systematic reviews published in the Cochrane Database of Systematic Reviews after January 1, 2009 by three Cochrane Review Groups relating to mental health were included. RESULTS One hundred ninety systematic reviews were considered. Missing outcome data were present in at least one included study in 175 systematic reviews. Of these 175 systematic reviews, 147 (84%) accounted for missing outcome data by considering a relevant primary or secondary outcome (e.g., dropout). Missing outcome data implications were reported only in 61 (35%) systematic reviews and primarily in the discussion section by commenting on the amount of the missing outcome data. One hundred forty eligible meta-analyses with missing data were scrutinized. Seventy-nine (56%) of them had studies with total dropout rate between 10 and 30%. One hundred nine (78%) meta-analyses reported to have performed intention-to-treat analysis by including trials with imputed outcome data. Sensitivity analysis for incomplete outcome data was implemented in less than 20% of the meta-analyses. CONCLUSIONS Reporting of the techniques for handling missing outcome data and their implications in the findings of the systematic reviews are suboptimal.

Dasatinib, nilotinib and standard-dose imatinib for the first line treatment of chronic myeloid leukaemia: Systematic reviews and economic analyses

- Background Nilotinib and dasatinib are now being considered as alternative treatments to imatinib as a first-line treatment of chronic myeloid leukaemia (CML). - Objective This technology assessment reviews the available evidence for the clinical effectiveness and cost-effectiveness of dasatinib, nilotinib and standard-dose imatinib for the first-line treatment of Philadelphia chromosome-positive CML. - Data sources Databases [including MEDLINE (Ovid), EMBASE, Current Controlled Trials, ClinicalTrials.gov, the US Food and Drug Administration website and the European Medicines Agency website] were searched from search end date of the last technology appraisal report on this topic in October 2002 to September 2011. - Review methods A systematic review of clinical effectiveness and cost-effectiveness studies; a review of surrogate relationships with survival; a review and critique of manufacturer submissions; and a model-based economic analysis. - Results Two clinical trials (dasatinib vs imatinib and nilotinib vs imatinib) were included in the effectiveness review. Survival was not significantly different for dasatinib or nilotinib compared with imatinib with the 24-month follow-up data available. The rates of complete cytogenetic response (CCyR) and major molecular response (MMR) were higher for patients receiving dasatinib than for those with imatinib for 12 months' follow-up (CCyR 83% vs 72%, p < 0.001; MMR 46% vs 28%, p < 0.0001). The rates of CCyR and MMR were higher for patients receiving nilotinib than for those receiving imatinib for 12 months' follow-up (CCyR 80% vs 65%, p < 0.001; MMR 44% vs 22%, p < 0.0001). An indirect comparison analysis showed no difference between dasatinib and nilotinib for CCyR or MMR rates for 12 months' follow-up (CCyR, odds ratio 1.09, 95% CI 0.61 to 1.92; MMR, odds ratio 1.28, 95% CI 0.77 to 2.16). There is observational association evidence from imatinib studies supporting the use of CCyR and MMR at 12 months as surrogates for overall all-cause survival and progression-free survival in patients with CML in chronic phase. In the cost-effectiveness modelling scenario, analyses were provided to reflect the extensive structural uncertainty and different approaches to estimating OS. First-line dasatinib is predicted to provide very poor value for money compared with first-line imatinib, with deterministic incremental cost-effectiveness ratios (ICERs) of between £256,000 and £450,000 per quality-adjusted life-year (QALY). Conversely, first-line nilotinib provided favourable ICERs at the willingness-to-pay threshold of £20,000-30,000 per QALY. - Limitations Immaturity of empirical trial data relative to life expectancy, forcing either reliance on surrogate relationships or cumulative survival/treatment duration assumptions. - Conclusions From the two trials available, dasatinib and nilotinib have a statistically significant advantage compared with imatinib as measured by MMR or CCyR. Taking into account the treatment pathways for patients with CML, i.e. assuming the use of second-line nilotinib, first-line nilotinib appears to be more cost-effective than first-line imatinib. Dasatinib was not cost-effective if decision thresholds of £20,000 per QALY or £30,000 per QALY were used, compared with imatinib and nilotinib. Uncertainty in the cost-effectiveness analysis would be substantially reduced with better and more UK-specific data on the incidence and cost of stem cell transplantation in patients with chronic CML. - Funding The Health Technology Assessment Programme of the National Institute for Health Research.

Researching 'what works' in early intervention and prevention of homelessness : From systematic reviews to collaborative inquiry

Mechanisms of intervention and the contexts they are used in interact in complex ways. This helps explain why we can’t overgeneralize about what works in respect of models of service designed to prevent or respond to homelessness. This said there are some key messages from the totality of evidence that has been accumulated to date. First homelessness would be a lot easier to prevent for first or subsequent episodes if adequate and appropriate (developmentally/ culturally) housing was available. Second (and often dependent on the first) timely support of a particular character ‘works’ both in a preventive sense and in periods when people experience ongoing challenges which may render them vulnerable to further homelessness. This paper reflects on some of the critical features of how we can generate and use evidence, and how these complement each other in important ways.

The benefits and challenges of conducting an overview of systematic reviews in public health: a focus on physical activity

Introduction The ultimate aim of Cochrane systematic reviews is to inform policy and practice decisions for better health outcomes. However, due to the increasing numbers of scientific publications, wading through the available evidence of both individual studies and systematic reviews can be challenging and overwhelming even for avid authors and readers. This paper briefly describes the first overview (a systematic review of reviews) of the Cochrane Public Health Group (CPHG) in development and proposes a way forward for the methodologies under consideration.

Public health interventions for increasing physical activity in children, adolescents and adults : an overview of systematic reviews

This is the protocol for a review and there is no abstract. The objectives are as follows: This overview intends to: a) summarise the existing evidence on interventions that aim to increase PA; b) explore whether any effects of the intervention are different within and between populations, and whether these differences form an equity gradient such as an effect that differs according the advantage/disadvantage (e.g. low income and ethnic minorities); c) highlight gaps in the present evidence base that may warrant a Cochrane systematic review to be completed; and c) identify 'up to date' Cochrane reviews. .

Pension structural reforms, re-reforms and their effects on social welfare: The case of Chile

This article fi rst summarizes the structural reforms of pensions (total or partial privatization) in Latin America and Central and Eastern Europe, identifying their advantages and disadvantages, and does the same with the international process of re-reforms of pensions with a greater role of the state. Second, chooses Chile as a case study, as a world pioneer in both types of reforms; describes their characteristics and effects on social welfare of the structural reform of 1981 and the re-reform of 2008. Such effects are evaluated based on ten basic principles of social security from the International Labour Offi ce (ILO): 1) social dialogue to approve the reforms, 2) universal coverage of the population, 3) equal treatment of insured persons, 4) social solidarity, 5) gender equity, 6) suffi ciency of benefi ts, 7) effi ciency and reasonable administrative cost, 8) social participation in the management of the system, 9) role of the state and supervision, and 10) fi nancial sustainability. Third, it summarizes the advantages and disadvantages-challenges of the re-reform and informs on the current debate for further reforms.

Public health interventions in midwifery: a systematic review of systematic reviews

Background: Maternity care providers, particularly midwives, have a window of opportunity to influence pregnant women about positive health choices. This aim of this paper is to identify evidence of effective public health interventions from good quality systematic reviews that could be conducted by midwives.

Methods: Relevant databases including MEDLINE, Pubmed, EBSCO, CRD, MIDIRS, Web of Science, The Cochrane Library and Econlit were searched to identify systematic reviews in October 2010. Quality assessment of all reviews was conducted.

Results: Thirty-six good quality systematic reviews were identified which reported on effective interventions. The reviews were conducted on a diverse range of interventions across the reproductive continuum and were categorised under: screening; supplementation; support; education; mental health; birthing environment; clinical care in labour and breast feeding. The scope and strength of the review findings are discussed in relation to current practice. A logic model was developed to provide an overarching framework of midwifery public health roles to inform research policy and practice.

Conclusions: This review provides a broad scope of high quality systematic review evidence and definitively highlights the challenge of knowledge transfer from research into practice. The review also identified gaps in knowledge around the impact of core midwifery practice on public health outcomes and the value of this contribution. This review provides evidence for researchers and funders as to the gaps in current knowledge and should be used to inform the strategic direction of the role of midwifery in public health in policy and practice.

A systematic review of systematic reviews to improve maternal mental health and well-being

Objective: to identify non-invasive interventions in the perinatal period that could enable midwives to offer effective support to women within the area of maternal mental health and well-being.

Methods: a total of 9 databases were searched: MEDLINE, PubMed, EBSCO (CINAHL/British Nursing Index), MIDIRS Online Database, Web of Science, The Cochrane library, CRD (NHS EED/DARE/HTA), Joanne Briggs Institute and EconLit. A systematic search strategy was formulated using key MeSH terms and related text words for midwifery, study aim, study design and mental health. Inclusion criteria were articles published from 1999 onwards, English language publications and articles originating from economically developed countries, indicated by membership of the Organisation for Economic Co-operation and Development (OECD). Data were independently extracted using a data collection form, which recorded data on the number of papers reviewed, time frame of the review, objectives, key findings and recommendations. Summary data tables were set up outlining key data for each study and findings were organised into related groups. The methodological quality of the reviews was assessed based on predefined quality assessment criteria for reviews.

Findings: 32 reviews were identified as examining interventions that could be used or co-ordinated by midwives in relation to some aspect of maternal mental health and well-being from the antenatal to the postnatal period and met the inclusion criteria. The review highlighted that based on current systematic review evidence it would be premature to consider introducing any of the identified interventions into midwifery training or practice. However there were a number of examples of possible interventions worthy of further research including midwifery led models of care in the prevention of postpartum depression, psychological and psychosocial interventions for treating postpartum depression and facilitation/co-ordination of parent-training programmes. No reviews were identified that supported a specific midwifery role in maternal mental health and well-being in pregnancy, and yet, this is the point of most intensive contact.

Key conclusions and implications for practice: This systematic review of systematic reviews provides a valuable overview of the current strengths and gaps in relation to maternal mental health interventions in the perinatal period. While there was little evidence identified to inform the current role of midwives in maternal mental health, the review provides the opportunity to reflect on what is achievable by midwives now and in the future and the need for high quality randomised controlled trials to inform a strategic approach to promoting maternal mental health in midwifery.

Not all systematic reviews are systematic : a meta-review of the quality of current systematic reviews and meta-analyses for remote monitoring in heart failure

Background/aims: Remote monitoring for heart failure has not only been evaluated in a large number of randomised controlled trials, but also in many systematic reviews and meta-analyses. The aim of this meta-review was to identify, appraise and synthesise existing systematic reviews that have evaluated the effects of remote monitoring in heart failure. Methods: Using a Cochrane methodology, we electronically searched all relevant online databases and search engines, performed a forward citation search as well as hand-searched bibliographies. Only fully published systematic reviews of invasive and/or non-invasive remote monitoring interventions were included. Two reviewers independently extracted data. Results: Sixty-five publications from 3333 citations were identified. Seventeen fulfilled the inclusion and exclusion criteria. Quality varied with A Measurement Tool to Assess Systematic Reviews (AMSTAR scores) ranging from 2 to 11 (mean 5.88). Seven reviews (41%) pooled results from individual studies for meta-analysis. Eight (47%) considered all non-invasive remote monitoring strategies. Four (24%) focused specifically on telemonitoring. Four (24%) included studies investigating both non-invasive and invasive technologies. Population characteristics of the included studies were not reported consistently. Mortality and hospitalisations were the most frequently reported outcomes 12 (70%). Only five reviews (29%) reported healthcare costs and compliance. A high degree of heterogeneity was reported in many of the meta-analyses. Conclusions: These results should be considered in context of two negative RCTs of remote monitoring for heart failure that have been published since the meta-analyses (TIM-HF and Tele-HF). However, high quality reviews demonstrated improved mortality, quality of life, reduction in hospitalisations and healthcare costs.

Queensland's budget austerity and its impact on social welfare : is the cure worse than the disease?

While considerable attention has been paid to the austerity experiments in Europe, much less attention has been paid to austerity case studies from other parts of the world. This paper examines the case of Queensland, Australia, where the government has pursued austerity measures, while making dire warnings that unless public debt was slashed and the public service sector downsized,Queensland risked becoming the Spain of Australia. The comparison is incomprehensible, given the very different economic situation in Queensland compared with Spain. This comparison constructed a sense of crisis that helped to mask standard neoliberal economic reform. While pursuing neoliberal economic policies,the Queensland Government has also been introducing draconian laws that limit civil liberties and political freedoms for ordinary citizens. This mix of authoritarianism and austerity has met considerable resistance, and this dynamic is discussed in the paper, along with the predictable and unequal impact that austerity measures have had on the general population and social services.