Cautionary tales in the interpretation of systematic reviews of therapy trials

This is the second in a series of articles emphasizing the cautions in the interpretation of health-care studies. Systematic reviews are presented as comprehensive, unbiased summaries of evidence and are often referred to by clinicians, guideline developers and health policy-makers. Their strengths and limitations, and how their results can be subject to bias and misinterpretation, are discussed.

Appraising qualitative research for inclusion in systematic reviews:A quantitative and qualitative comparison of three methods

Objective: Qualitative research is increasingly valued as part of the evidence for policy and practice, but how it should be appraised is contested. Various appraisal methods, including checklists and other structured approaches, have been proposed but rarely evaluated. We aimed to compare three methods for appraising qualitative research papers that were candidates for inclusion in a systematic review of evidence on support for breast-feeding. Method: A sample of 12 research papers on support for breast-feeding was appraised by six qualitative reviewers using three appraisal methods: unprompted judgement, based on expert opinion; a UK Cabinet Office quality framework; and CASP, a Critical Appraisal Skills Programme tool. Papers were assigned, following appraisals, to 1 of 5 categories, which were dichotomized to indicate whether or not papers should be included in a systematic review. Patterns of agreement in categorization of papers were assessed quantitatively using κ statistics, and qualitatively using cross-case analysis. Results: Agreement in categorizing papers across the three methods was slight (κ =0.13; 95% CI 0.06-0.24). Structured approaches did not appear to yield higher agreement than that by unprompted judgement. Qualitative analysis revealed reviewers' dilemmas in deciding between the potential impact of findings and the quality of the research execution or reporting practice. Structured instruments appeared to make reviewers more explicit about the reasons for their judgements. Conclusions: Structured approaches may not produce greater consistency of judgements about whether to include qualitative papers in a systematic review. Future research should address how appraisals of qualitative research should be incorporated in systematic reviews. © The Royal Society of Medicine Press Ltd 2007.

How can systematic reviews incorporate qualitative research? A critical perspective

Systematic review has developed as a specific methodology for searching for, appraising and synthesizing findings of primary studies, and has rapidly become a cornerstone of the evidence-based practice and policy movement. Qualitative research has traditionally been excluded from systematic reviews, and much effort is now being invested in resolving the daunting methodological and epistemological challenges associated with trying to move towards more inclusive forms of review. We describe our experiences, as a very diverse multidisciplinary group, in attempting to incorporate qualitative research in a systematic review of support for breastfeeding. We show how every stage of the review process, from asking the review question through to searching for and sampling the evidence, appraising the evidence and producing a synthesis, provoked profound questions about whether a review that includes qualitative research can remain consistent with the frame offered by current systematic review methodology. We conclude that more debate and dialogue between the different communities that wish to develop review methodology is needed, and that attempts to impose dominant views about the appropriate means of conducting reviews of qualitative research should be resisted so that innovation can be fostered. Copyright © 2006 SAGE Publications.

How can we use systematic reviews to inform clinical practice?

Aims/purpose: Systematic reviews provide the highest quality evidence and inform clinical practice. For patient benefit, it is imperative that nurses keep abreast of evidence-based practice. This presentation highlights where to find systematic reviews and how the information presented can be used to inform care.
Presentation description: Clinical research is increasing at an incredible rate. In the clinical trials database alone, more than 2000 new studies are registered/month and this does not include qualitative studies that do not require registration. Keeping abreast of current evidence can not only be a time consuming process, but can be problematic when studies produce conflicting results. Systematic reviews can be useful for summarizing the increasing amount of knowledge that is gained from scientific papers. In addition, combining individual studies in a meta-analysis increases statistical power, resulting in more precise effect estimates. This presentation draws upon a few systematic reviews relevant to ICU nursing practice, highlights their findings and demonstrates how the information can be used to inform translation of evidence into practice. Additionally, although these reviews include steps to minimize bias, nurses should be aware of some of the biases that may reduce confidence in the findings.
Conclusion: Systematic reviews can be useful tools for informing evidence based practice, although careful interpretation is necessary for understanding their relevance to local practice.

Systematic reviews as a tool for planning and interpreting trials

Background Systematic reviews followed by ameta-analysis are carried out in medical research to combine the results of two or more related studies. Stroke trials have struggled to show beneficial effects and meta-analysis should be used more widely throughout the research process to either speed up the development of useful interventions, or halt more quickly research with hazardous or ineffective interventions. Summary of review. This review summarises the clinical research process and illustrates how and when systematic reviews may be used throughout the development programme. Meta-analyses should be performed after observational studies, preclinical studies in experimental stroke, and after phase I, II, and III clinical trials and phase IV clinical surveillance studies. Although meta-analyses most commonly work with summary data, they may be performed to assess relationships between variables (meta-regression) and, ideally, should utilise individual patient data. Meta-analysis techniques may alsoworkwith ordered categorical outcome data (ordinal meta-analysis) and be used to perform indirect comparisons where original trial data do not exist. Conclusion Systematic review/meta-analyses are powerful tools in medical research and should be used throughout the development of all stroke and other interventions

Clarifying the abstracts of systematic literature reviews*

Background: There is a small body of research on improving the clarity of abstracts in general that is relevant to improving the clarity of abstracts of systematic reviews.

The mental health and wellbeing of first generation migrants: a systematic-narrative review of reviews

Background
First generation migrants are reportedly at higher risk of mental ill-health compared to the settled population. This paper systematically reviews and synthesizes all reviews on the mental health of first generation migrants in order to appraise the risk factors for, and explain differences in, the mental health of this population.
Methods
Scientific databases were searched for systematic reviews (inception-November 2015) which provided quantitative data on the mental ill-health of first generation migrants and associated risk factors. Two reviewers screened titles, abstracts and full text papers for their suitability against pre-specified criteria, methodological quality was assessed.
Results
One thousand eight hundred twenty articles were identified, eight met inclusion criteria, which were all moderate or low quality. Depression was mostly higher in first generation migrants in general, and in refugees/asylum seekers when analysed separately. However, for both groups there was wide variation in prevalence rates, from 5 to 44 % compared with prevalence rates of 8–12 % in the general population. Post-Traumatic Stress Disorder prevalence was higher for both first generation migrants in general and for refugees/asylum seekers compared with the settled majority. Post-Traumatic Stress Disorder prevalence in first generation migrants in general and refugees/ asylum seekers ranged from 9 to 36 % compared with reported prevalence rates of 1–2 % in the general population. Few studies presented anxiety prevalence rates in first generation migrants and there was wide variation in those that did. Prevalence ranged from 4 to 40 % compared with reported prevalence of 5 % in the general population. Two reviews assessed the psychotic disorder risk, reporting this was two to three times more likely in adult first generation migrants. However, one review on the risk of schizophrenia in refugees reported similar prevalence rates (2 %) to estimates of prevalence among the settled majority (3 %). Risk factors for mental ill-health included low Gross National Product in the host country, downward social mobility, country of origin, and host country.
Conclusion
First generation migrants may be at increased risk of mental illness and public health policy must account for this and influencing factors. High quality research in the area is urgently needed as is the use of culturally specific validated measurement tools for assessing migrant mental health.

Association Between Unilateral Tonsillar Enlargement And Lymphoma In Children: A Systematic Review And Meta-analysis.

Lymphoma is the most common head and neck malignancy in children, and palatine tonsils asymmetry is the most frequent clinical manifestation of tonsillar lymphoma. However, several studies with children with tonsillar asymmetry found no case of lymphoma, showing that the relationship of tonsillar asymmetry with lymphoma is unclear. In this review, we aimed to identify the association between tonsillar asymmetry and tonsillar lymphoma in children by conducting systematic reviews of the literature on children with palatine tonsil lymphoma and tonsillar asymmetry. Articles comprising the paediatric age group (up to 18 years) with information concerning clinical manifestations of tonsillar lymphoma or the diagnosis of the tonsillar asymmetry were included. The main cause of asymmetry of palatine tonsils was lymphoid hyperplasia, followed by lymphoma and nonspecific benign changes. The asymmetry of tonsils was present in 73.2% of cases of lymphoma. There was an association between asymmetric palatine tonsils and lymphoma, with a likelihood ratio of 43.5 for children with asymmetry of palatine tonsils and 8938.4 for children with asymmetry of tonsils and other signs of suspicion for malignancy. We also provide recommendations on the management of suspicious cases of palatine tonsil lymphoma.