Appraising qualitative research for inclusion in systematic reviews:A quantitative and qualitative comparison of three methods

Objective: Qualitative research is increasingly valued as part of the evidence for policy and practice, but how it should be appraised is contested. Various appraisal methods, including checklists and other structured approaches, have been proposed but rarely evaluated. We aimed to compare three methods for appraising qualitative research papers that were candidates for inclusion in a systematic review of evidence on support for breast-feeding. Method: A sample of 12 research papers on support for breast-feeding was appraised by six qualitative reviewers using three appraisal methods: unprompted judgement, based on expert opinion; a UK Cabinet Office quality framework; and CASP, a Critical Appraisal Skills Programme tool. Papers were assigned, following appraisals, to 1 of 5 categories, which were dichotomized to indicate whether or not papers should be included in a systematic review. Patterns of agreement in categorization of papers were assessed quantitatively using κ statistics, and qualitatively using cross-case analysis. Results: Agreement in categorizing papers across the three methods was slight (κ =0.13; 95% CI 0.06-0.24). Structured approaches did not appear to yield higher agreement than that by unprompted judgement. Qualitative analysis revealed reviewers' dilemmas in deciding between the potential impact of findings and the quality of the research execution or reporting practice. Structured instruments appeared to make reviewers more explicit about the reasons for their judgements. Conclusions: Structured approaches may not produce greater consistency of judgements about whether to include qualitative papers in a systematic review. Future research should address how appraisals of qualitative research should be incorporated in systematic reviews. © The Royal Society of Medicine Press Ltd 2007.

How can systematic reviews incorporate qualitative research? A critical perspective

Systematic review has developed as a specific methodology for searching for, appraising and synthesizing findings of primary studies, and has rapidly become a cornerstone of the evidence-based practice and policy movement. Qualitative research has traditionally been excluded from systematic reviews, and much effort is now being invested in resolving the daunting methodological and epistemological challenges associated with trying to move towards more inclusive forms of review. We describe our experiences, as a very diverse multidisciplinary group, in attempting to incorporate qualitative research in a systematic review of support for breastfeeding. We show how every stage of the review process, from asking the review question through to searching for and sampling the evidence, appraising the evidence and producing a synthesis, provoked profound questions about whether a review that includes qualitative research can remain consistent with the frame offered by current systematic review methodology. We conclude that more debate and dialogue between the different communities that wish to develop review methodology is needed, and that attempts to impose dominant views about the appropriate means of conducting reviews of qualitative research should be resisted so that innovation can be fostered. Copyright © 2006 SAGE Publications.

Pharmacy Education and Accreditation Reviews (PEARs) project:final report

Report commissioned by the Pharmaceutical Society of Ireland.

Book reviews

Amor Technologiae: Marshall McLuhan as Philosopher of Technology – Toward a Philosophy of Human-Media Relationships, Yoni Van Den Eede (2012) Brussels, Belgium: ASP VUB Press, 517 pp., ISBN: 978-9057181870, p/bk, €29.95 Titanic Century: Media, Myth and the Making of a Cultural Icon, Paul Heyer (2012) Santa Barbara, CA: Praeger, 211 pp., ISBN 978-0-313-39815-5, h/bk, $48.00 Media Environments, Barry Vacker (ed.) (2010) San Diego, CA: Cognella, 546 pp., ISBN: 978-1935551348, p/bk, $142.50 Networked Reenactments: Stories Transdisciplinary Knowledges Tell, Katie King (2012) Durham, NC: Duke University Press, 392 pp., ISBN: 978-0822350729, p/bk, $25.95

Book reviews

Book reviewed: The Brain and Emotion by E. Rolls (2000). ix + 367 pp., ISBN 0-19-852463-3, Price £17.99, Oxford: Oxford University Press. Anxiety, Depression and Emotion by R.D. Davidson (Ed.). (2000). ISBN 0-19-513358-7, New York: Oxford University Press.

Leveraging product adopter information from online reviews for product recommendation

The availability of the sheer volume of online product reviews makes it possible to derive implicit demographic information of product adopters from review documents. This paper proposes a novel approach to the extraction of product adopter mentions from online reviews. The extracted product adopters are the ncategorise into a number of different demographic user groups. The aggregated demographic information of many product adopters can be used to characterize both products and users, which can be incorporated into a recommendation method using weighted regularised matrix factorisation. Our experimental results on over 15 million reviews crawled from JINGDONG, the largest B2C e-commerce website in China, show the feasibility and effectiveness of our proposed frame work for product recommendation.

Mining product adopter information from online reviews for improving product recommendation

We present in this article an automated framework that extracts product adopter information from online reviews and incorporates the extracted information into feature-based matrix factorization formore effective product recommendation. In specific, we propose a bootstrapping approach for the extraction of product adopters from review text and categorize them into a number of different demographic categories. The aggregated demographic information of many product adopters can be used to characterize both products and users in the form of distributions over different demographic categories. We further propose a graphbased method to iteratively update user- and product-related distributions more reliably in a heterogeneous user-product graph and incorporate them as features into the matrix factorization approach for product recommendation. Our experimental results on a large dataset crawled from JINGDONG, the largest B2C e-commerce website in China, show that our proposed framework outperforms a number of competitive baselines for product recommendation.

The discourse of online consumer reviews

Review: The Discourse of Online Consumer Reviews, Camilla Vásquez. Bloomsbury, London and New York (2014), 210 pp., £75, Hardback, ISBN: 978-1-4411-9628-6