Appraising qualitative research for inclusion in systematic reviews:A quantitative and qualitative comparison of three methods

Objective: Qualitative research is increasingly valued as part of the evidence for policy and practice, but how it should be appraised is contested. Various appraisal methods, including checklists and other structured approaches, have been proposed but rarely evaluated. We aimed to compare three methods for appraising qualitative research papers that were candidates for inclusion in a systematic review of evidence on support for breast-feeding. Method: A sample of 12 research papers on support for breast-feeding was appraised by six qualitative reviewers using three appraisal methods: unprompted judgement, based on expert opinion; a UK Cabinet Office quality framework; and CASP, a Critical Appraisal Skills Programme tool. Papers were assigned, following appraisals, to 1 of 5 categories, which were dichotomized to indicate whether or not papers should be included in a systematic review. Patterns of agreement in categorization of papers were assessed quantitatively using κ statistics, and qualitatively using cross-case analysis. Results: Agreement in categorizing papers across the three methods was slight (κ =0.13; 95% CI 0.06-0.24). Structured approaches did not appear to yield higher agreement than that by unprompted judgement. Qualitative analysis revealed reviewers' dilemmas in deciding between the potential impact of findings and the quality of the research execution or reporting practice. Structured instruments appeared to make reviewers more explicit about the reasons for their judgements. Conclusions: Structured approaches may not produce greater consistency of judgements about whether to include qualitative papers in a systematic review. Future research should address how appraisals of qualitative research should be incorporated in systematic reviews. © The Royal Society of Medicine Press Ltd 2007.

The problem of appraising qualitative research

Qualitative research can make a valuable contribution to the study of quality and safety in health care. Sound ways of appraising qualitative research are needed, but currently there are many different proposals with few signs of an emerging consensus. One problem has been the tendency to treat qualitative research as a unified field. We distinguish universal features of quality from those specific to methodology and offer a set of minimally prescriptive prompts to assist with the assessment of generic features of qualitative research. In using these, account will need to be taken of the particular method of data collection and methodological approach being used. There may be a need for appraisal criteria suited to the different methods of qualitative data collection and to different methodological approaches. These more specific criteria would help to distinguish fatal flaws from more minor errors in the design, conduct, and reporting of qualitative research. There will be difficulties in doing this because some aspects of qualitative research, particularly those relating to quality of insight and interpretation, will remain difficult to appraise and will rely largely on subjective judgement.

Large scale organisational intervention to improve patient safety in four UK hospitals:mixed method evaluation

Objectives: To conduct an independent evaluation of the first phase of the Health Foundation's Safer Patients Initiative (SPI), and to identify the net additional effect of SPI and any differences in changes in participating and non-participating NHS hospitals. Design: Mixed method evaluation involving five substudies, before and after design. Setting: NHS hospitals in United Kingdom. Participants: Four hospitals (one in each country in the UK) participating in the first phase of the SPI (SPI1); 18 control hospitals. Intervention: The SPI1 was a compound (multicomponent) organisational intervention delivered over 18 months that focused on improving the reliability of specific frontline care processes in designated clinical specialties and promoting organisational and cultural change. Results: Senior staff members were knowledgeable and enthusiastic about SPI1. There was a small (0.08 points on a 5 point scale) but significant (P<0.01) effect in favour of the SPI1 hospitals in one of 11 dimensions of the staff questionnaire (organisational climate). Qualitative evidence showed only modest penetration of SPI1 at medical ward level. Although SPI1 was designed to engage staff from the bottom up, it did not usually feel like this to those working on the wards, and questions about legitimacy of some aspects of SPI1 were raised. Of the five components to identify patients at risk of deterioration - monitoring of vital signs (14 items); routine tests (three items); evidence based standards specific to certain diseases (three items); prescribing errors (multiple items from the British National Formulary); and medical history taking (11 items) - there was little net difference between control and SPI1 hospitals, except in relation to quality of monitoring of acute medical patients, which improved on average over time across all hospitals. Recording of respiratory rate increased to a greater degree in SPI1 than in control hospitals; in the second six hours after admission recording increased from 40% (93) to 69% (165) in control hospitals and from 37% (141) to 78% (296) in SPI1 hospitals (odds ratio for "difference in difference" 2.1, 99% confidence interval 1.0 to 4.3; P=0.008). Use of a formal scoring system for patients with pneumonia also increased over time (from 2% (102) to 23% (111) in control hospitals and from 2% (170) to 9% (189) in SPI1 hospitals), which favoured controls and was not significant (0.3, 0.02 to 3.4; P=0.173). There were no improvements in the proportion of prescription errors and no effects that could be attributed to SPI1 in non-targeted generic areas (such as enhanced safety culture). On some measures, the lack of effect could be because compliance was already high at baseline (such as use of steroids in over 85% of cases where indicated), but even when there was more room for improvement (such as in quality of medical history taking), there was no significant additional net effect of SPI1. There were no changes over time or between control and SPI1 hospitals in errors or rates of adverse events in patients in medical wards. Mortality increased from 11% (27) to 16% (39) among controls and decreased from17%(63) to13%(49) among SPI1 hospitals, but the risk adjusted difference was not significant (0.5, 0.2 to 1.4; P=0.085). Poor care was a contributing factor in four of the 178 deaths identified by review of case notes. The survey of patients showed no significant differences apart from an increase in perception of cleanliness in favour of SPI1 hospitals. Conclusions The introduction of SPI1 was associated with improvements in one of the types of clinical process studied (monitoring of vital signs) and one measure of staff perceptions of organisational climate. There was no additional effect of SPI1 on other targeted issues nor on other measures of generic organisational strengthening.

Focus group method and methodology:current practice and recent debate

This paper considers the contemporary use of focus groups as a method of data collection within qualitative research settings. The authors draw upon their own experiences of using focus groups in educational and 'community' user-group environments in order to provide an overview of recent issues and debates surrounding the deployment of focus group methods and to pick out specific areas of contention in relation to both their epistemological and practical implications. Accordingly, the paper reflects on some of the realities of 'doing' focus groups whilst, at the same time, highlighting common problems and dilemmas which beginning researchers might encounter in their application. In turn, the paper raises a number of related issues around which there appears to have been a lack of academic discussion to date.

Leontief Model Analysis with Fuzzy Parameters by Basic Matrixes Method

The basic matrixes method is suggested for the Leontief model analysis (LM) with some of its components indistinctly given. LM can be construed as a forecast task of product’s expenses-output on the basis of the known statistic information at indistinctly given several elements’ meanings of technological matrix, restriction vector and variables’ limits. Elements of technological matrix, right parts of restriction vector LM can occur as functions of some arguments. In this case the task’s dynamic analog occurs. LM essential complication lies in inclusion of variables restriction and criterion function in it.

Help-seeking for medically unexplained hearing difficulties:a qualitative study

Aims - Up to 10% of audiology patients report diffi culties hearing speech in noise even though clinical investigation reveals normal hearing thresholds, in other words, no evidence of physical pathology. The diagnostic category applied to these patients is known as King-Kopetzky Syndrome (KKS). This study aimed to gather descriptions of patients' experiences of the clinical encounter involving their KKS diagnosis and analyse the themes of help-seeking, as part of a larger study into the process of coping with medically unexplained hearing diffi culties. Method - A qualitative approach was employed, comprising unstructured interviews in the homes of 25 patients who had attended audiology services (and received a diagnosis of KKS) in Bath and Cardiff. Thematic analysis of transcripts was undertaken, infl uenced by grounded theory techniques. Findings - Informants characterized the clinical encounter as either negative or positive. Negative consultations were those in which patients' illness claims were dismissed and as such not validated. Positive encounters were typifi ed by the provision of meaningful information that reconciled clinical information with the patients' experiences of hearing loss. Conclusion - Successful management of medically unexplained illnesses requires the adoption of a patient-centred approach, rather than focusing on the absence of observable pathology

A qualitative investigation of decision making during help-seeking for adult hearing loss

Objective: The Any Qualified Provider framework in the National Health Service has changed the way adult audiology services are offered in England. Under the new rules, patients are being offered a choice in geographical location and audiology provider. This study aimed to explore how choices in treatment are presented and to identify what information patients need when they are seeking help with hearing loss. Design: This study adopted qualitative methods of ethnographic observations and focus group interviews to identify information needed prior to, and during, help-seeking. Observational data and focus group data were analysed using the constant comparison method of grounded theory. Study sample: Participants were recruited from a community Health and Social Care Trust in the west of England. This service incorporates both an Audiology and a Hearing Therapy service. Twenty seven participants were involved in focus groups or interviews. Results: Participants receive little information beyond the detail of hearing aids. Participants report little information that was not directly related to uptake of hearing aids. Conclusions: Participant preferences were not explored and limited information resulted in decisions that were clinician-led. The gaps in information reflect previous data on clinician communication and highlight the need for consistent information on a range of interventions to manage hearing loss.

Perceptions and experiences of the implementation, management, use and optimisation of electronic prescribing systems in hospital settings:protocol for a systematic review of qualitative studies

Introduction: There is increasing evidence that electronic prescribing (ePrescribing) or computerised provider/physician order entry (CPOE) systems can improve the quality and safety of healthcare services. However, it has also become clear that their implementation is not straightforward and may create unintended or undesired consequences once in use. In this context, qualitative approaches have been particularly useful and their interpretative synthesis could make an important and timely contribution to the field. This review will aim to identify, appraise and synthesise qualitative studies on ePrescribing/CPOE in hospital settings, with or without clinical decision support. Methods and analysis: Data sources will include the following bibliographic databases: MEDLINE, MEDLINE In Process, EMBASE, PsycINFO, Social Policy and Practice via Ovid, CINAHL via EBSCO, The Cochrane Library (CDSR, DARE and CENTRAL databases), Nursing and Allied Health Sources, Applied Social Sciences Index and Abstracts via ProQuest and SCOPUS. In addition, other sources will be searched for ongoing studies (ClinicalTrials.gov) and grey literature: Healthcare Management Information Consortium, Conference Proceedings Citation Index (Web of Science) and Sociological abstracts. Studies will be independently screened for eligibility by 2 reviewers. Qualitative studies, either standalone or in the context of mixed-methods designs, reporting the perspectives of any actors involved in the implementation, management and use of ePrescribing/CPOE systems in hospital-based care settings will be included. Data extraction will be conducted by 2 reviewers using a piloted form. Quality appraisal will be based on criteria from the Critical Appraisal Skills Programme checklist and Standards for Reporting Qualitative Research. Studies will not be excluded based on quality assessment. A postsynthesis sensitivity analysis will be undertaken. Data analysis will follow the thematic synthesis method. Ethics and dissemination: The study does not require ethical approval as primary data will not be collected. The results of the study will be published in a peer-reviewed journal and presented at relevant conferences.

Grounded theory, a kvalitatív kutatás klasszikus mérföldköve (Grounded theory, the classic milestone of qualitative research)

A grounded theory olyan kutatási módszer, ahol az elmélet az empirikus adatokból fejlődik ki és abban gyökerezik (Glaser - Strauss, 1967). Annak ellenére, hogy a módszer nemzetközileg rendkívül népszerű, a grounded theory nem igazán vert gyökeret a hazai menedzsment- és marketingkutatás talajában. A cikk célja a grounded theory kulcsfontosságú mozzanatainak bemutatása és a módszer népszerűsítése. / === / Grounded theory is a research method in which theory emerges from the data and is grounded in it (Glaser - Strauss, 1967). Despite the international popularity of the method, the grounded theory unfortunately hasn't taken roots in the Hungarian management and marketing research fields. The purpose of this article is to highlight some features and key points of the method and to popularize it among researchers.

A mixed method study of organizational culture dimensions of community college faculty library use and perceptions

Because past research has shown faculty as the driving force affecting student academic library use, librarians have tried for decades to engage classroom faculty in library activities. Nevertheless, a low rate of library use by faculty on behalf of their students persists. This study investigated the organizational culture dimensions affecting library faculty demand at a community college. The study employed a sequential quantitative-qualitative research design. A random sample of full-time faculty at a large urban community college responded to a 46-item survey. The survey data showed strong espoused support (84%) for the use of library-based materials but a much lower incidence of putting this construct into practice (46%). Interviews were conducted with 11 full-time faculty from two academic groups, English-Humanities and Engineering-Math-Science. These groups were selected because the survey data resulted in statistically significant differences between the groups pertaining to several key variables. These variables concerned the professors' perceptions of the importance of library research in their discipline, the amount of time spent on the course textbook during a term, the frequency of conversations about the library in the academic department, and the professors' ratings of the librarians' skill in instruction related to the academic discipline. All interviewees described the student culture as the predominant organizational culture at Major College. Although most interview subjects held to high information literacy standards in their courses, others were less convinced these could be realistically practiced, based on a perception of students' poor academic skills, lack of time for students to complete assignments due to their commuter and family responsibilities, and the need to focus on textbook content. Recommended future research would involve investigation of methods to bridge the gap between high espoused value toward information literacy and implementation of information-literate coursework.

Organizing and Interpreting Unstructured Qualitative Data

In the last several years there has been an increase in the amount of qualitative research using in-depth interviews and comprehensive content analyses in sport psychology. However, no explicit method has been provided to deal with the large amount of unstructured data. This article provides common guidelines for organizing and interpreting unstructured data. Two main operations are suggested and discussed: first, coding meaningful text segments, or creating tags, and second, regrouping similar text segments,or creating categories. Furthermore, software programs for the microcomputer are presented as away to facilitate the organization and interpretation of qualitative data

‘Potentially inappropriate or specifically appropriate?’ Qualitative evaluation of general practitioners views on prescribing, polypharmacy and potentially inappropriate prescribing in older people

Background: Potentially inappropriate prescribing (PIP) is common in older people in primary care, as evidenced by a significant body of quantitative research. However, relatively few qualitative studies have investigated the phenomenon of PIP and its underlying processes from the perspective of general practitioners (GPs). The aim of this paper is to explore qualitatively, GP perspectives regarding prescribing and PIP in older primary care patients.

Method: Semi-structured qualitative interviews were conducted with GPs participating in a randomised controlled trial (RCT) of an intervention to decrease PIP in older patients (≥70 years) in Ireland. Interviews were conducted with GP participants (both intervention and control) from the OPTI-SCRIPT cluster RCT as part of the trial process evaluation between January and July 2013. Interviews were conducted by one interviewer and audio recorded. Interviews were transcribed verbatim and a thematic analysis was conducted.

Results: Seventeen semi-structured interviews were conducted (13 male; 4 female). Three main, inter-related themes emerged (complex prescribing environment, paternalistic doctor-patient relationship, and relevance of PIP concept). Patient complexity (e.g. polypharmacy, multimorbidity), as well as prescriber complexity (e.g. multiple prescribers, poor communication, restricted autonomy) were all identified as factors contributing to a complex prescribing environment where PIP could occur, as was a paternalistic-doctor patient relationship. The concept of PIP was perceived to be of variable usefulness to GPs and the criteria to measure it may be at odds with the complex processes of prescribing for this patient population.

Conclusions: Several inter-related factors contributing to the occurrence of PIP were identified, some of which may be amenable to intervention. Improvement strategies focused on improved management of polypharmacy and multimorbidity, and communication across primary and secondary care could result in substantial improvements in PIP.

Qualitative Comparative Analysis as a Tool for Concept Clarification, Typology Building, and Contextualized Comparisons in Gender and Feminist Research

Qualitative Comparative Analysis (QCA) is a method for the systematic analysis of cases. A holistic view of cases and an approach to causality emphasizing complexity are some of its core features. Over the last decades, QCA has found application in many fields of the social sciences. In spite of this, its use in feminist research has been slower, and only recently QCA has been applied to topics related to social care, the political representation of women, and reproductive politics. In spite of the comparative turn in feminist studies, researchers still privilege qualitative methods, in particular case studies, and are often skeptical of quantitative techniques (Spierings 2012). These studies show that the meaning and measurement of many gender concepts differ across countries and that the factors leading to feminist success and failure are context specific. However, case study analyses struggle to systematically account for the ways in which these forces operate in different locations.

What is health and what is important for its achievement? : A qualitative study on adolescent boys’ perceptions and experiences of health

Few qualitative studies have explored adolescent boys’ perceptions of health. Aim: The aim of this study was therefore to explore how adolescent boys understand the concept of health and what they find important for its achievement Methods: Grounded theory was used as a method to analyse interviews with 33 adolescent boys aged 16 to 17 years attending three upper secondary schools in a relatively small town in Sweden. Results: There was a complexity in how health was perceived, experienced, dealt with, and valued. Although health on a conceptual level was described as ‘holistic’, health was experienced and dealt with in a more dualistic manner, one in which the boys were prone to differentiate between mind and body. Health was experienced as mainly emotional and relational, whereas the body had a subordinate value. The presence of positive emotions, experiencing self-esteem, balance in life, trustful relationships, and having a sense of belonging were important factors for health while the body was experienced as a tool to achieve health, as energy, and as a condition. Conclusion: Our findings indicate that young, masculine health is largely experienced through emotions and relationships and thus support theories on health as a social construction of interconnected processes. 

The Accuracy and Reliability of Traditional Surface Flow Type Mapping: Is It Time for A New Method of Characterising Physical River Habitat?

Surface flow types (SFT) are advocated as ecologically relevant hydraulic units, often mapped visually from the bankside to characterise rapidly the physical habitat of rivers. SFT mapping is simple, non-invasive and cost-efficient. However, it is also qualitative, subjective and plagued by difficulties in recording accurately the spatial extent of SFT units. Quantitative validation of the underlying physical habitat parameters is often lacking, and does not consistently differentiate between SFTs. Here, we investigate explicitly the accuracy, reliability and statistical separability of traditionally mapped SFTs as indicators of physical habitat, using independent, hydraulic and topographic data collected during three surveys of a c. 50m reach of the River Arrow, Warwickshire, England. We also explore the potential of a novel remote sensing approach, comprising a small unmanned aerial system (sUAS) and Structure-from-Motion photogrammetry (SfM), as an alternative method of physical habitat characterisation. Our key findings indicate that SFT mapping accuracy is highly variable, with overall mapping accuracy not exceeding 74%. Results from analysis of similarity (ANOSIM) tests found that strong differences did not exist between all SFT pairs. This leads us to question the suitability of SFTs for characterising physical habitat for river science and management applications. In contrast, the sUAS-SfM approach provided high resolution, spatially continuous, spatially explicit, quantitative measurements of water depth and point cloud roughness at the microscale (spatial scales ≤1m). Such data are acquired rapidly, inexpensively, and provide new opportunities for examining the heterogeneity of physical habitat over a range of spatial and temporal scales. Whilst continued refinement of the sUAS-SfM approach is required, we propose that this method offers an opportunity to move away from broad, mesoscale classifications of physical habitat (spatial scales 10-100m), and towards continuous, quantitative measurements of the continuum of hydraulic and geomorphic conditions which actually exists at the microscale.