Comparative analysis of ten assesment grids of quality criteria in qualitative research

The authors present 10 grids which are widely used in Health Sciences for the assessment of quality in research. They proceed through a comparative thematic analysis of these grids and show which points of view are preferred. They insist on the issues that differentiate these grids from each other and suggest the analysis of their differences by distinguishing the theoretical perspectives that underpin each one of these grids. Whilst the authors of the assessment grids rarely refer to the implicit theoretical backgrounds that guide their work, findings show that these grids convey varied epistemologies and research models. This gap renders the comparison of quality assessment in qualitative research a very difficult task, unless we shift our focus on the relationship between the grids, their theoretical backgrounds and their specific research subjects.

A Thematic Comparison of Eight Frameworks of Quality Criteria in Qualitative Health Research

Background: Despite the increasing incidences of the publication of assessment frameworks intending to establish the "standards" of the quality of qualitative research, the research conducted using such empirical methods are still facing difficulties in being published or recognised by funding agencies. Methods: We conducted a thematic content analysis of eight frameworks from psychology/psychiatry and general medicine. The frameworks and their criteria are then compared against each other. Findings: The results illustrated the difficulties in reaching consensus on the definition of quality criteria. This showed the differences between the frameworks from the point of views of the underlying epistemology and the criteria suggested. Discussion: The aforementioned differences reflect the diversity of paradigms implicitly referred to by the authors of the frameworks, although rarely explicitly mentioned in text. We conclude that the increase in qualitative research and publications has failed to overcome the difficulties in establishing shared criteria and the great heterogeneity of concepts raises methodological and epistemological problems.

The gendering of excellence through quality criteria: the case of the SNSF Professorships in Switzerland

The Swiss National Science Foundation Professorships Programme is presented as a scientific programme that aims to promote outstanding young scholars to professorial positions. Academic excellence is presented as the main selection criterion. The emphasis put on the research portfolio and on the age of the candidates means that the beneficiaries of these professorships put forward an image of excellence that is more embedded in data-based sciences than in the humanities and social sciences, thus strengthening the domination of a sector scientific activity essentially occupied by men over the sector that has opened up more widely to women. This paper aims to deconstruct the criteria of academic excellence as they appear in this programme and to show that what seem quality criteria are in fact inspired by a specific androcentric model. These biases tend to undermine the gender equality aims of the programme.

Critères de qualité en radiothérapie des cancers de la tête et du cou sous l'égide de l'Intergroupe ORL Quality criteria in radiotherapy for head and neck cancers under the aegis of Head and Neck Intergroup.

The aim of radiotherapy is to deliver enough radiation to the tumor in order to achieve maximum tumour control in the irradiated volume with as few serious complications as possible with an irradiation dose as low as possible to normal tissue. The quality of radiotherapy is essential for optimal treatment and quality control is to reduce the bias in clinical trials avoiding possible major deviations. The assurance and quality control programs have been developed in large european (EORTC, GORTEC) and american cooperative groups (RTOG) of radiation oncology since the 1980s. We insist here on the importance of quality assurance in radiotherapy and the current status in this domain and the criteria for quality control especially for current clinical trials within GORTEC are discussed here.

Diversity of the Quality Criteria in Qualitative Research in the Health Sciences: Lessons from a Lexicometric Analysis Composed of 133 Guidelines

A review of health sciences literature shows a substantial increase in qualitative publications. This work incorporates a certain number of research quality guidelines. We present the results of the Alceste® lexicometric analysis, which includes 133 quality grids for qualitative research covering five disciplinary fields of the health sciences: medicine and epidemiology, public health and health education, nursing, health sociology and anthropology, psychiatry and psychology. This analysis helped to cross-check the disciplinary fields with the various objectives assigned to the different criteria in the grids examined. The results obtained with Alceste® show the variability of the objectives sought by the authors of the guidelines. These discrepancies are not directly associated to disciplinary fields, and appear to be more closely linked to different qualitative research conceptualizations within the disciplines, and with essential qualitative research validation criteria. These conceptualizations must be clarified to help users better understand the objectives targeted by the grids, and promote more appreciation for qualitative research in the health sciences.

Complexity of the Paradigms Present in Quality Criteria of Qualitative Research Grids

With qualitative methods being increasingly used in health science fields, numerous grids proposing criteria to evaluate the quality of this type of research have been produced. Expert evaluators deem that there is a lack of consensual tools to evaluate qualitative research. Based on the review of 133 quality criteria grids for qualitative research in health sciences, the authors present the results of a computerized lexicometric analysis, which confirms the variety of intra- and inter-grid constructions, including within the same field. This variety is linked to the authors' paradigmatic references underlying the criteria proposed. These references seem to be built intuitively, reflecting internal representations of qualitative research, thus making the grids and their criteria hard to compare. Consequently, the consensus on the definitions and the number of criteria becomes problematic. The paradigmatic and theoretical references of the grids should be specified so that users could better assess their contributions and limitations.

Quality Assessment of Cardiac Magnetic Resonance Images at 1.5/3.0 T: Description and Validation of Standardized Criteria

PURPOSE: Cardiovascular magnetic resonance (CMR) has become a robust and important diagnostic imaging modality in cardiovascular medicine. However,insufficient image quality may compromise its diagnostic accuracy. No standardized criteria are available to assess the quality of CMR studies. We aimed todescribe and validate standardized criteria to evaluate the quality of CMR studies including: a) cine steady-state free precession, b) delayed gadoliniumenhancement, and c) adenosine stress first-pass perfusion. These criteria will serve for the assessment of the image quality in the setting of the Euro-CMR registry.METHOD AND MATERIALS: First, a total of 45 quality criteria were defined (35 qualitative criteria with a score from 0-3, and 10 quantitative criteria). Thequalitative score ranged from 0 to 105. The lower the qualitative score, the better the quality. The quantitative criteria were based on the absolute signal intensity (delayed enhancement) and on the signal increase (perfusion) of the anterior/posterior left ventricular wall after gadolinium injection. These criteria were then applied in 30 patients scanned with a 1.5T system and in 15 patients scanned with a 3.0T system. The examinations were jointly interpreted by 3 CMR experts and 1 study nurse. In these 45 patients the correlation between the results of the quality assessment obtained by the different readers was calculated.RESULTS: On the 1.5T machine, the mean quality score was 3.5. The mean difference between each pair of observers was 0.2 (5.7%) with a mean standarddeviation of 1.4. On the 3.0T machine, the mean quality score was 4.4. The mean difference between each pair of onservers was 0.3 (6.4%) with a meanstandard deviation of 1.6. The quantitative quality assessments between observers were well correlated for the 1.5T machine: R was between 0.78 and 0.99 (pCONCLUSION: The described criteria for the assessment of CMR image quality are robust and have a low inter-observer variability, especially on 1.5T systems.CLINICAL RELEVANCE/APPLICATION: These criteria will allow the standardization of CMR examinations. They will help to improve the overall quality ofexaminations and the comparison between clinical studies.

Quality assessment of cardiovascular magnetic resonance in the setting of the European CMR registry: description and validation of standardized criteria.

BACKGROUND: Cardiovascular magnetic resonance (CMR) has become an important diagnostic imaging modality in cardiovascular medicine. However, insufficient image quality may compromise its diagnostic accuracy. We aimed to describe and validate standardized criteria to evaluate a) cine steady-state free precession (SSFP), b) late gadolinium enhancement (LGE), and c) stress first-pass perfusion images. These criteria will serve for quality assessment in the setting of the Euro-CMR registry. METHODS: Thirty-five qualitative criteria were defined (scores 0-3) with lower scores indicating better image quality. In addition, quantitative parameters were measured yielding 2 additional quality criteria, i.e. signal-to-noise ratio (SNR) of non-infarcted myocardium (as a measure of correct signal nulling of healthy myocardium) for LGE and % signal increase during contrast medium first-pass for perfusion images. These qualitative and quantitative criteria were assessed in a total of 90 patients (60 patients scanned at our own institution at 1.5T (n=30) and 3T (n=30) and in 30 patients randomly chosen from the Euro-CMR registry examined at 1.5T). Analyses were performed by 2 SCMR level-3 experts, 1 trained study nurse, and 1 trained medical student. RESULTS: The global quality score was 6.7±4.6 (n=90, mean of 4 observers, maximum possible score 64), range 6.4-6.9 (p=0.76 between observers). It ranged from 4.0-4.3 for 1.5T (p=0.96 between observers), from 5.9-6.9 for 3T (p=0.33 between observers), and from 8.6-10.3 for the Euro-CMR cases (p=0.40 between observers). The inter- (n=4) and intra-observer (n=2) agreement for the global quality score, i.e. the percentage of assignments to the same quality tertile ranged from 80% to 88% and from 90% to 98%, respectively. The agreement for the quantitative assessment for LGE images (scores 0-2 for SNR <2, 2-5, >5, respectively) ranged from 78-84% for the entire population, and 70-93% at 1.5T, 64-88% at 3T, and 72-90% for the Euro-CMR cases. The agreement for perfusion images (scores 0-2 for %SI increase >200%, 100%-200%,<100%, respectively) ranged from 81-91% for the entire population, and 76-100% at 1.5T, 67-96% at 3T, and 62-90% for the Euro-CMR registry cases. The intra-class correlation coefficient for the global quality score was 0.83. CONCLUSIONS: The described criteria for the assessment of CMR image quality are robust with a good inter- and intra-observer agreement. Further research is needed to define the impact of image quality on the diagnostic and prognostic yield of CMR studies.

Quality of Qualitative Research in the Health Sciences : Analysis of the Common Criteria Present in 58 Assessment Guidelines by Expert Users

The number of qualitative research methods has grown substantially over the last twenty years, both in social sciences and, more recently, in the health sciences. This growth came with questions on the quality criteria needed to evaluate this work, and numerous guidelines were published. The latters include many discrepancies though, both in their vocabulary and construction. Many expert evaluators decry the absence of consensual and reliable evaluation tools. The authors present the results of an evaluation of 58 existing guidelines in 4 major health science fields (medicine and epidemiology; nursing and health education; social sciences and public health; psychology / psychiatry, research methods and organization) by expert users (article reviewers, experts allocating funds, editors, etc.). The results propose a toolbox containing 12 consensual criteria with the definitions given by expert users. They also indicate in which disciplinary field each type of criteria is known to be more or less essential. Nevertheless, the authors highlight the limitations of the criteria comparability, as soon as one focuses on their specific definitions. They conclude that each criterion in the toolbox must be explained to come to broader consensus and identify definitions that are consensual to all the fields examined and easily operational.

Risk assessment of herbicide mixtures in a large European lake

Lake Geneva is one of the largest European lakes with a surface area of 580 km2. Its catchment area covers 7400 km2, of which approximately 20% is arable land. Monitoring campaigns have been carried out in 2004 and 2005 to determine the contamination of the lake by pesticides. The results highlight the widespread presence of herbicides in water, the measured concentrations for most substances remaining constant in 2004 and 2005. However, for some individual herbicides the concentrations increased drastically (e.g., the herbicide foramsulfuron). We assessed the environmental risk of the herbicides detected in the lake using water quality criteria recently determined for the Swiss environmental protection agency. Furthermore, we assessed the risk of herbicide mixtures, grouped based upon their mode of action. Generally, the risk estimated for all single substances is low, except for some sulfonylurea compounds. For these substances, the measured concentrations are higher than the predicted no-effect concentration. Impact on the flora of the lake can therefore not be excluded. When mixtures of pesticides with similar mode of action are taken into account, the risk remains lower than the mixture water quality criteria for all groups, but can reach as high as one third of this quality criteria. A further step would therefore be to assess the risk of the total pesticide mixture, including similar and dissimilar modes of action

Scientific value of systematic reviews: survey of editors of core clinical journals.

BACKGROUND: Synthesizing research evidence using systematic and rigorous methods has become a key feature of evidence-based medicine and knowledge translation. Systematic reviews (SRs) may or may not include a meta-analysis depending on the suitability of available data. They are often being criticised as 'secondary research' and denied the status of original research. Scientific journals play an important role in the publication process. How they appraise a given type of research influences the status of that research in the scientific community. We investigated the attitudes of editors of core clinical journals towards SRs and their value for publication.¦METHODS: We identified the 118 journals labelled as "core clinical journals" by the National Library of Medicine, USA in April 2009. The journals' editors were surveyed by email in 2009 and asked whether they considered SRs as original research projects; whether they published SRs; and for which section of the journal they would consider a SR manuscript.¦RESULTS: The editors of 65 journals (55%) responded. Most respondents considered SRs to be original research (71%) and almost all journals (93%) published SRs. Several editors regarded the use of Cochrane methodology or a meta-analysis as quality criteria; for some respondents these criteria were premises for the consideration of SRs as original research. Journals placed SRs in various sections such as "Review" or "Feature article". Characterization of non-responding journals showed that about two thirds do publish systematic reviews.¦DISCUSSION: Currently, the editors of most core clinical journals consider SRs original research. Our findings are limited by a non-responder rate of 45%. Individual comments suggest that this is a grey area and attitudes differ widely. A debate about the definition of 'original research' in the context of SRs is warranted.

Risk Assessement of Herbicide Mixtures

Lake Geneva is one of the largest European lakes with a surface area of 580 km2. Its catchment area covers 7400 km2, of which approximately 20% is arable land. Monitoring campaigns have been carried out in 2004 and 2005 to determine the contamination of the lake by pesticides. The results highlight the widespread presence of herbicides in water, the measured concentrations for most substances remaining constant in 2004 and 2005. However, for some individual herbicides the concentrations increased drastically (e.g., the herbicide foramsulfuron). We assessed the environmental risk of the herbicides detected in the lake using water quality criteria recently determined for the Swiss environmental protection agency. Furthermore, we assessed the risk of herbicide mixtures, grouped based upon their mode of action. Generally, the risk estimated for all single substances is low, except for some sulfonylurea compounds. For these substances, the measured concentrations are higher than the predicted no-effect concentration. Impact on the flora of the lake can therefore not be excluded. When mixtures of pesticides with similar mode of action are taken into account, the risk remains lower than the mixture water quality criteria for all groups, but can reach as high as one third of this quality criteria. A further step would therefore be to assess the risk of the total pesticide mixture, including similar and dissimilar modes of action.

Communication du risque en médecine des voyages [Risk communication in travel medicine].

Les recommandations en termes de prévention contre la malaria pour les pays à risques modéré à faible diffèrent entre les pays, malgré le fait que les personnes soient exposées à un risque identique dans les pays qu'ils visitent. Pour inclure les voyageurs dans la réflexion, des outils de partage de la décision ont été développés et testés dans cette population. Leur utilisation a montré que la majorité des personnes choisissent de ne pas prendre de chimioprophylaxie en avançant des raisons valides. Le développement d'aides décisionnelles répondant à des critères reconnus est prévu et permettra d'améliorer la pertinence des recommandations. Les aides décisionnelles permettront aussi aux voyageurs de faire un choix de prévention avec les soignants au plus près de leurs valeurs et préférences, tout en respectant les règles de l'éthique médicale. Recommendations for malaria prevention for travelers planning a trip in medium to low risk countries differ between countries, despite the fact that people are exposed to the same risk in the travelled country. Decision aids have been developed and tested in a population of travelers planning a trip in such countries n order to present travelers the various prevention options and involve them in the decision. The use of the decision aid showed that he majority of people choose not to take chemoprophylaxis and that they could motivate their choice with valid reasons. The development of decision aids based on recognized quality criteria is foreseen; these will allow to improving the relevance of the recommendations and enable travelers to choose a prevention option that will be the closest to their values and preferences while following to the principles of medical ethics.