Quality of Qualitative Research in the Health Sciences : Analysis of the Common Criteria Present in 58 Assessment Guidelines by Expert Users

The number of qualitative research methods has grown substantially over the last twenty years, both in social sciences and, more recently, in the health sciences. This growth came with questions on the quality criteria needed to evaluate this work, and numerous guidelines were published. The latters include many discrepancies though, both in their vocabulary and construction. Many expert evaluators decry the absence of consensual and reliable evaluation tools. The authors present the results of an evaluation of 58 existing guidelines in 4 major health science fields (medicine and epidemiology; nursing and health education; social sciences and public health; psychology / psychiatry, research methods and organization) by expert users (article reviewers, experts allocating funds, editors, etc.). The results propose a toolbox containing 12 consensual criteria with the definitions given by expert users. They also indicate in which disciplinary field each type of criteria is known to be more or less essential. Nevertheless, the authors highlight the limitations of the criteria comparability, as soon as one focuses on their specific definitions. They conclude that each criterion in the toolbox must be explained to come to broader consensus and identify definitions that are consensual to all the fields examined and easily operational.

Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries

BACKGROUND: The synthesis of published research in systematic reviews is essential when providing evidence to inform clinical and health policy decision-making. However, the validity of systematic reviews is threatened if journal publications represent a biased selection of all studies that have been conducted (dissemination bias). To investigate the extent of dissemination bias we conducted a systematic review that determined the proportion of studies published as peer-reviewed journal articles and investigated factors associated with full publication in cohorts of studies (i) approved by research ethics committees (RECs) or (ii) included in trial registries. METHODS AND FINDINGS: Four bibliographic databases were searched for methodological research projects (MRPs) without limitations for publication year, language or study location. The searches were supplemented by handsearching the references of included MRPs. We estimated the proportion of studies published using prediction intervals (PI) and a random effects meta-analysis. Pooled odds ratios (OR) were used to express associations between study characteristics and journal publication. Seventeen MRPs (23 publications) evaluated cohorts of studies approved by RECs; the proportion of published studies had a PI between 22% and 72% and the weighted pooled proportion when combining estimates would be 46.2% (95% CI 40.2%-52.4%, I2 = 94.4%). Twenty-two MRPs (22 publications) evaluated cohorts of studies included in trial registries; the PI of the proportion published ranged from 13% to 90% and the weighted pooled proportion would be 54.2% (95% CI 42.0%-65.9%, I2 = 98.9%). REC-approved studies with statistically significant results (compared with those without statistically significant results) were more likely to be published (pooled OR 2.8; 95% CI 2.2-3.5). Phase-III trials were also more likely to be published than phase II trials (pooled OR 2.0; 95% CI 1.6-2.5). The probability of publication within two years after study completion ranged from 7% to 30%. CONCLUSIONS: A substantial part of the studies approved by RECs or included in trial registries remains unpublished. Due to the large heterogeneity a prediction of the publication probability for a future study is very uncertain. Non-publication of research is not a random process, e.g., it is associated with the direction of study findings. Our findings suggest that the dissemination of research findings is biased.

Underfunding of stroke research: a Europe-wide problem.

BACKGROUND AND PURPOSE: Previous studies in the United States and the United Kingdom have shown that stroke research is underfunded compared with coronary heart disease (CHD) and cancer research despite the high clinical and financial burden of stroke. We aimed to determine whether underfunding of stroke research is a Europe-wide problem. METHODS: Data for the financial year 2000 to 2001 were collected from 9 different European countries. Information on stroke, CHD, and cancer research funding awarded by disease-specific charities and nondisease-specific charity or government- funded organizations was obtained from annual reports, web sites, and by direct communication with organizations. RESULTS: There was marked and consistent underfunding of stroke research in all the countries studied. Stroke funding as a percentage of the total funding for stroke, CHD, and cancer was uniformly low, ranging from 2% to 11%. Funding for stroke was less than funding for cancer, usually by a factor of > or =10. In every country except Turkey, funding for stroke research was less than that for CHD. CONCLUSIONS: This study confirms that stroke research is grossly underfunded, compared with CHD and cancer, throughout Europe. Similar data have been obtained from the United States suggesting that relative underfunding of stroke research is likely to be a worldwide phenomenon.

Epidemiology of cleft palate in Europe: implications for genetic research.

OBJECTIVE: To describe the epidemiology of cleft palate (CP) in Europe. DESIGN AND SETTING: A descriptive epidemiological study on 3852 cases of CP, identified (1980 through 1996) from more than 6 million births from the EUROCAT network of 30 registers in 16 European countries. RESULTS: Significant differences in prevalence in Europe between registries and within countries were observed. A total of 2112 (54.8%) CP cases occurred as isolated, 694 (18.0%) were associated with other defects such as multiple congenital anomalies, and 1046 (27.2%) were in recognized conditions. The study confirmed the tendency toward female prevalence (sex ratio [SR] = 0.83), particularly among isolated cases (SR = 0.78) even if SR inversion is reported in some registries. A specific association with neural tube defects (NTDs) in some registers is reported. CONCLUSION: The differences identified in Europe (prevalence, sex, associated anomalies) can be only partially explained by methodological reasons because a common methodology was shared among all registries for case ascertainment and collection, and CP is an easy detectable condition with few induced abortions. The complex model of inheritance and the frequently conflicting results in different populations on the role of genes that constitute risk factors suggest the presence of real biological differences. The association of CP/NTD in an area with a high prevalence of NTDs can identify a group of conditions that can be considered etiologically homogeneous. The epidemiological evaluation can guide genetic research to specify the role of etiological factors in each different population

Albuterol delivery in an in vitro pediatric ventilator lung model: comparison of jet, ultrasonic, and mesh nebulizers.

OBJECTIVE: : To determine the influence of nebulizer types and nebulization modes on bronchodilator delivery in a mechanically ventilated pediatric lung model. DESIGN: : In vitro, laboratory study. SETTING: : Research laboratory of a university hospital. INTERVENTIONS: : Using albuterol as a marker, three nebulizer types (jet nebulizer, ultrasonic nebulizer, and vibrating-mesh nebulizer) were tested in three nebulization modes in a nonhumidified bench model mimicking the ventilatory pattern of a 10-kg infant. The amounts of albuterol deposited on the inspiratory filters (inhaled drug) at the end of the endotracheal tube, on the expiratory filters, and remaining in the nebulizers or in the ventilator circuit were determined. Particle size distribution of the nebulizers was also measured. MEASUREMENTS AND MAIN RESULTS: : The inhaled drug was 2.8% ± 0.5% for the jet nebulizer, 10.5% ± 2.3% for the ultrasonic nebulizer, and 5.4% ± 2.7% for the vibrating-mesh nebulizer in intermittent nebulization during the inspiratory phase (p < 0.01). The most efficient nebulizer was the vibrating-mesh nebulizer in continuous nebulization (13.3% ± 4.6%, p < 0.01). Depending on the nebulizers, a variable but important part of albuterol was observed as remaining in the nebulizers (jet and ultrasonic nebulizers), or being expired or lost in the ventilator circuit (all nebulizers). Only small particles (range 2.39-2.70 µm) reached the end of the endotracheal tube. CONCLUSIONS: : Important differences between nebulizer types and nebulization modes were seen for albuterol deposition at the end of the endotracheal tube in an in vitro pediatric ventilator-lung model. New aerosol devices, such as ultrasonic and vibrating-mesh nebulizers, were more efficient than the jet nebulizer.

Sex biased spatial strategies relying on the integration of multimodal cues in a rat model of schizophrenia: impairment in predicting future context?

Male and female Wistar rats were treated postnatally (PND 5-16) with BSO (l-buthionine-(S,R)-sulfoximine) to provide a rat model of schizophrenia based on transient glutathione deficit. In the watermaze, BSO-treated male rats perform very efficiently in conditions where a diversity of visual information is continuously available during orientation trajectories [1]. Our hypothesis is that the treatment impairs proactive strategies anticipating future sensory information, while supporting a tight visual adjustment on memorized snapshots, i.e. compensatory reactive strategies. To test this hypothesis, BSO rats' performance was assessed in two conditions using an 8-arm radial maze task: a semi-transparent maze with no available view on the environment from maze centre [2], and a modified 2-parallel maze known to induce a neglect of the parallel pair in normal rats [3-5]. Male rats, but not females, were affected by the BSO treatment. In the semi-transparent maze, BSO males expressed a higher error rate, especially in completing the maze after an interruption. In the 2-parallel maze shape, BSO males, unlike controls, expressed no neglect of the parallel arms. This second result was in accord with a reactive strategy using accurate memory images of the contextual environment instead of a representation based on integrating relative directions. These results are coherent with a treatment-induced deficit in proactive decision strategy based on multimodal cognitive maps, compensated by accurate reactive adaptations based on the memory of local configurations. Control females did not express an efficient proactive capacity in the semi-transparent maze, neither did they show the significant neglect of the parallel arms, which might have masked the BSO induced effect. Their reduced sensitivity to BSO treatment is discussed with regard to a sex biased basal cognitive style.

Zuckerman's reivsed alternative five-factor model: Validation of the Zuckerman-Kuhlman-Aluja Personality Questionnaire in four French-speaking countries

The aim of this study was to analyze the replicability of Zuckerman's revised Alternative Five-factor model in a French-speaking context by validating the Zuckerman-Kuhlman-Aluja Personality Questionnaire (ZKA-PQ) simultaneously in 4 French-speaking countries. The total sample was made up of 1,497 subjects from Belgium, Canada, France, and Switzerland. The internal consistencies for all countries were generally similar to those found for the normative U.S. and Spanish samples. A factor analysis confirmed that the normative structure replicated well and was stable within this French-speaking context. Moreover, multigroup confirmatory factor analyses have shown that the ZKA-PQ reaches scalar invariance across these 4 countries. Mean scores were slightly different for women and men, with women scoring higher on Neuroticism but lower on Sensation Seeking. Globally, mean score differences across countries were small. Overall, the ZKA-PQ seems an interesting alternative to assess both lower and higher order personality traits for applied or research purposes.

Patterns of Subject Mix in Higher Education Institutions : A First Empirical Analysis Using the AQUAMETH Database

Teaching and research are organised differently between subject domains: attempts to construct typologies of higher education institutions, however, often do not include quantitative indicators concerning subject mix which would allow systematic comparisons of large numbers of higher education institutions among different countries, as the availability of data for such indicators is limited. In this paper, we present an exploratory approach for the construction of such indicators. The database constructed in the AQUAMETH project, which includes also data disaggregated at the disciplinary level, is explored with the aim of understanding patterns of subject mix. For six European countries, an exploratory and descriptive analysis of staff composition divided in four large domains (medical sciences, engineering and technology, natural sciences and social sciences and humanities) is performed, which leads to a classification distinguishing between specialist and generalist institutions. Among the latter, a further distinction is made based on the presence or absence of a medical department. Preliminary exploration of this classification and its comparison with other indicators show the influence of long term dynamics on the subject mix of individual higher education institutions, but also underline disciplinary differences, for example regarding student to staff ratios, as well as national patterns, for example regarding the number of PhD degrees per 100 undergraduate students. Despite its many limitations, this exploratory approach allows defining a classification of higher education institutions that accounts for a large share of differences between the analysed higher education institutions.

A Thematic Comparison of Eight Frameworks of Quality Criteria in Qualitative Health Research

Background: Despite the increasing incidences of the publication of assessment frameworks intending to establish the "standards" of the quality of qualitative research, the research conducted using such empirical methods are still facing difficulties in being published or recognised by funding agencies. Methods: We conducted a thematic content analysis of eight frameworks from psychology/psychiatry and general medicine. The frameworks and their criteria are then compared against each other. Findings: The results illustrated the difficulties in reaching consensus on the definition of quality criteria. This showed the differences between the frameworks from the point of views of the underlying epistemology and the criteria suggested. Discussion: The aforementioned differences reflect the diversity of paradigms implicitly referred to by the authors of the frameworks, although rarely explicitly mentioned in text. We conclude that the increase in qualitative research and publications has failed to overcome the difficulties in establishing shared criteria and the great heterogeneity of concepts raises methodological and epistemological problems.

Differential vulnerability of neurochemically identified subpopulations of retinal neurons in a monkey model of glaucoma.

The vulnerability of subpopulations of retinal neurons delineated by their content of cytoskeletal or calcium-binding proteins was evaluated in the retinas of cynomolgus monkeys in which glaucoma was produced with an argon laser. We quantitatively compared the number of neurons containing either neurofilament (NF) protein, parvalbumin, calbindin or calretinin immunoreactivity in central and peripheral portions of the nasal and temporal quadrants of the retina from glaucomatous and fellow non-glaucomatous eyes. There was no significant difference between the proportion of amacrine, horizontal and bipolar cells labeled with antibodies to the calcium-binding proteins comparing the two eyes. NF triplet immunoreactivity was present in a subpopulation of retinal ganglion cells, many of which, but not all, likely correspond to large ganglion cells that subserve the magnocellular visual pathway. Loss of NF protein-containing retinal ganglion cells was widespread throughout the central (59-77% loss) and peripheral (96-97%) nasal and temporal quadrants and was associated with the loss of NF-immunoreactive optic nerve fibers in the glaucomatous eyes. Comparison of counts of NF-immunoreactive neurons with total cell loss evaluated by Nissl staining indicated that NF protein-immunoreactive cells represent a large proportion of the cells that degenerate in the glaucomatous eyes, particularly in the peripheral regions of the retina. Such data may be useful in determining the cellular basis for sensitivity to this pathologic process and may also be helpful in the design of diagnostic tests that may be sensitive to the loss of the subset of NF-immunoreactive ganglion cells.

Management of nanomaterials safety in research environment

Despite numerous discussions, workshops, reviews and reports about responsible development of nanotechnology, information describing health and environmental risk of engineered nanoparticles or nanomaterials is severely lacking and thus insufficient for completing rigorous risk assessment on their use. However, since preliminary scientific evaluations indicate that there are reasonable suspicions that activities involving nanomaterials might have damaging effects on human health; the precautionary principle must be applied. Public and private institutions as well as industries have the duty to adopt preventive and protective measures proportionate to the risk intensity and the desired level of protection. In this work, we present a practical, 'user-friendly' procedure for a university-wide safety and health management of nanomaterials, developed as a multi-stakeholder effort (government, accident insurance, researchers and experts for occupational safety and health). The process starts using a schematic decision tree that allows classifying the nano laboratory into three hazard classes similar to a control banding approach (from Nano 3 - highest hazard to Nano1 - lowest hazard). Classifying laboratories into risk classes would require considering actual or potential exposure to the nanomaterial as well as statistical data on health effects of exposure. Due to the fact that these data (as well as exposure limits for each individual material) are not available, risk classes could not be determined. For each hazard level we then provide a list of required risk mitigation measures (technical, organizational and personal). The target 'users' of this safety and health methodology are researchers and safety officers. They can rapidly access the precautionary hazard class of their activities and the corresponding adequate safety and health measures. We succeed in convincing scientist dealing with nano-activities that adequate safety measures and management are promoting innovation and discoveries by ensuring them a safe environment even in the case of very novel products. The proposed measures are not considered as constraints but as a support to their research. This methodology is being implemented at the Ecole Polytechnique de Lausanne in over 100 research labs dealing with nanomaterials. It is our opinion that it would be useful to other research and academia institutions as well. [Authors]

A joint back calculation model for the imputation of the date of HIV infection in a prevalent cohort.

In studies of the natural history of HIV-1 infection, the time scale of primary interest is the time since infection. Unfortunately, this time is very often unknown for HIV infection and using the follow-up time instead of the time since infection is likely to provide biased results because of onset confounding. Laboratory markers such as the CD4 T-cell count carry important information concerning disease progression and can be used to predict the unknown date of infection. Previous work on this topic has made use of only one CD4 measurement or based the imputation on incident patients only. However, because of considerable intrinsic variability in CD4 levels and because incident cases are different from prevalent cases, back calculation based on only one CD4 determination per person or on characteristics of the incident sub-cohort may provide unreliable results. Therefore, we propose a methodology based on the repeated individual CD4 T-cells marker measurements that use both incident and prevalent cases to impute the unknown date of infection. Our approach uses joint modelling of the time since infection, the CD4 time path and the drop-out process. This methodology has been applied to estimate the CD4 slope and impute the unknown date of infection in HIV patients from the Swiss HIV Cohort Study. A procedure based on the comparison of different slope estimates is proposed to assess the goodness of fit of the imputation. Results of simulation studies indicated that the imputation procedure worked well, despite the intrinsic high volatility of the CD4 marker.