The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content

Background: The COSMIN checklist (COnsensus-based Standards for the selection of health status Measurement INstruments) was developed in an international Delphi study to evaluate the methodological quality of studies on measurement properties of health-related patient reported outcomes (HR-PROs). In this paper, we explain our choices for the design requirements and preferred statistical methods for which no evidence is available in the literature or on which the Delphi panel members had substantial discussion. Methods: The issues described in this paper are a reflection of the Delphi process in which 43 panel members participated. Results: The topics discussed are internal consistency (relevance for reflective and formative models, and distinction with unidimensionality), content validity (judging relevance and comprehensiveness), hypotheses testing as an aspect of construct validity (specificity of hypotheses), criterion validity (relevance for PROs), and responsiveness (concept and relation to validity, and (in) appropriate measures).Conclusions: We expect that this paper will contribute to a better understanding of the rationale behind the items, thereby enhancing the acceptance and use of the COSMIN checklist.

Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist

Background: The COSMIN checklist is a tool for evaluating the methodological quality of studies on measurement properties of health-related patient-reported outcomes. The aim of this study is to determine the inter-rater agreement and reliability of each item score of the COSMIN checklist (n = 114). Methods: 75 articles evaluating measurement properties were randomly selected from the bibliographic database compiled by the Patient-Reported Outcome Measurement Group, Oxford, UK. Raters were asked to assess the methodological quality of three articles, using the COSMIN checklist. In a one-way design, percentage agreement and intraclass kappa coefficients or quadratic-weighted kappa coefficients were calculated for each item. Results: 88 raters participated. Of the 75 selected articles, 26 articles were rated by four to six participants, and 49 by two or three participants. Overall, percentage agreement was appropriate (68% was above 80% agreement), and the kappa coefficients for the COSMIN items were low (61% was below 0.40, 6% was above 0.75). Reasons for low inter-rater agreement were need for subjective judgement, and accustom to different standards, terminology and definitions.Conclusions: Results indicated that raters often choose the same response option, but that it is difficult on item level to distinguish between articles. When using the COSMIN checklist in a systematic review, we recommend getting some training and experience, completing it by two independent raters, and reaching consensus on one final rating. Instructions for using the checklist are improved.

Pseudogenes in the ENCODE regions: consensus annotation, analysis of transcription, and evolution

Arising from either retrotransposition or genomic duplication of functional genes, pseudogenes are “genomic fossils” valuable for exploring the dynamics and evolution of genes and genomes. Pseudogene identification is an important problem in computational genomics, and is also critical for obtaining an accurate picture of a genome’s structure and function. However, no consensus computational scheme for defining and detecting pseudogenes has been developed thus far. As part of the ENCyclopedia Of DNA Elements (ENCODE) project, we have compared several distinct pseudogene annotation strategies and found that different approaches and parameters often resulted in rather distinct sets of pseudogenes. We subsequently developed a consensus approach for annotating pseudogenes (derived from protein coding genes) in the ENCODE regions, resulting in 201 pseudogenes, two-thirds of which originated from retrotransposition. A survey of orthologs for these pseudogenes in 28 vertebrate genomes showed that a significant fraction (∼80%) of the processed pseudogenes are primate-specific sequences, highlighting the increasing retrotransposition activity in primates. Analysis of sequence conservation and variation also demonstrated that most pseudogenes evolve neutrally, and processed pseudogenes appear to have lost their coding potential immediately or soon after their emergence. In order to explore the functional implication of pseudogene prevalence, we have extensively examined the transcriptional activity of the ENCODE pseudogenes. We performed systematic series of pseudogene-specific RACE analyses. These, together with complementary evidence derived from tiling microarrays and high throughput sequencing, demonstrated that at least a fifth of the 201 pseudogenes are transcribed in one or more cell lines or tissues.

Médecines complémentaires: vers un consensus "evidence-based" à l'hôpital universitaire. [Complementary and alternative medicines (CAM): towards an "evidence-based" consensus in the University hospital]

Après la votation fédérale demandant de prendre en compte les médecines complémentaires, un consensus a été recherché dans quatorze services et unités du Centre hospitalier universitaire vaudois (CHUV). Confrontés aux données de la littérature (Plus de 2000 publications en "Evidence-based complementary medicine" depuis 1998), les soignants étaient tous surpris par l'ampleur des résultats cliniques disponibles actuellement. Tous identifiaient un besoin en formation et en informations sur le sujet. Une prise de position officielle de l'institution était aussi souhaitée, instituant l'enseignement et la recherche sur les médecines complémentaires et assurant la production d'informations rigoureuses et pertinentes pour la clinique. [Abstract] While a popular vote supported a new article on complementary and alternative medicines (CAM) in the Swiss Constitution, this assessment in 14 wards of the University Hospital of Lausanne, Switzerland, attempted at answering the question: How can CAM use be better taken into account and patients informed with more rigor and respect for their choices? Confronted with a review of the literature (> 2000 publications in "Evidence-based cornplementary medicine" since 1998), respondents declared their ignorance of the clinical data presently available on CAM. All were in favour of more teaching and information on the subject, plus an official statement from the Hospital direction, ensuring production and diffusion of rigorous and cJinically significant information on CAM.

Communication skills training in oncology: a position paper based on a consensus meeting among European experts in 2009.

BACKGROUND: Communication in cancer care has become a major topic of interest. Since there is evidence that ineffective communication affects both patients and oncology clinicians (physicians and nurses), so-called communication skills trainings (CSTs) have been developed over the last decade. While these trainings have been demonstrated to be effective, there is an important heterogeneity with regard to implementation and with regard to evidence of different aspects of CST. METHODS: In order to review and discuss the scientific literature on CST in oncology and to formulate recommendations, the Swiss Cancer League has organised a consensus meeting with European opinion leaders and experts in the field of CST, as well as oncology clinicians, representatives of oncology societies and patient organisations. On the basis of a systematic review and a meta-analysis, recommendations have been developed and agreed upon. RESULTS: Recommendations address (i) the setting, objectives and participants of CST, (ii) its content and pedagogic tools, (iii) organisational aspects, (iv) outcome and (v) future directions and research. CONCLUSION: This consensus meeting, on the basis of European expert opinions and a systematic review and meta-analysis, defines key elements for the current provision and future development and evaluation of CST in oncology.

Design and evaluation of a panel os single-nucleotide polymorphisms in microRNA genomic regions for association studies in human disease

MicroRNAs (miRNA) are recognized posttranscriptional gene repressors involved in the control of almost every biological process. Allelic variants in these regions may be an important source of phenotypic diversity and contribute to disease susceptibility. We analyzed the genomic organization of 325 human miRNAs (release 7.1, miRBase) to construct a panel of 768 single-nucleotide polymorphisms (SNPs) covering approximately 1 Mb of genomic DNA, including 131 isolated miRNAs (40%) and 194 miRNAs arranged in 48 miRNA clusters, as well as their 5-kb flanking regions. Of these miRNAs, 37% were inside known protein-coding genes, which were significantly associated with biological functions regarding neurological, psychological or nutritional disorders. SNP coverage analysis revealed a lower SNP density in miRNAs compared with the average of the genome, with only 24 SNPs located in the 325 miRNAs studied. Further genotyping of 340 unrelated Spanish individuals showed that more than half of the SNPs in miRNAs were either rare or monomorphic, in agreement with the reported selective constraint on human miRNAs. A comparison of the minor allele frequencies between Spanish and HapMap population samples confirmed the applicability of this SNP panel to the study of complex disorders among the Spanish population, and revealed two miRNA regions, hsa-mir-26a-2 in the CTDSP2 gene and hsa-mir-128-1 in the R3HDM1 gene, showing geographical allelic frequency variation among the four HapMap populations, probably because of differences in natural selection. The designed miRNA SNP panel could help to identify still hidden links between miRNAs and human disease.

Minimizing recombinations in consensus networks for phylogeographic studies

Background: We address the problem of studying recombinational variations in (human) populations. In this paper, our focus is on one computational aspect of the general task: Given two networks G1 and G2, with both mutation and recombination events, defined on overlapping sets of extant units the objective is to compute a consensus network G3 with minimum number of additional recombinations. We describe a polynomial time algorithm with a guarantee that the number of computed new recombination events is within ϵ = sz(G1, G2) (function sz is a well-behaved function of the sizes and topologies of G1 and G2) of the optimal number of recombinations. To date, this is the best known result for a network consensus problem.Results: Although the network consensus problem can be applied to a variety of domains, here we focus on structure of human populations. With our preliminary analysis on a segment of the human Chromosome X data we are able to infer ancient recombinations, population-specific recombinations and more, which also support the widely accepted 'Out of Africa' model. These results have been verified independently using traditional manual procedures. To the best of our knowledge, this is the first recombinations-based characterization of human populations. Conclusion: We show that our mathematical model identifies recombination spots in the individual haplotypes; the aggregate of these spots over a set of haplotypes defines a recombinational landscape that has enough signal to detect continental as well as population divide based on a short segment of Chromosome X. In particular, we are able to infer ancient recombinations, population-specific recombinations and more, which also support the widely accepted 'Out of Africa' model. The agreement with mutation-based analysis can be viewed as an indirect validation of our results and the model. Since the model in principle gives us more information embedded in the networks, in our future work, we plan to investigate more non-traditional questions via these structures computed by our methodology.

A multicenter study to validate the reproducibility of MSI testing with a panel of 5 quasimonomorphic mononucleotide repeats.

Microsatellite instability (MSI) testing in clinics is becoming increasingly widespread; therefore, there is an urgent need for methodology standardization and the availability of quality control. This study is aimed to assess the interlaboratory reproducibility of MSI testing in archive samples by using a panel of 5 recently introduced, mononucleotide repeats (MNR). The quality control involved 8 European institutions. Participants were supplied with DNA extracted from 15 archive colon carcinoma samples and from the corresponding normal tissues. Every group was asked to assess the MSI status of the samples by using the BAT25, BAT26, NR21, NR24, and NR27 mononucleotide markers. Four institutions repeated the analysis using the NCI reference panel to confirm the results obtained with the MNR markers. The overall concordance among institutions for MSI analyses at single locus level was 97.7% when using the MNR panel and 95.0% with the NCI one. The laboratories obtained a full agreement in scoring the MSI status of each patient sample, both using the mononucleotide and the NCI marker sets. With the NCI marker set, however, concordance was lowered to 85.7% when considering the MSI-Low phenotype. Concordance between the 2 panels in scoring the MSI status of each sample was complete if no discrimination was made between MSI-Stable and MSI-L, whereas it dropped to 76.7% if MSI-L was considered. In conclusion, the use of the MNR panel seems to be a robust approach that yields a very high level of reproducibility. The results obtained with the 5 MNR are diagnostically consistent with those obtained by the use of the NCI markers, except for the MSI-Low phenotype.

Hindsight judgement on ambiguous episodes of suspected infection in critically ill children: poor consensus amongst experts?

Few episodes of suspected infection observed in paediatric intensive care are classifiable without ambiguity by a priori defined criteria. Most require additional expert judgement. Recently, we observed a high variability in antibiotic prescription rates, not explained by the patients' clinical data or underlying diseases. We hypothesised that the disagreement of experts in adjudication of episodes of suspected infection could be one of the potential causes for this variability. During a 5-month period, we included all patients of a 19-bed multidisciplinary, tertiary, neonatal and paediatric intensive care unit, in whom infection was clinically suspected and antibiotics were prescribed ( n=183). Three experts (two senior ICU physicians and a specialist in infectious diseases) were provided with all patient data, laboratory and microbiological findings. All experts classified episodes according to a priori defined criteria into: proven sepsis, probable sepsis (negative cultures), localised infection and no infection. Episodes of proven viral infection and incomplete data sets were excluded. Of the remaining 167 episodes, 48 were classifiable by a priori criteria ( n=28 proven sepsis, n= 20 no infection). The three experts only achieved limited agreement beyond chance in the remaining 119 episodes (kappa = 0.32, and kappa = 0.19 amongst the ICU physicians). The kappa is a measure of the degree of agreement beyond what would be expected by chance alone, with 0 indicating the chance result and 1 indicating perfect agreement. CONCLUSION: agreement of specialists in hindsight adjudication of episodes of suspected infection is of questionable reliability.

Panel index VAR models: Specification, Estimation, Testing and Leading Indicators

This paper proposes a method to conduct inference in panel VAR models with cross unit interdependencies and time variations in the coefficients. The approach can be used to obtain multi-unit forecasts and leading indicators and to conduct policy analysis in a multiunit setups. The framework of analysis is Bayesian and MCMC methods are used to estimate the posterior distribution of the features of interest. The model is reparametrized to resemble an observable index model and specification searches are discussed. As an example, we construct leading indicators for inflation and GDP growth in the Euro area using G-7 information.