The inter-rater reliability of the Pulmonary Embolism Severity Index.

The Pulmonary Embolism Severity Index (PESI) is a validated clinical prognostic model for patients with acute pulmonary embolism (PE). Our goal was to assess the PESI's inter-rater reliability in patients diagnosed with PE. We prospectively identified consecutive patients diagnosed with PE in the emergency department of a Swiss teaching hospital. For all patients, resident and attending physician raters independently collected the 11 PESI variables. The raters then calculated the PESI total point score and classified patients into one of five PESI risk classes (I-V) and as low (risk classes I/II) versus higher-risk (risk classes III-V). We examined the inter-rater reliability for each of the 11 PESI variables, the PESI total point score, assignment to each of the five PESI risk classes, and classification of patients as low versus higher-risk using kappa (κ) and intra-class correlation coefficients (ICC). Among 48 consecutive patients with an objective diagnosis of PE, reliability coefficients between resident and attending physician raters were > 0.60 for 10 of the 11 variables comprising the PESI. The inter-rater reliability for the PESI total point score (ICC: 0.89, 95% CI: 0.81-0.94), PESI risk class assignment (κ: 0.81, 95% CI: 0.66-0.94), and the classification of patients as low versus higher-risk (κ: 0.92, 95% CI: 0.72-0.98) was near perfect. Our results demonstrate the high reproducibility of the PESI, supporting the use of the PESI for risk stratification of patients with PE.

Reliability of current estimates of schistosomiasis prevalence in the Rainforest Zone of the state of Pernambuco, Northeastern Brazil

The Program for Schistosomiasis Control within the Unified Health System (PCE-SUS) was implemented by 1999 in the Rainforest Zone or "Zona da Mata" of Pernambuco (ZMP) aiming to carry out biennial stool surveys of whole populations through municipal health organs followed by treatment of the positives through the local units of the Family Health Program (PSF). Yearly reports from the Health Department of Pernambuco State (SES/PE) from 2002 to 2004 on the PCE-SUS surveys were assessed to evaluate whether the current estimates of prevalence in the municipalities of the ZMP are based on reliable samples so as to allow considerations on the real situation of schistosomiasis in that area. The surveys carried out in that period did not follow the major principles underlying sampling design, thus posing problems in both precision and validity of the estimates. Only 12 out of 43 municipalities had minimally reliable estimates: five with moderate prevalence (10-50%) and seven with low prevalence (< 10%). Surveys with appropriate sampling procedures aimed either at representative target groups (school-aged children) or communities are recommended for the ZMP and other endemic areas not only to provide reliable information on the current situation of schistosomiasis but also to plan adequate control strategies.

Reliability of stable carbon and oxygen isotope compositions of pedogenic needle fibre calcite as environmental indicators: examples from Western Europe.

Stable carbon and oxygen isotope analyses were conducted on pedogenic needle fibre calcite (NFC) from seven sites in areas with roughly similar temperate climates in Western Europe, including the Swiss Jura Mountains, eastern and southern France, northern Wales, and north-eastern Spain. The δ(13)C values (-12.5 to-6.8 0/00 Vienna Pee Dee Belemnite (VPDB)) record the predominant C(3) vegetation cover at the sites. A good correlation was found between mean monthly climatic parameters (air temperature, number of frost days, humidity, and precipitation) and δ(18)O values (-7.8 to-3.40/00 VPDB) of all the NFC. Similar seasonal variations of δ(18)O values for monthly NFC samples from the Swiss sites and those of mean monthly δ(18)O values of local precipitation and meteorological data point out precipitation and preferential growth/or recrystallisation of the pedogenic needle calcite during dry seasons. These covariations indicate the potential of stable isotope compositions of preserved NFC in fossil soil horizons as a promising tool for palaeoenvironmental reconstructions.

Validity and reliability of the Spanish version of the DN4 (Douleur Neuropathique 4 questions) questionnaire for differential diagnosis of pain syndromes associated to a neuropathic or somatic component

BACKGROUND This study assesses the validity and reliability of the Spanish version of DN4 questionnaire as a tool for differential diagnosis of pain syndromes associated to a neuropathic (NP) or somatic component (non-neuropathic pain, NNP). METHODS A study was conducted consisting of two phases: cultural adaptation into the Spanish language by means of conceptual equivalence, including forward and backward translations in duplicate and cognitive debriefing, and testing of psychometric properties in patients with NP (peripheral, central and mixed) and NNP. The analysis of psychometric properties included reliability (internal consistency, inter-rater agreement and test-retest reliability) and validity (ROC curve analysis, agreement with the reference diagnosis and determination of sensitivity, specificity, and positive and negative predictive values in different subsamples according to type of NP). RESULTS A sample of 164 subjects (99 women, 60.4%; age: 60.4 +/- 16.0 years), 94 (57.3%) with NP (36 with peripheral, 32 with central, and 26 with mixed pain) and 70 with NNP was enrolled. The questionnaire was reliable [Cronbach's alpha coefficient: 0.71, inter-rater agreement coefficient: 0.80 (0.71-0.89), and test-retest intra-class correlation coefficient: 0.95 (0.92-0.97)] and valid for a cut-off value > or = 4 points, which was the best value to discriminate between NP and NNP subjects. DISCUSSION This study, representing the first validation of the DN4 questionnaire into another language different than the original, not only supported its high discriminatory value for identification of neuropathic pain, but also provided supplemental psychometric validation (i.e. test-retest reliability, influence of educational level and pain intensity) and showed its validity in mixed pain syndromes.

Diagnostic reliability of an immunochromatographic test for Chagas disease screening at a primary health care centre in a rural endemic area

Many patients with Chagas disease live in remote communities that lack both equipment and trained personnel to perform a diagnosis by conventional serology (CS). Thus, reliable tests suitable for use under difficult conditions are required. In this study, we evaluated the ability of personnel with and without laboratory skills to perform immunochromatographic (IC) tests to detect Chagas disease at a primary health care centre (PHCC). We examined whole blood samples from 241 patients and serum samples from 238 patients. Then, we calculated the percentage of overall agreement (POA) between the two groups of operators for the sensitivity (S), specificity (Sp) and positive (PPV) and negative (NPV) predictive values of IC tests compared to CS tests. We also evaluated the level of agreement between ELISAs and indirect haemagglutination (IHA) tests. The readings of the IC test results showed 100% agreement (POA = 1). The IC test on whole blood showed the following values: S = 87.3%; Sp = 98.8%; PPV = 96.9% and NPV = 95.9%. Additionally, the IC test on serum displayed the following results: S = 95.7%; Sp = 100%; PPV = 100% and NPV = 98.2%. Using whole blood, the agreement with ELISA was 96.3% and the agreement with IHA was 94.1%. Using serum, the agreement with ELISA was 97.8% and the agreement with IHA was 96.6%. The IC test performance with serum samples was excellent and demonstrated its usefulness in a PHCC with minimal equipment. If the IC test S value and NPV with whole blood are improved, then this test could also be used in areas lacking laboratories or specialised personnel.

Retest reliability of individual p3 topography assessed by high density electroencephalography.

BACKGROUND Some controversy remains about the potential applicability of cognitive potentials for evaluating the cerebral activity associated with cognitive capacity. A fundamental requirement is that these neurophysiological parameters show a high level of stability over time. Previous studies have shown that the reliability of diverse parameters of the P3 component (latency and amplitude) ranges between moderate and high. However, few studies have paid attention to the retest reliability of the P3 topography in groups or individuals. Considering that changes in P3 topography have been related to different pathologies and healthy aging, the main objective of this article was to evaluate in a longitudinal study (two sessions) the reliability of P3 topography in a group and at the individual level. RESULTS The correlation between sessions for P3 topography in the grand average of groups was high (r = 0.977, p<0.001). The within-subject correlation values ranged from 0.626 to 0.981 (mean: 0.888). In the between-subjects topography comparisons, the correlation was always lower for comparisons between different subjects than for within-subjects correlations in the first session but not in the second session. CONCLUSIONS The present study shows that P3 topography is highly reliable for group analysis (comprising the same subjects) in different sessions. The results also confirmed that retest reliability for individual P3 maps is suitable for follow-up studies for a particular subject. Moreover, P3 topography appears to be a specific marker considering that the between-subjects correlations were lower than the within-subject correlations. However, P3 topography appears more similar between subjects in the second session, demonstrating that is modulated by experience. Possible clinical applications of all these results are discussed.

Discriminant validity and test-retest reliability of a self-administered Internet-based questionnaire testing doctors' knowledge in evidence-based medicine.

Aims and objectives  This study aimed to determine the discriminant validity and the test-retest reliability of a questionnaire testing the impact of evidence-based medicine (EBM) training on doctors' knowledge and skills. Methods  Questionnaires were sent electronically to all doctors working as residents and chief residents in two French speaking hospital networks in Switzerland. Participants completed the questionnaire twice, within a 4-week interval. The discriminant validity was examined in comparing doctors' performance according to their reported EBM previous training. Proportion of agreement between both sessions of the questionnaire, Cohen's kappa and 'uniform kappa' determined its test-retest reliability. Results  The participation rate was 9.8%/7.1% to first/second session. Performance increased according to the level of doctors' previous training in EBM. The observed proportion of agreement between both sessions was over 70% for 14/19 questions, and the 'uniform kappa' was superior to 0.60 for 15/19 questions. Conclusion  The discriminant validity and test-retest reliability of the questionnaire were satisfying. The low participation rate did not prevent the study from achieving its aims.

Evidence for validity and reliability of a french version of the FAAM.

BACKGROUND: The Foot and Ankle Ability Measure (FAAM) is a self reported questionnaire for patients with foot and ankle disorders available in English, German, and Persian. This study plans to translate the FAAM from English to French (FAAM-F) and assess the validity and reliability of this new version.METHODS: The FAAM-F Activities of Daily Living (ADL) and sports subscales were completed by 105 French-speaking patients (average age 50.5 years) presenting various chronic foot and ankle disorders. Convergent and divergent validity was assessed by Pearson's correlation coefficients between the FAAM-F subscales and the SF-36 scales: Physical Functioning (PF), Physical Component Summary (PCS), Mental Health (MH) and Mental Component Summary (MCS). Internal consistency was calculated by Cronbach's Alpha (CA). To assess test re-test reliability, 22 patients filled out the questionnaire a second time to estimate minimal detectable changes (MDC) and intraclass correlation coefficients (ICC).RESULTS: Correlations for FAAM-F ADL subscale were 0.85 with PF, 0.81 with PCS, 0.26 with MH, 0.37 with MCS. Correlations for FAAM-F Sports subscale were 0.72 with PF, 0.72 with PCS, 0.21 with MH, 0.29 with MCS. CA estimates were 0.97 for both subscales. Respectively for the ADL and Sports subscales, ICC were 0.97 and 0.94, errors for a single measure were 8 and 10 points at 95% confidence and the MDC values at 95% confidence were 7 and 18 points.CONCLUSION: The FAAM-F is valid and reliable for the self-assessment of physical function in French-speaking patients with a wide range of chronic foot and ankle disorders.

Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist

Background: The COSMIN checklist is a tool for evaluating the methodological quality of studies on measurement properties of health-related patient-reported outcomes. The aim of this study is to determine the inter-rater agreement and reliability of each item score of the COSMIN checklist (n = 114). Methods: 75 articles evaluating measurement properties were randomly selected from the bibliographic database compiled by the Patient-Reported Outcome Measurement Group, Oxford, UK. Raters were asked to assess the methodological quality of three articles, using the COSMIN checklist. In a one-way design, percentage agreement and intraclass kappa coefficients or quadratic-weighted kappa coefficients were calculated for each item. Results: 88 raters participated. Of the 75 selected articles, 26 articles were rated by four to six participants, and 49 by two or three participants. Overall, percentage agreement was appropriate (68% was above 80% agreement), and the kappa coefficients for the COSMIN items were low (61% was below 0.40, 6% was above 0.75). Reasons for low inter-rater agreement were need for subjective judgement, and accustom to different standards, terminology and definitions.Conclusions: Results indicated that raters often choose the same response option, but that it is difficult on item level to distinguish between articles. When using the COSMIN checklist in a systematic review, we recommend getting some training and experience, completing it by two independent raters, and reaching consensus on one final rating. Instructions for using the checklist are improved.

Intraclass reliability of the Alberta Infant Motor Scale in the Brazilian version

This study had as its objective to analyze the intraclass reliability of the Alberta Infant Motor Scale (AIMS), in the Brazilian version, in preterm and term infants. It was a methodological study, conducted from November 2009 to April 2010, with 50 children receiving care in two public institutions in Fortaleza, Ceará, Brazil. Children were grouped according to gestational age as preterm and term, and evaluated by three evaluators in the communication laboratory of a public institution or at home. The intraclass correlation indices for the categories prone, supine, sitting and standing ranged from 0.553 to 0.952; most remained above 0.800, except for the standing category of the third evaluator, in which the index was 0.553. As for the total score and percentile, rates ranged from 0.843 to 0.954. The scale proved to be a reliable instrument for assessing gross motor performance of Brazilian children, particularly in Ceará, regardless of gestational age at birth.

Validity and reliability of a questionnaire on knowledge of cardiovascular risk factors for use in Brazil

Using a sample of patients with coronary artery disease, this methodological study aimed to conduct a cross-cultural adaptation and validation of a questionnaire on knowledge of cardiovascular risk factors (Q-FARCS), lifestyle changes, and treatment adherence for use in Brazil. The questionnaire has three scales: general knowledge of risk factors (RFs); specific knowledge of these RFs; and lifestyle changes achieved. Cross-cultural adaptation included translation, synthesis, back-translation, expert committee review, and pretesting. Face and content validity, reliability, and construct validity were measured. Cronbach’s alpha for the total sample (n = 240) was 0.75. Assessment of psychometric properties revealed adequate face and content validity, and the construct revealed seven components. It was concluded that the Brazilian version of Q-FARCS had adequate reliability and validity for the assessment of knowledge of cardiovascular RFs.

Impact of the disease: acceptability, ceiling and floor effects and reliability of an instrument on heart failure

This study evaluated the acceptability, ceiling/floor effects, and the reliability of the instrument for measuring the Impact of the Disease on the Daily Life of Patients with Valvular Disease (IDCV) when applied to 135 patients with heart failure (HF). Acceptability was evaluated by the percentage of unanswered items and by the proportion of patients who responded to all items; the ceiling/floor effects by the percentage of patients who scored in the top of 10% best and worst results of the scale, respectively. Reliability was estimated by internal consistency (Cronbach's alpha coefficient) and stability of the measure (intraclass correlation coefficient - ICC). All patients responded to all items. Ceiling/floor effects evidenced were of moderate magnitude. The Cronbach's alpha was satisfactory for the majority of the domains and ICC> 0.90 in all the domains. The IDCV proved to be an easy to understand questionnaire, with evidence of reliability in patients with HF.