The inter-rater reliability of the Pulmonary Embolism Severity Index

The Pulmonary Embolism Severity Index (PESI) is a validated clinical prognostic model for patients with acute pulmonary embolism (PE). Our goal was to assess the PESI's inter-rater reliability in patients diagnosed with PE. We prospectively identified consecutive patients diagnosed with PE in the emergency department of a Swiss teaching hospital. For all patients, resident and attending physician raters independently collected the 11 PESI variables. The raters then calculated the PESI total point score and classified patients into one of five PESI risk classes (I-V) and as low (risk classes I/II) versus higher-risk (risk classes III-V). We examined the inter-rater reliability for each of the 11 PESI variables, the PESI total point score, assignment to each of the five PESI risk classes, and classification of patients as low versus higher-risk using kappa ( ) and intra-class correlation coefficients (ICC). Among 48 consecutive patients with an objective diagnosis of PE, reliability coefficients between resident and attending physician raters were > 0.60 for 10 of the 11 variables comprising the PESI. The inter-rater reliability for the PESI total point score (ICC: 0.89, 95% CI: 0.81-0.94), PESI risk class assignment ( : 0.81, 95% CI: 0.66-0.94), and the classification of patients as low versus higher-risk ( : 0.92, 95% CI: 0.72-0.98) was near perfect. Our results demonstrate the high reproducibility of the PESI, supporting the use of the PESI for risk stratification of patients with PE.

The Lower Limb Functional Index: development and clinimetric properties

Background Existing lower-limb, region-specific, patient-reported outcome measures have clinimetric limitations, including limitations in psychometric characteristics (eg, lack of internal consistency, lack of responsiveness, measurement error) and the lack of reported practical and general characteristics. A new patient-reported outcome measure, the Lower Limb Functional Index (LLFI), was developed to address these limitations. Objective The purpose of this study was to overcome recognized deficiencies in existing lower-limb, region-specific, patient-reported outcome measures through: (1) development of a new lower-extremity outcome scale (ie, the LLFI) and (2) evaluation of the clinimetric properties of the LLFI using the Lower Extremity Functional Scale (LEFS) as a criterion measure. Design This was a prospective observational study. Methods The LLFI was developed in a 3-stage process of: (1) item generation, (2) item reduction with an expert panel, and (3) pilot field testing (n=18) for reliability, responsiveness, and sample size requirements for a larger study. The main study used a convenience sample (n=127) from 10 physical therapy clinics. Participants completed the LLFI and LEFS every 2 weeks for 6 weeks and then every 4 weeks until discharge. Data were used to assess the psychometric, practical, and general characteristics of the LLFI and the LEFS. The characteristics also were evaluated for overall performance using the Measurement of Outcome Measures and Bot clinimetric assessment scales. Results The LLFI and LEFS demonstrated a single-factor structure, comparable reliability (intraclass correlation coefficient [2,1]=.97), scale width, and high criterion validity (Pearson r=.88, with 95% confidence interval [CI]). Clinimetric performance was higher for the LLFI compared with the LEFS on the Measurement of Outcome Measures scale (96% and 95%, respectively) and the Bot scale (100% and 83%, respectively). The LLFI, compared with the LEFS, had improved responsiveness (standardized response mean=1.75 and 1.64, respectively), minimal detectable change with 90% CI (6.6% and 8.1%, respectively), and internal consistency (α=.91 and .95, respectively), as well as readability with reduced user error and completion and scoring times. Limitations Limitations of the study were that only participants recruited from outpatient physical therapy clinics were included and that no specific conditions or diagnostic subgroups were investigated. Conclusion The LLFI demonstrated sound clinimetric properties. There was lower response error, efficient completion and scoring, and improved responsiveness and overall performance compared with the LEFS. The LLFI is suitable for assessment of lower-limb function.

Multiplicity of data in trial reports and the reliability of meta-analyses: empirical study

Objectives To examine the extent of multiplicity of data in trial reports and to assess the impact of multiplicity on meta-analysis results. Design Empirical study on a cohort of Cochrane systematic reviews. Data sources All Cochrane systematic reviews published from issue 3 in 2006 to issue 2 in 2007 that presented a result as a standardised mean difference (SMD). We retrieved trial reports contributing to the first SMD result in each review, and downloaded review protocols. We used these SMDs to identify a specific outcome for each meta-analysis from its protocol. Review methods Reviews were eligible if SMD results were based on two to ten randomised trials and if protocols described the outcome. We excluded reviews if they only presented results of subgroup analyses. Based on review protocols and index outcomes, two observers independently extracted the data necessary to calculate SMDs from the original trial reports for any intervention group, time point, or outcome measure compatible with the protocol. From the extracted data, we used Monte Carlo simulations to calculate all possible SMDs for every meta-analysis. Results We identified 19 eligible meta-analyses (including 83 trials). Published review protocols often lacked information about which data to choose. Twenty-four (29%) trials reported data for multiple intervention groups, 30 (36%) reported data for multiple time points, and 29 (35%) reported the index outcome measured on multiple scales. In 18 meta-analyses, we found multiplicity of data in at least one trial report; the median difference between the smallest and largest SMD results within a meta-analysis was 0.40 standard deviation units (range 0.04 to 0.91). Conclusions Multiplicity of data can affect the findings of systematic reviews and meta-analyses. To reduce the risk of bias, reviews and meta-analyses should comply with prespecified protocols that clearly identify time points, intervention groups, and scales of interest.

Clinical application and reliability of a post abdominal surgery pain assessment scale (PASPAS) in horses

The aim of this study was to refine a multi-dimensional scale based on physiological and behavioural parameters, known as the post abdominal surgery pain assessment scale (PASPAS), to quantify pain after laparotomy in horses. After a short introduction, eight observers used the scale to assess eight horses at multiple time points after laparotomy. In addition, a single observer was used to test the correlation of each parameter with the total pain index in 34 patients, and the effect of general anaesthesia on PASPAS was investigated in a control group of eight horses. Inter-observer variability was low (coefficient of variation 0.3), which indicated good reliability of PASPAS. The correlation of individual parameters with the total pain index differed between parameters. PASPAS, which was not influenced by general anaesthesia, was a useful tool to evaluate pain in horses after abdominal surgery and may also be useful to investigate analgesic protocols or for teaching purposes.

Index of Complexity, Outcome and Need scored on plaster and digital models

The aim of this study was to compare standard plaster models with their digital counterparts for the applicability of the Index of Complexity, Outcome, and Need (ICON). Generated study models of 30 randomly selected patients: 30 pre- (T(0)) and 30 post- (T(1)) treatment. Two examiners, calibrated in the ICON, scored the digital and plaster models. The overall ICON scores were evaluated for reliability and reproducibility using kappa statistics and reliability coefficients. The values for reliability of the total and weighted ICON scores were generally high for the T(0) sample (range 0.83-0.95) but less high for the T(1) sample (range 0.55-0.85). Differences in total ICON score between plaster and digital models resulted in mostly statistically insignificant values (P values ranging from 0.07 to 0.19), except for observer 1 in the T(1) sample. No statistically different values were found for the total ICON score on either plaster or digital models. ICON scores performed on computer-based models appear to be as accurate and reliable as ICON scores on plaster models.

Oscillometric measurement of ankle-brachial index in patients with suspected peripheral disease: comparison with Doppler method

QUESTION UNDER STUDY: Purpose was to validate accuracy and reliability of automated oscillometric ankle-brachial (ABI) measurement prospectively against the current gold standard of Doppler-assisted ABI determination. METHODS: Oscillometric ABI was measured in 50 consecutive patients with peripheral arterial disease (n = 100 limbs, mean age 65 +/- 6 years, 31 men, 19 diabetics) after both high and low ABI had been determined conventionally by Doppler under standardised conditions. Correlation was assessed by linear regression and Pearson product moment correlation. Degree of inter-modality agreement was quantified by use of Bland and Altman method. RESULTS: Oscillometry was performed significantly faster than Doppler-assisted ABI (3.9 +/- 1.3 vs 11.4 +/- 3.8 minutes, P <0.001). Mean readings were 0.62 +/- 0.25, 0.70 +/- 0.22 and 0.63 +/- 0.39 for low, high and oscillometric ABI, respectively. Correlation between oscillometry and Doppler ABI was good overall (r = 0.76 for both low and high ABI) and excellent in oligo-symptomatic, non-diabetic patients (r = 0.81; 0.07 +/- 0.23); it was, however, limited in diabetic patients and in patients with critical limb ischaemia. In general, oscillometric ABI readings were slightly higher (+0.06), but linear regression analysis showed that correlation was sustained over the whole range of measurements. CONCLUSIONS: Results of automated oscillometric ABI determination correlated well with Doppler-assisted measurements and could be obtained in shorter time. Agreement was particularly high in oligo-symptomatic non-diabetic patients.

Reliability and stability of lucid dream and nightmare frequency scales

Lucid dream and nightmare frequencies vary greatly between individuals and to assess these differences reliable instruments are needed. The present study aimed to examine the reliability of eight-point scales for measuring lucid dream and nightmare frequencies. The scales were administered twice (with a four-week interval) to 93 sport students. A re-test reliability for the lucid dream frequency was found r=.89 (p<.001) and for the nightmare frequency r=.75 (p<.001). Both eight-point scales appear to be reliable measures for assessing individual differences in lucid dream and nightmare frequencies.

High-precision confocal reflection measurement for two dimensional refractive index mapping of optical fibers

We introduce a new fiber-optical approach for reflection based refractive index mapping. Our approach leads to improved stability and reliability over existing free-space confocal instruments and significantly cuts alignment efforts and reduces the number of components needed. Other than properly cleaved fiber end-faces, this setup requires no additional sample preparation. The instrument is calibrated by means of a set of samples with known refractive indices. The index steps of commercially available fibers are measured accurately down to < 10⁻³. The precision limit of the instrument is currently of the order of 10⁻⁴.