Interobserver variability in assessment of cranial ultrasound in very preterm infants

Cranial ultrasound (cUS) findings help doctors in the clinical management of preterm infants and in their discussion with parents regarding prediction of outcome. cUS is often used as outcome measure in clinical research studies. Accurate cUS performance and interpretation is therefore required.

Evaluation of filtering blebs using the 'Wuerzburg bleb classification score' compared to clinical findings

To determine the agreement between intraocular pressure and the 'Wuerzburg bleb classification score', as well as between single items of the score and intraocular pressure. Interobserver variability was analyzed.

The postoperative COFAS end-stage ankle arthritis classification system: interobserver and intraobserver reliability

End-stage ankle arthritis is operatively treated with numerous designs of total ankle replacement and different techniques for ankle fusion. For superior comparison of these procedures, outcome research requires a classification system to stratify patients appropriately. A postoperative 4-type classification system was designed by 6 fellowship-trained foot and ankle surgeons. Four surgeons reviewed blinded patient profiles and radiographs on 2 occasions to determine the interobserver and intraobserver reliability of the classification. Excellent interobserver reliability (κ = .89) and intraobserver reproducibility (κ = .87) were demonstrated for the postoperative classification system. In conclusion, the postoperative Canadian Orthopaedic Foot and Ankle Society (COFAS) end-stage ankle arthritis classification system appears to be a valid tool to evaluate the outcome of patients operated for end-stage ankle arthritis.

Good agreement between capillary and venous sampling for lectin pathway proteins

Background: The lectin pathway of complement activation, in particular mannose-binding lectin (MBL), has been extensively investigated over recent years. So far, studies were exclusively based on venous samples. The aim of this study was to investigate whether measurements of lectin pathway proteins obtained by capillary sampling are in agreement with venous samples. Methods: Prospective study including 31 infants that were admitted with suspected early-onset sepsis. Lectin pathway proteins were measured in simultaneously obtained capillary and venous samples. Bland–Altman plots of logarithmized results were constructed, and the mean capillary to venous ratios (ratiocap/ven) were calculated with their 95% confidence intervals (CI). Results: The agreement between capillary and venous sampling was very high for MBL (mean ratiocap/ven, 1.01; 95% CI, 0.85–1.19). Similarly, high agreement was observed for H-ficolin (mean ratiocap/ven, 1.02; 95% CI, 0.72–1.44), MASP-2 (1.04; 0.59–1.84), MASP-3 (0.96; 0.71–1.28), and MAp44 (1.01; 0.82–1.25), while the agreement was moderate for M-ficolin (mean ratiocap/ven, 0.78; 95% CI, 0.27–2.28). Conclusions: The results of this study show an excellent agreement between capillary and venous samples for most lectin pathway proteins. Except for M-ficolin, small volume capillary samples can thus be used when assessing lectin pathway proteins in neonates and young children.

Ocular pulse amplitude in healthy subjects as measured by dynamic contour tonometry

OBJECTIVES: To test whether dynamic contour tonometry yields ocular pulse amplitude (OPA) measurements that are independent of corneal thickness and curvature, and to assess variables of observer agreement. METHODS: In a multivariate cluster analysis on 223 eyes, the relationship between central corneal thickness, corneal curvature, axial length, anterior chamber depth, intraocular pressure, sex, age, and OPA measurements was assessed. Intraobserver and interobserver variabilities were calculated from repeated measurements obtained from 8 volunteers by 4 observers. RESULTS: The OPA readings were not affected by central corneal thickness (P = .08), corneal curvature (P = .47), anterior chamber depth (P = .80), age (P = .60), or sex (P = .73). There was a positive correlation between OPA and intraocular pressure (0.12 mm Hg/1 mm Hg of intraocular pressure; P<.001) and a negative correlation between OPA and axial length (0.27 mm Hg/1 mm of length; P<.001). Intraobserver and interobserver variabilities were 0.08 and 0.02 mm Hg, respectively, and the intraclass correlation coefficient was 0.89. CONCLUSIONS: The OPA readings obtained with dynamic contour tonometry in healthy subjects are not influenced by the structure of the anterior segment of the eye but are affected by intraocular pressure and axial length. We found a high amount of agreement within and between observers.

Agreement between HbA1c measured by DCA 2000 and by HPLC: effects of fetal hemoglobin concentrations

In subjects with type 1 diabetes, persisting elevations of fetal hemoglobin (HbF) have been demonstrated. This study evaluated whether HbF levels typically seen in type 1 diabetes (up to 3%) interfere with glycohemoglobin determinations using a common immunologic method (DCA 2000).

Orthogonal polarization spectroscopy to detect mesenteric hypoperfusion

OBJECTIVE: Orthogonal polarization spectral (OPS) imaging is used to assess mucosal microcirculation. We tested sensitivity and variability of OPS in the assessment of mesenteric blood flow (Q (sma)) reduction. SETTING: University Animal Laboratory. INTERVENTIONS: In eight pigs, Q (sma) was reduced in steps of 15% from baseline; five animals served as controls. Jejunal mucosal microcirculatory blood flow was recorded with OPS and laser Doppler flowmetry at each step. OPS data from each period were collected and randomly ordered. Samples from each period were individually chosen by two blinded investigators and quantified [capillary density (number of vessels crossing predefined lines), number of perfused villi] after agreement on the methodology. MEASUREMENT AND RESULTS: Interobserver coefficient of variation (CV) for capillary density from samples representing the same flow condition was 0.34 (0.04-1.41) and intraobserver CV was 0.10 (0.02-0.61). Only one investigator observed a decrease in capillary density [to 62% (48-82%) of baseline values at 45% Q (sma) reduction; P = 0.011], but comparisons with controls never revealed significant differences. In contrast, reduction in perfused villi was detected by both investigators at 75% of mesenteric blood flow reduction. Laser Doppler flow revealed heterogeneous microcirculatory perfusion. CONCLUSIONS: Assessment of capillary density did not reveal differences between animals with and without Q (sma) reduction, and evaluation of perfused villi revealed blood flow reduction only when Q (sma) was very low. Potential explanations are blood flow redistribution and heterogeneity, and suboptimal contrast of OPS images. Despite agreement on the method of analysis, interobserver differences in the quantification of vessel density on gut mucosa using OPS are high.

Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study

OBJECTIVE: To study the inter-observer variation related to extraction of continuous and numerical rating scale data from trial reports for use in meta-analyses. DESIGN: Observer agreement study. DATA SOURCES: A random sample of 10 Cochrane reviews that presented a result as a standardised mean difference (SMD), the protocols for the reviews and the trial reports (n=45) were retrieved. DATA EXTRACTION: Five experienced methodologists and five PhD students independently extracted data from the trial reports for calculation of the first SMD result in each review. The observers did not have access to the reviews but to the protocols, where the relevant outcome was highlighted. The agreement was analysed at both trial and meta-analysis level, pairing the observers in all possible ways (45 pairs, yielding 2025 pairs of trials and 450 pairs of meta-analyses). Agreement was defined as SMDs that differed less than 0.1 in their point estimates or confidence intervals. RESULTS: The agreement was 53% at trial level and 31% at meta-analysis level. Including all pairs, the median disagreement was SMD=0.22 (interquartile range 0.07-0.61). The experts agreed somewhat more than the PhD students at trial level (61% v 46%), but not at meta-analysis level. Important reasons for disagreement were differences in selection of time points, scales, control groups, and type of calculations; whether to include a trial in the meta-analysis; and data extraction errors made by the observers. In 14 out of the 100 SMDs calculated at the meta-analysis level, individual observers reached different conclusions than the originally published review. CONCLUSIONS: Disagreements were common and often larger than the effect of commonly used treatments. Meta-analyses using SMDs are prone to observer variation and should be interpreted with caution. The reliability of meta-analyses might be improved by having more detailed review protocols, more than one observer, and statistical expertise.