Diagnostic reproducibility of tumour budding in colorectal cancer: a multicentre, multinational study using virtual microscopy

Puppa G, Senore C, Sheahan K, Vieth M, Lugli A, Zlobec I, Pecori S, Wang L M, Langner C, Mitomi H, Nakamura T, Watanabe M, Ueno H, Chasle J, Conley S A, Herlin P, Lauwers G Y & Risio M (2012) Histopathology Diagnostic reproducibility of tumour budding in colorectal cancer: a multicentre, multinational study using virtual microscopy Aims:  Despite the established prognostic relevance of tumour budding in colorectal cancer, the reproducibility of the methods reported for its assessment has not yet been determined, limiting its use and reporting in routine pathology practice. Methods and results:  A morphometric system within telepathology was devised to evaluate the reproducibility of the various methods published for the assessment of tumour budding in colorectal cancer. Five methods were selected to evaluate the diagnostic reproducibility among 10 investigators, using haematoxylin and eosin (H&E) and AE1-3 cytokeratin-immunostained, whole-slide digital scans from 50 pT1-pT4 colorectal cancers. The overall interobserver agreement was fair for all methods, and increased to moderate for pT1 cancers. The intraobserver agreement was also fair for all methods and moderate for pT1 cancers. Agreement was dependent on the participants' experience with tumour budding reporting and performance time. Cytokeratin immunohistochemistry detected a higher percentage of tumour budding-positive cases with all methods compared to H&E-stained slides, but did not influence agreement levels. Conclusions:  An overall fair level of diagnostic agreement for tumour budding in colorectal cancer was demonstrated, which was significantly higher in early cancer and among experienced gastrointestinal pathologists. Cytokeratin immunostaining facilitated detection of budding cancer cells, but did not result in improved interobserver agreement.

Intrapatient reproducibility of the BISxp monitor

BACKGROUND: The Bispectral Index (BIS) reportedly reflects anesthetic depth. It is recommended that anesthetic agents should be titrated to maintain the BIS between 40 and 60 arbitrary BIS units during anesthesia. For anesthesia providers to follow this recommendation, the monitor should be predictably affected by different anesthetic agents and have good interpatient and intrapatient reproducibility. The authors hypothesized that when two BISxp devices (Aspect Medical Systems, Newton, MA) are placed concurrently on the same patient, their readings are concordant throughout the anesthetic period. METHODS: Simultaneous BIS recordings from two BISxp monitors were obtained during anesthesia at 5-s intervals from 12 participants. RESULTS: In total 22,860 concurrent paired BIS readings were obtained. For 10.7% of the time, there were sustained periods of 30 s or greater where the readings suggested a different depth of anesthesia. For 6% of the time, there were sustained periods of 30 s or greater where the readings differed by 10 or more arbitrary BIS units. The regression coefficient (R) for the two devices was 0.65 (range, 0.35-0.92). There was zero bias between the devices, and the 95% limits of agreement ranged between -18 and +17. CONCLUSION: A conflicting anesthetic management was suggested by the simultaneous BIS readings 10.7% of the time. These results suggest that BISxp does not always provide a reproducible single number. Anesthesia providers should not rely exclusively on the BIS reading when assessing depth of anesthesia.

Reproducibility of dGEMRIC in assessment of hip joint cartilage: a prospective study

PURPOSE: To investigate the reproducibility of dGEMRIC in the assessment of cartilage health of the adult asymptomatic hip joint. MATERIALS AND METHODS: Fifteen asymptomatic volunteers (mean age, 26.3 years +/- 3.0) were preliminarily studied. Any volunteer that was incidentally diagnosed with damaged cartilage on MRI (n = 5) was excluded. Ten patients that had no evidence of prior cartilage damage (mean age, 26.2 years +/- 3.4) were evaluated further in this study. The reproducibility of dGEMRIC was assessed with two T1(Gd) exams performed 4 weeks apart in these volunteers. The protocol involved an initial standard MRI to confirm healthy cartilage, which was then followed by dGEMRIC. The second scan included only the repeat dGEMRIC. Region of interest (ROI) analyses for T1(Gd)-measurement was performed in seven radial reformats. Statistical analysis included the student's t-test and intra-class correlation (ICC) measurement to assess reproducibility. RESULTS: Overall 70 ROIs were studied. Mean cartilage T1(Gd) values at various loci ranged from 560.9 ms to 684.4 ms at the first set of readings and 551.5 ms to 662.2 ms in the second one. The mean difference per region of interest between the two T1(Gd)-measurements ranged from 21.4 ms (3.7%) to 45.0 ms (6.8%), which was not found to be statistically significant (P = 0.153). There was a high reproducibility detected (ICC range, 0.667-0.915). Intra- and Inter-observer analyses proved a high agreement for T1(Gd) assessment (0.973 and 0.932). CONCLUSION: We found dGEMRIC to be a reliable tool in the assessment of cartilage health status in adult hip joints.

Variability of anti-PF4/heparin antibody results obtained by the rapid testing system ID-H/PF4-PaGIA

BACKGROUND: Recent studies have shown that a low clinical pretest probability may be adequate for excluding heparin-induced thrombocytopenia. However, for patients with intermediate or high pretest probability, laboratory testing is essential for confirming or refuting the diagnosis. Rapid assessment of anti-PF4/heparin-antibodies may assist clinical decision-making. OBJECTIVES: To evaluate the performance of rapid ID-H/PF4-PaGIA. In particular, we verified reproducibility of results between plasma and serum specimens, between fresh and frozen samples, and between different ID-H/PF4-polymer lots (polystyrene beads coated with heparin/PF4-complexes). PATIENTS/METHODS: The samples studied were 1376 plasma and 914 corresponding serum samples from patients investigated for suspected heparin-induced thrombocytopenia between January 2000 and October 2008. Anti-PF4/heparin-antibodies were assessed by ID-H/PF4-PaGIA, commercially available ELISAs and heparin-induced platelet aggregation test. RESULTS: Among 914 paired plasma/serum samples we noted discordant results (negative vs. low-titre positive) in nine instances (1%; 95%CI, 0.4-1.6%). Overall, agreement between titres assessed in plasma vs. serum was highly significant (Spearman correlation coefficient, 0.975; P < 0.0001). Forty-seven samples tested both fresh and after freezing/thawing showed a good agreement, with one discordant positive/negative result (Spearman correlation coefficient, 0.970; P < 0.0001). Among 1376 plasma samples we noted a strikingly variable incidence of false negative results (none - 82%; 95%CI, 66-98%), depending on the employed ID-H/PF4-polymer lot. Faulty lots can be recognized by titrating commercial positive controls and stored samples of HIT-patients. CONCLUSION: Laboratories performing the assay should implement stringent internal quality controls in order to recognize potentially faulty ID-H/PF4-polymer lots, thus avoiding false negative results.

Reproducibility of cerebral phenylalanine levels in patients with phenylketonuria determined by 1H-MR spectroscopy

The reproducibility of metabolite content determined by MR spectroscopy (MRS) is usually at best a few percent for the prominent singlets. When studying low-concentration metabolites, like phenylalanine (Phe), where tissue content can be <100 micromol/kg, better reproducibility is paramount-particularly in view of using MRS results for potential individual treatment advice. An optimized, targeted spectroscopy method was established at 1.5T and reproducibility was established in 21 patients with phenylketonuria (PKU) where three spectra were recorded in each of three independent sessions, two of which were in immediate succession to minimize physiologic variation. Intersession variation was found to be only 7 micromol/kg Phe for back-to-back repetition of sessions, in close agreement with the variation of 16 micromol/kg observed for single spectra within a session. Analysis of variance proved the individuality of the blood/brain Phe ratio-though this ratio seems to be influenced by physiologic factors that are not stable in time. The excellent reproducibility was achieved through optimization of various factors, including signal-to-noise ratio, repositioning, and prescan calibrations, but also by enforcing as much prior information as possible (e.g., lineshape and phase from reference scans, constant prior-knowledge-locked baseline). While the application of maximum general prior knowledge is a general method to reduce fluctuations, one should remember that it may introduce systematic errors.

Reproducibility of retinal thickness measurements in healthy subjects using spectralis optical coherence tomography

PURPOSE: To test the reproducibility of retinal thickness measurements in healthy volunteers of a new Frequency-domain optical coherence tomography (OCT) device (Spectralis OCT; Heidelberg Engineering, Heidelberg, Germany). DESIGN: Prospective, observational study. METHODS: Forty-one eyes of 41 healthy subjects were included into the study. Intraobserver reproducibility was tested with 20 x 15 degree raster scans consisting of 37 high-resolution line scans that were repeated three times by one examiner (M.N.M.). Mean retinal thickness was calculated for nine areas corresponding to the Early Treatment Diabetic Retinopathy Study (ETDRS) areas. Coefficients of variation (COV) were calculated. RESULTS: Retinal thickness measurements were highly reproducible for all ETDRS areas. Mean total retinal thickness was 342 +/- 15 microm. Mean foveal thickness was 286 +/- 17 microm. COVs ranged from 0.38% to 0.86%. Lowest COV was found for the temporal outer ETDRS area (area 7; COV, 0.38%). Highest COV was found for the temporal inner ETDRS area (area 3; COV, 0.86%). Mean difference between measurement 1 and 2, measurement 1 and 3, and measurement 2 and 3 for all ETDRS areas was 1.01 microm, 0.98 microm, and 0.99 microm, respectively. CONCLUSION: Spectralis OCT retinal thickness measurements in healthy volunteers showed excellent intraobserver reproducibility with virtually identical results between retinal thickness measurements performed by one operator.

Accuracy and reproducibility of aortic annulus sizing using a dedicated three-dimensional computed tomography reconstruction tool in patients evaluated for transcatheter aortic valve replacement

Aims: To evaluate the accuracy and reproducibility of aortic annulus sizing using a multislice computed tomography (MSCT) based aortic root reconstruction tool compared with conventional imaging among patients evaluated for transcatheter aortic valve replacement (TAVR). Methods and results: Patients referred for TAVR underwent standard preprocedural assessment of aortic annulus parameters using MSCT, angiography and transoesophageal echocardiography (TEE). Three-dimensional (3-D) reconstruction of MSCT images of the aortic root was performed using 3mensio (3mensio Medical Imaging BV, Bilthoven, The Netherlands), allowing for semi-automated delineation of the annular plane and assessment of annulus perimeter, area, maximum, minimum and virtual diameters derived from area and perimeter (aVD and pVD). A total of 177 patients were enrolled. We observed a good inter-observer variability of 3-D reconstruction assessments with concordance coefficients for agreement of 0.91 (95% CI: 0.87-0.93) and 0.91 (0.88-0.94) for annulus perimeter and area assessments, respectively. 3-D derived pVD and aVD correlated very closely with a concordance coefficient of 0.97 (0.96-0.98) with a mean difference of 0.5±0.3 mm (pVD-aVD). 3-D derived pVD showed the best, but moderate concordance with diameters obtained from coronal MSCT (0.67, 0.56-0.75; 0.3±1.8 mm), and the lowest concordance with diameters obtained from TEE (0.42, 0.31-0.52; 1.9±1.9 mm). Conclusions: MSCT-based 3-D reconstruction of the aortic annulus using the 3mensio software enables accurate and reproducible assessment of aortic annulus dimensions.

Reproducibility of effects of homeopathically potentised gibberellic acid on the growth of Lemna gibba L. in a randomised and blinded bioassay

BACKGROUND: Reproducibility of basic research investigations in homeopathy is challenging. This study investigated if formerly observed effects of homeopathically potentised gibberellic acid (GA3) on growth of duckweed (Lemna gibba L.) were reproducible. METHODS: Duckweed was grown in potencies (14x-30x) of GA3 and one time succussed and unsuccussed water controls. Outcome parameter area-related growth rate was determined by a computerised image analysis system. Three series including five independent blinded and randomised potency experiments (PE) each were carried out. System stability was controlled by three series of five systematic negative control (SNC) experiments. Gibbosity (a specific growth state of L. gibba) was investigated as possibly essential factor for reactivity of L. gibba towards potentised GA3 in one series of potency and SNC experiments, respectively. RESULTS: Only in the third series with gibbous L. gibba L. we observed a significant effect (p = 0.009, F-test) of the homeopathic treatment. However, growth rate increased in contrast to the former study, and most biologically active potency levels differed. Variability in PE was lower than in SNC experiments. The stability of the experimental system was verified by the SNC experiments. CONCLUSIONS: Gibbosity seems to be a necessary condition for reactivity of L. gibba to potentised GA3. Further still unknown conditions seem to govern effect direction and the pattern of active and inactive potency levels. When designing new reproducibility studies, the physiological state of the test organism must be considered. Variability might be an interesting parameter to investigate effects of homeopathic remedies in basic research.

Reproducibility of peripheral arterial tonometry measurements in male cardiovascular patients

BACKGROUND Assessment of endothelial function of the microvasculature by peripheral arterial tonometry (EndoPAT(®)) has gained increasing popularity in patients with cardiovascular risk factors. Only limited knowledge about its reproducibility in patients with coronary artery disease (CAD) is available. We therefore aimed to quantify reproducibility of EndoPAT(®) parameters in patients with stable CAD. DESIGN EndoPAT(®) measurements were performed repeatedly in 78 male patients (age 66 ± 8 years) with CAD on stable medication. We calculated overall mean, standard deviation (SD), coefficient of variation (CV) and intraclass correlation coefficient (ICC) of the following parameters: reactive hyperemic index (RHI), PAT ratio of the postocclusion period 90-150 s as used for calculation of the RHI (PAT ratio90-150 s) and 90-120 s (PAT ratio90-120 s) as used for the often employed Framingham RHI (F-RHI), as well as PAT ratio of the peak hyperemic response (PAT ratiopeak response). Additionally, least significant changes (LSC) for individual subjects and minimum sample sizes for parallel and cross-over design studies were calculated. RESULTS Mean RHI was 1·84 (SD 0·36). For RHI, PAT ratio90-150 s , PAT ratio90-120 s , and PAT ratiopeak response the CVs were 17·0%, 25·4%, 26·1%, and 25·0%, respectively. The ICCs were 0·45, 0·49, 0·48 and 0·51, respectively, and LSC for RHI was 47·2%. CONCLUSIONS CV of RHI in our population was moderate; however, we consider this precision insufficient to monitor changes in individual patients, as they would need to exceed 47% to show a significant change. Further, the poor ICCs reflect the difficulty of detecting treatment effects in homogenous populations, such as patients with stable CAD.

Good reproducibility of heart rate variability after orthostatic challenge in patients with a history of acute coronary syndrome

BACKGROUND: Several parameters of heart rate variability (HRV) have been shown to predict the risk of sudden cardiac death (SCD) in cardiac patients. There is consensus that risk prediction is increased when measuring HRV during specific provocations such as orthostatic challenge. For the first time, we provide data on reproducibility of such a test in patients with a history of acute coronary syndrome. METHODS: Sixty male patients (65+/-8years) with a history of acute coronary syndrome on stable medication were included. HRV was measured in supine (5min) and standing (5min) position on 2 occasions separated by two weeks. For risk assessment relevant time-domain [standard deviation of all R-R intervals (SDNN) and root mean squared standard differences between adjacent R-R intervals (RMSSD)], frequency domain [low-frequency power (LF), high-frequency power (HF) and LF/HF power ratio] and short-term fractal scaling component (DF1) were computed. Absolute reproducibility was assessed with the standard errors of the mean (SEM) and 95% limits of random variation, and relative reproducibility by the intraclass correlation coefficient (ICC). RESULTS: We found comparable SEMs and ICCs in supine position and after an orthostatic challenge test. All ICCs were good to excellent (ICCs between 0.636 and 0.869). CONCLUSIONS: Reproducibility of HRV parameters during orthostatic challenge is good and comparable with supine position.

A Semantic-Based Approach to Attain Reproducibility of Computational Environments in Scientic Work ows: A Case Study

Reproducible research in scientic work ows is often addressed by tracking the provenance of the produced results. While this approach allows inspecting intermediate and nal results, improves understanding, and permits replaying a work ow execution, it does not ensure that the computational environment is available for subsequent executions to reproduce the experiment. In this work, we propose describing the resources involved in the execution of an experiment using a set of semantic vocabularies, so as to conserve the computational environment. We dene a process for documenting the work ow application, management system, and their dependencies based on 4 domain ontologies. We then conduct an experimental evaluation sing a real work ow application on an academic and a public Cloud platform. Results show that our approach can reproduce an equivalent execution environment of a predened virtual machine image on both computing platforms.

A semantic-based approach to attain reproducibility of computational environments in scientific workflows: a case study

Reproducible research in scientific workflows is often addressed by tracking the provenance of the produced results. While this approach allows inspecting intermediate and final results, improves understanding, and permits replaying a workflow execution, it does not ensure that the computational environment is available for subsequent executions to reproduce the experiment. In this work, we propose describing the resources involved in the execution of an experiment using a set of semantic vocabularies, so as to conserve the computational environment. We define a process for documenting the workflow application, management system, and their dependencies based on 4 domain ontologies. We then conduct an experimental evaluation using a real workflow application on an academic and a public Cloud platform. Results show that our approach can reproduce an equivalent execution environment of a predefined virtual machine image on both computing platforms.

Repeatability and reproducibility of corneal thickness using SOCT Copernicus HR

Purpose: The aim of this study is to determine the reliability of corneal thickness measurements derived from SOCT Copernicus HR (Fourier domain OCT). Methods: Thirty healthy eyes of 30 subjects were evaluated. One eye of each patient was chosen randomly. Images were obtained of the central (up to 2.0 mm from the corneal apex) and paracentral (2.0 to 4.0 mm) cornea. We assessed corneal thickness (central and paracentral) and epithelium thickness. The intra-observer repeatability data were analysed using the intra-class correlation coefficient (ICC) for a range of 95 per cent within-subject standard deviation (SW) and the within-subject coefficient of variation (CW). The level of agreement by Bland–Altman analysis was also represented for the study of the reproducibility between observers and agreement between methods of measurement (automatic versus manual). Results: The mean value of the central corneal thickness (CCT) was 542.4 ± 30.1 μm (SD). There was a high intra-observer agreement, finding the best result in the central sector with an intra-class correlation coefficient of 0.99, 95 per cent CI (0.989 to 0.997) and the worst, in the minimum corneal thickness, with an intra-class correlation coefficient of 0.672, 95 per cent CI (0.417 to 0.829). Reproducibility between observers was very high. The best result was found in the central sector thickness obtained both manually and automatically with an intra-class correlation coefficient of 0.990 in both cases and the worst result in the maximum corneal thickness with an intra-class correlation coefficient of 0.827. The agreement between measurement methods was also very high with intra-class correlation coefficient greater than 0.91. On the other hand the repeatability and reproducibility for epithelial measurements was poor. Conclusion: Pachymetric mapping with SOCT Copernicus HR was found to be highly repeatable and reproducible. We found that the device lacks an appropriate ergonomic design as proper focusing of the laser beam onto the cornea for anterior segment scanning required that patients were positioned slightly farther away from the machine head-rest than in the setup for retinal imaging.

Multiple training interventions significantly improve reproducibility of PET/CT-based lung cancer radiotherapy target volume delineation using an IAEA study protocol

BACKGROUND AND PURPOSE: To assess the impact of a standardized delineation protocol and training interventions on PET/CT-based target volume delineation (TVD) in NSCLC in a multicenter setting.

MATERIAL AND METHODS: Over a one-year period, 11 pairs, comprised each of a radiation oncologist and nuclear medicine physician with limited experience in PET/CT-based TVD for NSCLC from nine different countries took part in a training program through an International Atomic Energy Agency (IAEA) study (NCT02247713). Teams delineated gross tumor volume of the primary tumor, during and after training interventions, according to a provided delineation protocol. In-house developed software recorded the performed delineations, to allow visual inspection of strategies and to assess delineation accuracy.

RESULTS: Following the first training, overall concordance indices for 3 repetitive cases increased from 0.57±0.07 to 0.66±0.07. The overall mean surface distance between observer and expert contours decreased from -0.40±0.03 cm to -0.01±0.33 cm. After further training overall concordance indices for another 3 repetitive cases further increased from 0.64±0.06 to 0.80±0.05 (p=0.01). Mean surface distances decreased from -0.34±0.16 cm to -0.05±0.20 cm (p=0.01).

CONCLUSION: Multiple training interventions improve PET/CT-based TVD delineation accuracy in NSCLC and reduces interobserver variation.