Reproducibility of cerebral phenylalanine levels in patients with phenylketonuria determined by 1H-MR spectroscopy

The reproducibility of metabolite content determined by MR spectroscopy (MRS) is usually at best a few percent for the prominent singlets. When studying low-concentration metabolites, like phenylalanine (Phe), where tissue content can be <100 micromol/kg, better reproducibility is paramount-particularly in view of using MRS results for potential individual treatment advice. An optimized, targeted spectroscopy method was established at 1.5T and reproducibility was established in 21 patients with phenylketonuria (PKU) where three spectra were recorded in each of three independent sessions, two of which were in immediate succession to minimize physiologic variation. Intersession variation was found to be only 7 micromol/kg Phe for back-to-back repetition of sessions, in close agreement with the variation of 16 micromol/kg observed for single spectra within a session. Analysis of variance proved the individuality of the blood/brain Phe ratio-though this ratio seems to be influenced by physiologic factors that are not stable in time. The excellent reproducibility was achieved through optimization of various factors, including signal-to-noise ratio, repositioning, and prescan calibrations, but also by enforcing as much prior information as possible (e.g., lineshape and phase from reference scans, constant prior-knowledge-locked baseline). While the application of maximum general prior knowledge is a general method to reduce fluctuations, one should remember that it may introduce systematic errors.

Reproducibility of retinal thickness measurements in healthy subjects using spectralis optical coherence tomography

PURPOSE: To test the reproducibility of retinal thickness measurements in healthy volunteers of a new Frequency-domain optical coherence tomography (OCT) device (Spectralis OCT; Heidelberg Engineering, Heidelberg, Germany). DESIGN: Prospective, observational study. METHODS: Forty-one eyes of 41 healthy subjects were included into the study. Intraobserver reproducibility was tested with 20 x 15 degree raster scans consisting of 37 high-resolution line scans that were repeated three times by one examiner (M.N.M.). Mean retinal thickness was calculated for nine areas corresponding to the Early Treatment Diabetic Retinopathy Study (ETDRS) areas. Coefficients of variation (COV) were calculated. RESULTS: Retinal thickness measurements were highly reproducible for all ETDRS areas. Mean total retinal thickness was 342 +/- 15 microm. Mean foveal thickness was 286 +/- 17 microm. COVs ranged from 0.38% to 0.86%. Lowest COV was found for the temporal outer ETDRS area (area 7; COV, 0.38%). Highest COV was found for the temporal inner ETDRS area (area 3; COV, 0.86%). Mean difference between measurement 1 and 2, measurement 1 and 3, and measurement 2 and 3 for all ETDRS areas was 1.01 microm, 0.98 microm, and 0.99 microm, respectively. CONCLUSION: Spectralis OCT retinal thickness measurements in healthy volunteers showed excellent intraobserver reproducibility with virtually identical results between retinal thickness measurements performed by one operator.

Accuracy and reproducibility of aortic annulus sizing using a dedicated three-dimensional computed tomography reconstruction tool in patients evaluated for transcatheter aortic valve replacement

Aims: To evaluate the accuracy and reproducibility of aortic annulus sizing using a multislice computed tomography (MSCT) based aortic root reconstruction tool compared with conventional imaging among patients evaluated for transcatheter aortic valve replacement (TAVR). Methods and results: Patients referred for TAVR underwent standard preprocedural assessment of aortic annulus parameters using MSCT, angiography and transoesophageal echocardiography (TEE). Three-dimensional (3-D) reconstruction of MSCT images of the aortic root was performed using 3mensio (3mensio Medical Imaging BV, Bilthoven, The Netherlands), allowing for semi-automated delineation of the annular plane and assessment of annulus perimeter, area, maximum, minimum and virtual diameters derived from area and perimeter (aVD and pVD). A total of 177 patients were enrolled. We observed a good inter-observer variability of 3-D reconstruction assessments with concordance coefficients for agreement of 0.91 (95% CI: 0.87-0.93) and 0.91 (0.88-0.94) for annulus perimeter and area assessments, respectively. 3-D derived pVD and aVD correlated very closely with a concordance coefficient of 0.97 (0.96-0.98) with a mean difference of 0.5±0.3 mm (pVD-aVD). 3-D derived pVD showed the best, but moderate concordance with diameters obtained from coronal MSCT (0.67, 0.56-0.75; 0.3±1.8 mm), and the lowest concordance with diameters obtained from TEE (0.42, 0.31-0.52; 1.9±1.9 mm). Conclusions: MSCT-based 3-D reconstruction of the aortic annulus using the 3mensio software enables accurate and reproducible assessment of aortic annulus dimensions.

Reproducibility of effects of homeopathically potentised gibberellic acid on the growth of Lemna gibba L. in a randomised and blinded bioassay

BACKGROUND: Reproducibility of basic research investigations in homeopathy is challenging. This study investigated if formerly observed effects of homeopathically potentised gibberellic acid (GA3) on growth of duckweed (Lemna gibba L.) were reproducible. METHODS: Duckweed was grown in potencies (14x-30x) of GA3 and one time succussed and unsuccussed water controls. Outcome parameter area-related growth rate was determined by a computerised image analysis system. Three series including five independent blinded and randomised potency experiments (PE) each were carried out. System stability was controlled by three series of five systematic negative control (SNC) experiments. Gibbosity (a specific growth state of L. gibba) was investigated as possibly essential factor for reactivity of L. gibba towards potentised GA3 in one series of potency and SNC experiments, respectively. RESULTS: Only in the third series with gibbous L. gibba L. we observed a significant effect (p = 0.009, F-test) of the homeopathic treatment. However, growth rate increased in contrast to the former study, and most biologically active potency levels differed. Variability in PE was lower than in SNC experiments. The stability of the experimental system was verified by the SNC experiments. CONCLUSIONS: Gibbosity seems to be a necessary condition for reactivity of L. gibba to potentised GA3. Further still unknown conditions seem to govern effect direction and the pattern of active and inactive potency levels. When designing new reproducibility studies, the physiological state of the test organism must be considered. Variability might be an interesting parameter to investigate effects of homeopathic remedies in basic research.

Reproducibility and accuracy of linear measurements on dental models derived from cone-beam computed tomography compared with digital dental casts.

INTRODUCTION The aim of this study was to determine the reproducibility and accuracy of linear measurements on 2 types of dental models derived from cone-beam computed tomography (CBCT) scans: CBCT images, and Anatomodels (InVivoDental, San Jose, Calif); these were compared with digital models generated from dental impressions (Digimodels; Orthoproof, Nieuwegein, The Netherlands). The Digimodels were used as the reference standard. METHODS The 3 types of digital models were made from 10 subjects. Four examiners repeated 37 linear tooth and arch measurements 10 times. Paired t tests and the intraclass correlation coefficient were performed to determine the reproducibility and accuracy of the measurements. RESULTS The CBCT images showed significantly smaller intraclass correlation coefficient values and larger duplicate measurement errors compared with the corresponding values for Digimodels and Anatomodels. The average difference between measurements on CBCT images and Digimodels ranged from -0.4 to 1.65 mm, with limits of agreement values up to 1.3 mm for crown-width measurements. The average difference between Anatomodels and Digimodels ranged from -0.42 to 0.84 mm with limits of agreement values up to 1.65 mm. CONCLUSIONS Statistically significant differences between measurements on Digimodels and Anatomodels, and between Digimodels and CBCT images, were found. Although the mean differences might be clinically acceptable, the random errors were relatively large compared with corresponding measurements reported in the literature for both Anatomodels and CBCT images, and might be clinically important. Therefore, with the CBCT settings used in this study, measurements made directly on CBCT images and Anatomodels are not as accurate as measurements on Digimodels.

Reproducibility of peripheral arterial tonometry measurements in male cardiovascular patients

BACKGROUND Assessment of endothelial function of the microvasculature by peripheral arterial tonometry (EndoPAT(®)) has gained increasing popularity in patients with cardiovascular risk factors. Only limited knowledge about its reproducibility in patients with coronary artery disease (CAD) is available. We therefore aimed to quantify reproducibility of EndoPAT(®) parameters in patients with stable CAD. DESIGN EndoPAT(®) measurements were performed repeatedly in 78 male patients (age 66 ± 8 years) with CAD on stable medication. We calculated overall mean, standard deviation (SD), coefficient of variation (CV) and intraclass correlation coefficient (ICC) of the following parameters: reactive hyperemic index (RHI), PAT ratio of the postocclusion period 90-150 s as used for calculation of the RHI (PAT ratio90-150 s) and 90-120 s (PAT ratio90-120 s) as used for the often employed Framingham RHI (F-RHI), as well as PAT ratio of the peak hyperemic response (PAT ratiopeak response). Additionally, least significant changes (LSC) for individual subjects and minimum sample sizes for parallel and cross-over design studies were calculated. RESULTS Mean RHI was 1·84 (SD 0·36). For RHI, PAT ratio90-150 s , PAT ratio90-120 s , and PAT ratiopeak response the CVs were 17·0%, 25·4%, 26·1%, and 25·0%, respectively. The ICCs were 0·45, 0·49, 0·48 and 0·51, respectively, and LSC for RHI was 47·2%. CONCLUSIONS CV of RHI in our population was moderate; however, we consider this precision insufficient to monitor changes in individual patients, as they would need to exceed 47% to show a significant change. Further, the poor ICCs reflect the difficulty of detecting treatment effects in homogenous populations, such as patients with stable CAD.

Loss on ignition as a method for estimating organic and carbonate content in sediments: reproducibility and comparability of results

Five test runs were performed to assess possible bias when performing the loss on ignition (LOI) method to estimate organic matter and carbonate content of lake sediments. An accurate and stable weight loss was achieved after 2 h of burning pure CaCO3 at 950 °C, whereas LOI of pure graphite at 530 °C showed a direct relation to sample size and exposure time, with only 40-70% of the possible weight loss reached after 2 h of exposure and smaller samples losing weight faster than larger ones. Experiments with a standardised lake sediment revealed a strong initial weight loss at 550 °C, but samples continued to lose weight at a slow rate at exposure of up to 64 h, which was likely the effect of loss of volatile salts, structural water of clay minerals or metal oxides, or of inorganic carbon after the initial burning of organic matter. A further test-run revealed that at 550 °C samples in the centre of the furnace lost more weight than marginal samples. At 950 °C this pattern was still apparent but the differences became negligible. Again, LOI was dependent on sample size. An analytical LOI quality control experiment including ten different laboratories was carried out using each laboratory's own LOI procedure as well as a standardised LOI procedure to analyse three different sediments. The range of LOI values between laboratories measured at 550 °C was generally larger when each laboratory used its own method than when using the standard method. This was similar for 950 °C, although the range of values tended to be smaller. The within-laboratory range of LOI measurements for a given sediment was generally small. Comparisons of the results of the individual and the standardised method suggest that there is a laboratory-specific pattern in the results, probably due to differences in laboratory equipment and/or handling that could not be eliminated by standardising the LOI procedure. Factors such as sample size, exposure time, position of samples in the furnace and the laboratory measuring affected LOI results, with LOI at 550 °C being more susceptible to these factors than LOI at 950 °C. We, therefore, recommend analysts to be consistent in the LOI method used in relation to the ignition temperatures, exposure times, and the sample size and to include information on these three parameters when referring to the method.

Good reproducibility of heart rate variability after orthostatic challenge in patients with a history of acute coronary syndrome

BACKGROUND: Several parameters of heart rate variability (HRV) have been shown to predict the risk of sudden cardiac death (SCD) in cardiac patients. There is consensus that risk prediction is increased when measuring HRV during specific provocations such as orthostatic challenge. For the first time, we provide data on reproducibility of such a test in patients with a history of acute coronary syndrome. METHODS: Sixty male patients (65+/-8years) with a history of acute coronary syndrome on stable medication were included. HRV was measured in supine (5min) and standing (5min) position on 2 occasions separated by two weeks. For risk assessment relevant time-domain [standard deviation of all R-R intervals (SDNN) and root mean squared standard differences between adjacent R-R intervals (RMSSD)], frequency domain [low-frequency power (LF), high-frequency power (HF) and LF/HF power ratio] and short-term fractal scaling component (DF1) were computed. Absolute reproducibility was assessed with the standard errors of the mean (SEM) and 95% limits of random variation, and relative reproducibility by the intraclass correlation coefficient (ICC). RESULTS: We found comparable SEMs and ICCs in supine position and after an orthostatic challenge test. All ICCs were good to excellent (ICCs between 0.636 and 0.869). CONCLUSIONS: Reproducibility of HRV parameters during orthostatic challenge is good and comparable with supine position.

Evaluation of New Cone-beam Computed Tomographic Criteria for Radiographic Healing Evaluation after Apical Surgery: Assessment of Repeatability and Reproducibility.

INTRODUCTION Conventional 2-dimensional radiography uses defined criteria for outcome assessment of apical surgery. However, these radiographic healing criteria are not applicable for 3-dimensional radiography. The present study evaluated the repeatability and reproducibility of new cone-beam computed tomographic (CBCT)-based healing criteria for the judgment of periapical healing 1 year after apical surgery. METHODS CBCT scans taken 1 year after apical surgery (61 roots of 54 teeth in 54 patients, mean age = 54.4 years) were evaluated by 3 blinded and calibrated observers using 4 different indices. Reformatted buccolingual CBCT sections through the longitudinal axis of the treated roots were analyzed. Radiographic healing was assessed at the resection plane (R index), within the apical area (A index), of the cortical plate (C index), and regarding a combined apical-cortical area (B index). All readings were performed twice to calculate the intraobserver agreement (repeatability). Second-time readings were used for analyzing the interobserver agreement (reproducibility). Various statistical tests (Cohen, kappa, Fisher, and Spearman) were performed to measure the intra- and interobserver concurrence, the variability of score ratios, and the correlation of indices. RESULTS For all indices, the rates of identical first- and second-time scores were always higher than 80% (intraobserver Cohen κ values ranging from 0.793 to 0.963). The B index (94.0%) showed the highest intraobserver agreement. Regarding interobserver agreement, the highest rate was found for the B index (72.1%). The Fleiss' κ values for R and B indices exhibited substantial agreement (0.626 and 0.717, respectively), whereas the values for A and C indices showed moderate agreement (0.561 and 0.573, respectively). The Spearman correlation coefficients for R, A, C, and B indices all exhibited a moderate to very strong correlation with the highest correlation found between C and B indices (rs = 0.8069). CONCLUSIONS All indices showed an excellent intraobserver agreement (repeatability). With regard to interobserver agreement (reproducibility), the B index (healing of apical and cortical defects combined) and the R index (healing on the resection plane) showed substantial congruence and thus are to be recommended in future studies when using buccolingual CBCT sections for radiographic outcome assessment of apical surgery.

Quantifying Reproducibility in Computational Biology: The Case of the Tuberculosis Drugome

How easy is it to reproduce the results found in a typical computational biology paper? Either through experience or intuition the reader will already know that the answer is with difficulty or not at all. In this paper we attempt to quantify this difficulty by reproducing a previously published paper for different classes of users (ranging from users with little expertise to domain experts) and suggest ways in which the situation might be improved. Quantification is achieved by estimating the time required to reproduce each of the steps in the method described in the original paper and make them part of an explicit workflow that reproduces the original results. Reproducing the method took several months of effort, and required using new versions and new software that posed challenges to reconstructing and validating the results. The quantification leads to “reproducibility maps” that reveal that novice researchers would only be able to reproduce a few of the steps in the method, and that only expert researchers with advance knowledge of the domain would be able to reproduce the method in its entirety. The workflow itself is published as an online resource together with supporting software and data. The paper concludes with a brief discussion of the complexities of requiring reproducibility in terms of cost versus benefit, and a desiderata with our observations and guidelines for improving reproducibility. This has implications not only in reproducing the work of others from published papers, but reproducing work from one’s own laboratory.

Leveraging Semantics to Improve Reproducibility in Scientific Workflows

Reproducibility of published results is a cornerstone in scientific publishing and progress. Therefore, the scientific community has been encouraging authors and editors to publish their contributions in a verifiable and understandable way. Efforts such as the Reproducibility Initiative [1], or the Reproducibility Projects on Biology [2] and Psychology [3] domains, have been defining standards and patterns to assess whether an experimental result is reproducible.

A Semantic-Based Approach to Attain Reproducibility of Computational Environments in Scientic Work ows: A Case Study

Reproducible research in scientic work ows is often addressed by tracking the provenance of the produced results. While this approach allows inspecting intermediate and nal results, improves understanding, and permits replaying a work ow execution, it does not ensure that the computational environment is available for subsequent executions to reproduce the experiment. In this work, we propose describing the resources involved in the execution of an experiment using a set of semantic vocabularies, so as to conserve the computational environment. We dene a process for documenting the work ow application, management system, and their dependencies based on 4 domain ontologies. We then conduct an experimental evaluation sing a real work ow application on an academic and a public Cloud platform. Results show that our approach can reproduce an equivalent execution environment of a predened virtual machine image on both computing platforms.