Effect of Population Heterogenization on the Reproducibility of Mouse Behavior: A Multi-Laboratory Study

In animal experiments, animals, husbandry and test procedures are traditionally standardized to maximize test sensitivity and minimize animal use, assuming that this will also guarantee reproducibility. However, by reducing within-experiment variation, standardization may limit inference to the specific experimental conditions. Indeed, we have recently shown in mice that standardization may generate spurious results in behavioral tests, accounting for poor reproducibility, and that this can be avoided by population heterogenization through systematic variation of experimental conditions. Here, we examined whether a simple form of heterogenization effectively improves reproducibility of test results in a multi-laboratory situation. Each of six laboratories independently ordered 64 female mice of two inbred strains (C57BL/6NCrl, DBA/2NCrl) and examined them for strain differences in five commonly used behavioral tests under two different experimental designs. In the standardized design, experimental conditions were standardized as much as possible in each laboratory, while they were systematically varied with respect to the animals' test age and cage enrichment in the heterogenized design. Although heterogenization tended to improve reproducibility by increasing within-experiment variation relative to between-experiment variation, the effect was too weak to account for the large variation between laboratories. However, our findings confirm the potential of systematic heterogenization for improving reproducibility of animal experiments and highlight the need for effective and practicable heterogenization strategies.

Diagnostic reproducibility of tumour budding in colorectal cancer: a multicentre, multinational study using virtual microscopy

Puppa G, Senore C, Sheahan K, Vieth M, Lugli A, Zlobec I, Pecori S, Wang L M, Langner C, Mitomi H, Nakamura T, Watanabe M, Ueno H, Chasle J, Conley S A, Herlin P, Lauwers G Y & Risio M (2012) Histopathology Diagnostic reproducibility of tumour budding in colorectal cancer: a multicentre, multinational study using virtual microscopy Aims:  Despite the established prognostic relevance of tumour budding in colorectal cancer, the reproducibility of the methods reported for its assessment has not yet been determined, limiting its use and reporting in routine pathology practice. Methods and results:  A morphometric system within telepathology was devised to evaluate the reproducibility of the various methods published for the assessment of tumour budding in colorectal cancer. Five methods were selected to evaluate the diagnostic reproducibility among 10 investigators, using haematoxylin and eosin (H&E) and AE1-3 cytokeratin-immunostained, whole-slide digital scans from 50 pT1-pT4 colorectal cancers. The overall interobserver agreement was fair for all methods, and increased to moderate for pT1 cancers. The intraobserver agreement was also fair for all methods and moderate for pT1 cancers. Agreement was dependent on the participants' experience with tumour budding reporting and performance time. Cytokeratin immunohistochemistry detected a higher percentage of tumour budding-positive cases with all methods compared to H&E-stained slides, but did not influence agreement levels. Conclusions:  An overall fair level of diagnostic agreement for tumour budding in colorectal cancer was demonstrated, which was significantly higher in early cancer and among experienced gastrointestinal pathologists. Cytokeratin immunostaining facilitated detection of budding cancer cells, but did not result in improved interobserver agreement.

Sensitivity Gains, Linearity, and Spectral Reproducibility in Nonuniformly Sampled Multidimensional MAS NMR Spectra of High Dynamic Range

Recently, we have demonstrated that considerable inherent sensitivity gains are attained in MAS NMR spectra acquired by nonuniform sampling (NUS) and introduced maximum entropy interpolation (MINT) processing that assures the linearity of transformation between the time and frequency domains. In this report, we examine the utility of the NUS/MINT approach in multidimensional datasets possessing high dynamic range, such as homonuclear C-13-C-13 correlation spectra. We demonstrate on model compounds and on 1-73-(U-C-13,N-15)/74-108-(U-N-15) E. coli thioredoxin reassembly, that with appropriately constructed 50 % NUS schedules inherent sensitivity gains of 1.7-2.1-fold are readily reached in such datasets. We show that both linearity and line width are retained under these experimental conditions throughout the entire dynamic range of the signals. Furthermore, we demonstrate that the reproducibility of the peak intensities is excellent in the NUS/MINT approach when experiments are repeated multiple times and identical experimental and processing conditions are employed. Finally, we discuss the principles for design and implementation of random exponentially biased NUS sampling schedules for homonuclear C-13-C-13 MAS correlation experiments that yield high-quality artifact-free datasets.

Non-invasive monitoring of renal oxygenation using BOLD-MRI: a reproducibility study

Blood oxygenation level-dependent (BOLD) MRI was shown to allow non-invasive observation of renal oxygenation in humans. However, clinical applications of this type of functional MRI of the kidney are still limited, most likely because of difficulties in obtaining reproducible and reliable information. The aim of this study was to evaluate the reproducibility and robustness of a BOLD method applied to the kidneys and to identify systematic physiological changes potentially influencing the renal oxygenation of healthy volunteers. To measure the BOLD effect, a modified multi-echo data image combination (MEDIC) sequence was used to acquire 12 T2*-weighted images within a single breath-hold. Three identical measurements were performed on three axial and three coronal slices of right and left kidneys in 18 volunteers. The mean R2* (1/T2*) values determined in medulla and cortex showed no significant differences over three repetitions and low intra-subject coefficients of variation (CV) (3 and 4% in medulla and cortex, respectively). The average R2* values were higher in the medulla (16.15 +/- 0.11) than in the cortex (11.69 +/- 0.18) (P < 0.001). Only a minor influence of slice orientation was observed. Mean R2* values were slightly higher (3%) in the left than in the right kidney (P < 0.001). Differences between volunteers were identified (P < 0.001). Part of these differences was attributable to age-dependent R2* values, since these values increased with age when medulla (P < 0.001, r = 0.67) or cortex (P < 0.020, r = 0.42) were considered. Thus, BOLD measurements in the kidney are highly reproducible and robust. The results allow one to identify the known cortico-medullary gradient of oxygenation evidenced by the gradient of R2* values and suggest that medulla is more hypoxic in older than younger individuals. BOLD-MRI is therefore a useful tool to study sequentially and non-invasively regional oxygenation of human kidneys.

Intrapatient reproducibility of the BISxp monitor

BACKGROUND: The Bispectral Index (BIS) reportedly reflects anesthetic depth. It is recommended that anesthetic agents should be titrated to maintain the BIS between 40 and 60 arbitrary BIS units during anesthesia. For anesthesia providers to follow this recommendation, the monitor should be predictably affected by different anesthetic agents and have good interpatient and intrapatient reproducibility. The authors hypothesized that when two BISxp devices (Aspect Medical Systems, Newton, MA) are placed concurrently on the same patient, their readings are concordant throughout the anesthetic period. METHODS: Simultaneous BIS recordings from two BISxp monitors were obtained during anesthesia at 5-s intervals from 12 participants. RESULTS: In total 22,860 concurrent paired BIS readings were obtained. For 10.7% of the time, there were sustained periods of 30 s or greater where the readings suggested a different depth of anesthesia. For 6% of the time, there were sustained periods of 30 s or greater where the readings differed by 10 or more arbitrary BIS units. The regression coefficient (R) for the two devices was 0.65 (range, 0.35-0.92). There was zero bias between the devices, and the 95% limits of agreement ranged between -18 and +17. CONCLUSION: A conflicting anesthetic management was suggested by the simultaneous BIS readings 10.7% of the time. These results suggest that BISxp does not always provide a reproducible single number. Anesthesia providers should not rely exclusively on the BIS reading when assessing depth of anesthesia.

Quantitative (1)H magnetic resonance spectroscopy of myoglobin de- and reoxygenation in skeletal muscle: reproducibility and effects of location and disease

1H-magnetic resonance spectroscopy ((1)H-MRS) of deoxymyoglobin (DMb) provides a means to noninvasively monitor the oxygenation state of human skeletal muscle in work and disease. As shown in this work, it also offers the opportunity to measure the absolute tissue content of DMb, the basic oxygen consumption of resting muscle, and the reperfusion characteristics after release of a pressure cuff. The methodology to determine these tissue properties simultaneously at two positions along the calf is presented. The obtained values are in agreement with invasive determinations. The reproducibility of the (1)H-MRS measurements is established for healthy controls and patients with peripheral arterial disease (PAD). A location dependence in axial direction, as well as differences between controls and patients are demonstrated for all parameters. The reoxygenation time in particular is expected to provide a means to quantitatively monitor therapies aimed at improving muscular perfusion in these patients.

High-resolution morphological and biochemical imaging of articular cartilage of the ankle joint at 3.0 T using a new dedicated phased array coil: in vivo reproducibility study

OBJECTIVE: The objective of this study was to evaluate the feasibility and reproducibility of high-resolution magnetic resonance imaging (MRI) and quantitative T2 mapping of the talocrural cartilage within a clinically applicable scan time using a new dedicated ankle coil and high-field MRI. MATERIALS AND METHODS: Ten healthy volunteers (mean age 32.4 years) underwent MRI of the ankle. As morphological sequences, proton density fat-suppressed turbo spin echo (PD-FS-TSE), as a reference, was compared with 3D true fast imaging with steady-state precession (TrueFISP). Furthermore, biochemical quantitative T2 imaging was prepared using a multi-echo spin-echo T2 approach. Data analysis was performed three times each by three different observers on sagittal slices, planned on the isotropic 3D-TrueFISP; as a morphological parameter, cartilage thickness was assessed and for T2 relaxation times, region-of-interest (ROI) evaluation was done. Reproducibility was determined as a coefficient of variation (CV) for each volunteer; averaged as root mean square (RMSA) given as a percentage; statistical evaluation was done using analysis of variance. RESULTS: Cartilage thickness of the talocrural joint showed significantly higher values for the 3D-TrueFISP (ranging from 1.07 to 1.14 mm) compared with the PD-FS-TSE (ranging from 0.74 to 0.99 mm); however, both morphological sequences showed comparable good results with RMSA of 7.1 to 8.5%. Regarding quantitative T2 mapping, measurements showed T2 relaxation times of about 54 ms with an excellent reproducibility (RMSA) ranging from 3.2 to 4.7%. CONCLUSION: In our study the assessment of cartilage thickness and T2 relaxation times could be performed with high reproducibility in a clinically realizable scan time, demonstrating new possibilities for further investigations into patient groups.

Kinematic biomechanical assessment of human articular cartilage transplants in the knee using 3-T MRI: an in vivo reproducibility study

The aims of this study were to examine the clinical feasibility and reproducibility of kinematic MR imaging with respect to changes in T (2) in the femoral condyle articular cartilage. We used a flexible knee coil, which allows acquisition of data in different positions from 40 degrees flexion to full extension during MR examinations. The reproducibility of T (2) measurements was evaluated for inter-rater and inter-individual variability and determined as a coefficient of variation (CV) for each volunteer and rater. Three different volunteers were measured twice and regions of interest (ROIs) were selected by three raters at different time points. To prove the clinical feasibility of this method, 20 subjects (10 patients and 10 age- and sex-matched volunteers) were enrolled in the study. Inter-rater variability ranged from 2 to 9 and from 2 to 10% in the deep and superficial zones, respectively. Mean inter-individual variability was 7% for both zones. Different T (2) values were observed in the superficial cartilage zone of patients compared with volunteers. Since repair tissue showed a different behavior in the contact zone compared with healthy cartilage, a possible marker for improved evaluation of repair tissue quality after matrix-associated autologous chondrocyte transplantation (MACT) may be available and may allow biomechanical assessment of cartilage transplants.

Evaluation of reproducibility of protein identification results after multidimensional human serum protein separation

We developed a gel- and label-free proteomics platform for comparative studies of human serum. The method involves the depletion of the six most abundant proteins, protein fractionation by Off-Gel IEF and RP-HPLC, followed by tryptic digestion, LC-MS/MS, protein identification, and relative quantification using probabilistic peptide match score summation (PMSS). We evaluated performance and reproducibility of the complete platform and the individual dimensions, by using chromatograms of the RP-HPLC runs, PMSS based abundance scores and abundance distributions as objective endpoints. We were interested if a relationship exists between the quantity ratio and the PMSS score ratio. The complete analysis was performed four times with two sets of serum samples containing different concentrations of spiked bovine beta-lactoglobulin (0.1 and 0.3%, w/w). The two concentrations resulted in significantly differing PMSS scores when compared to the variability in PMSS scores of all other protein identifications. We identified 196 proteins, of which 116 were identified four times in corresponding fractions whereof 73 qualified for relative quantification. Finally, we characterized the PMSS based protein abundance distributions with respect to the two dimensions of fractionation and discussed some interesting patterns representing discrete isoforms. We conclude that combination of Off-Gel electrophoresis (OGE) and HPLC is a reproducible protein fractionation technique, that PMSS is applicable for relative quantification, that the number of quantifiable proteins is always smaller than the number of identified proteins and that reproducibility of protein identifications should supplement probabilistic acceptance criteria.

Reproducibility of dGEMRIC in assessment of hip joint cartilage: a prospective study

PURPOSE: To investigate the reproducibility of dGEMRIC in the assessment of cartilage health of the adult asymptomatic hip joint. MATERIALS AND METHODS: Fifteen asymptomatic volunteers (mean age, 26.3 years +/- 3.0) were preliminarily studied. Any volunteer that was incidentally diagnosed with damaged cartilage on MRI (n = 5) was excluded. Ten patients that had no evidence of prior cartilage damage (mean age, 26.2 years +/- 3.4) were evaluated further in this study. The reproducibility of dGEMRIC was assessed with two T1(Gd) exams performed 4 weeks apart in these volunteers. The protocol involved an initial standard MRI to confirm healthy cartilage, which was then followed by dGEMRIC. The second scan included only the repeat dGEMRIC. Region of interest (ROI) analyses for T1(Gd)-measurement was performed in seven radial reformats. Statistical analysis included the student's t-test and intra-class correlation (ICC) measurement to assess reproducibility. RESULTS: Overall 70 ROIs were studied. Mean cartilage T1(Gd) values at various loci ranged from 560.9 ms to 684.4 ms at the first set of readings and 551.5 ms to 662.2 ms in the second one. The mean difference per region of interest between the two T1(Gd)-measurements ranged from 21.4 ms (3.7%) to 45.0 ms (6.8%), which was not found to be statistically significant (P = 0.153). There was a high reproducibility detected (ICC range, 0.667-0.915). Intra- and Inter-observer analyses proved a high agreement for T1(Gd) assessment (0.973 and 0.932). CONCLUSION: We found dGEMRIC to be a reliable tool in the assessment of cartilage health status in adult hip joints.

Reproducibility and accuracy of the ICDAS-II for occlusal caries detection

OBJECTIVES: The aim of this in vitro study was to assess the inter- and intra-examiner reproducibility and the accuracy of the International Caries Detection and Assessment System-II (ICDAS-II) in detecting occlusal caries. METHODS: One hundred and sixty-three molars were independently assessed twice by two experienced dentists using the 0- to 6-graded ICDAS-II. The teeth were histologically prepared and classified using two different histological systems [Ekstrand et al. (1997) Caries Research vol. 31, pp. 224-231; Lussi et al. (1999) Caries Research vol. 33, pp. 261-266] and assessed for caries extension. Sensitivity, specificity, accuracy and area under the ROC curve (A(z)) were obtained at D(2) and D(3) thresholds. Unweighted kappa coefficient was used to assess inter- and intra-examiner reproducibility. RESULTS: For the Ekstrand et al. histological classification the sensitivity was 0.99 and 1.00, specificity 1.00 and 0.69 and accuracy 0.99 and 0.76 at D(2) and D(3), respectively. For the Lussi et al. histological classification the sensitivity was 0.91 and 0.75, specificity 0.47 and 0.62 and accuracy 0.86 and 0.68 at D(2) and D(3), respectively. The A(z) varied from 0.54 to 0.73. The inter- and intra-examiner kappa values were 0.51 and 0.58, respectively. CONCLUSIONS: ICDAS-II presented good reproducibility and accuracy in detecting occlusal caries, especially caries lesions in the outer half of the enamel.