Effect of Population Heterogenization on the Reproducibility of Mouse Behavior: A Multi-Laboratory Study

In animal experiments, animals, husbandry and test procedures are traditionally standardized to maximize test sensitivity and minimize animal use, assuming that this will also guarantee reproducibility. However, by reducing within-experiment variation, standardization may limit inference to the specific experimental conditions. Indeed, we have recently shown in mice that standardization may generate spurious results in behavioral tests, accounting for poor reproducibility, and that this can be avoided by population heterogenization through systematic variation of experimental conditions. Here, we examined whether a simple form of heterogenization effectively improves reproducibility of test results in a multi-laboratory situation. Each of six laboratories independently ordered 64 female mice of two inbred strains (C57BL/6NCrl, DBA/2NCrl) and examined them for strain differences in five commonly used behavioral tests under two different experimental designs. In the standardized design, experimental conditions were standardized as much as possible in each laboratory, while they were systematically varied with respect to the animals' test age and cage enrichment in the heterogenized design. Although heterogenization tended to improve reproducibility by increasing within-experiment variation relative to between-experiment variation, the effect was too weak to account for the large variation between laboratories. However, our findings confirm the potential of systematic heterogenization for improving reproducibility of animal experiments and highlight the need for effective and practicable heterogenization strategies.

An inter-laboratory comparative study of serological tools employed in the diagnosis of Besnoitia besnoiti infection in bovines

Bovine besnoitiosis is considered an emerging chronic and debilitating disease in Europe. Many infections remain subclinical, and the only sign of disease is the presence of parasitic cysts in the sclera and conjunctiva. Serological tests are useful for detecting asymptomatic cattle/sub-clinical infections for control purposes, as there are no effective drugs or vaccines. For this purpose, diagnostic tools need to be further standardized. Thus, the aim of this study was to compare the serological tests available in Europe in a multi-centred study. A coded panel of 241 well-characterized sera from infected and non-infected bovines was provided by all participants (SALUVET-Madrid, FLI-Wusterhausen, ENV-Toulouse, IPB-Berne). The tests evaluated were as follows: an in-house ELISA, three commercial ELISAs (INGEZIM BES 12.BES.K1 INGENASA, PrioCHECK Besnoitia Ab V2.0, ID Screen Besnoitia indirect IDVET), two IFATs and seven Western blot tests (tachyzoite and bradyzoite extracts under reducing and non-reducing conditions). Two different definitions of a gold standard were used: (i) the result of the majority of tests ('Majority of tests') and (ii) the majority of test results plus pre-test information based on clinical signs ('Majority of tests plus pre-test info'). Relative to the gold standard 'Majority of tests', almost 100% sensitivity (Se) and specificity (Sp) were obtained with SALUVET-Madrid and FLI-Wusterhausen tachyzoite- and bradyzoite-based Western blot tests under non-reducing conditions. On the ELISAs, PrioCHECK Besnoitia Ab V2.0 showed 100% Se and 98.8% Sp, whereas ID Screen Besnoitia indirect IDVET showed 97.2% Se and 100% Sp. The in-house ELISA and INGEZIM BES 12.BES.K1 INGENASA showed 97.3% and 97.2% Se; and 94.6% and 93.0% Sp, respectively. IFAT FLI-Wusterhausen performed better than IFAT SALUVET-Madrid, with 100% Se and 95.4% Sp. Relative to the gold standard 'Majority of test plus pre-test info', Sp significantly decreased; this result was expected because of the existence of seronegative animals with clinical signs. All ELISAs performed very well and could be used in epidemiological studies; however, Western blot tests performed better and could be employed as a posteriori tests for control purposes in the case of uncertain results from valuable samples.

Reproducibility of retinal thickness measurements in patients with age-related macular degeneration using 3D Fourier-domain optical coherence tomography (OCT) (Topcon 3D-OCT 1000)

Conventional time-domain optical coherence tomography (OCT) has become an important tool for following dry or exudative age-related macular degeneration (AMD). Fourier-domain three-dimensional (3D) OCT was recently introduced. This study tested the reproducibility of 3D-OCT retinal thickness measurements in patients with dry and exudative AMD.

[Microbiological point of care tests]

It is well known that the early initiation of a specific antiinfective therapy is crucial to reduce the mortality in severe infection. Procedures culturing pathogens are the diagnostic gold standard in such diseases. However, these methods yield results earliest between 24 to 48 hours. Therefore, severe infections such as sepsis need to be treated with an empirical antimicrobial therapy, which is ineffective in an unknown fraction of these patients. Today's microbiological point of care tests are pathogen specific and therefore not appropriate for an infection with a variety of possible pathogens. Molecular nucleic acid diagnostics such as polymerase chain reaction (PCR) allow the identification of pathogens and resistances. These methods are used routinely to speed up the analysis of positive blood cultures. The newest PCR based system allows the identification of the 25 most frequent sepsis pathogens by PCR in parallel without previous culture in less than 6 hours. Thereby, these systems might shorten the time of possibly insufficient antiinfective therapy. However, these extensive tools are not suitable as point of care diagnostics. Miniaturization and automating of the nucleic acid based method is pending, as well as an increase of detectable pathogens and resistance genes by these methods. It is assumed that molecular PCR techniques will have an increasing impact on microbiological diagnostics in the future.

Accuracy of diagnostic tests for clinically suspected upper extremity deep vein thrombosis: a systematic review

The best available test for the diagnosis of upper extremity deep venous thrombosis (UEDVT) is contrast venography. The aim of this systematic review was to assess whether the diagnostic accuracy of other tests for clinically suspected UEDVT is high enough to justify their use in clinical practise and to evaluate if any test can replace venography.

Antibiotic use in adult outpatients in Switzerland in relation to regions, seasonality and point of care tests

The use of antibiotics is highest in primary care and directly associated with antibiotic resistance in the community. We assessed regional variations in antibiotic use in primary care in Switzerland and explored prescription patterns in relation to the use of point of care tests. Defined daily doses of antibiotics per 1000 inhabitants (DDD(1000pd) ) were calculated for the year 2007 from reimbursement data of the largest Swiss health insurer, based on the anatomic therapeutic chemical classification and the DDD methodology recommended by WHO. We present ecological associations by use of descriptive and regression analysis. We analysed data from 1 067 934 adults, representing 17.1% of the Swiss population. The rate of outpatient antibiotic prescriptions in the entire population was 8.5 DDD(1000pd) , and varied between 7.28 and 11.33 DDD(1000pd) for northwest Switzerland and the Lake Geneva region. DDD(1000pd) for the three most prescribed antibiotics were 2.90 for amoxicillin and amoxicillin-clavulanate, 1.77 for fluoroquinolones, and 1.34 for macrolides. Regions with higher DDD(1000pd) showed higher seasonal variability in antibiotic use and lower use of all point of care tests. In regression analysis for each class of antibiotics, the use of any point of care test was consistently associated with fewer antibiotic prescriptions. Prescription rates of primary care physicians showed variations between Swiss regions and were lower in northwest Switzerland and in physicians using point of care tests. Ecological studies are prone to bias and whether point of care tests reduce antibiotic use has to be investigated in pragmatic primary care trials.

Reproducibility of sensory nerve conduction studies of the sural nerve using ultrasound-guided needle positioning

In this study we sought to evaluate the reproducibility of sensory nerve conduction studies (NCS) using ultrasound-guided needle positioning (USNP).

Reproducibility of acute mountain sickness in children and adults: a prospective study

Although a history of previous acute mountain sickness (AMS) is commonly used for providing advice and recommending its prophylaxis during subsequent exposure, the intraindividual reproducibility of AMS during repeated high-altitude exposure has never been examined in a prospective controlled study.

Diagnostic reproducibility of tumour budding in colorectal cancer: a multicentre, multinational study using virtual microscopy

Puppa G, Senore C, Sheahan K, Vieth M, Lugli A, Zlobec I, Pecori S, Wang L M, Langner C, Mitomi H, Nakamura T, Watanabe M, Ueno H, Chasle J, Conley S A, Herlin P, Lauwers G Y & Risio M (2012) Histopathology Diagnostic reproducibility of tumour budding in colorectal cancer: a multicentre, multinational study using virtual microscopy Aims:  Despite the established prognostic relevance of tumour budding in colorectal cancer, the reproducibility of the methods reported for its assessment has not yet been determined, limiting its use and reporting in routine pathology practice. Methods and results:  A morphometric system within telepathology was devised to evaluate the reproducibility of the various methods published for the assessment of tumour budding in colorectal cancer. Five methods were selected to evaluate the diagnostic reproducibility among 10 investigators, using haematoxylin and eosin (H&E) and AE1-3 cytokeratin-immunostained, whole-slide digital scans from 50 pT1-pT4 colorectal cancers. The overall interobserver agreement was fair for all methods, and increased to moderate for pT1 cancers. The intraobserver agreement was also fair for all methods and moderate for pT1 cancers. Agreement was dependent on the participants' experience with tumour budding reporting and performance time. Cytokeratin immunohistochemistry detected a higher percentage of tumour budding-positive cases with all methods compared to H&E-stained slides, but did not influence agreement levels. Conclusions:  An overall fair level of diagnostic agreement for tumour budding in colorectal cancer was demonstrated, which was significantly higher in early cancer and among experienced gastrointestinal pathologists. Cytokeratin immunostaining facilitated detection of budding cancer cells, but did not result in improved interobserver agreement.

Bivariate meta-analysis of predictive values of diagnostic tests can be an alternative to bivariate meta-analysis of sensitivity and specificity

Meta-analysis of predictive values is usually discouraged because these values are directly affected by disease prevalence, but sensitivity and specificity sometimes show substantial heterogeneity as well. We propose a bivariate random-effects logitnormal model for the meta-analysis of the positive predictive value (PPV) and negative predictive value (NPV) of diagnostic tests.

Intrapatient reproducibility of the BISxp monitor

BACKGROUND: The Bispectral Index (BIS) reportedly reflects anesthetic depth. It is recommended that anesthetic agents should be titrated to maintain the BIS between 40 and 60 arbitrary BIS units during anesthesia. For anesthesia providers to follow this recommendation, the monitor should be predictably affected by different anesthetic agents and have good interpatient and intrapatient reproducibility. The authors hypothesized that when two BISxp devices (Aspect Medical Systems, Newton, MA) are placed concurrently on the same patient, their readings are concordant throughout the anesthetic period. METHODS: Simultaneous BIS recordings from two BISxp monitors were obtained during anesthesia at 5-s intervals from 12 participants. RESULTS: In total 22,860 concurrent paired BIS readings were obtained. For 10.7% of the time, there were sustained periods of 30 s or greater where the readings suggested a different depth of anesthesia. For 6% of the time, there were sustained periods of 30 s or greater where the readings differed by 10 or more arbitrary BIS units. The regression coefficient (R) for the two devices was 0.65 (range, 0.35-0.92). There was zero bias between the devices, and the 95% limits of agreement ranged between -18 and +17. CONCLUSION: A conflicting anesthetic management was suggested by the simultaneous BIS readings 10.7% of the time. These results suggest that BISxp does not always provide a reproducible single number. Anesthesia providers should not rely exclusively on the BIS reading when assessing depth of anesthesia.

Evaluation of reproducibility of protein identification results after multidimensional human serum protein separation

We developed a gel- and label-free proteomics platform for comparative studies of human serum. The method involves the depletion of the six most abundant proteins, protein fractionation by Off-Gel IEF and RP-HPLC, followed by tryptic digestion, LC-MS/MS, protein identification, and relative quantification using probabilistic peptide match score summation (PMSS). We evaluated performance and reproducibility of the complete platform and the individual dimensions, by using chromatograms of the RP-HPLC runs, PMSS based abundance scores and abundance distributions as objective endpoints. We were interested if a relationship exists between the quantity ratio and the PMSS score ratio. The complete analysis was performed four times with two sets of serum samples containing different concentrations of spiked bovine beta-lactoglobulin (0.1 and 0.3%, w/w). The two concentrations resulted in significantly differing PMSS scores when compared to the variability in PMSS scores of all other protein identifications. We identified 196 proteins, of which 116 were identified four times in corresponding fractions whereof 73 qualified for relative quantification. Finally, we characterized the PMSS based protein abundance distributions with respect to the two dimensions of fractionation and discussed some interesting patterns representing discrete isoforms. We conclude that combination of Off-Gel electrophoresis (OGE) and HPLC is a reproducible protein fractionation technique, that PMSS is applicable for relative quantification, that the number of quantifiable proteins is always smaller than the number of identified proteins and that reproducibility of protein identifications should supplement probabilistic acceptance criteria.

Reproducibility of dGEMRIC in assessment of hip joint cartilage: a prospective study

PURPOSE: To investigate the reproducibility of dGEMRIC in the assessment of cartilage health of the adult asymptomatic hip joint. MATERIALS AND METHODS: Fifteen asymptomatic volunteers (mean age, 26.3 years +/- 3.0) were preliminarily studied. Any volunteer that was incidentally diagnosed with damaged cartilage on MRI (n = 5) was excluded. Ten patients that had no evidence of prior cartilage damage (mean age, 26.2 years +/- 3.4) were evaluated further in this study. The reproducibility of dGEMRIC was assessed with two T1(Gd) exams performed 4 weeks apart in these volunteers. The protocol involved an initial standard MRI to confirm healthy cartilage, which was then followed by dGEMRIC. The second scan included only the repeat dGEMRIC. Region of interest (ROI) analyses for T1(Gd)-measurement was performed in seven radial reformats. Statistical analysis included the student's t-test and intra-class correlation (ICC) measurement to assess reproducibility. RESULTS: Overall 70 ROIs were studied. Mean cartilage T1(Gd) values at various loci ranged from 560.9 ms to 684.4 ms at the first set of readings and 551.5 ms to 662.2 ms in the second one. The mean difference per region of interest between the two T1(Gd)-measurements ranged from 21.4 ms (3.7%) to 45.0 ms (6.8%), which was not found to be statistically significant (P = 0.153). There was a high reproducibility detected (ICC range, 0.667-0.915). Intra- and Inter-observer analyses proved a high agreement for T1(Gd) assessment (0.973 and 0.932). CONCLUSION: We found dGEMRIC to be a reliable tool in the assessment of cartilage health status in adult hip joints.

The James Lind Library: explaining and illustrating the evolution of fair tests of medical treatments

The James Lind Library (www.jameslindlibrary.org) has been established to improve public and professional general knowledge about fair tests of treatments in healthcare and their history. Its foundation was laid ten years ago at the Royal College of Physicians of Edinburgh, and its administrative centre is in the College's Sibbald Library, one of the most important collections of historic medical manuscripts, papers and books in the world. The James Lind Library is a website that introduces visitors to the principles of fair tests of treatments, with a series of short, illustrated essays, which are currently available in English, Arabic, Chinese, French, Portuguese, Russian and Spanish. A 100-page book-- Testing Treatments--is now available free through the website, both in English and in Arabic and Spanish translations. To illustrate the evolution of ideas related to fair tests of treatments from 2000 BC to the present, the James Lind Library contains key passages and images from manuscripts, books and journal articles, many of them accompanied by commentaries, biographies, portraits and other relevant documents and images, including audio and video files. New material is being added to the website continuously, as relevant new records are identified and as methods for testing treatments evolve. A multinational, multilingual editorial team oversees the development of the website, which currently receives tens of thousands of visitors every month.