Assessment of the carotid artery by MRI at 3T: a study on reproducibility

PURPOSE: To examine the reproducibility of carotid artery dimension measurements using 3T MRI. MATERIALS AND METHODS: Ten healthy volunteers underwent three scans on two occasions for assessment of total vessel wall area (TVWA), total luminal area (TLA), and minimum (MinT) and maximum (MaxT) vessel wall thickness. A double inversion-recovery (IR) fast gradient-echo (FGRE) sequence was used on a commercial 3T system. During the first visit the subjects were scanned twice. The third scan was performed at least four days later. One observer traced all scans, and a second observer retraced the first scan series. RESULTS: For TVWA an interclass correlation (ICC) of 0.994 was calculated with all three scans taken into account. The interobserver ICC was 0.984. The agreement between the scans for TLA showed an ICC of 0.982 with an interobserver ICC of 0.998. For MinT and MaxT an ICC of 0.843 and 0.935 were calculated, with interobserver ICCs of 0.860 and 0.726, respectively. CONCLUSION: With the use of a commercial 3T MR system, TVWA, TLA, and wall thickness measurements of the carotid artery can be assessed with good reproducibility.

The QUASAR reproducibility study, Part II: Results from a multi-center Arterial Spin Labeling test-retest study.

Arterial Spin Labeling (ASL) is a method to measure perfusion using magnetically labeled blood water as an endogenous tracer. Being fully non-invasive, this technique is attractive for longitudinal studies of cerebral blood flow in healthy and diseased individuals, or as a surrogate marker of metabolism. So far, ASL has been restricted mostly to specialist centers due to a generally low SNR of the method and potential issues with user-dependent analysis needed to obtain quantitative measurement of cerebral blood flow (CBF). Here, we evaluated a particular implementation of ASL (called Quantitative STAR labeling of Arterial Regions or QUASAR), a method providing user independent quantification of CBF in a large test-retest study across sites from around the world, dubbed "The QUASAR reproducibility study". Altogether, 28 sites located in Asia, Europe and North America participated and a total of 284 healthy volunteers were scanned. Minimal operator dependence was assured by using an automatic planning tool and its accuracy and potential usefulness in multi-center trials was evaluated as well. Accurate repositioning between sessions was achieved with the automatic planning tool showing mean displacements of 1.87+/-0.95 mm and rotations of 1.56+/-0.66 degrees . Mean gray matter CBF was 47.4+/-7.5 [ml/100 g/min] with a between-subject standard variation SD(b)=5.5 [ml/100 g/min] and a within-subject standard deviation SD(w)=4.7 [ml/100 g/min]. The corresponding repeatability was 13.0 [ml/100 g/min] and was found to be within the range of previous studies.

A new technique with high reproducibility to estimate renal oxygenation using BOLD-MRI in chronic kidney disease.

OBJECTIVES: To assess inter-observer variability of renal blood oxygenation level-dependent MRI (BOLD-MRI) using a new method of analysis, called the concentric objects (CO) technique, in comparison with the classical ROI (region of interest)-based technique. METHODS: MR imaging (3T) was performed before and after furosemide in 10 chronic kidney disease (CKD) patients (mean eGFR 43±24ml/min/1.73m(2)) and 10 healthy volunteers (eGFR 101±28ml/min1.73m(2)), and R2* maps were determined on four coronal slices. In the CO-technique, R2* values were based on a semi-automatic procedure that divided each kidney in six equal layers, whereas in the ROI-technique, all circles (ROIs) were placed manually in the cortex and medulla. The mean R2*values as assessed by two independent investigators were compared. RESULTS: With the CO-technique, inter-observer variability was 0.7%-1.9% across all layers in non-CKD, versus 1.6%-3.8% in CKD. With the ROI-technique, median variability for cortical and medullary R2* values was 3.6 and 6.8% in non-CKD, versus 4.7 and 12.5% in CKD; similar results were observed after furosemide. CONCLUSION: The CO-technique offers a new, investigator-independent, highly reproducible alternative to the ROI-based technique to estimate renal tissue oxygenation in CKD.

Comparison of central macular thickness (CMT) measurement between time domain and spectral domain (SD) optical coherence tomography (OCT) and reproducibility of SD OCT in active neovascular AMD

Purpose: To evaluate the reproducibility of Cirrus-SD OCT measurements and to compare central macular thickness (CMT) measurements between TD-Stratus and SD-Cirrus OCT in patients with active exudative AMD. Methods: Consecutive case series of patients with active exudative AMD seen in the Medical Retina Department. Patients underwent 1 scan with Stratus (macular thickness map protocol) and 5 scans with Cirrus (Macular Cube protocol) at the same visit by the same experienced examiner. To be included, patients best-corrected visual acuity (BCVA) had to be >20/200 while all scans had to be of sufficient quality, well-centered and at least one Cirrus scan with CMT >300 microns. The repeatability of the SD Cirrus was estimated by using all 5 CMT measurements and the mean of the Cirrus measurements was compared with the CMT obtained by TD Stratus. Results: Cirrus OCT demonstrated high intraobserver repeatability at the central foveal region (ICC 96%). The mean of the CMT measurements was 321microns for Stratus and 387 microns for Cirrus. The average difference was 65m (SD=30). The coefficient of concordance between Stratus and Cirrus CMT measurements was rho=0,749 with a high precision and a moderate accuracy. The equation of the line of regression between Stratus and meanCirrus is given by the following: M_stratus = 0,848 x m_cirrus - 4,496 (1).Conclusions: The Cirrus macular cube protocol allows reproducible CMT measurements in patients with active exudative AMD. In cases of upgrading from TD to SD use and vice versa, there is the possibility to predict the measurements by using the equation (1). These real life data and conclusions can help in improving our clinical management of patients with neovascular AMD.

Reproducibility of diabetes quality of care indicators as reported by patients and physicians.

INTRODUCTION: Self-report of diabetes care has moderate validity and is prone to under- and over-reporting. We assessed reproducibility of a range of processes and outcomes of diabetes care as reported by patients and physicians. METHODS: In a Swiss community-based survey, patients with diabetes and physicians independently reported past 12 months processes of care (HbA1c, lipids, microalbuminuria, blood pressure, weight, foot and eye examinations) and last measured values of HbA1c, height, weight and blood pressure. For dichotomous variables, we assessed reliability by Cohen's kappa and agreement by uniform kappa. For continuous measures, we used Lin's concordance correlation coefficient and limits of agreement, respectively. RESULTS: Mean age of the 210 patients was 65 years; 40% were women, and 51% had diabetes for >10 years. Agreement was good for recommended processes of care such as blood pressure (uniform kappa = 0.94), HbA1c (0.93), weight (0.88) and lipid (0.78), but lower for microalbuminuria, foot and eye examinations (all <0.50). Cohen's kappa values were all low (<0.25). Comparisons of reported continuous variables showed large limits of agreement for height (±6 cm) and weight (8-10 kg) despite high concordance correlation coefficients (0.93 and 0.97). Concordance correlation coefficients were smaller for HbA1c (0.72) and blood pressure (0.5-0.6), with large limits of agreement (±2% and ±25 mmHg). CONCLUSION: While agreement of routine processes of care was good, agreement was less satisfactory for microalbuminuria, foot and eye examinations. Reports of continuous outcomes yielded good reliability but too wide limits of agreement. Quality of care evaluation relying on self-report only should be made cautiously.

Reproducibility of Vertebral Fracture Assessment Readings From Dual-energy X-ray Absorptiometry in Both a Population-based and Clinical Cohort: Cohen's and Uniform Kappa.

Vertebral fracture assessments (VFAs) using dual-energy X-ray absorptiometry increase vertebral fracture detection in clinical practice and are highly reproducible. Measures of reproducibility are dependent on the frequency and distribution of the event. The aim of this study was to compare 2 reproducibility measures, reliability and agreement, in VFA readings in both a population-based and a clinical cohort. We measured agreement and reliability by uniform kappa and Cohen's kappa for vertebral reading and fracture identification: 360 VFAs from a population-based cohort and 85 from a clinical cohort. In the population-based cohort, 12% of vertebrae were unreadable. Vertebral fracture prevalence ranged from 3% to 4%. Inter-reader and intrareader reliability with Cohen's kappa was fair to good (0.35-0.71 and 0.36-0.74, respectively), with good inter-reader and intrareader agreement by uniform kappa (0.74-0.98 and 0.76-0.99, respectively). In the clinical cohort, 15% of vertebrae were unreadable, and vertebral fracture prevalence ranged from 7.6% to 8.1%. Inter-reader reliability was moderate to good (0.43-0.71), and the agreement was good (0.68-0.91). In clinical situations, the levels of reproducibility measured by the 2 kappa statistics are concordant, so that either could be used to measure agreement and reliability. However, if events are rare, as in a population-based cohort, we recommend evaluating reproducibility using the uniform kappa, as Cohen's kappa may be less accurate.

Polygenic determinants of white matter volume derived from GWAS lack reproducibility in a replicate sample

A recent publication reported an exciting polygenic effect of schizophrenia (SCZ) risk variants, identified by a large genome-wide association study (GWAS), on total brain and white matter volumes in schizophrenic patients and, even more prominently, in healthy subjects. The aim of the present work was to replicate and then potentially extend these findings. According to the original publication, polygenic risk scores using single nucleotide polymorphism (SNP) information of SCZ GWAS (polygenic SCZ risk scores; PSS) were calculated in 122 healthy subjects, enrolled in a structural magnetic resonance imaging (MRI) study. These scores were computed based on P-values and odds ratios available through the Psychiatric GWAS Consortium. In addition, polygenic white matter scores (PWM) were calculated, using the respective SNP subset in the original publication. None of the polygenic scores, either PSS or PWM, were found to be associated with total brain, white matter or gray matter volume in our replicate sample. Minor differences between the original and the present study that might have contributed to lack of reproducibility (but unlikely explain it fully), are number of subjects, ethnicity, age distribution, array technology, SNP imputation quality and MRI scanner type. In contrast to the original publication, our results do not reveal the slightest signal of association of the described sets of GWAS-identified SCZ risk variants with brain volumes in adults. Caution is indicated in interpreting studies building on polygenic risk scores without replication sample.

Moderate-term reproducibility of heart rate variability during rest and light to moderate exercise in children

Previous studies have demonstrated the high reproducibility of heart rate variability (HRV) measures in adults while little information exists concerning HRV reproducibility in children. Subsequently, the aim of the current study was to examine the moderate-term reproducibility of heart rate and frequency domain measures of HRV during rest and light to moderate exercise in children. Ten healthy children (6 males, 4 females) aged between 7 and 12 years of age volunteered for this study with HRV recordings obtained during supine rest and three treadmill walking exercise work rates (≤60% maximum heart rate), initially and then 8 weeks later. Differences (P < 0.05) between variables were examined using paired t-tests or Wilcoxon signed rank tests while reliability and reproducibility were examined by intraclass correlation coefficients (ICC), coefficients of variation (CV), and mean bias ratio and ratio limits of agreement (LOA). Heart rate and all measures of HRV at rest and exercise were unchanged after 8 weeks. Significant ICC were documented primarily during rest (0.72-0.85) while weaker relationships (-0.02-0.87) were evident during exercise. A large range of CV was identified during rest (6-33%) and exercise (3-128%) while the ratio LOA were variable and substantial (1.04-2.73). Despite similar HRV over an 8-week period, variable ICC and sizable CV and ratio LOA indicate moderate to poor reproducibility of HRV in children, particularly during light to moderate exercise. Studies examining HRV in children should include age- or maturation stage-matched control participants to address the age-related change in HRV and inadequate HRV reliability.

Reproducibility of heart rate variability parameters measured in healthy subjects at rest and after a postural change maneuver

Heart rate variability (HRV) provides important information about cardiac autonomic modulation. Since it is a noninvasive and inexpensive method, HRV has been used to evaluate several parameters of cardiovascular health. However, the internal reproducibility of this method has been challenged in some studies. Our aim was to determine the intra-individual reproducibility of HRV parameters in short-term recordings obtained in supine and orthostatic positions. Electrocardiographic (ECG) recordings were obtained from 30 healthy subjects (20-49 years, 14 men) using a digital apparatus (sampling ratio = 250 Hz). ECG was recorded for 10 min in the supine position and for 10 min in the orthostatic position. The procedure was repeated 2-3 h later. Time and frequency domain analyses were performed. Frequency domain included low (LF, 0.04-0.15 Hz) and high frequency (HF, 0.15-0.4 Hz) bands. Power spectral analysis was performed by the autoregressive method and model order was set at 16. Intra-subject agreement was assessed by linear regression analysis, test of difference in variances and limits of agreement. Most HRV measures (pNN50, RMSSD, LF, HF, and LF/HF ratio) were reproducible independent of body position. Better correlation indexes (r > 0.6) were obtained in the orthostatic position. Bland-Altman plots revealed that most values were inside the agreement limits, indicating concordance between measures. Only SDNN and NNv in the supine position were not reproducible. Our results showed reproducibility of HRV parameters when recorded in the same individual with a short time between two exams. The increased sympathetic activity occurring in the orthostatic position probably facilitates reproducibility of the HRV indexes.

Assessing the Reproducibility of Clustering of Molecular Dynamics Conformations on Self-Organizing Maps

The goal of most clustering algorithms is to find the optimal number of clusters (i.e. fewest number of clusters). However, analysis of molecular conformations of biological macromolecules obtained from computer simulations may benefit from a larger array of clusters. The Self-Organizing Map (SOM) clustering method has the advantage of generating large numbers of clusters, but often gives ambiguous results. In this work, SOMs have been shown to be reproducible when the same conformational dataset is independently clustered multiple times (~100), with the help of the Cramérs V-index (C_v). The ability of C_v to determine which SOMs are reproduced is generalizable across different SOM source codes. The conformational ensembles produced from MD (molecular dynamics) and REMD (replica exchange molecular dynamics) simulations of the penta peptide Met-enkephalin (MET) and the 34 amino acid protein human Parathyroid Hormone (hPTH) were used to evaluate SOM reproducibility. The training length for the SOM has a huge impact on the reproducibility. Analysis of MET conformational data definitively determined that toroidal SOMs cluster data better than bordered maps due to the fact that toroidal maps do not have an edge effect. For the source code from MATLAB, it was determined that the learning rate function should be LINEAR with an initial learning rate factor of 0.05 and the SOM should be trained by a sequential algorithm. The trained SOMs can be used as a supervised classification for another dataset. The toroidal 10×10 hexagonal SOMs produced from the MATLAB program for hPTH conformational data produced three sets of reproducible clusters (27%, 15%, and 13% of 100 independent runs) which find similar partitionings to those of smaller 6×6 SOMs. The χ^2 values produced as part of the C_v calculation were used to locate clusters with identical conformational memberships on independently trained SOMs, even those with different dimensions. The χ^2 values could relate the different SOM partitionings to each other.

Reproducibility of a short semi-quantitative food group questionnaire and its performance in estimating nutrient intake compared with a 7-day diet diary in the Million Women Study

Objectives: To assess the short- and long-term reproducibility of a short food group questionnaire, and to compare its performance for estimating nutrient intakes in comparison with a 7-day diet diary. Design: Participants for the reproducibility study completed the food group questionnaire at two time points, up to 2 years apart. Participants for the performance study completed both the food group questionnaire and a 7-day diet diary a few months apart. Reproducibility was assessed by kappa statistics and percentage change between the two questionnaires; performance was assessed by kappa statistics, rank correlations and percentages of participants classified into the same and opposite thirds of intake. Setting: A random sample of participants in the Million Women Study, a population-based prospective study in the UK. Subjects: In total, 12 221 women aged 50-64 years. Results: in the reproducibility study, 75% of the food group items showed at least moderate agreement for all four time-point comparisons. Items showing fair agreement or worse tended to be those where few respondents reported eating them more than once a week, those consumed in small amounts and those relating to types of fat consumed. Compared with the diet diary, the food group questionnaire showed consistently reasonable performance for the nutrients carbohydrate, saturated fat, cholesterol, total sugars, alcohol, fibre, calcium, riboflavin, folate and vitamin C. Conclusions: The short food group questionnaire used in this study has been shown to be reproducible over time and to perform reasonably well for the assessment of a number of dietary nutrients.

Error, reproducibility and sensitivity: a pipeline for data processing of Agilent oligonucleotide expression arrays

Background: Expression microarrays are increasingly used to obtain large scale transcriptomic information on a wide range of biological samples. Nevertheless, there is still much debate on the best ways to process data, to design experiments and analyse the output. Furthermore, many of the more sophisticated mathematical approaches to data analysis in the literature remain inaccessible to much of the biological research community. In this study we examine ways of extracting and analysing a large data set obtained using the Agilent long oligonucleotide transcriptomics platform, applied to a set of human macrophage and dendritic cell samples. Results: We describe and validate a series of data extraction, transformation and normalisation steps which are implemented via a new R function. Analysis of replicate normalised reference data demonstrate that intrarray variability is small (only around 2 of the mean log signal), while interarray variability from replicate array measurements has a standard deviation (SD) of around 0.5 log(2) units (6 of mean). The common practise of working with ratios of Cy5/Cy3 signal offers little further improvement in terms of reducing error. Comparison to expression data obtained using Arabidopsis samples demonstrates that the large number of genes in each sample showing a low level of transcription reflect the real complexity of the cellular transcriptome. Multidimensional scaling is used to show that the processed data identifies an underlying structure which reflect some of the key biological variables which define the data set. This structure is robust, allowing reliable comparison of samples collected over a number of years and collected by a variety of operators. Conclusions: This study outlines a robust and easily implemented pipeline for extracting, transforming normalising and visualising transcriptomic array data from Agilent expression platform. The analysis is used to obtain quantitative estimates of the SD arising from experimental (non biological) intra- and interarray variability, and for a lower threshold for determining whether an individual gene is expressed. The study provides a reliable basis for further more extensive studies of the systems biology of eukaryotic cells.

Online dietary intake estimation : reproducibility and validity of the Food4Me food frequency questionnaire against a 4-day weighed food record

Background: Advances in nutritional assessment are continuing to embrace developments in computer technology. The online Food4Me food frequency questionnaire (FFQ) was created as an electronic system for the collection of nutrient intake data. To ensure its accuracy in assessing both nutrient and food group intake, further validation against data obtained using a reliable, but independent, instrument and assessment of its reproducibility are required. Objective: The aim was to assess the reproducibility and validity of the Food4Me FFQ against a 4-day weighed food record (WFR). Methods: Reproducibility of the Food4Me FFQ was assessed using test-retest methodology by asking participants to complete the FFQ on 2 occasions 4 weeks apart. To assess the validity of the Food4Me FFQ against the 4-day WFR, half the participants were also asked to complete a 4-day WFR 1 week after the first administration of the Food4Me FFQ. Level of agreement between nutrient and food group intakes estimated by the repeated Food4Me FFQ and the Food4Me FFQ and 4-day WFR were evaluated using Bland-Altman methodology and classification into quartiles of daily intake. Crude unadjusted correlation coefficients were also calculated for nutrient and food group intakes. Results: In total, 100 people participated in the assessment of reproducibility (mean age 32, SD 12 years), and 49 of these (mean age 27, SD 8 years) also took part in the assessment of validity. Crude unadjusted correlations for repeated Food4Me FFQ ranged from .65 (vitamin D) to .90 (alcohol). The mean cross-classification into “exact agreement plus adjacent” was 92% for both nutrient and food group intakes, and Bland-Altman plots showed good agreement for energy-adjusted macronutrient intakes. Agreement between the Food4Me FFQ and 4-day WFR varied, with crude unadjusted correlations ranging from .23 (vitamin D) to .65 (protein, % total energy) for nutrient intakes and .11 (soups, sauces and miscellaneous foods) to .73 (yogurts) for food group intake. The mean cross-classification into “exact agreement plus adjacent” was 80% and 78% for nutrient and food group intake, respectively. There were no significant differences between energy intakes estimated using the Food4Me FFQ and 4-day WFR, and Bland-Altman plots showed good agreement for both energy and energy-controlled nutrient intakes. Conclusions: The results demonstrate that the online Food4Me FFQ is reproducible for assessing nutrient and food group intake and has moderate agreement with the 4-day WFR for assessing energy and energy-adjusted nutrient intakes. The Food4Me FFQ is a suitable online tool for assessing dietary intake in healthy adults.

Reproducibility of the online Food4Me food-frequency questionnaire for estimating dietary intakes across Europe

Background: Accurate dietary assessment is key to understanding nutrition-related outcomes and is essential for estimating dietary change in nutrition-based interventions. Objective: The objective of this study was to assess the pan-European reproducibility of the Food4Me food-frequency questionnaire (FFQ) in assessing the habitual diet of adults. Methods: Participantsfromthe Food4Me study, a 6-mo,Internet-based, randomizedcontrolled trial of personalized nutrition conducted in the United Kingdom, Ireland, Spain, Netherlands, Germany, Greece, and Poland were included. Screening and baseline data (both collected before commencement of the intervention) were used in the present analyses, and participants were includedonly iftheycompleted FFQs at screeningand at baselinewithin a 1-mo timeframebeforethe commencement oftheintervention. Sociodemographic (e.g., sex andcountry) andlifestyle[e.g.,bodymass index(BMI,inkg/m2)and physical activity] characteristics were collected. Linear regression, correlation coefficients, concordance (percentage) in quartile classification, and Bland-Altman plots for daily intakes were used to assess reproducibility. Results: In total, 567 participants (59% female), with a mean 6 SD age of 38.7 6 13.4 y and BMI of 25.4 6 4.8, completed bothFFQswithin 1 mo(mean 6 SD: 19.26 6.2d).Exact plus adjacent classification oftotal energy intakeinparticipants was highest in Ireland (94%) and lowest in Poland (81%). Spearman correlation coefficients (r) in total energy intake between FFQs ranged from 0.50 for obese participants to 0.68 and 0.60 in normal-weight and overweight participants, respectively. Bland-Altman plots showed a mean difference between FFQs of 210 kcal/d, with the agreement deteriorating as energy intakes increased. There was little variation in reproducibility of total energy intakes between sex and age groups. Conclusions: The online Food4Me FFQ was shown to be reproducible across 7 European countries when administered within a 1-mo period to a large number of participants. The results support the utility of the online Food4Me FFQ as a reproducible tool across multiple European populations. This trial was registered at clinicaltrials.gov as NCT01530139.