Looking for Validity or Testing It? The Perils of Stepwise Regression, Extreme-Scores Analysis, Heteroscedasticity, and Measurement Error

When researchers introduce a new test they have to demonstrate that it is valid, using unbiased designs and suitable statistical procedures. In this article we use Monte Carlo analyses to highlight how incorrect statistical procedures (i.e., stepwise regression, extreme scores analyses) or ignoring regression assumptions (e.g., heteroscedasticity) contribute to wrong validity estimates. Beyond these demonstrations, and as an example, we re-examined the results reported by Warwick, Nettelbeck, and Ward (2010) concerning the validity of the Ability Emotional Intelligence Measure (AEIM). Warwick et al. used the wrong statistical procedures to conclude that the AEIM was incrementally valid beyond intelligence and personality traits in predicting various outcomes. In our re-analysis, we found that the reliability-corrected multiple correlation of their measures with personality and intelligence was up to .69. Using robust statistical procedures and appropriate controls, we also found that the AEIM did not predict incremental variance in GPA, stress, loneliness, or well-being, demonstrating the importance for testing validity instead of looking for it.

Correlation between microtensile bond strength data and clinical outcome of Class V restorations.

OBJECTIVE: To determine if the results of resin-dentin microtensile bond strength (µTBS) is correlated with the outcome parameters of clinical studies on non-retentive Class V restorations. METHODS: Resin-dentin µTBS data were obtained from one test center; the in vitro tests were all performed by the same operator. The µTBS testing was performed 8h after bonding and after 6 months of storing the specimens in water. Pre-test failures (PTFs) of specimens were included in the analysis, attributing them a value of 1MPa. Prospective clinical studies on cervical restorations (Class V) with an observation period of at least 18 months were searched in the literature. The clinical outcome variables were retention loss, marginal discoloration and marginal integrity. Furthermore, an index was formulated to be better able to compare the laboratory and clinical results. Estimates of adhesive effects in a linear mixed model were used to summarize the clinical performance of each adhesive between 12 and 36 months. Spearman correlations between these clinical performances and the µTBS values were calculated subsequently. RESULTS: Thirty-six clinical studies with 15 adhesive/restorative systems for which µTBS data were also available were included in the statistical analysis. In general 3-step and 2-step etch-and-rinse systems showed higher bond strength values than the 2-step/3-step self-etching systems, which, however, produced higher values than the 1-step self-etching and the resin modified glass ionomer systems. Prolonged water storage of specimens resulted in a significant decrease of the mean bond strength values in 5 adhesive systems (Wilcoxon, p<0.05). There was a significant correlation between µTBS values both after 8h and 6 months of storage and marginal discoloration (r=0.54 and r=0.67, respectively). However, the same correlation was not found between µTBS values and the retention rate, clinical index or marginal integrity. SIGNIFICANCE: As µTBS data of adhesive systems, especially after water storage for 6 months, showed a good correlation with marginal discoloration in short-term clinical Class V restorations, longitudinal clinical trials should explore whether early marginal staining is predictive for future retention loss in non-carious cervical restorations.

Image Analysis for Evaluating Air Void Parameters of Concrete, HR-396, 1998

This research project investigated the use of image analysis to measure the air void parameters of concrete specimens produced under standard laboratory conditions. The results obtained from the image analysis technique were compared to results obtained from plastic air content tests, Danish air meter tests (also referred to as Air Void Analyzer tests), high-pressure air content tests on hardened concrete, and linear traverse tests (as per ASTM C-457). Hardened concrete specimens were sent to three different laboratories for the linear traverse tests. The samples that were circulated to the three labs consisted of specimens that needed different levels of surface preparation. The first set consisted of approximately 18 specimens that had been sectioned from a 4 in. by 4 in. by 18 in. (10 cm by 10 cm by 46 cm) beam using a saw equipped with a diamond blade. These specimens were subjected to the normal sample preparation techniques that were commonly employed by the three different labs (each lab practiced slightly different specimen preparation techniques). The second set of samples consisted of eight specimens that had been ground and polished at a single laboratory. The companion labs were only supposed to retouch the sample surfaces if they exhibited major flaws. In general, the study indicated that the image analysis test results for entrained air content exhibited good to strong correlation to the average values determined via the linear traverse technique. Specimens ground and polished in a single laboratory and then circulated to the other participating laboratories for the air content determinations exhibited the strongest correlation between the image analysis and linear traverse techniques (coefficient of determination, r-squared = 0.96, for n=8). Specimens ground and polished at each of the individual laboratories exhibited considerably more scatter (coefficient of determination, r-squared = 0.78, for n=16). The image analysis technique tended to produce low estimates of the specific surface of the voids when compared to the results from the linear traverse method. This caused the image analysis spacing factor calculations to produce larger values than those obtained from the linear traverse tests. The image analysis spacing factors were still successful at distinguishing between the frost-prone test specimens and the other (more durable) test specimens that were studied in this research project.

Correlation of the Roughometer to the High-Speed Laser Profiler, MLR-97-7, 1997

The Iowa Department of Transportation has been using the Bureau of Public Roads (BPR) Roughometer as part of its detour analysis process for more than 20 years. Advances in technology have made the BPR Roughometer obsolete for ride quality testing. High-speed profilers that can collect the profile of the road at highway speeds are the standard ride instruments for determining ride quality on pavements. The objective of the project was to develop a correlation between the BPR Roughometer and the high-speed laser South Dakota type Profiler (SD Profiler). Nineteen pavement sections were chosen to represent the range of types and conditions for detours. Three computer simulation models were tested on the profiler profiles. The first model is the International Ride Index (IRI) which is considered the standard index for reporting ride quality in the United States. The second model is the Ride Number (RN) developed by the University of Michigan Transportation Research Institute and the third model used is a quarter-car simulation of the BPR Roughometer (ASTM E-1170) which should match the speed and range of roadway features experienced by Iowa's BPR Roughometer Unit. The BPR Roughometer quarter-car model provided the best overall correlation with Iowa's BPR Roughometer.

Correlation of Locally-Based Performance of Asphalts with their Physicochemical Parameters, Task 1 Report, HR-298, 1988

This is the first report of 6 tasks to be performed in an effort to establish locally-based quality and performance criteria for asphalts, and ultimately to develop performance-related specifications based on simple physicochemical methods. Three of the most promising chemical methods (high performance liquid chromatography (HPLC), thermal analysis, and X-ray diffraction were selected to analyze 4 different types of samples. The results will indicate the fundamental asphalt property variables that directly affect the field performance in Iowa. The details of the materials and procedures employed are described, and the results of the tests are presented.

Comparative functional analysis of two fibroblast growth factor receptor 1 (FGFR1) mutations affecting the same residue (R254W and R254Q) in isolated hypogonadotropic hypogonadism (IHH).

FGFR1 mutations have been identified in both Kallmann syndrome and normosmic HH (nIHH). To date, few mutations in the FGFR1 gene have been structurally or functionally characterized in vitro to identify molecular mechanisms that contribute to the disease pathogenesis. We attempted to define the in vitro functionality of two FGFR1 mutants (R254W and R254Q), resulting from two different amino acid substitutions of the same residue, and to correlate the in vitro findings to the patient phenotypes. Two unrelated GnRH deficient probands were found to harbor mutations in FGFR1 (R254W and R254Q). Mutant signaling activity and expression levels were evaluated in vitro and compared to a wild type (WT) receptor. Signaling activity was determined by a FGF2/FGFR1 dependent transcription reporter assay. Receptor total expression levels were assessed by Western blot and cell surface expression was measured by a radiolabeled antibody binding assay. The R254W maximal receptor signaling capacity was reduced by 45% (p<0.01) while R254Q activity was not different from WT. However, both mutants displayed diminished total protein expression levels (40 and 30% reduction relative to WT, respectively), while protein maturation was unaffected. Accordingly, cell surface expression levels of the mutant receptors were also significantly reduced (35% p<0.01 and 15% p<0.05, respectively). The p.R254W and p.R254Q are both loss-of-function mutations as demonstrated by their reduced overall and cell surface expression levels suggesting a deleterious effect on receptor folding and stability. It appears that a tryptophan substitution at R254 is more disruptive to receptor structure than the more conserved glutamine substitution. No clear correlation between the severity of in vitro loss-of-function and phenotypic presentation could be assigned.

Statistical Analysis of Highway Needs Condition Data: Manual vs. Automated, TR-494

This project examines similarities and differences between the automated condition data collected on and off county paved roads and the manual condition data collected by Iowa Department of Transportation (DOT) staff in 2000 and 2001. Also, the researchers will provide staff support to the advisory committee in exploring other options to the highway need process. The results show that the automated condition data can be used in a converted highway needs process with no major differences between the two methods. Even though the foundation rating difference was significant, the foundation rating weighting factor in HWYNEEDS is minimal and should not have a major impact. In terms of RUTF formula based distribution, the results clearly show the superiority of the condition-based analysis compared to the non-condition based. That correlation can be further enhanced by adding more distress variables to the analysis.

Violent adolescents and their educational environment: a multilevel analysis.

OBJECTIVE: This study examined the respective roles of personal and environmental factors in youth violence in a nationally representative sample of 7548 postmandatory school students and apprentices ages 16-20 years in Switzerland. METHODS: Youth violence was defined as having committed at least one of the following in the previous 12 months: attacking an adult, snatching something, carrying a weapon, or using a weapon in a fight. Different ecological levels were tested, resulting in a three-level model only in males (individual, classroom, and school) as the low prevalence of female violence did not allow for a multilevel analysis. Dependent variables were attributed to each level. For males, the classroom level (10%) and the school level (24%) accounted for more than one third in interindividual variance. RESULTS: Factors associated with violence perpetration in females were being a victim of physical violence and sensation seeking at the individual level. In males, practicing unsafe sex, sensation seeking, being a victim of physical violence, having a poor relationship with parents, being depressed, and living in a single-parent household at the individual level; violence and antisocial acts at the classroom level; and being in a vocational school at the school level showed a correlation with violence perpetration. CONCLUSION: Interventions at the classroom level as well as an explicit school policy on violence and other risk behaviors should be considered a priority when dealing with the problem of youth violence. Furthermore, prevention should take into account gender differences.

Optimization of HS-SPME/GC-MS analysis and its use in the profiling of illicit ecstasy tablets (Part 1)

A headspace solid-phase microextraction procedure (HS-SPME) was developed for the profiling of traces present in 3,4-methylenedioxymethylampethamine (MDMA). Traces were first extracted using HS-SPME and then analyzed by gas chromatography-mass spectroscopy (GC-MS). The HS-SPME conditions were optimized using varying conditions. Optimal results were obtained when 40 mg of crushed MDMA sample was heated at 80 °C for 15 min, followed by extraction at 80 °C for 15 min with a polydimethylsiloxane/divinylbenzene coated fibre. A total of 31 compounds were identified as traces related to MDMA synthesis, namely precursors, intermediates or by-products. In addition some fatty acids used as tabletting materials and caffeine used as adulterant, were also detected. The use of a restricted set of 10 target compounds was also proposed for developing a screening tool for clustering samples having close profile. 114 seizures were analyzed using an SPME auto-sampler (MultiPurpose Samples MPS2), purchased from Gerstel GMBH & Co. (Germany), and coupled to GC-MS. The data was handled using various pre-treatment methods, followed by the study of similarities between sample pairs based on the Pearson correlation. The results show that HS-SPME, coupled with the suitable statistical method is a powerful tool for distinguishing specimens coming from the same seizure and specimens coming from different seizures. This information can be used by law enforcement personnel to visualize the ecstasy distribution network as well as the clandestine tablet manufacturing.

On-line desorption of dried blood spot: A novel approach for the direct LC/MS analysis of micro-whole blood samples.

The aim of this work is to present a new concept, called on-line desorption of dried blood spots (on-line DBS), allowing the direct analysis of a dried blood spot coupled to liquid chromatography mass spectrometry device (LC/MS). The system is based on an inox cell which can receive a blood sample (10 microL) previously spotted on a filter paper. The cell is then integrated into LC/MS system where the analytes are desorbed out of the paper towards a column switching system ensuring the purification and separation of the compounds before their detection on a single quadrupole MS coupled to atmospheric pressure chemical ionisation (APCI) source. The described procedure implies that no pretreatment is necessary in spite the analysis is based on whole blood sample. To ensure the applicability of the concept, saquinavir, imipramine, and verapamil were chosen. Despite the use of a small sampling volume and a single quadrupole detector, on-line DBS allowed the analyses of these three compounds over their therapeutic concentrations from 50 to 500 ng/mL for imipramine and verapamil and from 100 to 1000 ng/mL for saquinavir. Moreover, the method showed good repeatability with relative standard deviation (RSD) lower than 15% based on two levels of concentration (low and high). Function responses were found to be linear over the therapeutic concentration for each compound and were used to determine the concentrations of real patient samples for saquinavir. Comparison of the founded values with those of a validated method used routinely in a reference laboratory showed a good correlation between the two methods. Moreover, good selectivity was observed ensuring that no endogenous or chemical components interfered with the quantitation of the analytes. This work demonstrates the feasibility and applicability of the on-line DBS procedure for bioanalysis.

A simple gas chromatography method for the analysis of monoethanolamine in air

A simple method determining airborne monoethanolamine has been developed. Monoethanolamine determination has traditionally been difficult due to analytical separation problems. Even in recent sophisticated methods, this difficulty remains as the major issue often resulting in time-consuming sample preparations. Impregnated glass fiber filters were used for sampling. Desorption of monoethanolamine was followed by capillary GC analysis and nitrogen phosphorous selective detection. Separation was achieved using a specific column for monoethanolamines (35% diphenyl and 65% dimethyl polysiloxane). The internal standard was quinoline. Derivatization steps were not needed. The calibration range was 0.5-80 μg/mL with a good correlation (R(2) = 0.996). Averaged overall precisions and accuracies were 4.8% and -7.8% for intraday (n = 30), and 10.5% and -5.9% for interday (n = 72). Mean recovery from spiked filters was 92.8% for the intraday variation, and 94.1% for the interday variation. Monoethanolamine on stored spiked filters was stable for at least 4 weeks at 5°C. This newly developed method was used among professional cleaners and air concentrations (n = 4) were 0.42 and 0.17 mg/m(3) for personal and 0.23 and 0.43 mg/m(3) for stationary measurements. The monoethanolamine air concentration method described here was simple, sensitive, and convenient both in terms of sampling and analytical analysis.

Deoxyribonucleic acid content as an indicator of progression of squamous cell carcinogenesis in the esophagus: comparative analysis on imprint-cytospin and tissue section preparation.

BACKGROUND: The purpose of this study was to explore the potential use of image analysis on tissue sections preparation as a predictive marker of early malignant changes during squamous cell (SC) carcinogenesis in the esophagus. Results of DNA ploidy quantification on formalin-fixed, paraffin-embedded tissue using two different techniques were compared: imprint-cytospin and 6 microm thick tissue sections preparation. METHODS: This retrospective study included 26 surgical specimens of squamous cell carcinoma (SCC) from patients who underwent surgery alone at the Department of Surgery in CHUV Hospital in Lausanne between January 1993 and December 2000. We analyzed 53 samples of healthy tissue, 43 tumors and 7 lymph node metastases. RESULTS: Diploid DNA histogram patterns were observed in all histologically healthy tissues, either distant or proximal to the lesion. Aneuploidy was observed in 34 (79%) of 43 carcinomas, namely 24 (75%) of 32 early squamous cell carcinomas and 10 (91%) of 11 advanced carcinomas. DNA content was similar in the different tumor stages, whether patients presented with single or multiple synchronous tumors. All lymph node metastases had similar DNA content as their primary tumor. CONCLUSIONS: Early malignant changes in the esophagus are associated with alteration in DNA content, and aneuploidy tends to correlate with progression of invasive SCC. A very good correlation between imprint-cytospin and tissue section analysis was observed. Although each method used here showed advantages and disadvantages; tissue sections preparation provided useful information on aberrant cell-cycle regulation and helped select the optimal treatment for the individual patient along with consideration of other clinical parameters.

Sample size for measurement of root traits on common bean by image analysis

Evaluation of root traits may be facilitated if they are assessed on samples of the root system. The objective of this work was to determine the sample size of the root system in order to estimate root traits of common bean (Phaseolus vulgaris L.) cultivars by digital image analysis. One plant was grown per pot and harvested at pod setting, with 64 and 16 pots corresponding to two and four cultivars in the first and second experiments, respectively. Root samples were scanned up to the completeness of the root system and the root area and length were estimated. Scanning a root sample demanded 21 minutes, and scanning the entire root system demanded 4 hours and 53 minutes. In the first experiment, root area and length estimated with two samples showed, respectively, a correlation of 0.977 and 0.860, with these traits measured in the entire root. In the second experiment, the correlation was 0.889 and 0.915. The increase in the correlation with more than two samples was negligible. The two samples corresponded to 13.4% and 16.9% of total root mass (excluding taproot and nodules) in the first and second experiments. Taproot stands for a high proportion of root mass and must be deducted on root trait estimations. Samples with nearly 15% of total root mass produce reliable root trait estimates.

Correlation of CXCL10, tumor necrosis factor receptor type II, and galectin 9 with disease activity in juvenile dermatomyositis.

OBJECTIVE: Juvenile dermatomyositis (DM) is a systemic autoimmune disorder of unknown immunopathogenesis in which the immune system targets the microvasculature of skeletal muscles, skin, and other organs. The current mainstay of therapy is a steroid regimen in combination with other immunosuppressive treatments. To date, no validated markers for monitoring disease activity have been identified, which hampers personalized treatment. This study was undertaken to identify a panel of proteins specifically related to active disease in juvenile DM. METHODS: We performed a multiplex immunoassay for plasma levels of 45 proteins related to inflammation in 25 patients with juvenile DM in 4 clinically well-defined groups, as determined by clinical activity and treatment. We compared them to 14 age-matched healthy children and 8 age-matched children with nonautoimmune muscle disease. RESULTS: Cluster analysis of circulating proteins showed distinct profiles for juvenile DM patients and controls based on a group of 10 proteins. In addition to CXCL10, tumor necrosis factor receptor type II (TNFRII) and galectin 9 were significantly increased in active juvenile DM. The levels of these 3 proteins were tightly linked to active disease and correlated with clinical scores (as measured by the Childhood Myositis Assessment Scale and physician's global assessment of disease activity on a visual analog scale). CONCLUSION: Our findings indicate that CXCL10, TNFRII, and galectin 9 correspond to disease status in juvenile DM and thus could be helpful in monitoring disease activity and guiding treatment. Furthermore, they might provide new knowledge about the pathogenesis of this autoimmune disease.

Mineralocorticoid effects in the kidney: correlation between alphaENaC, GILZ, and Sgk-1 mRNA expression and urinary excretion of Na+ and K+.

Aldosterone exerts its effects through interactions with two types of binding sites, the mineralocorticoid (MR) and the glucocorticoid (GR) receptors. Although both receptors are known to be involved in the anti-natriuretic response to aldosterone, the mechanisms of signal transduction leading to modulation of electrolyte transport are not yet fully understood. This study measured the Na(+) and K(+) urinary excretion and the mRNA levels of three known aldosterone-induced transcripts, the serum and glucocorticoid-induced kinase (Sgk-1), the alpha subunit of the epithelial Na(+) channel (alphaENaC), and the glucocorticoid-induced-leucine-zipper protein (GILZ) in the whole kidney and in isolated cortical collecting tubules of adrenalectomized rats treated with low doses of aldosterone and/or dexamethasone. The resulting plasma concentrations of both steroids were close to 1 nmol/L. Aldosterone, given with or without dexamethasone, induced anti-natriuresis and kaliuresis, whereas dexamethasone alone did not. GILZ and alphaENaC transcripts were higher after treatment with either or both hormones, whereas the mRNA abundance of Sgk-1 was increased in the cortical collecting tubule by aldosterone but not by dexamethasone. We conclude the increased expression of Sgk-1 in the cortical collecting tubules is a primary event in the early antinatriuretic and kaliuretic responses to physiologic concentrations of aldosterone. Induction of alphaENaC and/or GILZ mRNAs may play a permissive role in the enhancement of the early and/or late responses; these effects may be necessary for a full response but do not by themselves promote early changes in urinary Na(+) and K(+) excretion.