Looking for Validity or Testing It? The Perils of Stepwise Regression, Extreme-Scores Analysis, Heteroscedasticity, and Measurement Error

When researchers introduce a new test they have to demonstrate that it is valid, using unbiased designs and suitable statistical procedures. In this article we use Monte Carlo analyses to highlight how incorrect statistical procedures (i.e., stepwise regression, extreme scores analyses) or ignoring regression assumptions (e.g., heteroscedasticity) contribute to wrong validity estimates. Beyond these demonstrations, and as an example, we re-examined the results reported by Warwick, Nettelbeck, and Ward (2010) concerning the validity of the Ability Emotional Intelligence Measure (AEIM). Warwick et al. used the wrong statistical procedures to conclude that the AEIM was incrementally valid beyond intelligence and personality traits in predicting various outcomes. In our re-analysis, we found that the reliability-corrected multiple correlation of their measures with personality and intelligence was up to .69. Using robust statistical procedures and appropriate controls, we also found that the AEIM did not predict incremental variance in GPA, stress, loneliness, or well-being, demonstrating the importance for testing validity instead of looking for it.

Direct three-dimensional myocardial strain tensor quantification and tracking using zHARP.

Images of myocardial strain can be used to diagnose heart disease, plan and monitor treatment, and to learn about cardiac structure and function. Three-dimensional (3D) strain is typically quantified using many magnetic resonance (MR) images obtained in two or three orthogonal planes. Problems with this approach include long scan times, image misregistration, and through-plane motion. This article presents a novel method for calculating cardiac 3D strain using a stack of two or more images acquired in only one orientation. The zHARP pulse sequence encodes in-plane motion using MR tagging and out-of-plane motion using phase encoding, and has been previously shown to be capable of computing 3D displacement within a single image plane. Here, data from two adjacent image planes are combined to yield a 3D strain tensor at each pixel; stacks of zHARP images can be used to derive stacked arrays of 3D strain tensors without imaging multiple orientations and without numerical interpolation. The performance and accuracy of the method is demonstrated in vitro on a phantom and in vivo in four healthy adult human subjects.

The career indecision profile : measurement equivalence in two international samples

This study tested for the measurement equivalence of a four-factor measure of career indecision (Career Indecision Profile-65 [CIP-65]) between a U.S. sample and two international samples; one composed of French-speaking young adults from France and Switzerland and the other of Italian ado- lescents. Previous research had supported the four-factor structure of the CIP-65 in both the United States and Iceland but also showed that items on two of the four scales may be interpreted differently by young adults growing up in these two countries. This study extends previous research by testing whether the four CIP-65 factors are measured equivalently in two additional international samples. Results largely supported the configural and metric invariance of the CIP-65 in the United States and international samples, but several scales showed a lack of scalar invariance. Some explanations are offered for these findings along with suggestions for future research and implications for practice.

Radiographic total disc replacement angle measurement accuracy using the Oxford Cobbometer: precision and bias.

Total disc replacement (TDR) clinical success has been reported to be related to the residual motion of the operated level. Thus, accurate measurement of TDR range of motion (ROM) is of utmost importance. One commonly used tool in measuring ROM is the Oxford Cobbometer. Little is known however on its accuracy (precision and bias) in measuring TDR angles. The aim of this study was to assess the ability of the Cobbometer to accurately measure radiographic TDR angles. An anatomically accurate synthetic L4-L5 motion segment was instrumented with a CHARITE artificial disc. The TDR angle and anatomical position between L4 and L5 was fixed to prohibit motion while the motion segment was radiographically imaged in various degrees of rotation and elevation, representing a sample of possible patient placement positions. An experienced observer made ten readings of the TDR angle using the Cobbometer at each different position. The Cobbometer readings were analyzed to determine measurement accuracy at each position. Furthermore, analysis of variance was used to study rotation and elevation of the motion segment as treatment factors. Cobbometer TDR angle measurements were most accurate (highest precision and lowest bias) at the centered position (95.5%), which placed the TDR directly inline with the x-ray beam source without any rotation. In contrast, the lowest accuracy (75.2%) was observed in the most rotated and off-centered view. A difference as high as 4 degrees between readings at any individual position, and as high as 6 degrees between all the positions was observed. Furthermore, the Cobbometer was unable to detect the expected trend in TDR angle projection with changing position. Although the Cobbometer has been reported to be reliable in different clinical applications, it lacks the needed accuracy to measure TDR angles and ROM. More accurate ROM measurement methods need to be developed to help surgeons and researchers assess radiological success of TDRs.

Temporal quantification of MAPK induced expression in single yeast cells.

The quantification of gene expression at the single cell level uncovers novel regulatory mechanisms obscured in measurements performed at the population level. Two methods based on microscopy and flow cytometry are presented to demonstrate how such data can be acquired. The expression of a fluorescent reporter induced upon activation of the high osmolarity glycerol MAPK pathway in yeast is used as an example. The specific advantages of each method are highlighted. Flow cytometry measures a large number of cells (10,000) and provides a direct measure of the dynamics of protein expression independent of the slow maturation kinetics of the fluorescent protein. Imaging of living cells by microscopy is by contrast limited to the measurement of the matured form of the reporter in fewer cells. However, the data sets generated by this technique can be extremely rich thanks to the combinations of multiple reporters and to the spatial and temporal information obtained from individual cells. The combination of these two measurement methods can deliver new insights on the regulation of protein expression by signaling pathways.

Refinement of Rapid PCC Coarse Aggregate Testing Program, HR-1071, 1999

The characterization and categorization of coarse aggregates for use in portland cement concrete (PCC) pavements is a highly refined process at the Iowa Department of Transportation. Over the past 10 to 15 years, much effort has been directed at pursuing direct testing schemes to supplement or replace existing physical testing schemes. Direct testing refers to the process of directly measuring the chemical and mineralogical properties of an aggregate and then attempting to correlate those measured properties to historical performance information (i.e., field service record). This is in contrast to indirect measurement techniques, which generally attempt to extrapolate the performance of laboratory test specimens to expected field performance. The purpose of this research project was to investigate and refine the use of direct testing methods, such as X-ray analysis techniques and thermal analysis techniques, to categorize carbonate aggregates for use in portland cement concrete. The results of this study indicated that the general testing methods that are currently used to obtain data for estimating service life tend to be very reliable and have good to excellent repeatability. Several changes in the current techniques were recommended to enhance the long-term reliability of the carbonate database. These changes can be summarized as follows: (a) Limits that are more stringent need to be set on the maximum particle size in the samples subjected to testing. This should help to improve the reliability of all three of the test methods studied during this project. (b) X-ray diffraction testing needs to be refined to incorporate the use of an internal standard. This will help to minimize the influence of sample positioning errors and it will also allow for the calculation of the concentration of the various minerals present in the samples. (c) Thermal analysis data needs to be corrected for moisture content and clay content prior to calculating the carbonate content of the sample.

LiveCount Assay: concomitant measurement of cytolytic activity and phenotypic characterisation of CD8(+) T-cells by flow cytometry.

Tumour immunologists strive to develop efficient tumour vaccination and adoptive transfer therapies that enlarge the pool of tumour-specific and -reactive effector T-cells in vivo. To assess the efficiency of the various strategies, ex vivo assays are needed for the longitudinal monitoring of the patient's specific immune responses providing both quantitative and qualitative data. In particular, since tumour cell cytolysis is the end goal of tumour immunotherapy, routine immune monitoring protocols need to include a read-out for the cytolytic efficiency of Ag-specific cells. We propose to combine current immune monitoring techniques in a highly sensitive and reproducible multi-parametric flow cytometry based cytotoxicity assay that has been optimised to require low numbers of Ag-specific T-cells. The possibility of re-analysing those T-cells that have undergone lytic activity is illustrated by the concomitant detection of CD107a upregulation on the surface of degranulated T-cells. To date, the LiveCount Assay provides the only possibility of assessing the ex vivo cytolytic activity of low-frequency Ag-specific cytotoxic T-lymphocytes from patient material.

Measurement of the specific heat and determination of the thermodynamic functions of relaxed amorphous silicon

The specific heat, cp, of two amorphous silicon (a-Si) samples has been measured by differential scanning calorimetry in the 100–900K temperature range. When the hydrogen content is reduced by thermal annealing, cp approaches the value of crystalline Si (c-Si). Within experimental accuracy, we conclude that cp of relaxed pure a-Si coincides with that of c-Si. This result is used to determine the enthalpy, entropy, and Gibbs free energy of defect-free relaxed a-Si. Finally, the contribution of structural defects on these quantities is calculated and the melting point of several states of a-Si is predicted

Quantifying Uncertainty in Real Time Performance Measurement for Highway Winter Maintenance Operations – Phase 2, RB03-013, 2014

Winter weather in Iowa is often unpredictable and can have an adverse impact on traffic flow. The Iowa Department of Transportation (Iowa DOT) attempts to lessen the impact of winter weather events on traffic speeds with various proactive maintenance operations. In order to assess the performance of these maintenance operations, it would be beneficial to develop a model for expected speed reduction based on weather variables and normal maintenance schedules. Such a model would allow the Iowa DOT to identify situations in which speed reductions were much greater than or less than would be expected for a given set of storm conditions, and make modifications to improve efficiency and effectiveness. The objective of this work was to predict speed changes relative to baseline speed under normal conditions, based on nominal maintenance schedules and winter weather covariates (snow type, temperature, and wind speed), as measured by roadside weather stations. This allows for an assessment of the impact of winter weather covariates on traffic speed changes, and estimation of the effect of regular maintenance passes. The researchers chose events from Adair County, Iowa and fit a linear model incorporating the covariates mentioned previously. A Bayesian analysis was conducted to estimate the values of the parameters of this model. Specifically, the analysis produces a distribution for the parameter value that represents the impact of maintenance on traffic speeds. The effect of maintenance is not a constant, but rather a value that the researchers have some uncertainty about and this distribution represents what they know about the effects of maintenance. Similarly, examinations of the distributions for the effects of winter weather covariates are possible. Plots of observed and expected traffic speed changes allow a visual assessment of the model fit. Future work involves expanding this model to incorporate many events at multiple locations. This would allow for assessment of the impact of winter weather maintenance across various situations, and eventually identify locations and times in which maintenance could be improved.

On-line desorption of dried blood spot: A novel approach for the direct LC/MS analysis of micro-whole blood samples.

The aim of this work is to present a new concept, called on-line desorption of dried blood spots (on-line DBS), allowing the direct analysis of a dried blood spot coupled to liquid chromatography mass spectrometry device (LC/MS). The system is based on an inox cell which can receive a blood sample (10 microL) previously spotted on a filter paper. The cell is then integrated into LC/MS system where the analytes are desorbed out of the paper towards a column switching system ensuring the purification and separation of the compounds before their detection on a single quadrupole MS coupled to atmospheric pressure chemical ionisation (APCI) source. The described procedure implies that no pretreatment is necessary in spite the analysis is based on whole blood sample. To ensure the applicability of the concept, saquinavir, imipramine, and verapamil were chosen. Despite the use of a small sampling volume and a single quadrupole detector, on-line DBS allowed the analyses of these three compounds over their therapeutic concentrations from 50 to 500 ng/mL for imipramine and verapamil and from 100 to 1000 ng/mL for saquinavir. Moreover, the method showed good repeatability with relative standard deviation (RSD) lower than 15% based on two levels of concentration (low and high). Function responses were found to be linear over the therapeutic concentration for each compound and were used to determine the concentrations of real patient samples for saquinavir. Comparison of the founded values with those of a validated method used routinely in a reference laboratory showed a good correlation between the two methods. Moreover, good selectivity was observed ensuring that no endogenous or chemical components interfered with the quantitation of the analytes. This work demonstrates the feasibility and applicability of the on-line DBS procedure for bioanalysis.

Sample size for measurement of root traits on common bean by image analysis

Evaluation of root traits may be facilitated if they are assessed on samples of the root system. The objective of this work was to determine the sample size of the root system in order to estimate root traits of common bean (Phaseolus vulgaris L.) cultivars by digital image analysis. One plant was grown per pot and harvested at pod setting, with 64 and 16 pots corresponding to two and four cultivars in the first and second experiments, respectively. Root samples were scanned up to the completeness of the root system and the root area and length were estimated. Scanning a root sample demanded 21 minutes, and scanning the entire root system demanded 4 hours and 53 minutes. In the first experiment, root area and length estimated with two samples showed, respectively, a correlation of 0.977 and 0.860, with these traits measured in the entire root. In the second experiment, the correlation was 0.889 and 0.915. The increase in the correlation with more than two samples was negligible. The two samples corresponded to 13.4% and 16.9% of total root mass (excluding taproot and nodules) in the first and second experiments. Taproot stands for a high proportion of root mass and must be deducted on root trait estimations. Samples with nearly 15% of total root mass produce reliable root trait estimates.

Inequality measurement for ordered response health data

When health status is an ordered response variable, Allison and Foster (2004)postulate that a distribution Q exhibits more inequality than a distribution P if Q is obtained from P via a sequence of median preserving spreads. This paper introduces a parametric family of inequality indices which are founded on the Allison and Foster ordering. [Authors]

Transcranial direct current stimulation for the treatment of focal hand dystonia.

The treatment of writer's cramp, a task-specific focal hand dystonia, needs new approaches. A deficiency of inhibition in the motor cortex might cause writer's cramp. Transcranial direct current stimulation modulates cortical excitability and may provide a therapeutic alternative. In this randomized, double-blind, sham-controlled study, we investigated the efficacy of cathodal stimulation of the contralateral motor cortex in 3 sessions in 1 week. Assessment over a 2-week period included clinical scales, subjective ratings, kinematic handwriting analysis, and neurophysiological evaluation. Twelve patients with unilateral dystonic writer's cramp were investigated; 6 received transcranial direct current and 6 sham stimulation. Cathodal transcranial direct current stimulation had no favorable effects on clinical scales and failed to restore normal handwriting kinematics and cortical inhibition. Subjective worsening remained unexplained, leading to premature study termination. Repeated sessions of cathodal transcranial direct current stimulation of the motor cortex yielded no favorable results supporting a therapeutic potential in writer's cramp.