Looking for Validity or Testing It? The Perils of Stepwise Regression, Extreme-Scores Analysis, Heteroscedasticity, and Measurement Error

When researchers introduce a new test they have to demonstrate that it is valid, using unbiased designs and suitable statistical procedures. In this article we use Monte Carlo analyses to highlight how incorrect statistical procedures (i.e., stepwise regression, extreme scores analyses) or ignoring regression assumptions (e.g., heteroscedasticity) contribute to wrong validity estimates. Beyond these demonstrations, and as an example, we re-examined the results reported by Warwick, Nettelbeck, and Ward (2010) concerning the validity of the Ability Emotional Intelligence Measure (AEIM). Warwick et al. used the wrong statistical procedures to conclude that the AEIM was incrementally valid beyond intelligence and personality traits in predicting various outcomes. In our re-analysis, we found that the reliability-corrected multiple correlation of their measures with personality and intelligence was up to .69. Using robust statistical procedures and appropriate controls, we also found that the AEIM did not predict incremental variance in GPA, stress, loneliness, or well-being, demonstrating the importance for testing validity instead of looking for it.

T Cells Bearing a Chimeric Antigen Receptor against Prostate-Specific Membrane Antigen Mediate Vascular Disruption and Result in Tumor Regression.

Aberrant blood vessels enable tumor growth, provide a barrier to immune infiltration, and serve as a source of protumorigenic signals. Targeting tumor blood vessels for destruction, or tumor vascular disruption therapy, can therefore provide significant therapeutic benefit. Here, we describe the ability of chimeric antigen receptor (CAR)-bearing T cells to recognize human prostate-specific membrane antigen (hPSMA) on endothelial targets in vitro as well as in vivo. CAR T cells were generated using the anti-PSMA scFv, J591, and the intracellular signaling domains: CD3ζ, CD28, and/or CD137/4-1BB. We found that all anti-hPSMA CAR T cells recognized and eliminated PSMA(+) endothelial targets in vitro, regardless of the signaling domain. T cells bearing the third-generation anti-hPSMA CAR, P28BBζ, were able to recognize and kill primary human endothelial cells isolated from gynecologic cancers. In addition, the P28BBζ CAR T cells mediated regression of hPSMA-expressing vascular neoplasms in mice. Finally, in murine models of ovarian cancers populated by murine vessels expressing hPSMA, the P28BBζ CAR T cells were able to ablate PSMA(+) vessels, cause secondary depletion of tumor cells, and reduce tumor burden. Taken together, these results provide a strong rationale for the use of CAR T cells as agents of tumor vascular disruption, specifically those targeting PSMA. Cancer Immunol Res; 3(1); 68-84. ©2014 AACR.

Giant warts in a kidney transplant patient: regression with sirolimus.

Purpose: Recent reports have suggested that intraabdominal postoperative infection is associated with higher rates of overall and local recurrence and cancer-specific mortality. However, the mechanisms responsible for this association are unknown. We hypothesized that the greater inflammatory response in patients with postoperative intraabdominal infection is associated to an increase in local and systemic angiogenesis. Methods: We designed a prospective cohorts study with matched controls. Patients with postoperative intra-abdominal infection (abscess and/or anastomotic leakage) (group 1; n=17) after elective colorectal cancer resection operated on for cure were compared to patients with an uncomplicated postoperative course (group 2; n=17). IL-6 and VEGF levels were determined by ELISA in serum and peritoneal fluid at baseline, 48 hours and postoperative day 4 or at the time the peritoneal infection occurred. Results: No differences were observed in age, gender, preoperative CEA, tumor stage and location and type of procedure performed. Although there were no differences in serum IL-6 levels at 48 hours, this pro-inflammatory cytokine was higher in group 1 on postoperative day 4 (group 1: 21533 + 27900 vs. group 2: 1130 + 3563 pg/ml; p < 0.001). Serum VEGF levels were higher in group 1 on postoperative day 4 (group 1: 1212 + 1025 vs. group 2: 408 + 407 pg/ml; p < 0.01). Peritoneal fluid VEGF levels were also higher in group 1 at 48 hours (group 1: 4857 + 4384 vs. group 2: 630 + 461 pg/ml; p < 0.001) and postoperative day 4 (group 1: 32807 + 98486 vs. group 2: 1002 + 1229 pg/ml; p < 0.001). A positive correlation between serum IL-6 and VEGF serum levels was observed on postoperative day 4 (r=0.7; p<0.01). Conclusions: These results suggest that not only the inflammatory response but also the angiogenic pathways are stimulated in patients with intra-abdominal infection after surgery for colorectal cancer. The implications of this finding on long-term follow-up need to be evaluated.

Long-term outcome after cognitive and behavioral regression in nonlesional epilepsy with continuous spike-waves during slow-wave sleep.

PURPOSE: To present the long-term follow-up of 10 adolescents and young adults with documented cognitive and behavioral regression as children due to nonlesional focal, mainly frontal, epilepsy with continuous spike-waves during slow wave sleep (CSWS). METHODS: Past medical and electroencephalography (EEG) data were reviewed and neuropsychological tests exploring main cognitive functions were administered. KEY FINDINGS: After a mean duration of follow-up of 15.6 years (range, 8-23 years), none of the 10 patients had recovered fully, but four regained borderline to normal intelligence and were almost independent. Patients with prolonged global intellectual regression had the worst outcome, whereas those with more specific and short-lived deficits recovered best. The marked behavioral disorders resolved in all but one patient. Executive functions were neither severely nor homogenously affected. Three patients with a frontal syndrome during the active phase (AP) disclosed only mild residual executive and social cognition deficits. The main cognitive gains occurred shortly after the AP, but qualitative improvements continued to occur. Long-term outcome correlated best with duration of CSWS. SIGNIFICANCE: Our findings emphasize that cognitive recovery after cessation of CSWS depends on the severity and duration of the initial regression. None of our patients had major executive and social cognition deficits with preserved intelligence, as reported in adults with early destructive lesions of the frontal lobes. Early recognition of epilepsy with CSWS and rapid introduction of effective therapy are crucial for a best possible outcome.

Factors associated with latent tuberculosis among asylum seekers in Switzerland: a cross-sectional study in Vaud County.

BACKGROUND: Screening and treatment of latent tuberculosis infection (LTBI) in asylum seekers (AS) may prevent future cases of tuberculosis. As the screening with Interferon Gamma Release Assay (IGRA) is costly, the objective of this study was to assess which factors were associated with LTBI and to define a score allowing the selection of AS with the highest risk of LTBI. METHODS: In across-sectional study, AS seekers recently arrived in Vaud County, after screening for tuberculosis at the border were offered screening for LTBI with T-SPOT.TB and questionnaire on potentially risk factors. The factors associated with LTBI were analyzed by univariate and multivariate regression. RESULTS: Among 393 adult AS, 98 (24.93%) had a positive IGRA response, five of them with active tuberculosis previously undetected. Six factors associated with LTBI were identified in multivariate analysis: origin, travel conditions, marital status, cough, age and prior TB exposure. Their combination leads to a robust LTBI predictive score. CONCLUSIONS: The prevalence of LTBI and active tuberculosis in AS is high. A predictive score integrating six factors could identify the asylum seekers with the highest risk for LTBI.

Evaluating sampling strategies and logistic regression methods for modelling complex land cover changes

The role of land cover change as a significant component of global change has become increasingly recognized in recent decades. Large databases measuring land cover change, and the data which can potentially be used to explain the observed changes, are also becoming more commonly available. When developing statistical models to investigate observed changes, it is important to be aware that the chosen sampling strategy and modelling techniques can influence results. We present a comparison of three sampling strategies and two forms of grouped logistic regression models (multinomial and ordinal) in the investigation of patterns of successional change after agricultural land abandonment in Switzerland. Results indicated that both ordinal and nominal transitional change occurs in the landscape and that the use of different sampling regimes and modelling techniques as investigative tools yield different results. Synthesis and applications. Our multimodel inference identified successfully a set of consistently selected indicators of land cover change, which can be used to predict further change, including annual average temperature, the number of already overgrown neighbouring areas of land and distance to historically destructive avalanche sites. This allows for more reliable decision making and planning with respect to landscape management. Although both model approaches gave similar results, ordinal regression yielded more parsimonious models that identified the important predictors of land cover change more efficiently. Thus, this approach is favourable where land cover change pattern can be interpreted as an ordinal process. Otherwise, multinomial logistic regression is a viable alternative.

MART-1 peptide vaccination plus IMP321 (LAG-3Ig fusion protein) in patients receiving autologous PBMCs after lymphodepletion: results of a Phase I trial.

BACKGROUND: Immunotherapy offers a promising novel approach for the treatment of cancer and both adoptive T-cell transfer and immune modulation lead to regression of advanced melanoma. However, the potential synergy between these two strategies remains unclear. METHODS: We investigated in 12 patients with advanced stage IV melanoma the effect of multiple MART-1 analog peptide vaccinations with (n = 6) or without (n = 6) IMP321 (LAG-3Ig fusion protein) as an adjuvant in combination with lymphodepleting chemotherapy and adoptive transfer of autologous PBMCs at day (D) 0 (Trial registration No: NCT00324623). All patients were selected on the basis of ex vivo detectable MART-1-specific CD8 T-cell responses and immunized at D0, 8, 15, 22, 28, 52, and 74 post-reinfusion. RESULTS: After immunization, a significant expansion of MART-1-specific CD8 T cells was measured in 83% (n = 5/6) and 17% (n = 1/6) of patients from the IMP321 and control groups, respectively (P < 0.02). Compared to the control group, the mean fold increase of MART-1-specific CD8 T cells in the IMP321 group was respectively >2-, >4- and >6-fold higher at D15, D30 and D60 (P < 0.02). Long-lasting MART-1-specific CD8 T-cell responses were significantly associated with IMP321 (P < 0.02). At the peak of the response, MART-1-specific CD8 T cells contained higher proportions of effector (CCR7⁻ CD45RA⁺/⁻) cells in the IMP321 group (P < 0.02) and showed no sign of exhaustion (i.e. were mostly PD1⁻CD160⁻TIM3⁻LAG3⁻2B4⁺/⁻). Moreover, IMP321 was associated with a significantly reduced expansion of regulatory T cells (P < 0.04); consistently, we observed a negative correlation between the relative expansion of MART-1-specific CD8 T cells and of regulatory T cells. Finally, although there were no confirmed responses as per RECIST criteria, a transient, 30-day partial response was observed in a patient from the IMP321 group. CONCLUSIONS: Vaccination with IMP321 as an adjuvant in combination with lymphodepleting chemotherapy and adoptive transfer of autologous PBMCs induced more robust and durable cellular antitumor immune responses, supporting further development of IMP321 as an adjuvant for future immunotherapeutic strategies.

Reversible behavioural autistic-like regression: A manifestation of a special (new?) epileptic syndrome in a 28-month-old child. A 2-year longitudinal study .

A 28-month-old boy was referred for acute onset of abnormal head movements. History revealed an insidious progressive regression in behaviour and communication over several months. Head and shoulder 'spasms' with alteration of consciousness and on one occasion ictal laughter were seen. The electroencephalograph (EEG) showed repeated bursts of brief generalized polyspikes and spike-wave during the 'spasms', followed by flattening, a special pattern which never recurred after treatment. Review of family videos showed a single 'minor' identical seizure 6 months previously. Magnetic resonance imaging was normal. Clonazepam brought immediate cessation of seizures, normalization of the EEG and a parallel spectacular improvement in communication, mood and language. Follow-up over the next 10 months showed a new regression unaccompained by recognized seizures, although numerous seizures were discovered during the videotaped neuropsychological examination, when stereotyped subtle brief paroxysmal changes in posture and behaviour could be studied in slow motion and compared with the 'prototypical' initial ones. The EEG showed predominant rare left-sided fronto-temporal discharges. Clonazepam was changed to carbamazepin with marked improvement in behaviour, language and cognition which has been sustained up to the last control at 51 months. Videotaped home observations allowed the documentation of striking qualitative and quantitative variations in social interaction and play of autistic type in relation to the epileptic activity. We conclude that this child has a special characteristic epileptic syndrome with subtle motor and vegetative symptomatology associated with an insidious catastrophic 'autistic-like' regression which could be overlooked. The methods used to document such fluctuating epileptic behavioural manifestations are discussed.

A robust genomic signature for the detection of colorectal cancer patients with microsatellite instability phenotype and high mutation frequency.

Microsatellite instability (MSI) occurs in 10-20% of colorectal tumours and is associated with good prognosis. Here we describe the development and validation of a genomic signature that identifies colorectal cancer patients with MSI caused by DNA mismatch repair deficiency with high accuracy. Microsatellite status for 276 stage II and III colorectal tumours has been determined. Full-genome expression data was used to identify genes that correlate with MSI status. A subset of these samples (n = 73) had sequencing data for 615 genes available. An MSI gene signature of 64 genes was developed and validated in two independent validation sets: the first consisting of frozen samples from 132 stage II patients; and the second consisting of FFPE samples from the PETACC-3 trial (n = 625). The 64-gene MSI signature identified MSI patients in the first validation set with a sensitivity of 90.3% and an overall accuracy of 84.8%, with an AUC of 0.942 (95% CI, 0.888-0.975). In the second validation, the signature also showed excellent performance, with a sensitivity 94.3% and an overall accuracy of 90.6%, with an AUC of 0.965 (95% CI, 0.943-0.988). Besides correct identification of MSI patients, the gene signature identified a group of MSI-like patients that were MSS by standard assessment but MSI by signature assessment. The MSI-signature could be linked to a deficient MMR phenotype, as both MSI and MSI-like patients showed a high mutation frequency (8.2% and 6.4% of 615 genes assayed, respectively) as compared to patients classified as MSS (1.6% mutation frequency). The MSI signature showed prognostic power in stage II patients (n = 215) with a hazard ratio of 0.252 (p = 0.0145). Patients with an MSI-like phenotype had also an improved survival when compared to MSS patients. The MSI signature was translated to a diagnostic microarray and technically and clinically validated in FFPE and frozen samples.

Robust discrimination between EEG responses to categories of environmental sounds in early coma.

Humans can recognize categories of environmental sounds, including vocalizations produced by humans and animals and the sounds of man-made objects. Most neuroimaging investigations of environmental sound discrimination have studied subjects while consciously perceiving and often explicitly recognizing the stimuli. Consequently, it remains unclear to what extent auditory object processing occurs independently of task demands and consciousness. Studies in animal models have shown that environmental sound discrimination at a neural level persists even in anesthetized preparations, whereas data from anesthetized humans has thus far provided null results. Here, we studied comatose patients as a model of environmental sound discrimination capacities during unconsciousness. We included 19 comatose patients treated with therapeutic hypothermia (TH) during the first 2 days of coma, while recording nineteen-channel electroencephalography (EEG). At the level of each individual patient, we applied a decoding algorithm to quantify the differential EEG responses to human vs. animal vocalizations as well as to sounds of living vocalizations vs. man-made objects. Discrimination between vocalization types was accurate in 11 patients and discrimination between sounds from living and man-made sources in 10 patients. At the group level, the results were significant only for the comparison between vocalization types. These results lay the groundwork for disentangling truly preferential activations in response to auditory categories, and the contribution of awareness to auditory category discrimination.

Evaluating the influence of spatial uncertainty in locality points for species distributional modeling

1. Species distribution modelling is used increasingly in both applied and theoretical research to predict how species are distributed and to understand attributes of species' environmental requirements. In species distribution modelling, various statistical methods are used that combine species occurrence data with environmental spatial data layers to predict the suitability of any site for that species. While the number of data sharing initiatives involving species' occurrences in the scientific community has increased dramatically over the past few years, various data quality and methodological concerns related to using these data for species distribution modelling have not been addressed adequately. 2. We evaluated how uncertainty in georeferences and associated locational error in occurrences influence species distribution modelling using two treatments: (1) a control treatment where models were calibrated with original, accurate data and (2) an error treatment where data were first degraded spatially to simulate locational error. To incorporate error into the coordinates, we moved each coordinate with a random number drawn from the normal distribution with a mean of zero and a standard deviation of 5 km. We evaluated the influence of error on the performance of 10 commonly used distributional modelling techniques applied to 40 species in four distinct geographical regions. 3. Locational error in occurrences reduced model performance in three of these regions; relatively accurate predictions of species distributions were possible for most species, even with degraded occurrences. Two species distribution modelling techniques, boosted regression trees and maximum entropy, were the best performing models in the face of locational errors. The results obtained with boosted regression trees were only slightly degraded by errors in location, and the results obtained with the maximum entropy approach were not affected by such errors. 4. Synthesis and applications. To use the vast array of occurrence data that exists currently for research and management relating to the geographical ranges of species, modellers need to know the influence of locational error on model quality and whether some modelling techniques are particularly robust to error. We show that certain modelling techniques are particularly robust to a moderate level of locational error and that useful predictions of species distributions can be made even when occurrence data include some error.

The use of classification and regression trees to predict the likelihood of seasonal influenza.

Background Individual signs and symptoms are of limited value for the diagnosis of influenza. Objective To develop a decision tree for the diagnosis of influenza based on a classification and regression tree (CART) analysis. Methods Data from two previous similar cohort studies were assembled into a single dataset. The data were randomly divided into a development set (70%) and a validation set (30%). We used CART analysis to develop three models that maximize the number of patients who do not require diagnostic testing prior to treatment decisions. The validation set was used to evaluate overfitting of the model to the training set. Results Model 1 has seven terminal nodes based on temperature, the onset of symptoms and the presence of chills, cough and myalgia. Model 2 was a simpler tree with only two splits based on temperature and the presence of chills. Model 3 was developed with temperature as a dichotomous variable (≥38°C) and had only two splits based on the presence of fever and myalgia. The area under the receiver operating characteristic curves (AUROCC) for the development and validation sets, respectively, were 0.82 and 0.80 for Model 1, 0.75 and 0.76 for Model 2 and 0.76 and 0.77 for Model 3. Model 2 classified 67% of patients in the validation group into a high- or low-risk group compared with only 38% for Model 1 and 54% for Model 3. Conclusions A simple decision tree (Model 2) classified two-thirds of patients as low or high risk and had an AUROCC of 0.76. After further validation in an independent population, this CART model could support clinical decision making regarding influenza, with low-risk patients requiring no further evaluation for influenza and high-risk patients being candidates for empiric symptomatic or drug therapy.

La régression tumorale n'est pas un facteur de risque d'atteinte du ganglion sentinelle dans les mélanomes fins (indice de Breslow < or = 1 mm) [Tumour regression is not predictive for higher risk of sentinel node involvement in thin melanomas (Breslow thickness < or = 1 mm)]

Background: Thin melanomas (Breslow thickness <= 1 mm) are considered highly curable. The aim of this study was to evaluate the correlation between histological tumour regression and sentinel lymph node (SLN) involvement in thin melanomas. Patients and methods: This was a retrospective single-centre study of 34 patients with thin melanomas undergoing SLN biopsy between April 1998 and January 2005. Results: The study included 14 women and 20 men of mean age 56.3 years. Melanomas were located on the neck (n = 3), soles (n = 4), trunk (n = 13) and extremities (n = 14). Pathological examination showed 25 SSM, four acral lentiginous melanomas, three in situ melanomas, one nodular melanoma and one unclassified melanoma with a mean Breslow thickness of 0.57 mm. Histological tumour regression was observed in 26 over 34 cases and ulceration was found in one case. Clark levels were as follows: I (n = 3), II (n = 20), III (n = 9), IV (n = 2). Growth phase was available in 15 cases (seven radial and eight vertical). Mitotic rates, available in 24 cases, were: 0 (n = 9), 1 (n = 11), 2 (n = 2), 3 (n = 1), 6 (n = 1). One patient with histological tumour regression (2.9% of cases and 3.8% of cases with regressing tumours) had a metastatic SLN. One patient negative for SLN had a lung relapse and died of the disease. Mean follow-up was 26.2 months. Conclusion: The results of the present study and the analysis of the literature show that histological regression of the primary tumour does not seem predictive of higher risk of SLN involvement in thin melanomas. This suggests that screening for SLN is not indicated in thin melanomas, even those with histological regression.