On making causal claims: A review and recommendations

Social scientists often estimate models from correlational data, where the independent variable has not been exogenously manipulated; they also make implicit or explicit causal claims based on these models. When can these claims be made? We answer this question by first discussing design and estimation conditions under which model estimates can be interpreted, using the randomized experiment as the gold standard. We show how endogeneity--which includes omitted variables, omitted selection, simultaneity, common methods bias, and measurement error--renders estimates causally uninterpretable. Second, we present methods that allow researchers to test causal claims in situations where randomization is not possible or when causal interpretation is confounded, including fixed-effects panel, sample selection, instrumental variable, regression discontinuity, and difference-in-differences models. Third, we take stock of the methodological rigor with which causal claims are being made in a social sciences discipline by reviewing a representative sample of 110 articles on leadership published in the previous 10 years in top-tier journals. Our key finding is that researchers fail to address at least 66 % and up to 90 % of design and estimation conditions that make causal claims invalid. We conclude by offering 10 suggestions on how to improve non-experimental research.

On making causal claims: A review and recommendations

Social scientists often estimate models from correlational data, where the independent variable has not been exogenously manipulated; they also make implicit or explicit causal claims based on these models. When can these claims be made? We answer this question by first discussing design and estimation conditions under which model estimates can be interpreted, using the randomized experiment as the gold standard. We show how endogeneity--which includes omitted variables, omitted selection, simultaneity, common methods bias, and measurement error--renders estimates causally uninterpretable. Second, we present methods that allow researchers to test causal claims in situations where randomization is not possible or when causal interpretation is confounded, including fixed-effects panel, sample selection, instrumental variable, regression discontinuity, and difference-in-differences models. Third, we take stock of the methodological rigor with which causal claims are being made in a social sciences discipline by reviewing a representative sample of 110 articles on leadership published in the previous 10 years in top-tier journals. Our key finding is that researchers fail to address at least 66 % and up to 90 % of design and estimation conditions that make causal claims invalid. We conclude by offering 10 suggestions on how to improve non-experimental research.

The use of classification and regression trees to predict the likelihood of seasonal influenza.

Background Individual signs and symptoms are of limited value for the diagnosis of influenza. Objective To develop a decision tree for the diagnosis of influenza based on a classification and regression tree (CART) analysis. Methods Data from two previous similar cohort studies were assembled into a single dataset. The data were randomly divided into a development set (70%) and a validation set (30%). We used CART analysis to develop three models that maximize the number of patients who do not require diagnostic testing prior to treatment decisions. The validation set was used to evaluate overfitting of the model to the training set. Results Model 1 has seven terminal nodes based on temperature, the onset of symptoms and the presence of chills, cough and myalgia. Model 2 was a simpler tree with only two splits based on temperature and the presence of chills. Model 3 was developed with temperature as a dichotomous variable (≥38°C) and had only two splits based on the presence of fever and myalgia. The area under the receiver operating characteristic curves (AUROCC) for the development and validation sets, respectively, were 0.82 and 0.80 for Model 1, 0.75 and 0.76 for Model 2 and 0.76 and 0.77 for Model 3. Model 2 classified 67% of patients in the validation group into a high- or low-risk group compared with only 38% for Model 1 and 54% for Model 3. Conclusions A simple decision tree (Model 2) classified two-thirds of patients as low or high risk and had an AUROCC of 0.76. After further validation in an independent population, this CART model could support clinical decision making regarding influenza, with low-risk patients requiring no further evaluation for influenza and high-risk patients being candidates for empiric symptomatic or drug therapy.

Assessing the Relationship between the Baseline Value of a Continuous Variable and Subsequent Change over Time

Analyzing the relationship between the baseline value and subsequent change of a continuous variable is a frequent matter of inquiry in cohort studies. These analyses are surprisingly complex, particularly if only two waves of data are available. It is unclear for non-biostatisticians where the complexity of this analysis lies and which statistical method is adequate.With the help of simulated longitudinal data of body mass index in children,we review statistical methods for the analysis of the association between the baseline value and subsequent change, assuming linear growth with time. Key issues in such analyses are mathematical coupling, measurement error, variability of change between individuals, and regression to the mean. Ideally, it is better to rely on multiple repeated measurements at different times and a linear random effects model is a standard approach if more than two waves of data are available. If only two waves of data are available, our simulations show that Blomqvist's method - which consists in adjusting for measurement error variance the estimated regression coefficient of observed change on baseline value - provides accurate estimates. The adequacy of the methods to assess the relationship between the baseline value and subsequent change depends on the number of data waves, the availability of information on measurement error, and the variability of change between individuals.

Behaviour evaluation for risk-taking adolescents (BERTA): an easy to use and assess instrument to detect adolescent risky behaviours in a clinical setting.

To create an instrument to be used in an outpatient clinic to detect adolescents prone to risk-taking behaviours. Based on previous research, five identified variables (relationship with parents and teachers, liking going to school, average grades, and level of religiosity) were used to create a screening tool to detect at least one of ten risky behaviours (tobacco, alcohol, cannabis and other illegal drugs use; sexual intercourse and sexual risky behaviour; driving while intoxicated, riding with an intoxicated driver, not always using a seat belt, and not always using a helmet). The instrument was tested using the Barcelona Adolescent Health Survey 1993. A Receiver Operating Characteristics curve was used to find the best cut-off point between high and low risk score. Odds ratios and 95% confidence intervals were calculated to detect at least one risky behaviour and for each individual behaviour. In order to assess its predictive value, the analysis was repeated using the Barcelona Adolescent Health Survey 1999. In both cases, analyses were conducted for the whole sample and for younger and older adolescents. Adolescents with a high-risk score were more likely to take at least one risky behaviour both when the whole sample was analysed and by age groups. With very few exceptions, the Behaviour Evaluation for Risk-Taking Adolescents showed significant odds ratios for each individual variable. CONCLUSION: The Behaviour Evaluation for Risk-Taking Adolescents has shown its potential as an easy to use instrument to screen for risk-taking behaviours. Future research must aim towards assessing this instrument's predictive value in the clinical setting and it's application to other populations.

Characteristics of eyes with secondary loss of visual acuity receiving variable dosing ranibizumab for neovascular age-related macular degeneration.

PURPOSE: The aim of this work is to investigate the characteristics of eyes failing to maintain visual acuity (VA) receiving variable dosing ranibizumab for neovascular age-related macular degeneration (nAMD) after three initial loading doses. METHODS: A consecutive series of patients with nAMD, who, after three loading doses of intravitreal ranibizumab (0.5 mg each), were re-treated for fluid seen on optical coherence tomography. After exclusion of eyes with previous treatment, follow-up less than 12 months, or missed visits, 99 patients were included in the analysis. The influence of baseline characteristics, initial VA response, and central retinal thickness (CRT) fluctuations on the VA stability from month 3 to month 24 were analyzed using subgroups and multiple regression analyses. RESULTS: Mean follow-up duration was 21.3 months (range 12-40 months, 32 patients followed-up for ≥24 months). Secondary loss of VA (loss of five letters or more) after month 3 was seen in 30 patients (mean VA improvement from baseline +5.8 letters at month 3, mean loss from baseline -5.3 letters at month 12 and -9.7 at final visit up to month 24), while 69 patients maintained vision (mean gain +8.9 letters at month 3, +10.4 letters at month 12, and +12.8 letters at final visit up to month 24). Secondary loss of VA was associated with the presence of pigment epithelial detachment (PED) at baseline (p 0.01), but not with baseline fibrosis/atrophy/hemorrhage, CRT fluctuations, or initial VA response. Chart analysis revealed additional individual explanations for the secondary loss of VA, including retinal pigment epithelial tears, progressive fibrosis, and atrophy. CONCLUSIONS: Tissue damage due to degeneration of PED, retinal pigment epithelial tears, progressive fibrosis, progressive atrophy, or massive hemorrhage, appears to be relevant in causing secondary loss of VA despite vascular endothelial growth factor suppression. PED at baseline may represent a risk factor.

Improving generalized regression analysis for the spatial prediction of forest communities

Aim This study used data from temperate forest communities to assess: (1) five different stepwise selection methods with generalized additive models, (2) the effect of weighting absences to ensure a prevalence of 0.5, (3) the effect of limiting absences beyond the environmental envelope defined by presences, (4) four different methods for incorporating spatial autocorrelation, and (5) the effect of integrating an interaction factor defined by a regression tree on the residuals of an initial environmental model. Location State of Vaud, western Switzerland. Methods Generalized additive models (GAMs) were fitted using the grasp package (generalized regression analysis and spatial predictions, http://www.cscf.ch/grasp). Results Model selection based on cross-validation appeared to be the best compromise between model stability and performance (parsimony) among the five methods tested. Weighting absences returned models that perform better than models fitted with the original sample prevalence. This appeared to be mainly due to the impact of very low prevalence values on evaluation statistics. Removing zeroes beyond the range of presences on main environmental gradients changed the set of selected predictors, and potentially their response curve shape. Moreover, removing zeroes slightly improved model performance and stability when compared with the baseline model on the same data set. Incorporating a spatial trend predictor improved model performance and stability significantly. Even better models were obtained when including local spatial autocorrelation. A novel approach to include interactions proved to be an efficient way to account for interactions between all predictors at once. Main conclusions Models and spatial predictions of 18 forest communities were significantly improved by using either: (1) cross-validation as a model selection method, (2) weighted absences, (3) limited absences, (4) predictors accounting for spatial autocorrelation, or (5) a factor variable accounting for interactions between all predictors. The final choice of model strategy should depend on the nature of the available data and the specific study aims. Statistical evaluation is useful in searching for the best modelling practice. However, one should not neglect to consider the shapes and interpretability of response curves, as well as the resulting spatial predictions in the final assessment.

Random measurement error and regression dilution bias.

Summary points: - The bias introduced by random measurement error will be different depending on whether the error is in an exposure variable (risk factor) or outcome variable (disease) - Random measurement error in an exposure variable will bias the estimates of regression slope coefficients towards the null - Random measurement error in an outcome variable will instead increase the standard error of the estimates and widen the corresponding confidence intervals, making results less likely to be statistically significant - Increasing sample size will help minimise the impact of measurement error in an outcome variable but will only make estimates more precisely wrong when the error is in an exposure variable

A comparison of three methods of Mendelian randomization when the genetic instrument, the risk factor and the outcome are all binary.

The method of instrumental variable (referred to as Mendelian randomization when the instrument is a genetic variant) has been initially developed to infer on a causal effect of a risk factor on some outcome of interest in a linear model. Adapting this method to nonlinear models, however, is known to be problematic. In this paper, we consider the simple case when the genetic instrument, the risk factor, and the outcome are all binary. We compare via simulations the usual two-stages estimate of a causal odds-ratio and its adjusted version with a recently proposed estimate in the context of a clinical trial with noncompliance. In contrast to the former two, we confirm that the latter is (under some conditions) a valid estimate of a causal odds-ratio defined in the subpopulation of compliers, and we propose its use in the context of Mendelian randomization. By analogy with a clinical trial with noncompliance, compliers are those individuals for whom the presence/absence of the risk factor X is determined by the presence/absence of the genetic variant Z (i.e., for whom we would observe X = Z whatever the alleles randomly received at conception). We also recall and illustrate the huge variability of instrumental variable estimates when the instrument is weak (i.e., with a low percentage of compliers, as is typically the case with genetic instruments for which this proportion is frequently smaller than 10%) where the inter-quartile range of our simulated estimates was up to 18 times higher compared to a conventional (e.g., intention-to-treat) approach. We thus conclude that the need to find stronger instruments is probably as important as the need to develop a methodology allowing to consistently estimate a causal odds-ratio.

Spatial prediction of monthly wind speeds in complex terrain with adaptive general regression neural networks

This paper presents the general regression neural networks (GRNN) as a nonlinear regression method for the interpolation of monthly wind speeds in complex Alpine orography. GRNN is trained using data coming from Swiss meteorological networks to learn the statistical relationship between topographic features and wind speed. The terrain convexity, slope and exposure are considered by extracting features from the digital elevation model at different spatial scales using specialised convolution filters. A database of gridded monthly wind speeds is then constructed by applying GRNN in prediction mode during the period 1968-2008. This study demonstrates that using topographic features as inputs in GRNN significantly reduces cross-validation errors with respect to low-dimensional models integrating only geographical coordinates and terrain height for the interpolation of wind speed. The spatial predictability of wind speed is found to be lower in summer than in winter due to more complex and weaker wind-topography relationships. The relevance of these relationships is studied using an adaptive version of the GRNN algorithm which allows to select the useful terrain features by eliminating the noisy ones. This research provides a framework for extending the low-dimensional interpolation models to high-dimensional spaces by integrating additional features accounting for the topographic conditions at multiple spatial scales. Copyright (c) 2012 Royal Meteorological Society.

Improved predictive mapping of indoor radon concentrations using ensemble regression trees based on automatic clustering of geological units.

PURPOSE: According to estimations around 230 people die as a result of radon exposure in Switzerland. This public health concern makes reliable indoor radon prediction and mapping methods necessary in order to improve risk communication to the public. The aim of this study was to develop an automated method to classify lithological units according to their radon characteristics and to develop mapping and predictive tools in order to improve local radon prediction. METHOD: About 240 000 indoor radon concentration (IRC) measurements in about 150 000 buildings were available for our analysis. The automated classification of lithological units was based on k-medoids clustering via pair-wise Kolmogorov distances between IRC distributions of lithological units. For IRC mapping and prediction we used random forests and Bayesian additive regression trees (BART). RESULTS: The automated classification groups lithological units well in terms of their IRC characteristics. Especially the IRC differences in metamorphic rocks like gneiss are well revealed by this method. The maps produced by random forests soundly represent the regional difference of IRCs in Switzerland and improve the spatial detail compared to existing approaches. We could explain 33% of the variations in IRC data with random forests. Additionally, the influence of a variable evaluated by random forests shows that building characteristics are less important predictors for IRCs than spatial/geological influences. BART could explain 29% of IRC variability and produced maps that indicate the prediction uncertainty. CONCLUSION: Ensemble regression trees are a powerful tool to model and understand the multidimensional influences on IRCs. Automatic clustering of lithological units complements this method by facilitating the interpretation of radon properties of rock types. This study provides an important element for radon risk communication. Future approaches should consider taking into account further variables like soil gas radon measurements as well as more detailed geological information.

T Cells Bearing a Chimeric Antigen Receptor against Prostate-Specific Membrane Antigen Mediate Vascular Disruption and Result in Tumor Regression.

Aberrant blood vessels enable tumor growth, provide a barrier to immune infiltration, and serve as a source of protumorigenic signals. Targeting tumor blood vessels for destruction, or tumor vascular disruption therapy, can therefore provide significant therapeutic benefit. Here, we describe the ability of chimeric antigen receptor (CAR)-bearing T cells to recognize human prostate-specific membrane antigen (hPSMA) on endothelial targets in vitro as well as in vivo. CAR T cells were generated using the anti-PSMA scFv, J591, and the intracellular signaling domains: CD3ζ, CD28, and/or CD137/4-1BB. We found that all anti-hPSMA CAR T cells recognized and eliminated PSMA(+) endothelial targets in vitro, regardless of the signaling domain. T cells bearing the third-generation anti-hPSMA CAR, P28BBζ, were able to recognize and kill primary human endothelial cells isolated from gynecologic cancers. In addition, the P28BBζ CAR T cells mediated regression of hPSMA-expressing vascular neoplasms in mice. Finally, in murine models of ovarian cancers populated by murine vessels expressing hPSMA, the P28BBζ CAR T cells were able to ablate PSMA(+) vessels, cause secondary depletion of tumor cells, and reduce tumor burden. Taken together, these results provide a strong rationale for the use of CAR T cells as agents of tumor vascular disruption, specifically those targeting PSMA. Cancer Immunol Res; 3(1); 68-84. ©2014 AACR.

Giant warts in a kidney transplant patient: regression with sirolimus.

Purpose: Recent reports have suggested that intraabdominal postoperative infection is associated with higher rates of overall and local recurrence and cancer-specific mortality. However, the mechanisms responsible for this association are unknown. We hypothesized that the greater inflammatory response in patients with postoperative intraabdominal infection is associated to an increase in local and systemic angiogenesis. Methods: We designed a prospective cohorts study with matched controls. Patients with postoperative intra-abdominal infection (abscess and/or anastomotic leakage) (group 1; n=17) after elective colorectal cancer resection operated on for cure were compared to patients with an uncomplicated postoperative course (group 2; n=17). IL-6 and VEGF levels were determined by ELISA in serum and peritoneal fluid at baseline, 48 hours and postoperative day 4 or at the time the peritoneal infection occurred. Results: No differences were observed in age, gender, preoperative CEA, tumor stage and location and type of procedure performed. Although there were no differences in serum IL-6 levels at 48 hours, this pro-inflammatory cytokine was higher in group 1 on postoperative day 4 (group 1: 21533 + 27900 vs. group 2: 1130 + 3563 pg/ml; p < 0.001). Serum VEGF levels were higher in group 1 on postoperative day 4 (group 1: 1212 + 1025 vs. group 2: 408 + 407 pg/ml; p < 0.01). Peritoneal fluid VEGF levels were also higher in group 1 at 48 hours (group 1: 4857 + 4384 vs. group 2: 630 + 461 pg/ml; p < 0.001) and postoperative day 4 (group 1: 32807 + 98486 vs. group 2: 1002 + 1229 pg/ml; p < 0.001). A positive correlation between serum IL-6 and VEGF serum levels was observed on postoperative day 4 (r=0.7; p<0.01). Conclusions: These results suggest that not only the inflammatory response but also the angiogenic pathways are stimulated in patients with intra-abdominal infection after surgery for colorectal cancer. The implications of this finding on long-term follow-up need to be evaluated.

Long-term outcome after cognitive and behavioral regression in nonlesional epilepsy with continuous spike-waves during slow-wave sleep.

PURPOSE: To present the long-term follow-up of 10 adolescents and young adults with documented cognitive and behavioral regression as children due to nonlesional focal, mainly frontal, epilepsy with continuous spike-waves during slow wave sleep (CSWS). METHODS: Past medical and electroencephalography (EEG) data were reviewed and neuropsychological tests exploring main cognitive functions were administered. KEY FINDINGS: After a mean duration of follow-up of 15.6 years (range, 8-23 years), none of the 10 patients had recovered fully, but four regained borderline to normal intelligence and were almost independent. Patients with prolonged global intellectual regression had the worst outcome, whereas those with more specific and short-lived deficits recovered best. The marked behavioral disorders resolved in all but one patient. Executive functions were neither severely nor homogenously affected. Three patients with a frontal syndrome during the active phase (AP) disclosed only mild residual executive and social cognition deficits. The main cognitive gains occurred shortly after the AP, but qualitative improvements continued to occur. Long-term outcome correlated best with duration of CSWS. SIGNIFICANCE: Our findings emphasize that cognitive recovery after cessation of CSWS depends on the severity and duration of the initial regression. None of our patients had major executive and social cognition deficits with preserved intelligence, as reported in adults with early destructive lesions of the frontal lobes. Early recognition of epilepsy with CSWS and rapid introduction of effective therapy are crucial for a best possible outcome.