On making causal claims: A review and recommendations

Social scientists often estimate models from correlational data, where the independent variable has not been exogenously manipulated; they also make implicit or explicit causal claims based on these models. When can these claims be made? We answer this question by first discussing design and estimation conditions under which model estimates can be interpreted, using the randomized experiment as the gold standard. We show how endogeneity--which includes omitted variables, omitted selection, simultaneity, common methods bias, and measurement error--renders estimates causally uninterpretable. Second, we present methods that allow researchers to test causal claims in situations where randomization is not possible or when causal interpretation is confounded, including fixed-effects panel, sample selection, instrumental variable, regression discontinuity, and difference-in-differences models. Third, we take stock of the methodological rigor with which causal claims are being made in a social sciences discipline by reviewing a representative sample of 110 articles on leadership published in the previous 10 years in top-tier journals. Our key finding is that researchers fail to address at least 66 % and up to 90 % of design and estimation conditions that make causal claims invalid. We conclude by offering 10 suggestions on how to improve non-experimental research.

Les symbioses agro-industrielles en Afrique de l'Ouest : évaluation du potentiel d'une nouvelle stratégie pour un développement durable

RésuméCette thèse traite de l'utilisation des concepts de Symbiose Industrielle dans les pays en développement et étudie le potentiel de cette stratégie pour stimuler un développement régional durable dans les zones rurales d'Afrique de l'Ouest. En particulier, lorsqu'une Symbiose Industrielle est instaurée entre une usine et sa population alentour, des outils d'évaluation sont nécessaires pour garantir que le projet permette d'atteindre un réel développement durable. Les outils existants, développés dans les pays industrialisés, ne sont cependant pas complètement adaptés pour l'évaluation de projets dans les pays en développement. En effet, les outils sont porteurs d'hypothèses implicites propres au contexte socio-économique dans lequel ils ont été conçus.L'objectif de cette thèse est de développer un cadre méthodologique pour l'évaluation de la durabilité de projets de Symbiose Industrielle dans les pays en développement.Pour ce faire, je m'appuie sur une étude de cas de la mise en place d'une Symbiose Industrielle au nord du Nigéria, à laquelle j'ai participé en tant qu'observatrice dès 2007. AshakaCem, une usine productrice de ciment du groupe Lafarge, doit faire face à de nombreuses tensions avec la population rurale alentour. L'entreprise a donc décidé d'adopter une nouvelle méthode inspirée des concepts de Symbiose Industrielle. Le projet consiste à remplacer jusqu'à 10% du carburant fossile utilisé pour la cuisson de la matière crue (calcaire et additifs) par de la biomasse produite par les paysans locaux. Pour ne pas compromettre la fragile sécurité alimentaire régionale, des techniques de lutte contre l'érosion et de fertilisation naturelle des sols sont enseignées aux paysans, qui peuvent ainsi utiliser la culture de biomasse pour améliorer leurs cultures vivrières. A travers cette Symbiose Industrielle, l'entreprise poursuit des objectifs sociaux (poser les bases nécessaires à un développement régional), mais également environnementaux (réduire ses émissions de CO2 globales) et économiques (réduire ses coûts énergétiques). Elle s'ancre ainsi dans une perspective de développement durable qui est conditionnelle à la réalisation du projet.A travers l'observation de cette Symbiose et par la connaissance des outils existants je constate qu'une évaluation de la durabilité de projets dans les pays en développement nécessite l'utilisation de critères d'évaluation propres à chaque projet. En effet, dans ce contexte, l'emploi de critères génériques apporte une évaluation trop éloignée des besoins et de la réalité locale. C'est pourquoi, en m'inspirant des outils internationalement reconnus comme l'Analyse du Cycle de Vie ou la Global Reporting Initiative, je définis dans cette thèse un cadre méthodologique qui peut, lui, être identique pour tous les projets. Cette stratégie suit six étapes, qui se réalisent de manière itérative pour permettre une auto¬amélioration de la méthodologie d'évaluation et du projet lui-même. Au cours de ces étapes, les besoins et objectifs en termes sociaux, économiques et environnementaux des différents acteurs sont déterminés, puis regroupés, hiérarchisés et formulés sous forme de critères à évaluer. Des indicateurs quantitatifs ou qualitatifs sont ensuite définis pour chacun de ces critères. Une des spécificités de cette stratégie est de définir une échelle d'évaluation en cinq graduations, identique pour chaque indicateur, témoignant d'un objectif totalement atteint (++) ou pas du tout atteint (--).L'application de ce cadre méthodologique à la Symbiose nigériane a permis de déterminer quatre critères économiques, quatre critères socio-économiques et six critères environnementaux à évaluer. Pour les caractériser, 22 indicateurs ont été définis. L'évaluation de ces indicateurs a permis de montrer que le projet élaboré atteint les objectifs de durabilité fixés pour la majorité des critères. Quatre indicateurs ont un résultat neutre (0), et un cinquième montre qu'un critère n'est pas atteint (--). Ces résultats s'expliquent par le fait que le projet n'en est encore qu'à sa phase pilote et n'a donc pas encore atteint la taille et la diffusion optimales. Un suivi sur plusieurs années permettra de garantir que ces manques seront comblés.Le cadre méthodologique que j'ai développé dans cette thèse est un outil d'évaluation participatif qui pourra être utilisé dans un contexte plus large que celui des pays en développement. Son caractère générique en fait un très bon outil pour la définition de critères et indicateurs de suivi de projet en terme de développement durable.SummaryThis thesis examines the use of industrial symbiosis in developing countries and studies its potential to stimulate sustainable regional development in rural areas across Western Africa. In particular, when industrial symbiosis is instituted between a factory and the surrounding population, evaluation tools are required to ensure the project achieves truly sustainable development. Existing tools developed in industrialized countries are not entirely suited to assessing projects in developing countries. Indeed, the implicit hypotheses behind such tools reflect the socioeconomic context in which they were designed. The goal of this thesis is to develop a methodological framework for evaluating the sustainability of industrial symbiosis projects in developing countries.To accomplish this, I followed a case study about the implementation of industrial symbiosis in northern Nigeria by participating as an observer since 2007. AshakaCem, a cement works of Lafarge group, must confront many issues associated with violence committed by the local rural population. Thus, the company decided to adopt a new approach inspired by the concepts of industrial symbiosis.The project involves replacing up to 10% of the fossil fuel used to heat limestone with biomass produced by local farmers. To avoid jeopardizing the fragile security of regional food supplies, farmers are taught ways to combat erosion and naturally fertilize the soil. They can then use biomass cultivation to improve their subsistence crops. Through this industrial symbiosis, AshakaCem follows social objectives (to lay the necessary foundations for regional development), but also environmental ones (to reduce its overall CO2 emissions) and economical ones (to reduce its energy costs). The company is firmly rooted in a view of sustainable development that is conditional upon the project's execution.By observing this symbiosis and by being familiar with existing tools, I note that assessing the sustainability of projects in developing countries requires using evaluation criteria that are specific to each project. Indeed, using generic criteria results in an assessment that is too far removed from what is needed and from the local reality. Thus, by drawing inspiration from such internationally known tools as Life Cycle Analysis and the Global Reporting Initiative, I define a generic methodological framework for the participative establishment of an evaluation methodology specific to each project.The strategy follows six phases that are fulfilled iteratively so as to improve the evaluation methodology and the project itself as it moves forward. During these phases, the social, economic, and environmental needs and objectives of the stakeholders are identified, grouped, ranked, and expressed as criteria for evaluation. Quantitative or qualitative indicators are then defined for each of these criteria. One of the characteristics of this strategy is to define a five-point evaluation scale, the same for each indicator, to reflect a goal that was completely reached (++) or not reached at all (--).Applying the methodological framework to the Nigerian symbiosis yielded four economic criteria, four socioeconomic criteria, and six environmental criteria to assess. A total of 22 indicators were defined to characterize the criteria. Evaluating these indicators made it possible to show that the project meets the sustainability goals set for the majority of criteria. Four indicators had a neutral result (0); a fifth showed that one criterion had not been met (--). These results can be explained by the fact that the project is still only in its pilot phase and, therefore, still has not reached its optimum size and scope. Following up over several years will make it possible to ensure these gaps will be filled.The methodological framework presented in this thesis is a highly effective tool that can be used in a broader context than developing countries. Its generic nature makes it a very good tool for defining criteria and follow-up indicators for sustainable development.

Evaluating the influence of spatial uncertainty in locality points for species distributional modeling

1. Species distribution modelling is used increasingly in both applied and theoretical research to predict how species are distributed and to understand attributes of species' environmental requirements. In species distribution modelling, various statistical methods are used that combine species occurrence data with environmental spatial data layers to predict the suitability of any site for that species. While the number of data sharing initiatives involving species' occurrences in the scientific community has increased dramatically over the past few years, various data quality and methodological concerns related to using these data for species distribution modelling have not been addressed adequately. 2. We evaluated how uncertainty in georeferences and associated locational error in occurrences influence species distribution modelling using two treatments: (1) a control treatment where models were calibrated with original, accurate data and (2) an error treatment where data were first degraded spatially to simulate locational error. To incorporate error into the coordinates, we moved each coordinate with a random number drawn from the normal distribution with a mean of zero and a standard deviation of 5 km. We evaluated the influence of error on the performance of 10 commonly used distributional modelling techniques applied to 40 species in four distinct geographical regions. 3. Locational error in occurrences reduced model performance in three of these regions; relatively accurate predictions of species distributions were possible for most species, even with degraded occurrences. Two species distribution modelling techniques, boosted regression trees and maximum entropy, were the best performing models in the face of locational errors. The results obtained with boosted regression trees were only slightly degraded by errors in location, and the results obtained with the maximum entropy approach were not affected by such errors. 4. Synthesis and applications. To use the vast array of occurrence data that exists currently for research and management relating to the geographical ranges of species, modellers need to know the influence of locational error on model quality and whether some modelling techniques are particularly robust to error. We show that certain modelling techniques are particularly robust to a moderate level of locational error and that useful predictions of species distributions can be made even when occurrence data include some error.

Coronary heart disease in primary care: accuracy of medical history and physical findings in patients with chest pain - a study protocol for a systematic review with individual patient data.

BACKGROUND: Chest pain is a common complaint in primary care, with coronary heart disease (CHD) being the most concerning of many potential causes. Systematic reviews on the sensitivity and specificity of symptoms and signs summarize the evidence about which of them are most useful in making a diagnosis. Previous meta-analyses are dominated by studies of patients referred to specialists. Moreover, as the analysis is typically based on study-level data, the statistical analyses in these reviews are limited while meta-analyses based on individual patient data can provide additional information. Our patient-level meta-analysis has three unique aims. First, we strive to determine the diagnostic accuracy of symptoms and signs for myocardial ischemia in primary care. Second, we investigate associations between study- or patient-level characteristics and measures of diagnostic accuracy. Third, we aim to validate existing clinical prediction rules for diagnosing myocardial ischemia in primary care. This article describes the methods of our study and six prospective studies of primary care patients with chest pain. Later articles will describe the main results. METHODS/DESIGN: We will conduct a systematic review and IPD meta-analysis of studies evaluating the diagnostic accuracy of symptoms and signs for diagnosing coronary heart disease in primary care. We will perform bivariate analyses to determine the sensitivity, specificity and likelihood ratios of individual symptoms and signs and multivariate analyses to explore the diagnostic value of an optimal combination of all symptoms and signs based on all data of all studies. We will validate existing clinical prediction rules from each of the included studies by calculating measures of diagnostic accuracy separately by study. DISCUSSION: Our study will face several methodological challenges. First, the number of studies will be limited. Second, the investigators of original studies defined some outcomes and predictors differently. Third, the studies did not collect the same standard clinical data set. Fourth, missing data, varying from partly missing to fully missing, will have to be dealt with.Despite these limitations, we aim to summarize the available evidence regarding the diagnostic accuracy of symptoms and signs for diagnosing CHD in patients presenting with chest pain in primary care. REVIEW REGISTRATION: Centre for Reviews and Dissemination (University of York): CRD42011001170.