The External validity in economic experiments

One of the most important questions regarding experimental economics is the external validity of laboratory experiments. This paper goes through a study that tests the generalizability of a Dictator Game as a laboratory analogue for a naturally occurringdecision-making context of teacher absenteeism. Because lab and naturally-occurring environments systematically differ we then discuss other factors that might strongly affect the choices that individuals make. We conclude that the dichotomy drawn between labexperiments and data collected from natural settings is a false one. A combination of the two would provide deeper and better insights than either separately.

On the external validity of laboratory experiments

Thèse numérisée par la Division de la gestion de documents et des archives de l'Université de Montréal

Sociality and external validity in experimental economics

It is sometimes argued that experimental economists do not have to worry about external validity so long as the design sticks closely to a theoretical model. This position mistakes the model for the theory. As a result, applied economics designs often study phenomena distinct from their stated objects of inquiry. Because the implemented models are abstract, they may provide improbable analogues to their stated subject matter. This problem is exacerbated by the relational character of the social world, which also sets epistemic limits for the social science laboratory more generally.

How to assess the external validity of therapeutic trials: a conceptual approach

Background External validity of study results is an important issue from a clinical point of view. From a methodological point of view, however, the concept of external validity is more complex than it seems to be at first glance. Methods Methodological review to address the concept of external validity. Results External validity refers to the question whether results are generalizable to persons other than the population in the original study. The only formal way to establish the external validity would be to repeat the study for that specific target population. We propose a three-way approach for assessing the external validity for specified target populations. (i) The study population might not be representative for the eligibility criteria that were intended. It should be addressed whether the study population differs from the intended source population with respect to characteristics that influence outcome. (ii) The target population will, by definition, differ from the study population with respect to geographical, temporal and ethnical conditions. Pondering external validity means asking the question whether these differences may influence study results. (iii) It should be assessed whether the study's conclusions can be generalized to target populations that do not meet all the eligibility criteria. Conclusion Judging the external validity of study results cannot be done by applying given eligibility criteria to a single target population. Rather, it is a complex reflection in which prior knowledge, statistical considerations, biological plausibility and eligibility criteria all have place.

The 12-Item General Health Questionnaire (GHQ-12): Reliability,external validity and factor structure in the Spanish population

The purpose of this study was to analyze the internal consistency and the external and structure validity of the 12-Item General Health Questionnaire (GHQ-12) in the Spanish general population. A stratified sample of 1001 subjects, ages between 25 and 65 years, taken from the general Spanish population was employed. The GHQ-12 and the Inventory of Situations and Responses of Anxiety-ISRA were administered. A Cronbach’s alpha of .76 (Standardized Alpha: .78) and a 3-factor structure (with oblique rotation and maximum likelihood procedure) were obtained. External validity of Factor I (Successful Coping) with the ISRA is very robust (.82; Factor II, .70; Factor III, .75). The GHQ-12 shows adequate reliability and validity in the Spanish population. Therefore, the GHQ-12 can be used with efficacy to assess people’s overall psychological well-being and to detect non-psychotic psychiatric problems. Additionally, our results confirm that the GHQ-12 can best be thought of as a multidimensional scale that assesses several distinct aspects of distress, rather than just a unitary screening measure.

Evaluating the external validity of comorbid disorders in youth using rating scales and behavior observations

To evaluate the theoretical underpinnings of current categorical approaches to classify childhood psychopathological conditions, this dissertation examined whether children with a single diagnosis of an anxiety disorder (ANX only) and children with an anxiety diagnosis comorbid with other diagnoses (i.e., anxiety + anxiety disorder [ANX + ANX], anxiety + depressive disorder [ANX + DEP], and anxiety + disruptive disorder [ANX + EXT]) could be differentiated using external validation criteria of clinical phenomenology (i.e., levels of anxiety, depression, and internalizing, externalizing and total behavior problems). This study further examined whether the four groups could be differentiated in terms of their interaction patterns with their parents and peers, respectively. The sample consisted of 129 youth and their parents who presented to the Child Anxiety and Phobia Program (CAPP) housed within the Child and Family Psychosocial Research Center at Florida International University, Miami. Youth were between the ages of 8 and 14 years old. A battery of questionnaires was used to assess participants' clinical presentation in terms of levels of anxiety, depression, and internalizing and externalizing symptoms. Family and peer interaction were evaluated through rating scales and through behavior observation tasks. Statistics based on the parameter estimates of the structured equation models indicated that all the comorbid groups were significantly different from the pure anxiety disorder group when it came to depression indices of clinical phenomenology. Further, significant differences appeared mainly in terms of the ANX + DEP comorbid group relative to the other comorbid groups. In terms of Parent-child interaction the ANX + EXT and the ANX + DEP comorbid groups were differentiated from the pure anxiety disorder and ANX + ANX comorbid group when it came to the appraisal of the parent/child relationship by the parent, and the acceptance subscale according to the mother report. In terms of peer-child interaction the ANX + EXT and the ANX + DEP comorbid groups were statistically significantly different from the pure anxiety disorder only when it came to the positive interactions and the social skills as rated by mother. Limitations and future research recommendations are discussed.

External validation study of a clinical prediction rule for ambulation outcomes after traumatic spinal cord injury

The main objective of this study was to determine the external validity of a clinical prediction rule developed by the European Multicenter Study on Human Spinal Cord Injury (EM-SCI) to predict the ambulation outcomes 12 months after traumatic spinal cord injury. Data from the North American Clinical Trials Network (NACTN) data registry with approximately 500 SCI cases were used for this validity study. The predictive accuracy of the EM-SCI prognostic model was evaluated using calibration and discrimination based on 231 NACTN cases. The area under the receiver-operating-characteristics curve (ROC) curve was 0.927 (95% CI 0.894 – 0.959) for the EM-SCI model when applied to NACTN population. This is lower than the AUC of 0.956 (95% CI 0.936 – 0.976) reported for the EM-SCI population, but suggests that the EM-SCI clinical prediction rule distinguished well between those patients in the NACTN population who were able to achieve independent ambulation and those who did not achieve independent ambulation. The calibration curve suggests that higher the prediction score is, the better the probability of walking with the best prediction for AIS D patients. In conclusion, the EM-SCI clinical prediction rule was determined to be generalizable to the adult NACTN SCI population.^

Quality of Reporting of Observational Longitudinal Research

Observational longitudinal research is particularly useful for assessing etiology and prognosis and for providing evidence for clinical decision making. However, there are no structured reporting requirements for studies of this design to assist authors, editors, and readers. The authors developed and tested a checklist of criteria related to threats to the internal and external validity of observational longitudinal studies. The checklist criteria concerned recruitment, data collection, biases, and data analysis and descriptive issues relevant to study rationale, study population, and generalizability. Two raters independently assessed 49 randomly selected articles describing stroke research published from 1999 to 2003 in six journals: American Journal of Epidemiology, Journal of Epidemiology and Community Health, Stroke, Annals of Neurology, Archives of Physical Medicine and Rehabilitation, and American Journal of Physical Medicine and Rehabilitation. On average, 17 of the 33 checklist criteria were reported. Criteria describing the study design were better reported than those related to internal validity. No relation was found between study type (etiologic or prognostic) or word count and quality of reporting. A flow diagram for summarizing participant flow through a study was developed. Editors and authors should consider using a checklist and flow diagram when reporting on observational longitudinal research.

Systematic reviews on leptospirosis

OBJECTIVES: To find the existing clinical evidence on interventions for leptospirosis. The objective is to evaluate the effectiveness and safety of any intervention on leptospirosis through systematic reviews of randomized controlled trials (RCTs). DATA SOURCE: The sources of studies used (where there were no limitations concerning language, date, or other restrictions) were: EMBASE, LILACS, MEDLINE, the Cochrane Controlled Clinical Trials Database, and the Cochrane Hepato-Biliary Group Randomized Trials register. SELECTION OF STUDIES: Type of Study: All systematic reviews of randomized controlled trials. Participants: patients with clinical and/or laboratorial diagnosis of leptospirosis, and subjects potencially exposed to leptospirosis as defined by the authors Interventions: any intervention for leptospirosis (as antibiotics or vaccines for prevention or treatment). DATA COLLECTION: The assessment will be independently made by the reviewers and cross-checked. The external validity was assessed by analysis of: studies, interventions, and outcomes. DATA SYNTHESIS: Located 163 studies using the search strategy described above, at the electronic databases above. Only 2 hits were selected, which are protocols of systematic reviews of Cochrane Collaboration, and not full reviews. One of the protocols evaluates antibiotics for treatment, and the other evaluates antibiotics for prevention of leptospirosis. CONCLUSIONS: There were not complete systematic reviews on interventions for leptospirosis. Any interventions for leptospirosis, such as prevention and treatment remains unclear for guidelines and practice.

Web-assisted tobacco intervention in Portuguese : intentions to make behavioural changes and behavioural changes

ABSTRACT - The problem of how to support “intentions to make behavioural changes” (IBC) and “behaviour changes” (BC) in smoking cessation when there is a scarcity of resources is a pressing issue in public health terms. The present research focuses on the use of information and communications technologies and their role in smoking cessation. It is developed in Portugal after the ratification of WHO Framework Convention on Tobacco Control (on 8 November 2005). The prevalence of smokers over fifteen years of age within the population stood at 20.9% (30.9% for men and 11.8% for women). While the strategy of helping people to quit smoking has been emphasised at National Health Service (NHS) level, the uptake of cessation assistance has exceeded the capacity of the service. This induced the search of new theoretical and practical venues to offer alternative options to people willing to stop smoking. Among these, the National Health Plan (NHP) of Portugal (2004-2010), identifies the use of information technologies in smoking cessation. eHealth and the importance of health literacy as a means of empowering people to make behavioural changes is recurrently considered an option worth investigating. The overall objective of this research is to understand, in the Portuguese context, the use of the Internet to help people to stop smoking. Research questions consider factors that may contribute to “intentions to make behavioural changes” (IBC) and “behavioural changes” (BC) while using a Web-Assisted Tobacco Intervention Probe (WATIP). Also consideration is given to the trade-off on the use of the Web as a tool for smoking cessation: can it reach a vast number of people for a small cost (efficiency) demonstrating to work in the domain of smoking cessation (efficacy)”? In addition to the introduction, there is a second chapter in which the use of tobacco is discussed as a public health menace. The health gains achieved by stopping smoking and the means of quitting are also examined, as is the use of the Internet in smoking cessation. Then, several research issues are introduced. These include background theory and the theoretical framework for the Sense of Coherence. The research model is also discussed. A presentation of the methods, materials and of the Web-Assisted Tobacco Intervention Probe (WATIP) follows. In chapter four the results of the use of the Web-Assisted Tobacco Intervention Probe (WATIP) are presented. This study is divided into two sections. The first describes results related to quality control in relation to the Web-Assisted Tobacco Intervention Probe (WATIP) and gives an overview of its users. Of these, 3,150 answered initial eligibility questions. In the end, 1,463 met all eligibility requirements, completed intake, decided on a day to quit smoking (Dday) and declared their “intentions to make behavioural changes” (IBC) while a second targeted group of 650 did not decide on a Dday. With two quit attempts made before joining the platform, most of the participants had experienced past failures while wanting to stop. The smoking rate averaged 21 cigarettes per day. With a mean age of 35, of the participants 55% were males. Among several other considerations, gender and the Sense of Coherence (SOC) influenced the success of participants in their IBC and endeavour to set quit dates. The results of comparing males and females showed that, for current smokers, establishing a Dday was related to gender differences, not favouring males (OR=0.76, p<0.005). Belonging to higher Socio-economic strata (SES) was associated with the intention to consider IBC (when compared to lower SES condition) (OR=1.57, p<0.001) and higher number of school years (OR=0.70, p<0.005) favoured the decision to smoking cessation. Those who demonstrated higher confidence in their likelihood of success in stopping in the shortest time had a higher rate of setting a Dday (OR=0.51, p<0.001). There were differences between groups in IBC reflecting the high and low levels of the SOC score (OR=1.43, p=0.006), as those who considered setting a Dday had higher levels of SOC. After adjusting for all variables, stages of readiness to change and SOC were kept in the model. This is the first Arm of this research where the focus is a discussion of the system’s implications for the participants’ “intentions to make behavioural changes” (IBC). Moreover, a second section of this study (second Arm) offers input collected from 77 in-depth interviews with the Web-Assisted Tobacco Intervention Probe (WATIP) users. Here, “Behaviour Change” (BC) and the usability of the platform are explored a year after IBC was declared. A percentage of 32.9% of self-reported, 12-month quitters in continuous abstinence from smoking from Dday to the 12-month follow- up point of the use of the Web-Assisted Tobacco Intervention Probe (WATIP) has been assessed. Comparing the Sense of Coherence (SOC) scores of participants by their respective means, according to the two groups, there was a significant difference in these scores of non smokers (BC) (M=144,66, SD=22,52) and Sense of Coherence (SOC) of smokers (noBC) (M=131,51, SD=21,43) p=0.014. This WATIP strategy and its contents benefit from the strengthening of the smoker’s sense of coherence (SOC), so that the person’s progress towards a life without tobacco may be experienced as comprehensible, manageable and meaningful. In this sample the sense of coherence (SOC) effect is moderate although it is associated with the day to quit smoking (Dday). Some of the limitations of this research have to do with self-selection bias, sample size (power) and self-reporting (no biochemical validation). The enrolment of participants was therefore not representative of the smoking population. It is not possible to verify the Web-Assisted Tobacco Intervention Probe (WATIP) evaluation of external validity; consequently, the results obtained cannot be applied generalized. No participation bias is provided. Another limitation of this study is the associated limitations of interviews. Interviewees’ perception that fabricating answers could benefit them more than telling the simple truth in response to questions is a risk that is not evaluated (with no external validation like measuring participants’ carbon monoxide levels). What emerges in this analysis is the relevance of the process that leads to the establishment of the quit day (Dday) to stop using tobacco. In addition, technological issues, when tailoring is the focus, are key elements for scrutiny. The high number of dropouts of users of the web platform mandates future research that should concentrate on the matters of the user-centred design of portals. The focus on gains in health through patient-centred care needs more research, so that technology usability be considered within the context of best practices in smoking cessation.