More on Testing for Validity Instead of Looking for It

Using Monte Carlo simulations and reanalyzing the data of a validation study of the AEIM emotional intelligence test, we demonstrated that an atheoretical approach and the use of weak statistical procedures can result in biased validity estimates. These procedures included stepwise regression-and the general case of failing to include important theoretical controls-extreme scores analysis, and ignoring heteroscedasticity as well as measurement error. The authors of the AEIM test responded by offering more complete information about their analyses, allowing us to further examine the perils of ignoring theory and correct statistical procedures. In this paper we show with extended analyses that the AEIM test is invalid.

Construct validity of the French version of the PRWE (Patient Rated Wrist Evaluation) with the French version of the DASH (Disabilities Arm Shoulder and Hand) is good to very good in a population of patients with wrist injuries in an inpatient rehabilitation unit

Objective.- The Patient-Rated Wrist Evaluation is a specific questionnaire for the wrist [1]. It consists of 15 questions with a total score of 100. It was recently translated into French [2]. However, its validity has not been tested in this language. The Disabilities Arm Shoulder and Hand (DASH), with well-established psychometric properties, is considered as the reference questionnaire for the evaluation of upper extremities. The objective of this study is to measure the construct validity of the PRWE-F with the DASH-F in patients with wrist pathology.Patients and methods.- Fifty-one patients (40 m, 11 w, mean age 42 years), 25 fractures of the radius and 26 lesions of the carpus.Questionnaires PRWE-F and DASH-F at entry and at discharge (0 to 100). Calculation of the construct validity of the PRWE-F comparing with the DASH-F with Pearson correlation coefficients (r) at entry and at discharge. Level of significance (alpha) was set at 5%.Results.- Correlation DASH/PRWE at entry: r = 0.799 (95% CI 0.671 to 0.881), P < 0.0001. Correlation DASH/PRWE at discharge: r = 0.847 (95% CI: 0.745 to 0.910), P < 0.0001.Discussion.- The construct validity of the two instruments indicates that they measure the same concept. Our correlation between DASH-F and PRWE-F, going from 0.799 to 0.847, are comparable to those published in different languages (0.71 to 0.84) [3,4]. The questionnaires PRWE-F can thus be used in rehabilitation patients presenting with wrist pathologies; it is comparable to the DASH but described by MacDermid [1] to be more specific. Compared to the DASH it has the advantage of consisting of two dimensions. Its construct validity is excellent. This questionnaire should be evaluated in other populations, and it should be compared with hand questionnaires more specific than the DASH.

Validity of an arterial pressure waveform analysis device: does the puncture site play a role in the agreement with intermittent pulmonary artery catheter thermodilution measurements?

OBJECTIVE: The measurement of cardiac output is a key element in the assessment of cardiac function. Recently, a pulse contour analysis-based device without need for calibration became available (FloTrac/Vigileo, Edwards Lifescience, Irvine, CA). This study was conducted to determine if there is an impact of the arterial catheter site and to investigate the accuracy of this system when compared with the pulmonary artery catheter using the bolus thermodilution technique (PAC). DESIGN: Prospective study. SETTING: The operating room of 1 university hospital. PARTICIPANTS: Twenty patients undergoing cardiac surgery. INTERVENTIONS: CO was determined in parallel by the use of the Flotrac/Vigileo systems in the radial and femoral position (CO_rad and CO_fem) and by PAC as the reference method. Data triplets were recorded at defined time points. The primary endpoint was the comparison of CO_rad and CO_fem, and the secondary endpoint was the comparison with the PAC. MEASUREMENTS AND MAIN RESULTS: Seventy-eight simultaneous data recordings were obtained. The Bland-Altman analysis for CO_fem and CO_rad showed a bias of 0.46 L/min, precision was 0.85 L/min, and the percentage error was 34%. The Bland-Altman analysis for CO_rad and PAC showed a bias of -0.35 L/min, the precision was 1.88 L/min, and the percentage error was 76%. The Bland-Altman analysis for CO_fem and PAC showed a bias of 0.11 L/min, the precision was 1.8 L/min, and the percentage error was 69%. CONCLUSION: The FloTrac/Vigileo system was shown to not produce exactly the same CO data when used in radial and femoral arteries, even though the percentage error was close to the clinically acceptable range. Thus, the impact of the introduction site of the arterial catheter is not negligible. The agreement with thermodilution was low.

Validity of Charlson Comorbidity Index in patients hospitalised with acute coronary syndrome. Insights from the nationwide AMIS Plus registry 2002-2012.

OBJECTIVE: This study aimed to assess the impact of individual comorbid conditions as well as the weight assignment, predictive properties and discriminating power of the Charlson Comorbidity Index (CCI) on outcome in patients with acute coronary syndrome (ACS). METHODS: A prospective multicentre observational study (AMIS Plus Registry) from 69 Swiss hospitals with 29 620 ACS patients enrolled from 2002 to 2012. The main outcome measures were in-hospital and 1-year follow-up mortality. RESULTS: Of the patients, 27% were female (age 72.1 ± 12.6 years) and 73% were male (64.2 ± 12.9 years). 46.8% had comorbidities and they were less likely to receive guideline-recommended drug therapy and reperfusion. Heart failure (adjusted OR 1.88; 95% CI 1.57 to 2.25), metastatic tumours (OR 2.25; 95% CI 1.60 to 3.19), renal diseases (OR 1.84; 95% CI 1.60 to 2.11) and diabetes (OR 1.35; 95% CI 1.19 to 1.54) were strong predictors of in-hospital mortality. In this population, CCI weighted the history of prior myocardial infarction higher (1 instead of -0.4, 95% CI -1.2 to 0.3 points) but heart failure (1 instead of 3.7, 95% CI 2.6 to 4.7) and renal disease (2 instead of 3.5, 95% CI 2.7 to 4.4) lower than the benchmark, where all comorbidities, age and gender were used as predictors. However, the model with CCI and age has an identical discrimination to this benchmark (areas under the receiver operating characteristic curves were both 0.76). CONCLUSIONS: Comorbidities greatly influenced clinical presentation, therapies received and the outcome of patients admitted with ACS. Heart failure, diabetes, renal disease or metastatic tumours had a major impact on mortality. CCI seems to be an appropriate prognostic indicator for in-hospital and 1-year outcomes in ACS patients. ClinicalTrials.gov Identifier: NCT01305785.

Induced quantum metric fluctuations and the validity of semiclassical gravity

We propose a criterion for the validity of semiclassical gravity (SCG) which is based on the stability of the solutions of SCG with respect to quantum metric fluctuations. We pay special attention to the two-point quantum correlation functions for the metric perturbations, which contain both intrinsic and induced fluctuations. These fluctuations can be described by the Einstein-Langevin equation obtained in the framework of stochastic gravity. Specifically, the Einstein-Langevin equation yields stochastic correlation functions for the metric perturbations which agree, to leading order in the large N limit, with the quantum correlation functions of the theory of gravity interacting with N matter fields. The homogeneous solutions of the Einstein-Langevin equation are equivalent to the solutions of the perturbed semiclassical equation, which describe the evolution of the expectation value of the quantum metric perturbations. The information on the intrinsic fluctuations, which are connected to the initial fluctuations of the metric perturbations, can also be retrieved entirely from the homogeneous solutions. However, the induced metric fluctuations proportional to the noise kernel can only be obtained from the Einstein-Langevin equation (the inhomogeneous term). These equations exhibit runaway solutions with exponential instabilities. A detailed discussion about different methods to deal with these instabilities is given. We illustrate our criterion by showing explicitly that flat space is stable and a description based on SCG is a valid approximation in that case.

Bilateral symmetry of radial pulse in high-level tennis players : implications for the validity of central aortic pulse wave analysis

Rapport de synthèse Ce travail de thèse s'articule autour de l'importance de l'évaluation de la fonction vasculaire et des répercussions au niveau central, cardiaque, des perturbations du réseau vasculaire. Les maladies cardiovasculaires sont prédominantes dans notre société et causes de morbidité et mortalité importante. La mesure de la pression artérielle classique reste le moyen le plus utilisé pour suivre la santé des vaisseaux, mais ne reflète pas directement ce qui se passe au niveau du coeur. La tonométrie d'aplanation permet depuis quelques années de mesurer l'onde de pouls radial, et par le biais d'une fonction mathématique de transfert validée, il est possible d'en déduire la forme et Γ amplitude de l'onde de pouls central, donc de la pression aortique centrale. Cette dernière est un reflet bien plus direct de la post-charge cardiaque, et de nombreuses études cliniques actuelles s'intéressent à cette mesure pour stratifier le risque ou évaluer l'effet d'un traitement vasculaire. Toutefois, bien que cet outil soit de plus en plus utilisé, il est rarement précisé si la latéralité de la mesure joue un rôle, sachant que certaines propriétés des membres supérieurs peuvent être affectées par un usage préférentiel (masse musculaire, densité osseuse, diamètre des artères, capillarisation musculaire, et même fonction endothéliale). On a en effet observé que ces divers paramètre étaient tous augmentés sur un bras entraîné. Dès lors on peut se poser la question de l'influence de ces adaptations physiologiques sur la mesure indirecte effectuée par le biais du pouls radial. Nous avons investigué les deux membres supérieurs de sujets jeunes et sédentaires (SED), ainsi que ceux de sujets sportifs avec un développement fortement asymétrique des bras, soit des joueurs de tennis de haut niveau (TEN). Des mesures anthropométriques incluant la composition corporelle et la circonférence des bras et avant-bras ont montré que TEN présente une asymétrie hautement significative aux deux mesures entre le bras dominant (entraîné) et l'autre, ce qui est aussi présent pour la force de serrage (mesurée au dynamomètre de Jamar). L'analyse des courbes centrales de pouls ne montre aucune différence entre les deux membres dans chaque groupe, par contre on peut observer une différence entre SED et TEN, avec un index d'augmentation diastolique qui est 50 % plus élevé chez TEN. Les index d'augmentation systolique sont identiques dans les deux groupes. On peut retenir de cette étude la validité de la méthode de tonométrie d'aplanation quel que soit le bras utilisé (dominant ou non-dominant) et ce même si une asymétrie conséquente est présente. Ces données sont clairement nouvelles et permettent de s'affranchir de cette variable dans la mesure d'un paramètre cardiovasculaire dont l'importance est actuellement grandissante. Les différences d'index diastolique sont expliquées par la fréquence cardiaque et la vitesse de conduction de l'onde de pouls plus basses chez TEN, causant un retard diastolique du retour de l'onde au niveau central, phénomène précédemment bien décrit dans la littérature.

Construct validity and internal reliability of a French version of FACES III in adolescents and adults

Family cohesion and adaptability, as operationalised in the Family Adaptability and Cohesion Scales III (FACES III), are two hypothesised dimensions of family functioning. We tested the properties of a French version of FACES III in school-children (mean age: 13 years; S.D:0.85) recruited from the general population and their parents. Separate confirmatory factor analyses were performed for adolescents and adults. The results of both analyses were compatible with a two-factor structure similar to that proposed by the authors of the original instrument. However, orthogonality between the two factors was only supported in the adult data. Internal reliability estimates were 0.78 and 0.68 in adolescents and 0.82 and 0.65 in adults, for cohesion and adaptability respectively.

Reproducibility and relative validity of a food-frequency questionnaire for French-speaking Swiss adults.

BACKGROUND: Due to the distinct cultural and language differences that exist in Switzerland, there is little information on the dietary intake among the general Swiss population. Adequately assessing dietary intake is thus paramount if nutritional epidemiological studies are to be conducted. OBJECTIVE: To assess the reproducibility and validity of a food-frequency questionnaire (FFQ) developed for French-speaking Swiss adults. DESIGN: A total of 23 men and 17 women (43.1+/-2.0 years) filled out one FFQ and completed one 24-hour dietary recall at baseline and 1 month afterward. RESULTS: Crude Pearson's correlation coefficients between the first and the second FFQ ranged from 0.58 to 0.90, intraclass correlation coefficient (ICC) ranged between 0.53 and 0.92. Lin's concordance coefficients ranged between 0.55 and 0.87. Over 80% of participants were classified in the same or adjacent tertile using each FFQ. Macronutrient intakes estimated by both FFQs were significantly higher than those estimated from the 24-hour recall for protein and water, while no significant differences were found for energy, carbohydrate, fats (five groups), and alcohol. De-attenuated Pearson's correlation coefficients between the 24-hour recall and the first FFQ ranged between 0.31 and 0.49, while for the second FFQ the values ranged between 0.38 and 0.59. Over 40 and 95% of participants fell into the same or the adjacent energy and nutrient tertiles, respectively, using the FFQs and the 24-hour recall. CONCLUSIONS: This FFQ shows good reproducibility and can be used determining macronutrient intake in a French-speaking Swiss population in an epidemiological setting.

Detecting and measuring deprivation in primary care: development, validity and reliability of a self-reported questionnaire - the DiPCare-Q

Background: General practitioners play a central role in taking deprivation into consideration when caring for patients in primary care. Validated questions to identify deprivation in primary-care practices are still lacking. For both clinical and research purposes, this study therefore aims to develop and validate a standardized instrument measuring both material and social deprivation at an individual level. Methods: The Deprivation in Primary Care Questionnaire (DiPCare-Q) was developed using qualitative and quantitative approaches between 2008 and 2011. A systematic review identified 199 questions related to deprivation. Using judgmental item quality, these were reduced to 38 questions. Two focus groups (primary-care physicians, and primary-care researchers), structured interviews (10 laymen), and think aloud interviews (eight cleaning staff) assured face validity. Item response theory analysis was then used to derive the DiPCare-Q index using data obtained from a random sample of 200 patients who were to complete the questionnaire a second time over the phone. For construct and criterion validity, the final 16 questions were administered to a random sample of 1,898 patients attending one of 47 different private primary-care practices in western Switzerland (validation set) along with questions on subjective social status (subjective SES ladder), education, source of income, welfare status, and subjective poverty. Results: Deprivation was defined in three distinct dimensions (table); material deprivation (eight items), social deprivation (five items) and health deprivation (three items). Item consistency was high in both the derivation (KR20 = 0.827) and the validation set (KR20 = 0.778). The DiPCare-Q index was reliable (ICC = 0.847). For construct validity, we showed the DiPCare-Q index to be correlated to patients' estimation of their position on the subjective SES ladder (rs = 0.539). This position was correlated to both material and social deprivation independently suggesting two separate mechanisms enhancing the feeling of deprivation. Conclusion: The DiPCare-Q is a rapid, reliable and validated instrument useful for measuring both material and social deprivation in primary care. Questions from the DiPCare-Q are easy to use when investigating patients' social history and could improve clinicians' ability to detect underlying social distress related to deprivation.

Looking for Validity or Testing It? The Perils of Stepwise Regression, Extreme-Scores Analysis, Heteroscedasticity, and Measurement Error

When researchers introduce a new test they have to demonstrate that it is valid, using unbiased designs and suitable statistical procedures. In this article we use Monte Carlo analyses to highlight how incorrect statistical procedures (i.e., stepwise regression, extreme scores analyses) or ignoring regression assumptions (e.g., heteroscedasticity) contribute to wrong validity estimates. Beyond these demonstrations, and as an example, we re-examined the results reported by Warwick, Nettelbeck, and Ward (2010) concerning the validity of the Ability Emotional Intelligence Measure (AEIM). Warwick et al. used the wrong statistical procedures to conclude that the AEIM was incrementally valid beyond intelligence and personality traits in predicting various outcomes. In our re-analysis, we found that the reliability-corrected multiple correlation of their measures with personality and intelligence was up to .69. Using robust statistical procedures and appropriate controls, we also found that the AEIM did not predict incremental variance in GPA, stress, loneliness, or well-being, demonstrating the importance for testing validity instead of looking for it.

The family alliance assessment scales : steps toward validity and reliability of an observational assessment tool for early family interactions

We present the first steps in the validation of an observational tool for father-mother-infant interactions: the FAAS (Family Alliance Assessment Scales). Family-level variables are acknowledged as unique contributors to the understanding of the socio-affective development of the child, yet producing reliable assessments of family-level interactions poses a methodological challenge. There is, therefore, a clear need for a validated and clinically relevant tool. This validation study has been carried out on three samples: one non-referred sample, of families taking part in a study on the transition to parenthood (normative sample; n = 30), one referred for medically assisted procreation (infertility sample; n = 30) and one referred for a psychiatric condition in one parent (clinical sample; n = 15). Results show that the FAAS scales have (1) good inter-rater reliability and (2) good validity, as assessed through known-group validity by comparing the three samples and through concurrent validity by checking family interactions against parents' self-reported marital satisfaction.

The External validity in economic experiments

One of the most important questions regarding experimental economics is the external validity of laboratory experiments. This paper goes through a study that tests the generalizability of a Dictator Game as a laboratory analogue for a naturally occurringdecision-making context of teacher absenteeism. Because lab and naturally-occurring environments systematically differ we then discuss other factors that might strongly affect the choices that individuals make. We conclude that the dichotomy drawn between labexperiments and data collected from natural settings is a false one. A combination of the two would provide deeper and better insights than either separately.