Discriminant validity and test-retest reliability of a self-administered Internet-based questionnaire testing doctors' knowledge in evidence-based medicine.

Aims and objectives  This study aimed to determine the discriminant validity and the test-retest reliability of a questionnaire testing the impact of evidence-based medicine (EBM) training on doctors' knowledge and skills. Methods  Questionnaires were sent electronically to all doctors working as residents and chief residents in two French speaking hospital networks in Switzerland. Participants completed the questionnaire twice, within a 4-week interval. The discriminant validity was examined in comparing doctors' performance according to their reported EBM previous training. Proportion of agreement between both sessions of the questionnaire, Cohen's kappa and 'uniform kappa' determined its test-retest reliability. Results  The participation rate was 9.8%/7.1% to first/second session. Performance increased according to the level of doctors' previous training in EBM. The observed proportion of agreement between both sessions was over 70% for 14/19 questions, and the 'uniform kappa' was superior to 0.60 for 15/19 questions. Conclusion  The discriminant validity and test-retest reliability of the questionnaire were satisfying. The low participation rate did not prevent the study from achieving its aims.

Evidence for validity and reliability of a french version of the FAAM.

BACKGROUND: The Foot and Ankle Ability Measure (FAAM) is a self reported questionnaire for patients with foot and ankle disorders available in English, German, and Persian. This study plans to translate the FAAM from English to French (FAAM-F) and assess the validity and reliability of this new version.METHODS: The FAAM-F Activities of Daily Living (ADL) and sports subscales were completed by 105 French-speaking patients (average age 50.5 years) presenting various chronic foot and ankle disorders. Convergent and divergent validity was assessed by Pearson's correlation coefficients between the FAAM-F subscales and the SF-36 scales: Physical Functioning (PF), Physical Component Summary (PCS), Mental Health (MH) and Mental Component Summary (MCS). Internal consistency was calculated by Cronbach's Alpha (CA). To assess test re-test reliability, 22 patients filled out the questionnaire a second time to estimate minimal detectable changes (MDC) and intraclass correlation coefficients (ICC).RESULTS: Correlations for FAAM-F ADL subscale were 0.85 with PF, 0.81 with PCS, 0.26 with MH, 0.37 with MCS. Correlations for FAAM-F Sports subscale were 0.72 with PF, 0.72 with PCS, 0.21 with MH, 0.29 with MCS. CA estimates were 0.97 for both subscales. Respectively for the ADL and Sports subscales, ICC were 0.97 and 0.94, errors for a single measure were 8 and 10 points at 95% confidence and the MDC values at 95% confidence were 7 and 18 points.CONCLUSION: The FAAM-F is valid and reliable for the self-assessment of physical function in French-speaking patients with a wide range of chronic foot and ankle disorders.

Self-reported social class in adolescents: validity and relationship with gradients in self-reported health

Background: Analyzing social differences in the health of adolescents is a challenge. The accuracy of adolescent's report on familial socio-economic position is unknown. The aims of the study were to examine the validity of measuring occupational social class and family level of education reported by adolescents aged 12 to 18, and the relationship between social position and self-reported health.Methods: A sample of 1453 Spanish adolescents 12 to 18 years old from urban and rural areas completed a self-administered questionnaire including the Child Health and Illness Profile-Adolescent Edition (CHIP-AE), and data on parental occupational social class (OSC) and level of education (LE). The responsible person for a sub-sample of teenagers (n = 91) were interviewed by phone. Kappa coefficients were estimated to analyze agreement between adolescents and proxy-respondents, and logistic regression models were adjusted to analyze factors associated with missing answers and disagreements. Effect size (ES) was calculated to analyze the relationship between OSC, LE and the CHIP-AE domain scores.Results: Missing answers were higher for father's (24.2%) and mother's (45.7%) occupational status than for parental education (8.4%, and 8.1% respectively), and belonging to a non-standard family was associated with more incomplete reporting of social position (OR = 4,98; 95%CI = 1,3–18,8) as was agreement between a parent and the adolescent. There were significant social class gradients, most notably for aspects of health related to resilience to threats to illness.ConclusionAdolescents can acceptably self-report on family occupation and level of education. Social class gradients are present in important aspects of health in adolescents.

Assessment of construct validity of a virtual reality laparoscopy simulator.

BACKGROUND: The aim of this study was to assess whether virtual reality (VR) can discriminate between the skills of novices and intermediate-level laparoscopic surgical trainees (construct validity), and whether the simulator assessment correlates with an expert's evaluation of performance. METHODS: Three hundred and seven (307) participants of the 19th-22nd Davos International Gastrointestinal Surgery Workshops performed the clip-and-cut task on the Xitact LS 500 VR simulator (Xitact S.A., Morges, Switzerland). According to their previous experience in laparoscopic surgery, participants were assigned to the basic course (BC) or the intermediate course (IC). Objective performance parameters recorded by the simulator were compared to the standardized assessment by the course instructors during laparoscopic pelvitrainer and conventional surgery exercises. RESULTS: IC participants performed significantly better on the VR simulator than BC participants for the task completion time as well as the economy of movement of the right instrument, not the left instrument. Participants with maximum scores in the pelvitrainer cholecystectomy task performed the VR trial significantly faster, compared to those who scored less. In the conventional surgery task, a significant difference between those who scored the maximum and those who scored less was found not only for task completion time, but also for economy of movement of the right instrument. CONCLUSIONS: VR simulation provides a valid assessment of psychomotor skills and some basic aspects of spatial skills in laparoscopic surgery. Furthermore, VR allows discrimination between trainees with different levels of experience in laparoscopic surgery establishing construct validity for the Xitact LS 500 clip-and-cut task. Virtual reality may become the gold standard to assess and monitor surgical skills in laparoscopic surgery.

Limits on the validity of infinite length assumptions for modelling shallow landslides

The infinite slope method is widely used as the geotechnical component of geomorphic and landscape evolution models. Its assumption that shallow landslides are infinitely long (in a downslope direction) is usually considered valid for natural landslides on the basis that they are generally long relative to their depth. However, this is rarely justified, because the critical length/depth (L/H) ratio below which edge effects become important is unknown. We establish this critical L/H ratio by benchmarking infinite slope stability predictions against finite element predictions for a set of synthetic two-dimensional slopes, assuming that the difference between the predictions is due to error in the infinite slope method. We test the infinite slope method for six different L/H ratios to find the critical ratio at which its predictions fall within 5% of those from the finite element method. We repeat these tests for 5000 synthetic slopes with a range of failure plane depths, pore water pressures, friction angles, soil cohesions, soil unit weights and slope angles characteristic of natural slopes. We find that: (1) infinite slope stability predictions are consistently too conservative for small L/H ratios; (2) the predictions always converge to within 5% of the finite element benchmarks by a L/H ratio of 25 (i.e. the infinite slope assumption is reasonable for landslides 25 times longer than they are deep); but (3) they can converge at much lower ratios depending on slope properties, particularly for low cohesion soils. The implication for catchment scale stability models is that the infinite length assumption is reasonable if their grid resolution is coarse (e.g. >25?m). However, it may also be valid even at much finer grid resolutions (e.g. 1?m), because spatial organization in the predicted pore water pressure field reduces the probability of short landslides and minimizes the risk that predicted landslides will have L/H ratios less than 25. Copyright (c) 2012 John Wiley & Sons, Ltd.

Reliability and validity of the cross-culturally adapted French version of the Core Outcome Measures Index (COMI) in patients with low back pain.

PURPOSE: To conduct a cross-cultural adaptation of the Core Outcome Measures Index (COMI) into French according to established guidelines. METHODS: Seventy outpatients with chronic low back pain were recruited from six spine centres in Switzerland and France. They completed the newly translated COMI, and the Roland Morris disability (RMQ), Dallas Pain (DPQ), adjectival pain rating scale, WHO Quality of Life, and EuroQoL-5D questionnaires. After ~14 days RMQ and COMI were completed again to assess reproducibility; a transition question (7-point Likert scale; "very much worse" through "no change" to "very much better") indicated any change in status since the first questionnaire. RESULTS: COMI whole scores displayed no floor effects and just 1.5% ceiling effects. The scores for the individual COMI items correlated with their corresponding full-length reference questionnaire with varying strengths of correlation (0.33-0.84, P < 0.05). COMI whole scores showed a very good correlation with the "multidimensional" DPQ global score (Rho = 0.71). 55 patients (79%) returned a second questionnaire with no/minimal change in their back status. The reproducibility of individual COMI 5-point items was good, with test-retest differences within one grade ranging from 89% for 'social/work disability' to 98% for 'symptom-specific well-being'. The intraclass correlation coefficient for the COMI whole score was 0.85 (95% CI 0.76-0.91). CONCLUSIONS: In conclusion, the French version of this short, multidimensional questionnaire showed good psychometric properties, comparable to those reported for German and Spanish versions. The French COMI represents a valuable tool for future multicentre clinical studies and surgical registries (e.g. SSE Spine Tango) in French-speaking countries.

A neuropsychological instrument measuring age-related cerebral decline in older drivers: development, reliability, and validity of MedDrive.

When facing age-related cerebral decline, older adults are unequally affected by cognitive impairment without us knowing why. To explore underlying mechanisms and find possible solutions to maintain life-space mobility, there is a need for a standardized behavioral test that relates to behaviors in natural environments. The aim of the project described in this paper was therefore to provide a free, reliable, transparent, computer-based instrument capable of detecting age-related changes on visual processing and cortical functions for the purposes of research into human behavior in computational transportation science. After obtaining content validity, exploring psychometric properties of the developed tasks, we derived (Study 1) the scoring method for measuring cerebral decline on 106 older drivers aged ≥70 years attending a driving refresher course organized by the Swiss Automobile Association to test the instrument's validity against on-road driving performance (106 older drivers). We then validated the derived method on a new sample of 182 drivers (Study 2). We then measured the instrument's reliability having 17 healthy, young volunteers repeat all tests included in the instrument five times (Study 3) and explored the instrument's psychophysical underlying functions on 47 older drivers (Study 4). Finally, we tested the instrument's responsiveness to alcohol and effects on performance on a driving simulator in a randomized, double-blinded, placebo, crossover, dose-response, validation trial including 20 healthy, young volunteers (Study 5). The developed instrument revealed good psychometric properties related to processing speed. It was reliable (ICC = 0.853) and showed reasonable association to driving performance (R (2) = 0.053), and responded to blood alcohol concentrations of 0.5 g/L (p = 0.008). Our results suggest that MedDrive is capable of detecting age-related changes that affect processing speed. These changes nevertheless do not necessarily affect driving behavior.

Validity and reliability of a questionnaire on knowledge of cardiovascular risk factors for use in Brazil

Using a sample of patients with coronary artery disease, this methodological study aimed to conduct a cross-cultural adaptation and validation of a questionnaire on knowledge of cardiovascular risk factors (Q-FARCS), lifestyle changes, and treatment adherence for use in Brazil. The questionnaire has three scales: general knowledge of risk factors (RFs); specific knowledge of these RFs; and lifestyle changes achieved. Cross-cultural adaptation included translation, synthesis, back-translation, expert committee review, and pretesting. Face and content validity, reliability, and construct validity were measured. Cronbach’s alpha for the total sample (n = 240) was 0.75. Assessment of psychometric properties revealed adequate face and content validity, and the construct revealed seven components. It was concluded that the Brazilian version of Q-FARCS had adequate reliability and validity for the assessment of knowledge of cardiovascular RFs.

Scientific Literature on the Reliability and Validity of the Manchester Triage System (MTS) Protocol: A Integrative Literature Review

OBJECTIVE To analyze the scientific production about the validity and reliability of the Manchester Triage System (MTS) protocol. METHOD A descriptive study of an integrative literature review. Articles on the validity and reliability of the MTS developed with children and adults published between 1999 and 2013 were included. RESULTS 14 articles were selected from a total of 8438, nine of validity and five of reliability. The reliability of the MTS ranged from moderate to almost perfect, with higher intra-evaluation. Regarding validity, the results seem to point to equivalent and satisfactory sensibility and specificity levels of the MTS. The instrument proved to be a good predictor of the need for hospitalization and of hospital mortality. CONCLUSION The reliability and validity of the MTS obtained in the studies is varied. It is recommended that new studies indicate necessary modifications to the MTS so that it is more safely used by nurses.

Validity of the Katz Index to assess activities of daily living by informants in neuropathological studies

Abstract OBJECTIVE To analyze the evidences of construct validity of the Katz Index for the retrospective assessment of activities of daily living (ADL) by informants, to assist neuropathological studies in the elderly. METHOD A cross-sectional study analyzed the functional ability of ADL measure by the Katz Index, of 650 cases randomly selected from the Brazilian Brain Bank of the Ageing Brain Study Group (BBBABSG) database. Sample was divided in two subsamples for the analysis (N=325, each) and then stratified according to cognitive decline assessed by the Clinical Dementia Rating Scale (CDR). Factor analyses with calculations of internal consistency and invariance were performed. RESULTS Factor analysis evidenced a unidimensional instrument with optimal internal consistency, in all subgroups. Goodness of fit indices were obtained after two treatments of covariance, indicating adequacy of the scale for assessing ADL by informants. The scale is invariant to cognitive decline meaning that it can be used for subjects with or without cognitive impairment. CONCLUSION Katz Index is valid for the retrospective assessment of basic ADL by informants, with optimal reliability.

Multi-sample analysis of moment-structures: Asymptotic validity of inferences based on second-order moments

In moment structure analysis with nonnormal data, asymptotic valid inferences require the computation of a consistent (under general distributional assumptions) estimate of the matrix $\Gamma$ of asymptotic variances of sample second--order moments. Such a consistent estimate involves the fourth--order sample moments of the data. In practice, the use of fourth--order moments leads to computational burden and lack of robustness against small samples. In this paper we show that, under certain assumptions, correct asymptotic inferences can be attained when $\Gamma$ is replaced by a matrix $\Omega$ that involves only the second--order moments of the data. The present paper extends to the context of multi--sample analysis of second--order moment structures, results derived in the context of (simple--sample) covariance structure analysis (Satorra and Bentler, 1990). The results apply to a variety of estimation methods and general type of statistics. An example involving a test of equality of means under covariance restrictions illustrates theoretical aspects of the paper.

A test of the predictive validity of non-linear QALY models using time trade-off utilities

This paper presents a test of the predictive validity of various classes ofQALY models (i.e., linear, power and exponential models). We first estimatedTTO utilities for 43 EQ-5D chronic health states and next these states wereembedded in health profiles. The chronic TTO utilities were then used topredict the responses to TTO questions with health profiles. We find that thepower QALY model clearly outperforms linear and exponential QALY models.Optimal power coefficient is 0.65. Our results suggest that TTO-based QALYcalculations may be biased. This bias can be avoided using a power QALY model.

Incremental Validity and Indirect effect of Ethical Development on Work Performance

We modeled work performance as outcomes of individual-differences mediated by technical performance. Beyond the "usual suspects" (e.g., general mental ability, and personality), we also measured the ethical development of participants (n = 460). We surmised that ethical development - which has not been extensively studied as a predictor of work performance while controlling for established predictors - captures unique variance in both technical and work performance. Results demonstrated incremental validity for ethical development in predicting technical performance, which in turn predicted work performance. The indirect effect of ethical development was significant too. Our results highlight the importance of process models of performance, which include proximal as well as distal individual differences.