Validity evidence based on test content

Resumen tomado de la publicación

Validity evidence based on internal structure

Resumen tomado de la publicación

Validity evidence based on testing consequences

Resumen tomado de la publicación

Validity evidence based on response processes

Resumen tomado de la publicación

Further validity evidence of the Behavioral Inhibition Observation System (BIOS)

Resumen tomado de la publicaci??n

Validity evidence and reliability of a Simulated Patient Feedback Instrument

In the training of healthcare professionals, one of the advantages of communication training with simulated patients (SPs) is the SP's ability to provide direct feedback to students after a simulated clinical encounter. The quality of SP feedback must be monitored, especially because it is well known that feedback can have a profound effect on student performance. Due to the current lack of valid and reliable instruments to assess the quality of SP feedback, our study examined the validity and reliability of one potential instrument, the 'modified Quality of Simulated Patient Feedback Form' (mQSF). Methods Content validity of the mQSF was assessed by inviting experts in the area of simulated clinical encounters to rate the importance of the mQSF items. Moreover, generalizability theory was used to examine the reliability of the mQSF. Our data came from videotapes of clinical encounters between six simulated patients and six students and the ensuing feedback from the SPs to the students. Ten faculty members judged the SP feedback according to the items on the mQSF. Three weeks later, this procedure was repeated with the same faculty members and recordings. Results All but two items of the mQSF received importance ratings of > 2.5 on a four-point rating scale. A generalizability coefficient of 0.77 was established with two judges observing one encounter. Conclusions The findings for content validity and reliability with two judges suggest that the mQSF is a valid and reliable instrument to assess the quality of feedback provided by simulated patients.

The Questionnaire about Interpersonal Difficulties for Adolescents: Reliability and Validity Evidence in Colombian Adolescents

The aim of this study was to analyze reliability and validity evidence of scores on the Colombian version of the Questionnaire about Interpersonal Difficulties for Adolescents (QIDA) in a sample of 1,628 adolescents (65.7% boys), ranging in age from 12 to 18 years. Confirmatory factor analyses replicated the correlated five-factor structure of the QIDA: Assertiveness, Heterosexual Relationships, Public Speaking, Family Relationships, and Close Friendships. Internal consistency for the QIDA and subscales scores was excellent. Girls reported higher level of perceived anxiety in heterosexual relationships, whereas boys showed more anxiety in close friendships and decreased interpersonal anxiety during adolescence. Results support the reliability and validity of the scores on the Colombian version of the QIDA.

Reliability and Validity Evidence of Scores on the French Version of the Questionnaire about Interpersonal Difficulties for Adolescents

This study examined the reliability and validity evidence drawn from the scores of the French version of the Questionnaire about Interpersonal Difficulties for Adolescents (QIDA) in a sample of 957 adolescents (48.5% boys) ranging in age from 11 to 18 years (M = 14.48, SD = 1.85). A principal axis factoring (PAF) and confirmatory factor analyses (CFA) were performed to determine the fit of the factor structure of scores on the QIDA. PAF and CFA replicated the previously identified correlated five-factor structure of the QIDA: Assertiveness, Heterosexual Relationships, Public Speaking, Family Relationships, and Close Friendships. The QIDA yielded acceptable reliability scores for French adolescents. Validity evidence of QIDA was also established through correlations with scores on the School Anxiety Inventory and the Social Anxiety Scale for Adolescents. Most of the correlations were positive and exceeded the established criteria of statistical significance, but the magnitude of these varied according to the scales of the QIDA. Results supported the reliability and validity evidence drawn from the scores of the French version of the QIDA.

Claims requiring development of additional medical evidence /

"February 1976."

School Anxiety Inventory: Reliability and Validity Evidence in a Sample of Slovenian Adolescents

The purpose of this study was to examine the reliability and validity of the School Anxiety Inventory (SAI) using a sample of 646 Slovenian adolescents (48% boys), ranging in age from 12 to 19 years. Single confirmatory factor analyses replicated the correlated four-factor structure of scores on the SAI for anxiety-provoking school situations (Anxiety about School Failure and Punishment, Anxiety about Aggression, Anxiety about Social Evaluation, and Anxiety about Academic Evaluation), and the three-factor structure of the anxiety response systems (Physiological Anxiety, Cognitive Anxiety, and Behavioral Anxiety). Equality of factor structures was compared using multigroup confirmatory factor analyses. Measurement invariance for the four- and three-factor models was obtained across gender and school-level samples. The scores of the instrument showed high internal reliability and adequate test–retest reliability. The concurrent validity of the SAI scores was also examined through its relationship with the Social Anxiety Scale for Adolescents (SASA) scores and the Questionnaire about Interpersonal Difficulties for Adolescents (QIDA) scores. Correlations of the SAI scores with scores on the SASA and the QIDA were of low to moderate effect sizes.

Evaluating two morningness scales with item response theory

Using a student sample (n = 692) and an organization sample (n = 180), we scrutinized two morning-evening orientation scales using item response theory (IRT) methods. We used IRT to compare the measurement precision of the Composite Scale (CS) and the Early/Late Preferences Scale (PS). The CS had slightly higher measurement precision at all ranges of orientations, except for extreme morning and evening orientations for which the PS had slightly higher precision. IRT item-level statistics were also computed to try to understand how morning-orientation items functioned. Items that asked questions about morning activities tended to be more discriminating indicators of morning-orientation than items that asked about evening or peak performance activities. Items that involved unpleasant activities were less frequently endorsed than items that involved neutral or enjoyable activities. Implications for measurement of morning-evening orientation are discussed. (C) 2002 Elsevier Science Ltd. All rights reserved.

(Endogenous) occupational choices and job satisfaction among recent PhD recipients: evidence from Catalonia

Drawing on data from two successive cohorts of PhD graduates, this paper analyses differences in overall job satisfaction and specific job domain satisfaction among PhDs employed in different sectors four years after completing their doctorate degrees. Covariate-adjusted job satisfaction differentials suggest that, compared to faculty members, PhD holders employed outside traditional academic and research jobs are more satisfied with the pecuniary facets of their work (principally, because of higher earnings), but significantly less satisfied with the content of their job and with how well the job matches their skills (and, in the case of public sector workers, with their prospects of promotion). The evidence regarding the overall job satisfaction of the PhD holders indicates that working in the public or private sectors is associated with less work well-being, which cannot be fully compensated by the better pecuniary facets of the job. It also appears that being employed in academia or in research centres provides almost the same perceived degree of satisfaction with the job and with its four specific domains. We also take into account the endogenous sorting of PhD holders into different occupations based on latent personal traits that might be related to job satisfaction. The selectivity-corrected job satisfaction differentials reveal the importance of self-selection based on unobservable traits, and confirm the existence of a certain penalisation for working in occupations other than academia or research, which is especially marked in the case of satisfaction with job content and job-skills match. The paper presents additional interesting evidence about the determinants of occupational choice among PhD holders, highlighting the relevance of certain academic attributes (especially PhD funding and pre-and-post-doc research mobility) in affecting the likelihood of being employed in academia, in a research centre or in other public or private sector job four years after completing their doctorate programme.

Comparing the validity and output of the GT1M and GT3X accelerometer in 5- to 9-year-old children

The purpose of this study was to compare the validity and output of the biaxial ActiGraph GT1M and the triaxial GT3X (ActiGraph, LLC, Pensacola, FL, USA)accelerometer in 5- to 9-year-old children. Thirty-two children wore the two monitors while their energy expenditure was measured with indirect calorimetry. They performed four locomotor and four play activities in an exercise laboratory and were further measured during 12 minutes of a sports lesson. Validity evidence in relation to indirect calorimetry was examined with linear regression equations applied to the laboratory data. During the sports lessons predicted energy expenditure according to the regression equations was compared to measured energy expenditure with the Wilcoxon-signed rank test and the Spearman correlation. To compare the output, agreement between counts of the two monitors during the laboratory activities was assessed with Bland-Altman plots. The evidence of validity was similar for both monitors. Agreement between the output of the two monitors was good for vertical counts (mean bias = −14 ± 22 counts) but not for horizontal counts (−17 ± 32 counts). The current results indicate that the two accelerometer models are able to estimate energy expenditure of a range of physical activities equally well in young children. However, they show output differences for movement in the horizontal direction.

Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: The design of Virtual Patients (VPs) is essential. So far there are no validated evaluation instruments for VP design published. Summary of work: We examined three sources of validity evidence of an instrument to be filled out by students aimed at measuring the quality of VPs with a special emphasis on fostering clinical reasoning: (1) Content was examined based on theory of clinical reasoning and an international VP expert team. (2) Response process was explored in think aloud pilot studies with students and content analysis of free text questions accompanying each item of the instrument. (3) Internal structure was assessed by confirmatory factor analysis (CFA) using 2547 student evaluations and reliability was examined utilizing generalizability analysis. Summary of results: Content analysis was supported by theory underlying Gruppen and Frohna’s clinical reasoning model on which the instrument is based and an international VP expert team. The pilot study and analysis of free text comments supported the validity of the instrument. The CFA indicated that a three factor model comprising 6 items showed a good fit with the data. Alpha coefficients per factor were 0,74 - 0,82. The findings of the generalizability studies indicated that 40-200 student responses are needed in order to obtain reliable data on one VP. Conclusions: The described instrument has the potential to provide faculty with reliable and valid information about VP design. Take-home messages: We present a short instrument which can be of help in evaluating the design of VPs.

Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: Virtual patients (VPs) are increasingly used to train clinical reasoning. So far, no validated evaluation instruments for VP design are available. Aims: We examined the validity of an instrument for assessing the perception of VP design by learners. Methods: Three sources of validity evidence were examined: (i) Content was examined based on theory of clinical reasoning and an international VP expert team. (ii) The response process was explored in think-aloud pilot studies with medical students and in content analyses of free text questions accompanying each item of the instrument. (iii) Internal structure was assessed by exploratory factor analysis (EFA) and inter-rater reliability by generalizability analysis. Results: Content analysis was reasonably supported by the theoretical foundation and the VP expert team. The think-aloud studies and analysis of free text comments supported the validity of the instrument. In the EFA, using 2547 student evaluations of a total of 78 VPs, a three-factor model showed a reasonable fit with the data. At least 200 student responses are needed to obtain a reliable evaluation of a VP on all three factors. Conclusion: The instrument has the potential to provide valid information about VP design, provided that many responses per VP are available.