Validity evidence based on test content

Resumen tomado de la publicación

Validity evidence based on internal structure

Resumen tomado de la publicación

Validity evidence based on testing consequences

Resumen tomado de la publicación

Validity evidence based on response processes

Resumen tomado de la publicación

Further validity evidence of the Behavioral Inhibition Observation System (BIOS)

Resumen tomado de la publicaci??n

Validity evidence and reliability of a Simulated Patient Feedback Instrument

In the training of healthcare professionals, one of the advantages of communication training with simulated patients (SPs) is the SP's ability to provide direct feedback to students after a simulated clinical encounter. The quality of SP feedback must be monitored, especially because it is well known that feedback can have a profound effect on student performance. Due to the current lack of valid and reliable instruments to assess the quality of SP feedback, our study examined the validity and reliability of one potential instrument, the 'modified Quality of Simulated Patient Feedback Form' (mQSF). Methods Content validity of the mQSF was assessed by inviting experts in the area of simulated clinical encounters to rate the importance of the mQSF items. Moreover, generalizability theory was used to examine the reliability of the mQSF. Our data came from videotapes of clinical encounters between six simulated patients and six students and the ensuing feedback from the SPs to the students. Ten faculty members judged the SP feedback according to the items on the mQSF. Three weeks later, this procedure was repeated with the same faculty members and recordings. Results All but two items of the mQSF received importance ratings of > 2.5 on a four-point rating scale. A generalizability coefficient of 0.77 was established with two judges observing one encounter. Conclusions The findings for content validity and reliability with two judges suggest that the mQSF is a valid and reliable instrument to assess the quality of feedback provided by simulated patients.

The Questionnaire about Interpersonal Difficulties for Adolescents: Reliability and Validity Evidence in Colombian Adolescents

The aim of this study was to analyze reliability and validity evidence of scores on the Colombian version of the Questionnaire about Interpersonal Difficulties for Adolescents (QIDA) in a sample of 1,628 adolescents (65.7% boys), ranging in age from 12 to 18 years. Confirmatory factor analyses replicated the correlated five-factor structure of the QIDA: Assertiveness, Heterosexual Relationships, Public Speaking, Family Relationships, and Close Friendships. Internal consistency for the QIDA and subscales scores was excellent. Girls reported higher level of perceived anxiety in heterosexual relationships, whereas boys showed more anxiety in close friendships and decreased interpersonal anxiety during adolescence. Results support the reliability and validity of the scores on the Colombian version of the QIDA.

Reliability and Validity Evidence of Scores on the French Version of the Questionnaire about Interpersonal Difficulties for Adolescents

This study examined the reliability and validity evidence drawn from the scores of the French version of the Questionnaire about Interpersonal Difficulties for Adolescents (QIDA) in a sample of 957 adolescents (48.5% boys) ranging in age from 11 to 18 years (M = 14.48, SD = 1.85). A principal axis factoring (PAF) and confirmatory factor analyses (CFA) were performed to determine the fit of the factor structure of scores on the QIDA. PAF and CFA replicated the previously identified correlated five-factor structure of the QIDA: Assertiveness, Heterosexual Relationships, Public Speaking, Family Relationships, and Close Friendships. The QIDA yielded acceptable reliability scores for French adolescents. Validity evidence of QIDA was also established through correlations with scores on the School Anxiety Inventory and the Social Anxiety Scale for Adolescents. Most of the correlations were positive and exceeded the established criteria of statistical significance, but the magnitude of these varied according to the scales of the QIDA. Results supported the reliability and validity evidence drawn from the scores of the French version of the QIDA.

School Anxiety Inventory: Reliability and Validity Evidence in a Sample of Slovenian Adolescents

The purpose of this study was to examine the reliability and validity of the School Anxiety Inventory (SAI) using a sample of 646 Slovenian adolescents (48% boys), ranging in age from 12 to 19 years. Single confirmatory factor analyses replicated the correlated four-factor structure of scores on the SAI for anxiety-provoking school situations (Anxiety about School Failure and Punishment, Anxiety about Aggression, Anxiety about Social Evaluation, and Anxiety about Academic Evaluation), and the three-factor structure of the anxiety response systems (Physiological Anxiety, Cognitive Anxiety, and Behavioral Anxiety). Equality of factor structures was compared using multigroup confirmatory factor analyses. Measurement invariance for the four- and three-factor models was obtained across gender and school-level samples. The scores of the instrument showed high internal reliability and adequate test–retest reliability. The concurrent validity of the SAI scores was also examined through its relationship with the Social Anxiety Scale for Adolescents (SASA) scores and the Questionnaire about Interpersonal Difficulties for Adolescents (QIDA) scores. Correlations of the SAI scores with scores on the SASA and the QIDA were of low to moderate effect sizes.

Comparing the validity and output of the GT1M and GT3X accelerometer in 5- to 9-year-old children

The purpose of this study was to compare the validity and output of the biaxial ActiGraph GT1M and the triaxial GT3X (ActiGraph, LLC, Pensacola, FL, USA)accelerometer in 5- to 9-year-old children. Thirty-two children wore the two monitors while their energy expenditure was measured with indirect calorimetry. They performed four locomotor and four play activities in an exercise laboratory and were further measured during 12 minutes of a sports lesson. Validity evidence in relation to indirect calorimetry was examined with linear regression equations applied to the laboratory data. During the sports lessons predicted energy expenditure according to the regression equations was compared to measured energy expenditure with the Wilcoxon-signed rank test and the Spearman correlation. To compare the output, agreement between counts of the two monitors during the laboratory activities was assessed with Bland-Altman plots. The evidence of validity was similar for both monitors. Agreement between the output of the two monitors was good for vertical counts (mean bias = −14 ± 22 counts) but not for horizontal counts (−17 ± 32 counts). The current results indicate that the two accelerometer models are able to estimate energy expenditure of a range of physical activities equally well in young children. However, they show output differences for movement in the horizontal direction.

Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: The design of Virtual Patients (VPs) is essential. So far there are no validated evaluation instruments for VP design published. Summary of work: We examined three sources of validity evidence of an instrument to be filled out by students aimed at measuring the quality of VPs with a special emphasis on fostering clinical reasoning: (1) Content was examined based on theory of clinical reasoning and an international VP expert team. (2) Response process was explored in think aloud pilot studies with students and content analysis of free text questions accompanying each item of the instrument. (3) Internal structure was assessed by confirmatory factor analysis (CFA) using 2547 student evaluations and reliability was examined utilizing generalizability analysis. Summary of results: Content analysis was supported by theory underlying Gruppen and Frohna’s clinical reasoning model on which the instrument is based and an international VP expert team. The pilot study and analysis of free text comments supported the validity of the instrument. The CFA indicated that a three factor model comprising 6 items showed a good fit with the data. Alpha coefficients per factor were 0,74 - 0,82. The findings of the generalizability studies indicated that 40-200 student responses are needed in order to obtain reliable data on one VP. Conclusions: The described instrument has the potential to provide faculty with reliable and valid information about VP design. Take-home messages: We present a short instrument which can be of help in evaluating the design of VPs.

Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: Virtual patients (VPs) are increasingly used to train clinical reasoning. So far, no validated evaluation instruments for VP design are available. Aims: We examined the validity of an instrument for assessing the perception of VP design by learners. Methods: Three sources of validity evidence were examined: (i) Content was examined based on theory of clinical reasoning and an international VP expert team. (ii) The response process was explored in think-aloud pilot studies with medical students and in content analyses of free text questions accompanying each item of the instrument. (iii) Internal structure was assessed by exploratory factor analysis (EFA) and inter-rater reliability by generalizability analysis. Results: Content analysis was reasonably supported by the theoretical foundation and the VP expert team. The think-aloud studies and analysis of free text comments supported the validity of the instrument. In the EFA, using 2547 student evaluations of a total of 78 VPs, a three-factor model showed a reasonable fit with the data. At least 200 student responses are needed to obtain a reliable evaluation of a VP on all three factors. Conclusion: The instrument has the potential to provide valid information about VP design, provided that many responses per VP are available.

An examination of the cross-cultural reliability and construct validity of a measure of loneliness

Loneliness is a pervasive, rather common experience in American culture, particularly notable among adolescents. However, the phenomenon is not well documented in the cross-cultural psychiatric literature. For psychiatric epidemiology to encompass a wide array of psychopathologic phenomena, it is important to develop useful measures to characterize and classify both non-clinical and clinical dysfunction in diverse subgroups and cultures.^ The goal of this research was to examine the cross-cultural reliability and construct validity of a scale designed to measure loneliness. The Roberts Loneliness Scale (RLS-8) was administered to 4,060 adolescents ages 10-19 years enrolled in high schools along either side of the Texas-Tamaulipas border region between the U.S. and Mexico. Data collected in 1988 from a study focusing on substance use and psychological distress among adolescents in these regions were used to examine the operating characteristics of the RLS-8. A sample stratified by nationality and language, age, gender, and grade was used for analysis.^ Results indicated that in general the RLS-8 has moderate reliability in the U.S. sample, but not in the Mexican sample. Validity analyses demonstrated that there was evidence for convergent validity of the RLS-8 in the U.S. sample, but none in the Mexican sample. Discriminant validity of the measures in neither sample could be established. Based on the factor structure of the RLS-8, two subscales were created and analyzed for construct validity. Evidence for convergent validity was established for both subscales in both national samples. However, the discriminant validity of the measure remains unsubstantiated in both national samples. Also, the dimensionality of the scale is unresolved.^ One primary goal for future cross-cultural research would be to develop and test better defined culture-specific models of loneliness within the two cultures. From such scientific endeavor, measures of loneliness can be developed or reconstructed to classify the phenomenon in the same manner across cultures. Since estimates of prevalence and incidence are contingent upon reliable and valid screening or diagnostic measures, this objective would serve as an important foundation for future psychiatric epidemiologic inquiry into loneliness. ^

Evaluating two morningness scales with item response theory

Using a student sample (n = 692) and an organization sample (n = 180), we scrutinized two morning-evening orientation scales using item response theory (IRT) methods. We used IRT to compare the measurement precision of the Composite Scale (CS) and the Early/Late Preferences Scale (PS). The CS had slightly higher measurement precision at all ranges of orientations, except for extreme morning and evening orientations for which the PS had slightly higher precision. IRT item-level statistics were also computed to try to understand how morning-orientation items functioned. Items that asked questions about morning activities tended to be more discriminating indicators of morning-orientation than items that asked about evening or peak performance activities. Items that involved unpleasant activities were less frequently endorsed than items that involved neutral or enjoyable activities. Implications for measurement of morning-evening orientation are discussed. (C) 2002 Elsevier Science Ltd. All rights reserved.

Aceitação e rejeição entre pares e habilidades sociais em universitários

Nesse estudo, investigaram-se as associações entre habilidades sociais e aceitação-rejeição em estudantes universitários. Foram estudados 126 estudantes universitários do curso de Educação Física de uma instituição de ensino superior particular de uma cidade do interior do estado de São Paulo. Para fins de avaliação foi utilizado um Inventário de Habilidades Sociais (IHS) e uma Medida Sociométrica que avalia a aceitação ou rejeição do estudante por seus colegas de classe para as atividades de sair e estudar. A aceitação para ambas as situações não foi explicada por nenhuma faceta das habilidades sociais para os homens. Já nas mulheres, foi explicada somente pela medida de autoexposição a desconhecidos. Essa pesquisa elucida algumas das condutas socialmente hábeis de homens e mulheres universitários que podem ser empregadas em treinamentos em habilidades sociais para se evitar a rejeição entre pares, e maximizar a aceitação.