Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: The design of Virtual Patients (VPs) is essential. So far there are no validated evaluation instruments for VP design published. Summary of work: We examined three sources of validity evidence of an instrument to be filled out by students aimed at measuring the quality of VPs with a special emphasis on fostering clinical reasoning: (1) Content was examined based on theory of clinical reasoning and an international VP expert team. (2) Response process was explored in think aloud pilot studies with students and content analysis of free text questions accompanying each item of the instrument. (3) Internal structure was assessed by confirmatory factor analysis (CFA) using 2547 student evaluations and reliability was examined utilizing generalizability analysis. Summary of results: Content analysis was supported by theory underlying Gruppen and Frohna’s clinical reasoning model on which the instrument is based and an international VP expert team. The pilot study and analysis of free text comments supported the validity of the instrument. The CFA indicated that a three factor model comprising 6 items showed a good fit with the data. Alpha coefficients per factor were 0,74 - 0,82. The findings of the generalizability studies indicated that 40-200 student responses are needed in order to obtain reliable data on one VP. Conclusions: The described instrument has the potential to provide faculty with reliable and valid information about VP design. Take-home messages: We present a short instrument which can be of help in evaluating the design of VPs.

Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: Virtual patients (VPs) are increasingly used to train clinical reasoning. So far, no validated evaluation instruments for VP design are available. Aims: We examined the validity of an instrument for assessing the perception of VP design by learners. Methods: Three sources of validity evidence were examined: (i) Content was examined based on theory of clinical reasoning and an international VP expert team. (ii) The response process was explored in think-aloud pilot studies with medical students and in content analyses of free text questions accompanying each item of the instrument. (iii) Internal structure was assessed by exploratory factor analysis (EFA) and inter-rater reliability by generalizability analysis. Results: Content analysis was reasonably supported by the theoretical foundation and the VP expert team. The think-aloud studies and analysis of free text comments supported the validity of the instrument. In the EFA, using 2547 student evaluations of a total of 78 VPs, a three-factor model showed a reasonable fit with the data. At least 200 student responses are needed to obtain a reliable evaluation of a VP on all three factors. Conclusion: The instrument has the potential to provide valid information about VP design, provided that many responses per VP are available.

Adolescent tobacco use: Studies of prevalence, factorial validity, and factorial invariance between African Americans and Whites

In the last thirty years, increasing efforts have been made to reduce the prevalence of adolescent tobacco use in the United States. Although the prevalence has declined dramatically over the past decade, there are still sharp differences in adolescent smoking-initiation rates across racial/ethnic groups. Large-scale surveys frequently assess smoking-related attitudes, self-efficacy, and intentions to explain the differences in smoking rates between African Americans and Whites. However, there is little agreement about which constructs are significant. Moreover, the psychometric properties of smoking-related attitude, self-efficacy, and intention constructs have not been fully examined. More studies are needed to understand existing patterns of tobacco use and to validate and fully exploit the constructs' relationship to adolescent smoking initiation across racial/ethnic groups. ^ This dissertation reports on a secondary analysis of data from a large multi-ethnic convenience sample of sixth- through eighth-grade students in 22 schools in East Texas and the city of Houston. The specific aims of this dissertation were to (1) describe smoking and alternate tobacco product use rates by race/ethnicity, gender, age, and grade level (Article 1); (2) test the factorial validity of smoking-related attitudes, self-efficacy, and intentions using confirmatory factor analysis techniques (Article 2); and (3) test the factorial invariance of smoking-related attitudes, self-efficacy, and intentions between African Americans and Whites (Article 3). ^ The prevalence findings confirm the disparities in tobacco use among African American, Hispanic, and White adolescents that other surveys have reported (Article 1). This study also demonstrates the usefulness of examining use patterns of not only cigarettes but also alternative tobacco products in younger multiethnic populations, as well as of providing epidemiological data estimates about different phases of smoking. The confirmatory factor analysis provides evidence of construct validity of attitude, self-efficacy, and intention scales for the multiethnic sample (Article 2). Finally, the factorial invariance analyses indicates that some measures representing smoking-related attitudes, self-efficacy, and intentions may not be appropriate for use among both African Americans and Whites (Article 3). Additional research is needed to further our understanding of the patterns and predictors of youth tobacco use initiation. ^

Affect and cognition measures in preference-based decisions: Validity testing of the Ottawa Decisional Conflict Scale and a decision-specific anxiety measure with men eligible for prostate cancer screening

Background. At present, prostate cancer screening (PCS) guidelines require a discussion of risks, benefits, alternatives, and personal values, making decision aids an important tool to help convey information and to help clarify values. Objective: The overall goal of this study is to provide evidence of the reliability and validity of a PCS anxiety measure and the Decisional Conflict Scale (DCS). Methods. Using data from a randomized, controlled PCS decision aid trial that measured PCS anxiety at baseline and DCS at baseline (T0) and at two-weeks (T2), four psychometric properties were assessed: (1) internal consistency reliability, indicated by factor analysis intraclass correlations and Cronbach's α; (2) construct validity, indicated by patterns of Pearson correlations among subscales; (3) discriminant validity, indicated by the measure's ability to discriminate between undecided men and those with a definite screening intention; and (4) factor validity and invariance using confirmatory factor analyses (CFA). Results. The PCS anxiety measure had adequate internal consistency reliability and good construct and discriminant validity. CFAs indicated that the 3-factor model did not have adequate fit. CFAs for a general PCS anxiety measure and a PSA anxiety measure indicated adequate fit. The general PCS anxiety measure was invariant across clinics. The DCS had adequate internal consistency reliability except for the support subscale and had adequate discriminate validity. Good construct validity was found at the private clinic, but was only found for the feeling informed subscale at the public clinic. The traditional DCS did not have adequate fit at T0 or at T2. The alternative DCS had adequate fit at T0 but was not identified at T2. Factor loadings indicated that two subscales, feeling informed and feeling clear about values, were not distinct factors. Conclusions. Our general PCS anxiety measure can be used in PCS decision aid studies. The alternative DCS may be appropriate for men eligible for PCS. Implications: More emphasis needs to be placed on the development of PCS anxiety items relating to testing procedures. We recommend that the two DCS versions be validated in other samples of men eligible for PCS and in other health care decisions that involve uncertainty. ^

Mistreatment and self-neglect in community dwelling older adults: Studies of construct validity, measurement invariance and latent classes

Mistreatment and self-neglect significantly increase the risk of dying in older adults. It is estimated that 1 to 2 million older adults experience elder mistreatment and self-neglect every year in the United States. Currently, there are no elder mistreatment and self-neglect assessment tools with construct validity and measurement invariance testing and no studies have sought to identify underlying latent classes of elder self-neglect that may have differential mortality rates. Using data from 11,280 adults with Texas APS substantiated elder mistreatment and self-neglect 3 studies were conducted to: (1) test the construct validity and (2) the measurement invariance across gender and ethnicity of the Texas Adult Protective Services (APS) Client Assessment and Risk Evaluation (CARE) tool and (3) identify latent classes associated with elder self-neglect. Study 1 confirmed the construct validity of the CARE tool following adjustments to the initial hypothesized CARE tool. This resulted in the deletion of 14 assessment items and a final assessment with 5 original factors and 43 items. Cross-validation for this model was achieved. Study 2 provided empirical evidence for factor loading and item-threshold invariance of the CARE tool across gender and between African-Americans and Caucasians. The financial status domain of the CARE tool did not function properly for Hispanics and thus, had to be deleted. Subsequent analyses showed factor loading and item-threshold invariance across all 3 ethnic groups with the exception of some residual errors. Study 3 identified 4-latent classes associated with elder self-neglect behaviors which included individuals with evidence of problems in the areas of (1) their environment, (2) physical and medical status, (3) multiple domains and (4) finances. Overall, these studies provide evidence supporting the use of APS CARE tool for providing unbiased and valid investigations of mistreatment and neglect in older adults with different demographic characteristics. Furthermore, the findings support the underlying notion that elder self-neglect may not only occur along a continuum, but that differential types may exist. All of which, have very important potential implications for social and health services distributed to vulnerable mistreated and neglected older adults.^

The Spanish version of the Prenatal Breast-feeding Self-efficacy Scale: Reliability and validity assessment

Background: Only a minority of infants are exclusively breastfed for the recommended 6 months postpartum. Breast-feeding self-efficacy is a mother's confidence in her ability to breastfeed and is predictive of breastfeeding behaviors. The Prenatal Breast-feeding Self-efficacy Scale (PBSES) was developed among English-speaking mothers to measure breastfeeding self-efficacy before delivery. Objectives: To translate the PBSES into Spanish and assess its psychometric properties. Design: Reliability and validity assessment. Setting: A public hospital in Yecla, Spain. Participants: A convenience sample of 234 pregnant women in their third trimester of pregnancy. Methods: The PBSES was translated into Spanish using forward and back translation. A battery of self-administered questionnaires was completed by participants, including a questionnaire on sociodemographic variables, breastfeeding experience and intention, as well as the Spanish version of the PBSES. Also, data on exclusive breastfeeding at discharge were collected from hospital database. Dimensional structure, internal consistency and construct validity of the Spanish version of PBSES were assessed. Results: Confirmatory factor analysis suggested the presence of one construct, self-efficacy, with four dimensions or latent variables. Cronbach's alpha coefficient for internal consistency was 0.91. Response patterns based on decision to breastfeed during pregnancy provided evidence of construct validity. In addition, the scores of the Spanish version of the PBSES significantly predicted exclusive breastfeeding at discharge. Conclusions: The Spanish version of PBSES shows evidences of reliability, and contrasting group and predictive validity. Confirmatory factor analysis indicated marginal fit and further studies are needed to provide new evidence on the structure of the scale. The Spanish version of the PBSES can be considered a reliable measure and shows validity evidences.

The Questionnaire about Interpersonal Difficulties for Adolescents: Reliability and Validity Evidence in Colombian Adolescents

The aim of this study was to analyze reliability and validity evidence of scores on the Colombian version of the Questionnaire about Interpersonal Difficulties for Adolescents (QIDA) in a sample of 1,628 adolescents (65.7% boys), ranging in age from 12 to 18 years. Confirmatory factor analyses replicated the correlated five-factor structure of the QIDA: Assertiveness, Heterosexual Relationships, Public Speaking, Family Relationships, and Close Friendships. Internal consistency for the QIDA and subscales scores was excellent. Girls reported higher level of perceived anxiety in heterosexual relationships, whereas boys showed more anxiety in close friendships and decreased interpersonal anxiety during adolescence. Results support the reliability and validity of the scores on the Colombian version of the QIDA.

Validity of self-reported skin screening histories

Screening by whole-body clinical skin examination may improve early diagnosis of melanoma and reduce mortality, but objective scientific evidence of this is lacking. As part of a randomized controlled trial of population screening for melanoma in Queensland, Australia, the authors assessed the validity of self-reported history of whole-body skin examination and factors associated with accuracy of recall among 2,704 participants in 2001. Approximately half of the participants were known to have undergone whole-body skin examination within the past 3 years at skin screening clinics conducted as part of the randomized trial. All positive and negative self-reports were compared with screening clinic records. Where possible, reports of skin examinations conducted outside the clinics were compared with private medical records. The validity of self-reports of whole-body skin examination in the past 3 years was high: Concordance between self-reports and medical records was 93.7%, sensitivity was 92.0%, and specificity was 96.3%. Concordance was lower (74.3%) for self-reports of examinations conducted in the past 12 months, and there was evidence of telescoping in recall for this more recent time frame. In multivariate analysis, women and younger participants more accurately recalled their history of skin examinations. Participants with a history of melanoma did not differ from other participants in their accuracy of recall.

The development of rubrics to measure undergraduate students' global awareness and global perspective: A validity study

Higher education institutions across the United States have developed global learning initiatives to support student achievement of global awareness and global perspective, but assessment options for these outcomes are extremely limited. A review of research for a global learning initiative at a large, Hispanic-serving, urban, public, research university in South Florida found a lack of instruments designed to measure global awareness and global perspective in the context of an authentic performance assessment. This quasi-experimental study explored the development of two rubrics for the global learning initiative and the extent to which evidence supported the rubrics' validity and reliability. One holistic rubric was developed to measure students' global awareness and the second to measure their global perspective. The study utilized a pretest/posttest nonequivalent group design. Multiple linear regression was used to ascertain the rubrics' ability to discern and compare average learning gains of undergraduate students enrolled in two global learning courses and students enrolled in two non-global learning courses. Parallel pretest/posttest forms of the performance task required students to respond to two open-ended questions, aligned with the learning outcomes, concerning a complex case narrative. Trained faculty raters read responses and used the rubrics to measure students' global awareness and perspective. Reliability was tested by calculating the rates of agreement among raters. Evidence supported the finding that the global awareness and global perspective rubrics yielded scores that were highly reliable measures of students' development of these learning outcomes. Chi-square tests of frequency found significant rates of inter-rater agreement exceeding the study's .80 minimum requirement. Evidence also supported the finding that the rubrics yielded scores that were valid measures of students' global awareness and global perspective. Regression analyses found little evidence of main effects; however, post hoc analyses revealed a significant interaction between global awareness pretest scores and the treatment, the global learning course. Significant interaction was also found between global perspective pretest scores and the treatment. These crossover interactions supported the finding that the global awareness and global perspective rubrics could be used to detect learning differences between the treatment and control groups as well as differences within the treatment group.

Syphilis at the Crossroad of Phylogenetics and Paleopathology

The origin of syphilis is still controversial. Different research avenues explore its fascinating history. Here we employed a new integrative approach, where paleopathology and molecular analyses are combined. As an exercise to test the validity of this approach we examined different hypotheses on the origin of syphilis and other human diseases caused by treponemes (treponematoses). Initially, we constructed a worldwide map containing all accessible reports on palaeopathological evidences of treponematoses before Columbus's return to Europe. Then, we selected the oldest ones to calibrate the time of the most recent common ancestor of Treponema pallidum subsp. pallidum, T. pallidum subsp. endemicum and T. pallidum subsp. pertenue in phylogenetic analyses with 21 genetic regions of different T. pallidum strains previously reported. Finally, we estimated the treponemes' evolutionary rate to test three scenarios: A) if treponematoses accompanied human evolution since Homo erectus; B) if venereal syphilis arose very recently from less virulent strains caught in the New World about 500 years ago, and C) if it emerged in the Americas between 16,500 and 5,000 years ago. Two of the resulting evolutionary rates were unlikely and do not explain the existent osseous evidence. Thus, treponematoses, as we know them today, did not emerge with H. erectus, nor did venereal syphilis appear only five centuries ago. However, considering 16,500 years before present (yBP) as the time of the first colonization of the Americas, and approximately 5,000 yBP as the oldest probable evidence of venereal syphilis in the world, we could not entirely reject hypothesis C. We confirm that syphilis seems to have emerged in this time span, since the resulting evolutionary rate is compatible with those observed in other bacteria. In contrast, if the claims of precolumbian venereal syphilis outside the Americas are taken into account, the place of origin remains unsolved. Finally, the endeavor of joining paleopathology and phylogenetics proved to be a fruitful and promising approach for the study of infectious diseases.

Direct evidence for the expression of multiple endogenous retroviruses in the synovial compartment in rheumatoid arthritis

Objective. Circumstantial evidence links retroviruses (RVs) with human autoimmune diseases, The aim of the present study was to obtain direct evidence of RV gene expression in rheumatoid arthritis (RA). Methods. Synovial samples were obtained from patients with RA, patients with osteoarthritis (OA), and normal control subjects, Reverse transcription-polymerase chain reaction (RT-PCR) was performed using synovial RNA and primers to conserved sequences in the polymerase (pol) genes of known RVs. Results. PCR products (n = 857) were cloned and sequenced, Multiple pol transcripts, many with open reading frames, were expressed in every sample, Sequences were aligned and classified into 6 families (F1-F6) that contained 33 groups of known and unknown endogenous RVs (ERVs), each distinguished by a specific, deduced peptide motif, The frequency of sequences in each family was similar between RA, OA, and normal synovial tissue, but differed significantly in RA synovial fluid cells, F1 sequences (undefined, but related to murine and primate type C RVs) were lower in frequency, F2 (ERV-9-related), F4 (HERV-K-related), and F6 (HERV-L-related) sequences were higher in frequency, and F3 (RTVL-H-related) sequences were not detected, in the RA synovial fluid cells compared with the RA synovial tissues. Conclusion. Multiple ERVs are expressed in normal and diseased synovial compartments, but specific transcripts can be differentially expressed in RA.

Feasibility of an evidence-based literature search service for general practitioners

Objective: To test the feasibility of an evidence-based clinical literature search service to help answer general practitioners' (GPs') clinical questions. Design: Two search services supplied GPs who submitted questions with the best available empirical evidence to answer these questions. The GPs provided feedback on the value of the service, and concordance of answers from the two search services was assessed. Setting: Two literature search services (Queensland and Victoria), operating for nine months from February 1999. Main outcome measures: Use of the service; time taken to locate answers; availability of evidence; value of the service to GPs; and consistency of answers from the two services. Results: 58 GPs asked 160 questions (29 asked one, 11 asked five or more). The questions concerned treatment (65%), aetiology (17%), prognosis (13%), and diagnosis (5%). Answering a question took a mean of 3 hours 32 minutes of personnel time (95% Cl, 2.67-3.97); nine questions took longer than 10 hours each to answer, the longest taking 23 hours 30 minutes. Evidence of suitable quality to provide a sound answer was available for 126 (79%) questions. Feedback data for 84 (53%) questions, provided by 42 GPs, showed that they appreciated the service, and asking the questions changed clinical care. There were many minor differences between the answers from the two centres, and substantial differences in the evidence found for 4/14 questions. However, conclusions reached were largely similar, with no or only minor differences for all questions. Conclusions: It is feasible to provide a literature search service, but further assessment is needed to establish its cost effectiveness.

A molecular comparison of T lymphocyte populations infiltrating the liver and circulating in the blood of patients with chronic hepatitis B: Evidence for antigen-driven selection of a public complementarity-determining region 3 (CDR3) motif

Despite a large number of T cells infiltrating the liver of patients with chronic hepatitis B, little is known about their complexity or specificity. To characterize the composition of these T cells involved with the pathogenesis of chronic hepatitis B (CHB), we have studied the clonality of V beta T cell receptor (TCR)-bearing populations in liver tissue by size spectratyping the complementarity-determining region (CDR3) lengths of TCR transcripts. We have also compared the CDR3 profiles of the lymphocytes infiltrating the liver with those circulating in the blood to see whether identical clonotypes may be detected that would indicate a virus-induced expansion in both compartments. Our studies show that in most of the patients examined, the T cell composition of liver infiltrating lymphocytes is highly restricted, with evidence of clonotypic expansions in 4 to 9 TCR V beta subfamilies. In contrast, the blood compartment contains an average of 1 to 3 expansions. This pattern is seen irrespective of the patient's viral load or degree of liver pathology. Although the TCR repertoire profiles between the 2 compartments are generally distinct, there is evidence of some T cell subsets being equally distributed between the blood and the liver. Finally, we provide evidence for a putative public binding motif within the CDR3 region with the sequence G-X-S, which may be involved with hepatitis B virus recognition.

A cross-sectional study of the association between Heberden's nodes, radiographic osteoarthritis of the hands, grip strength, disability and pain

Objective: To describe the associations between hand osteoarthritis (OA), pain and disability in males and females and to further validate the Australian/Canadian CA hand index (AUSCAN LK3.0). Design: Cross-sectional study of 522 subjects from 101 Tasmanian families (males N=174, females N=348). Hand OA was assessed by two observers using the Altman atlas for joint space narrowing and osteophytes at distal interphalangeal and first carpometacarpal joints as well as a score for Heberden's nodes based on hand photography. Hand pain and function were assessed by the AUSCAN LK3.0 and grip strength by dynamometry in both hands on two occasions. Results: The prevalence of hand CA was high in this sample at 44-71% (depending on site). Pain and dysfunction increased with age while grip strength decreased (all P <0.001). All three measures were markedly worse in women, even after taking the severity of arthritis into account. Hand CA explained 5.7-10% of the variation in function, grip strength and pain scores, even after adjustment for age and sex. Further adjustment suggested that the osteoarthritic associations with function and grip strength were largely mediated by pain. Severity of disease was more strongly associated with these scores than presence or absence. Lastly, the AUSCAN LK3.0 showed a comparable association to grip strength with structural damage providing further evidence of index validity. Conclusions: Hand CA at these two sites makes substantial contributions to hand function, strength and pain. The associations with function and strength measures appear mediated by pain. Gender differences in all three measures persist after adjustment for variation in age and CA severity indicating that factors apart from radiographic disease are responsible. (C) 2001 OsteoArthritis Research Society International.

The rheology of flotation froths

A literature review has highlighted the need to measure flotation froth rheology in order to fully characterise the role of the froth in the flotation process. The initial investigation using a coaxial cylinder viscometer for froth rheology measurement led to the development of a new device employing a vane measuring head. The modified rheometer was used in industrial scale flotation tests at Mt. Isa Copper Concentrator. The measured froth rheograms show a non-Newtonian nature for the flotation froths (pseudoplastic flow). The evidence of the non-Newtonian flow has questioned the validity of application of the Laplace equation in froth motion modelling as used by a number of researchers, since the assumption of irrotational flow is violated. Correlations between the froth rheology and the froth retention time, water hold-up in the froth and concentrate grades have been found. These correlations are independent of air flow rate (test data at various air flow rates fall on one similar trend line). This implies that froth rheology may be used as a lumped parameter for other operating variables in flotation modelling and scale up. (C) 2003 Elsevier Science B.V. All rights reserved.