Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: The design of Virtual Patients (VPs) is essential. So far there are no validated evaluation instruments for VP design published. Summary of work: We examined three sources of validity evidence of an instrument to be filled out by students aimed at measuring the quality of VPs with a special emphasis on fostering clinical reasoning: (1) Content was examined based on theory of clinical reasoning and an international VP expert team. (2) Response process was explored in think aloud pilot studies with students and content analysis of free text questions accompanying each item of the instrument. (3) Internal structure was assessed by confirmatory factor analysis (CFA) using 2547 student evaluations and reliability was examined utilizing generalizability analysis. Summary of results: Content analysis was supported by theory underlying Gruppen and Frohna’s clinical reasoning model on which the instrument is based and an international VP expert team. The pilot study and analysis of free text comments supported the validity of the instrument. The CFA indicated that a three factor model comprising 6 items showed a good fit with the data. Alpha coefficients per factor were 0,74 - 0,82. The findings of the generalizability studies indicated that 40-200 student responses are needed in order to obtain reliable data on one VP. Conclusions: The described instrument has the potential to provide faculty with reliable and valid information about VP design. Take-home messages: We present a short instrument which can be of help in evaluating the design of VPs.

Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: Virtual patients (VPs) are increasingly used to train clinical reasoning. So far, no validated evaluation instruments for VP design are available. Aims: We examined the validity of an instrument for assessing the perception of VP design by learners. Methods: Three sources of validity evidence were examined: (i) Content was examined based on theory of clinical reasoning and an international VP expert team. (ii) The response process was explored in think-aloud pilot studies with medical students and in content analyses of free text questions accompanying each item of the instrument. (iii) Internal structure was assessed by exploratory factor analysis (EFA) and inter-rater reliability by generalizability analysis. Results: Content analysis was reasonably supported by the theoretical foundation and the VP expert team. The think-aloud studies and analysis of free text comments supported the validity of the instrument. In the EFA, using 2547 student evaluations of a total of 78 VPs, a three-factor model showed a reasonable fit with the data. At least 200 student responses are needed to obtain a reliable evaluation of a VP on all three factors. Conclusion: The instrument has the potential to provide valid information about VP design, provided that many responses per VP are available.

Conservation of the Cinereous Vulture Aegypius monachus in Spain (1966–2011): a bibliometric review of threats, research and adaptive management

Detecting and quantifying threats and researching and implementing management actions are key to improving the conservation status of endangered species. Bibliometric analysis can constitute a useful tool for the evaluation of such questions from a long-term perspective. Taking as a case study the Cinereous Vulture Aegypius monachus in Spain, we tested relationships between population dynamics, research efforts, existing threats and conservation milestones. The population growth of the species (from 206 pairs in 1976 to 2,068 in 2011) was parallelled by the increase in the total number of publications, the number of articles in SCI journals and the number of published works dealing with aspects of conservation, threats and management. These results are discussed in terms of cause-effect relationships taking into account that the influence of other non-mutually exclusive factors could also probably explain such associations. Similarly, we analysed the trend of the Cinereous Vulture breeding population with respect to different threats and indices of food availability, obtaining a positive correlation with the increase in big-game hunting bags in Spain. With respect to conservation milestones, we concluded that the current situation is positive in terms of the protection of the species and its habitat, with the situation in relation to food availability being unclear. Finally, we reviewed the main conservation actions that have been taken for the species in Spain and how these have been progressively modified based on new scientific and technical evidence, as an example of adaptive management applied to conservation.

Tracing the Evidence: The Use of Evaluation Results in Direct-Democratic Campaigns

Although research on direct-democratic campaigns in Switzerland has intensified in the last decade, detailed information on the use of evidence in campaigns is still lacking. Our research aims to contribute both to research on direct democracy and to research on evidence-based policy making, by analyzing how evaluation results are used in directdemocratic campaigns. In this conceptual paper, the formulation of our hypothesis is based on a model of evaluation influence that traces the different uses of evaluation results in the process of a direct-democratic campaign. We assume that the policy analytical capacity of individual members in parliament, government and administration in the (pre)-parliamentary process fosters the use of evidence in campaigns. In the course of the campaign, symbolic use of evaluation in the form of justification, persuasion or mobilization prevails. We assume that the media is an important player in making transparent how political actors use evidence to support their positions. Evidence itself often remains ambiguous and uncertain, and evaluations are influenced by the values of the evaluator. To be able to make the right decisions, therefore, citizens should learn about possible interpretations in argumentative processes. For us, the context of direct democracy in Switzerland provides the setting for such a discourse that, besides evidence, brings up different opinions, values and beliefs.

Quality evaluation of tandem mass spectral libraries

Tandem mass spectral libraries are gaining more and more importance for the identification of unknowns in different fields of research, including metabolomics, forensics, toxicology, and environmental analysis. Particularly, the recent invention of reliable, robust, and transferable libraries has increased the general acceptance of these tools. Herein, we report on results obtained from thorough evaluation of the match reliabilities of two tandem mass spectral libraries: the MSforID library established by the Oberacher group in Innsbruck and the Weinmann library established by the Weinmann group in Freiburg. Three different experiments were performed: (1) Spectra of the libraries were searched against their corresponding library after excluding either this single compound-specific spectrum or all compound-specific spectra prior to searching; (2) the libraries were searched against each other using either library as reference set or sample set; (3) spectra acquired on different mass spectrometric instruments were matched to both libraries. Almost 13,000 tandem mass spectra were included in this study. The MSforID search algorithm was used for spectral matching. Statistical evaluation of the library search results revealed that principally both libraries enable the sensitive and specific identification of compounds. Due to higher mass accuracy of the QqTOF compared with the QTrap instrument, matches to the MSforID library were more reliable when comparing spectra with both libraries. Furthermore, only the MSforID library was shown to be efficiently transferable to different kinds of tandem mass spectrometers, including "tandem-in-time" instruments; this is due to the coverage of a large range of different collision energy settings-including the very low range-which is an outstanding characteristics of the MSforID library.

Retrospective adjustment of self-assessed medical competencies - noteworthy in the evaluation of postgraduate practical training courses

The efficacy of postgraduate practical training courses is frequently evaluated by self-assessment instruments. The present study analyses the effect of a basic course in laparoscopic surgery on self-assessed medical competencies.

A pain assessment scale for population-based studies: Development and validation of the Pain Module of the Standard Evaluation Questionnaire

The objectives of this study were to develop and validate a tool for assessing pain in population-based observational studies and to develop three subscales for back/neck, upper extremity and lower extremity pain. Based on a literature review, items were extracted from validated questionnaires and reviewed by an expert panel. The initial questionnaire consisted of a pain manikin and 34 items relating to (i) intensity of pain in different body regions (7 items), (ii) pain during activities of daily living (18 items) and (iii) various pain modalities (9 items). Psychometric validation of the initial questionnaire was performed in a random sample of the German-speaking Swiss population. Analyses included tests for reliability, correlation analysis, principal components factor analysis, tests for internal consistency and validity. Overall, 16,634 of 23,763 eligible individuals participated (70%). Test-retest reliability coefficients ranged from 0.32 to 0.97, but only three coefficients were below 0.60. Subscales were constructed combining four items for each of the subscales. Item-total coefficients ranged from 0.76 to 0.86 and Cronbach's alpha were 0.75 or higher for all subscales. Correlation coefficients between subscales and three validated instruments (WOMAC, SPADI and Oswestry) ranged from 0.62 to 0.79. The final Pain Standard Evaluation Questionnaire (SEQ Pain) included 28 items and the pain manikin and accounted for the multidimensionality of pain by assessing pain location and intensity, pain during activity, triggers and time of onset of pain and frequency of pain medication. It was found to be reliable and valid for the assessment of pain in population-based observational studies.

Posttraumatic stress disorder after implantation of a mechanical assist device followed by heart transplantation: evaluation of patients and partners

AIM: We sought to investigate the prevalence of posttraumatic stress disorder, anxiety, and depression in patients and their partners after implantation of a mechanical assist device as a bridge to heart transplantation. METHODS: This was a retrospective assessment of 41 patients (age 46.3 +/- 12.0 years; male-female ratio, 38:3; time since transplantation, 55.3 +/- 34.2 months [range, 7-122 months) and 27 partners (male-female ratio 2:25) by standardized instruments (Impact of Event Scale, Hospital Anxiety and Depression Scale), in 2 University Heart Transplant Centers (Vienna, Austria, Munster, Germany). The duration of the support systems (MicroMed DeBakey-VAD in 17 patients, Novacor in 10, Thoratec in 8, TCI HeartMate in 5, and Berlin Heart Incor in 1 patient) ranged from 28 to 711 (176 +/- 146) days. RESULTS: None of the patients, but 23% of the partners (n = 6), met the criteria for posttraumatic stress disorder (Maercker cutoff >0). The Impact of Event Scale (IES) sum scales differed significantly between the 2 groups (21.2 +/- 15.1, mean +/- SD) for the patients versus 38.1 +/- 27.8 for the partners, respectively; P = .001). Two percent of the patients, but 19% of the partners, showed mild to moderate depression; 4% of patients, but 23% of their partners, reported mild to moderate anxiety. None of the results were significantly influenced by the time since transplantation, patient age, diagnoses, type of assist device, or indication for heart transplantation. CONCLUSIONS: Despite patients being much closer to a life threat, their partners experience significantly more psychologic distress even in the long run. Our findings highlight the need for attention to the supporting persons.

Histological evaluation of human intrabony periodontal defects treated with an unsintered nanocrystalline hydroxyapatite paste

OBJECTIVES The aim of the study was to clinically and histologically evaluate the healing of human intrabony defects treated with open flap surgery (OFD) and application of a new, resorbable, fully synthetic, unsintered, nanocrystalline, phase-pure hydroxyapatite (nano-HA). MATERIALS AND METHODS Six patients, each of them displaying very advanced intrabony defects around teeth scheduled for extraction due to advanced chronic periodontitis and further prosthodontic considerations, were included in the study. Following local anaesthesia, mucoperiosteal flaps were reflected; the granulation tissue was removed, and the roots were meticulously debrided by hand and ultrasonic instruments. A notch was placed at the most apical extent of the calculus present on the root surface or at the most apical part of the defect (if no calculus was present) in order to serve as a reference for the histological evaluation. Following defect fill with nano-HA, the flaps were sutured by means of mattress sutures to allow primary intention healing. At 7 months after regenerative surgery, the teeth were extracted together with some of their surrounding soft and hard tissues and processed for histological analysis. RESULTS The postoperative healing was uneventful in all cases. At 7 months following surgery, mean PPD reduction and mean CAL gain measured 4.0 ± 0.8 and 2.5 ± 0.8 mm, respectively. The histological analysis revealed a healing predominantly characterized by epithelial downgrowth. Limited formation of new cementum with inserting connective tissue fibers and bone regeneration occurred in three out of the six biopsies (i.e. 0-0.86 and 0-1.33 mm, respectively). Complete resorption of the nano-HA was found in four out of the six biopsies. A few remnants of the graft particles (either surrounded by newly formed mineralized tissue or encapsulated in connective tissue) were found in two out of the six biopsies. CONCLUSION Within their limits, the present results indicate that nano-HA has limited potential to promote periodontal regeneration in human intrabony defects. CLINICAL RELEVANCE The clinical outcomes obtained following surgery with OFD + nano-HA may not reflect true periodontal regeneration.

Self-Reported Capabilities Among Young Male Adults in Switzerland: Translation and Psychometric Evaluation of a German, French and Italian Version of a Closed Survey Instrument

There is a shortage of empirical applications of the capability approach that employ closed survey instruments to assess self-reported capabilities. However, for those few instruments that have been designed and administered through surveys until now, no psychometric properties (reliability, validity, and factor structure) were reported. The purpose of this study is the assessment of the psychometric properties of three new language versions (German, French, and Italian) of an established (English) set of eight self-reported capability items. The set of items is taken from a previously published British study by Anand and van Hees (J Soc Econ 35(2):268–284, 2006). Our sample consists of 17,152 young male adults aged 18–25 years from the three major language regions in Switzerland. The results indicate good reliability of the three language versions. The results from the exploratory factor analyses suggest a one-dimensional factor structure for seven domain specific items. Furthermore, the results from multiple regression analyses suggest that a global summary item on overall capabilities represents a measurement alternative to the set of seven domain specific capability items. Finally, the results confirm the applicability of the closed capability instrument in a large scale survey questionnaire and represent the first attempt to measure self-reported capabilities in Switzerland.

Development and evaluation of a new instrument to measure visual exploration behavior

Effective visual exploration is required for many activities of daily living and instruments to assess visual exploration are important for the evaluation of the visual and the oculomotor system. In this article, the development of a new instrument to measure central and peripheral target recognition is described. The measurement setup consists of a hemispherical projection which allows presenting images over a large area of ±90° horizontal and vertical angle. In a feasibility study with 14 younger (21–49 years) and 12 older (50–78 years) test persons, 132 targets and 24 distractors were presented within naturalistic color photographs of everyday scenes at 10°, 30°, and 50° eccentricity. After the experiment, both younger and older participants reported in a questionnaire that the task is easy to understand, fun and that it measures a competence that is relevant for activities of daily living. A main result of the pilot study was that younger participants recognized more targets with smaller reaction times than older participants. The group differences were most pronounced for peripheral target detection. This test is feasible and appropriate to assess the functional field of view in younger and older adults.

A novel multidimensional geriatric screening tool in the ED: evaluation of feasibility and clinical relevance

PURPOSES Geriatric problems frequently go undetected in older patients in emergency departments (EDs), thus increasing their risk of adverse outcomes. We evaluated a novel emergency geriatric screening (EGS) tool designed to detect geriatric problems. BASIC PROCEDURES The EGS tool consisted of short validated instruments used to screen 4 domains (cognition, falls, mobility, and activities of daily living). Emergency geriatric screening was introduced for ED patients 75 years or older throughout a 4-month period. We analyzed the prevalence of abnormal EGS and whether EGS increased the number of EGS-related diagnoses in the ED during the screening, as compared with a preceding control period. MAIN FINDINGS Emergency geriatric screening was performed on 338 (42.5%) of 795 patients presenting during screening. Emergency geriatric screening was unfeasible in 175 patients (22.0%) because of life-threatening conditions and was not performed in 282 (35.5%) for logistical reasons. Emergency geriatric screening took less than 5 minutes to perform in most (85.8%) cases. Among screened patients, 285 (84.3%) had at least 1 abnormal EGS finding. In 270 of these patients, at least 1 abnormal EGS finding did not result in a diagnosis in the ED and was reported for further workup to subsequent care. During screening, 142 patients (42.0%) had at least 1 diagnosis listed within the 4 EGS domains, significantly more than the 29.3% in the control period (odds ratio 1.75; 95% confidence interval, 1.34-2.29; P<.001). Emergency geriatric screening predicted nursing home admission after the in-hospital stay (odds ratio for ≥3 vs <3 abnormal domains 12.13; 95% confidence interval, 2.79-52.72; P=.001). PRINCIPAL CONCLUSIONS The novel EGS is feasible, identifies previously undetected geriatric problems, and predicts determinants of subsequent care.

Evaluation of Medical Students’ Attitudes and Performance of Basic Surgery Skills in a Training Program Using Fresh Human skin, Excised During Body Contouring Surgeries

BACKGROUND: Learning surgical skills in the operating room may be a challenge for medical students. Therefore, more approaches using simulation to enable students to develop their practical skills are required. OBJECTIVES: We hypothesized that (1) there would be a need for additional surgical training for medical students in the pre-final year, and (2) our basic surgery skills training program using fresh human skin would improve medical students' surgical skills. DESIGN: We conducted a preliminary survey of medical students to clarify the need for further training in basic surgery procedures. A new approach using simulation to teach surgical skills on human skin was set up. The procedural skills of 15 randomly selected students were assessed in the operating room before and after participation in the simulation, using Objective Structured Assessment of Technical Skills. Furthermore, subjective assessment was performed based on students' self-evaluation. The data were analyzed using SPSS, version 21 (SPSS, Inc., Chicago, IL). SETTING: The study took place at the Inselspital, Bern University Hospital. PARTICIPANTS: A total of 186 pre-final-year medical students were enrolled into the preliminary survey; 15 randomly selected medical students participated in the basic surgical skills training course on the fresh human skin operating room. RESULTS: The preliminary survey revealed the need for a surgical skills curriculum. The simulation approach we developed showed significant (p < 0.001) improvement for all 12 surgical skills, with mean cumulative precourse and postcourse values of 31.25 ± 5.013 and 45.38 ± 3.557, respectively. The self-evaluation contained positive feedback as well. CONCLUSION: Simulation of surgery using human tissue samples could help medical students become more proficient in handling surgical instruments before stepping into a real surgical situation. We suggest further studies evaluating our proposed teaching method and the possibility of integrating this simulation approach into the medical school curriculum.