Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: Virtual patients (VPs) are increasingly used to train clinical reasoning. So far, no validated evaluation instruments for VP design are available. Aims: We examined the validity of an instrument for assessing the perception of VP design by learners. Methods: Three sources of validity evidence were examined: (i) Content was examined based on theory of clinical reasoning and an international VP expert team. (ii) The response process was explored in think-aloud pilot studies with medical students and in content analyses of free text questions accompanying each item of the instrument. (iii) Internal structure was assessed by exploratory factor analysis (EFA) and inter-rater reliability by generalizability analysis. Results: Content analysis was reasonably supported by the theoretical foundation and the VP expert team. The think-aloud studies and analysis of free text comments supported the validity of the instrument. In the EFA, using 2547 student evaluations of a total of 78 VPs, a three-factor model showed a reasonable fit with the data. At least 200 student responses are needed to obtain a reliable evaluation of a VP on all three factors. Conclusion: The instrument has the potential to provide valid information about VP design, provided that many responses per VP are available.

European undergraduate curriculum in geriatric medicine developed using an international modified Delphi technique.

INTRODUCTION the rise in the number of older, frail adults necessitates that future doctors are adequately trained in the skills of geriatric medicine. Few countries have dedicated curricula in geriatric medicine at the undergraduate level. The aim of this project was to develop a consensus among geriatricians on a curriculum with the minimal requirements that a medical student should achieve by the end of medical school. METHODS a modified Delphi process was used. First, educational experts and geriatricians proposed a set of learning objectives based on a literature review. Second, three Delphi rounds involving a panel with 49 experts representing 29 countries affiliated to the European Union of Medical Specialists (UEMS) was used to gain consensus for a final curriculum. RESULTS the number of disagreements following Delphi Rounds 1 and 2 were 81 and 53, respectively. Complete agreement was reached following the third round. The final curriculum consisted of detailed objectives grouped under 10 overarching learning outcomes. DISCUSSION a consensus on the minimum requirements of geriatric learning objectives for medical students has been agreed by European geriatricians. Major efforts will be needed to implement these requirements, given the large variation in the quality of geriatric teaching in medical schools. This curriculum is a first step to help improve teaching of geriatrics in medical schools, and will also serve as a basis for advancing postgraduate training in geriatrics across Europe.

Lernzielkatalog für die studentische Lehre im Fachbereich „Geriatrie“ an Fakultäten für Humanmedizin.

Zusammenfassung Die Betreuung geriatrischer Patientinnen und Patienten setzt, nebst einer entsprechenden Haltung, fundierte Kenntnisse in Diagnostik und Behandlung praktisch aller medizinischen Fachgebiete voraus. Daher ist es wichtig, dass die Kompetenz von Studierenden der Humanmedizin im Bereich Geriatrie entsprechend gefördert wird. Bis heute hat jedoch die studentische Ausbildung im Fach Geriatrie an vielen europäischen Universitäten einen unklaren oder untergeordneten Stellenwert. Als ersten Schritt zur Förderung der Lehre in der Geriatrie hat die Europäische Facharztvereinigung Geriatrie (UEMS-GMS) in einem Delphi-Prozess einen Lernzielkatalog entwickelt. Dieser Katalog enthält die Mindestanforderungen mit spezifischen Lernzielen (Wissen, Fertigkeiten und Haltungen), welche die Studierenden der Humanmedizin bezüglich Geriatrie bis zum Abschluss des Medizinstudiums erwerben sollen. Zur Förderung der Implementierung dieses neuen, kompetenzbasierten Lernzielkatalogs an den deutschsprachigen Universitäten wurde eine an den Sprachgebrauch des „DACH-Raums“ (Deutschland, Österreich und Schweiz) angepasste deutsche Version erstellt. Im vorliegenden Beitrag wird diese Übersetzung vorgestellt. Die Fachgesellschaften für Geriatrie aus Deutschland, Österreich und der Schweiz empfehlen den medizinischen Fakultäten der jeweiligen Länder, diesen Katalog umzusetzen.

Comparing three CPR feedback devices and standard BLS in a single rescuer scenario: a randomised simulation study

BACKGROUND Efficiently performed basic life support (BLS) after cardiac arrest is proven to be effective. However, cardiopulmonary resuscitation (CPR) is strenuous and rescuers' performance declines rapidly over time. Audio-visual feedback devices reporting CPR quality may prevent this decline. We aimed to investigate the effect of various CPR feedback devices on CPR quality. METHODS In this open, prospective, randomised, controlled trial we compared three CPR feedback devices (PocketCPR, CPRmeter, iPhone app PocketCPR) with standard BLS without feedback in a simulated scenario. 240 trained medical students performed single rescuer BLS on a manikin for 8min. Effective compression (compressions with correct depth, pressure point and sufficient decompression) as well as compression rate, flow time fraction and ventilation parameters were compared between the four groups. RESULTS Study participants using the PocketCPR performed 17±19% effective compressions compared to 32±28% with CPRmeter, 25±27% with the iPhone app PocketCPR, and 35±30% applying standard BLS (PocketCPR vs. CPRmeter p=0.007, PocketCPR vs. standard BLS p=0.001, others: ns). PocketCPR and CPRmeter prevented a decline in effective compression over time, but overall performance in the PocketCPR group was considerably inferior to standard BLS. Compression depth and rate were within the range recommended in the guidelines in all groups. CONCLUSION While we found differences between the investigated CPR feedback devices, overall BLS quality was suboptimal in all groups. Surprisingly, effective compression was not improved by any CPR feedback device compared to standard BLS. All feedback devices caused substantial delay in starting CPR, which may worsen outcome.

Sustained impact of a short small group course with systematic feedback in addition to regular clinical clerkship activities on musculoskeletal examination skills-a controlled study.

BACKGROUND The discrepancy between the extensive impact of musculoskeletal complaints and the common deficiencies in musculoskeletal examination skills lead to increased emphasis on structured teaching and assessment. However, studies of single interventions are scarce and little is known about the time-dependent effect of assisted learning in addition to a standard curriculum. We therefore evaluated the immediate and long-term impact of a small group course on musculoskeletal examination skills. METHODS All 48 Year 4 medical students of a 6 year curriculum, attending their 8 week clerkship of internal medicine at one University department in Berne, participated in this controlled study. Twenty-seven students were assigned to the intervention of a 6×1 h practical course (4-7 students, interactive hands-on examination of real patients; systematic, detailed feedback to each student by teacher, peers and patients). Twenty-one students took part in the regular clerkship activities only and served as controls. In all students clinical skills (CS, 9 items) were assessed in an Objective Structured Clinical Examination (OSCE) station, including specific musculoskeletal examination skills (MSES, 7 items) and interpersonal skills (IPS, 2 items). Two raters assessed the skills on a 4-point Likert scale at the beginning (T0), the end (T1) and 4-12 months after (T2) the clerkship. Statistical analyses included Friedman test, Wilcoxon rank sum test and Mann-Whitney U test. RESULTS At T0 there were no significant differences between the intervention and control group. At T1 and T2 the control group showed no significant changes of CS, MSES and IPS compared to T0. In contrast, the intervention group significantly improved CS, MSES and IPS at T1 (p < 0.001). This enhancement was sustained for CS and MSES (p < 0.05), but not for IPS at T2. CONCLUSIONS Year 4 medical students were incapable of improving their musculoskeletal examination skills during regular clinical clerkship activities. However, an additional small group, interactive clinical skills course with feedback from various sources, improved these essential examination skills immediately after the teaching and several months later. We conclude that supplementary specific teaching activities are needed. Even a single, short-lasting targeted module can have a long lasting effect and is worth the additional effort.

Improving Pediatric Basic Life Support Performance Through Blended Learning With Web-Based Virtual Patients: Randomized Controlled Trial

BACKGROUND E-learning and blended learning approaches gain more and more popularity in emergency medicine curricula. So far, little data is available on the impact of such approaches on procedural learning and skill acquisition and their comparison with traditional approaches. OBJECTIVE This study investigated the impact of a blended learning approach, including Web-based virtual patients (VPs) and standard pediatric basic life support (PBLS) training, on procedural knowledge, objective performance, and self-assessment. METHODS A total of 57 medical students were randomly assigned to an intervention group (n=30) and a control group (n=27). Both groups received paper handouts in preparation of simulation-based PBLS training. The intervention group additionally completed two Web-based VPs with embedded video clips. Measurements were taken at randomization (t0), after the preparation period (t1), and after hands-on training (t2). Clinical decision-making skills and procedural knowledge were assessed at t0 and t1. PBLS performance was scored regarding adherence to the correct algorithm, conformance to temporal demands, and the quality of procedural steps at t1 and t2. Participants' self-assessments were recorded in all three measurements. RESULTS Procedural knowledge of the intervention group was significantly superior to that of the control group at t1. At t2, the intervention group showed significantly better adherence to the algorithm and temporal demands, and better procedural quality of PBLS in objective measures than did the control group. These aspects differed between the groups even at t1 (after VPs, prior to practical training). Self-assessments differed significantly only at t1 in favor of the intervention group. CONCLUSIONS Training with VPs combined with hands-on training improves PBLS performance as judged by objective measures.

Cost-effectiveness of peer role play and standardized patients in undergraduate communication training.

BACKGROUND The few studies directly comparing the methodological approach of peer role play (RP) and standardized patients (SP) for the delivery of communication skills all suggest that both methods are effective. In this study we calculated the costs of both methods (given comparable outcomes) and are the first to generate a differential cost-effectiveness analysis of both methods. METHODS Medical students in their prefinal year were randomly assigned to one of two groups receiving communication training in Pediatrics either with RP (N = 34) or 19 individually trained SP (N = 35). In an OSCE with standardized patients using the Calgary-Cambridge Referenced Observation Guide both groups achieved comparable high scores (results published). In this study, corresponding costs were assessed as man-hours resulting from hours of work of SP and tutors. A cost-effectiveness analysis was performed. RESULTS Cost-effectiveness analysis revealed a major advantage for RP as compared to SP (112 vs. 172 man hours; cost effectiveness ratio .74 vs. .45) at comparable performance levels after training with both methods. CONCLUSIONS While both peer role play and training with standardized patients have their value in medical curricula, RP has a major advantage in terms of cost-effectiveness. This could be taken into account in future decisions.

Computer-based summative assessment of clinical reasoning in clerkships: A mixed-method comparison of key feature problems with case-based multiple choice questions

Background: It is yet unclear if there are differences between using electronic key feature problems (KFPs) or electronic case-based multiple choice questions (cbMCQ) for the assessment of clinical decision making. Summary of Work: Fifth year medical students were exposed to clerkships which ended with a summative exam. Assessment of knowledge per exam was done by 6-9 KFPs, 9-20 cbMCQ and 9-28 MC questions. Each KFP consisted of a case vignette and three key features (KF) using “long menu” as question format. We sought students’ perceptions of the KFPs and cbMCQs in focus groups (n of students=39). Furthermore statistical data of 11 exams (n of students=377) concerning the KFPs and (cb)MCQs were compared. Summary of Results: The analysis of the focus groups resulted in four themes reflecting students’ perceptions of KFPs and their comparison with (cb)MCQ: KFPs were perceived as (i) more realistic, (ii) more difficult, (iii) more motivating for the intense study of clinical reasoning than (cb)MCQ and (iv) showed an overall good acceptance when some preconditions are taken into account. The statistical analysis revealed that there was no difference in difficulty; however KFP showed a higher discrimination and reliability (G-coefficient) even when corrected for testing times. Correlation of the different exam parts was intermediate. Conclusions: Students perceived the KFPs as more motivating for the study of clinical reasoning. Statistically KFPs showed a higher discrimination and higher reliability than cbMCQs. Take-home messages: Including KFPs with long menu questions into summative clerkship exams seems to offer positive educational effects.

Improving the Assessment of Communication Skills at the Swiss Federal Licensing Exam: Results of a Student's Needs Assessment

Introduction With a three year project the assessment of communication skills within the Swiss Federal Licensing Examinations (FLE) shall be improved. As a first step a needs assessment among communication experts and medical students of the Swiss Medical Faculties will be performed. In this presentation the results of the students’ needs assessment will be presented. Methods A bilingual student’s online questionnaire will be developed by an expert panel taking relevant literature, the Swiss Catalogue of Learning Objectives and other consensus statements for communication (e.g., the European and Basler consensus statements) into account. With a think aloud study response process validity evidence will be sought. The questionnaire will focus on the following topics related to communication skills: (1) What has been taught?, (2) What has been assessed in the faculty exams?, (3) What has been assessed in the FLE?, (4) What should have been assessed in the FLE and how should the assessment be improved? Results Results of the students’ needs assessment will be available by the end of 2015 and be presented. Conclusions/ Take-home message We hope for valuable input for improving the assessment of communications skills within the FLE also from the students’ side. Results of the needs assessment from the students and experts will be combined and taken as input for an international expert symposium on how to improve the communication skills assessment within the FLE.

Short structured feedback training is equivalent to a mechanical feedback device in two-rescuer BLS: a randomised simulation study.

BACKGROUND Resuscitation guidelines encourage the use of cardiopulmonary resuscitation (CPR) feedback devices implying better outcomes after sudden cardiac arrest. Whether effective continuous feedback could also be given verbally by a second rescuer ("human feedback") has not been investigated yet. We, therefore, compared the effect of human feedback to a CPR feedback device. METHODS In an open, prospective, randomised, controlled trial, we compared CPR performance of three groups of medical students in a two-rescuer scenario. Group "sCPR" was taught standard BLS without continuous feedback, serving as control. Group "mfCPR" was taught BLS with mechanical audio-visual feedback (HeartStart MRx with Q-CPR-Technology™). Group "hfCPR" was taught standard BLS with human feedback. Afterwards, 326 medical students performed two-rescuer BLS on a manikin for 8 min. CPR quality parameters, such as "effective compression ratio" (ECR: compressions with correct hand position, depth and complete decompression multiplied by flow-time fraction), and other compression, ventilation and time-related parameters were assessed for all groups. RESULTS ECR was comparable between the hfCPR and the mfCPR group (0.33 vs. 0.35, p = 0.435). The hfCPR group needed less time until starting chest compressions (2 vs. 8 s, p < 0.001) and showed fewer incorrect decompressions (26 vs. 33 %, p = 0.044). On the other hand, absolute hands-off time was higher in the hfCPR group (67 vs. 60 s, p = 0.021). CONCLUSIONS The quality of CPR with human feedback or by using a mechanical audio-visual feedback device was similar. Further studies should investigate whether extended human feedback training could further increase CPR quality at comparable costs for training.

Publication activities of German junior researchers in academic medicine: which factors impact impact factors?

Previous studies have shown medical students in Germany to have little interest in research while at the same time there is a lack of physician scientists. This study’s aim is to investigate factors influencing publication productivity of physicians during and after finishing their medical doctorate. We conducted a PubMed search for physicians having received their doctoral degree at Ludwig-Maxmilians-University Munich Faculty of Medicine between 2011 and 2013 (N = 924) and identified the appropriate impact factor (IF) for each journal the participants had published in. Gender, age, final grade of the doctorate, participation in a structured doctoral study program and joint publication activities between graduate and academic supervisor were defined as factors. For analyses we used nonparametric procedures. Men show significantly more publications than women. Before their doctoral graduation men publish 1.98 (SD ± 3.64) articles on average, women 1.15 (±2.67) (p < 0.0001, d = 0.27). After completion of the doctorate (up to 06/2015), 40 % of men still publish, while only 24.3 % of women (p < 0.0001, φ = 0.17) continue to publish. No differences were found concerning the value of IFs. Similar results were found regarding the variable ‘participation in a structured doctoral study program’. Until doctoral graduation, program participants publish 2.82 (±5.41) articles, whereas participants doing their doctorate individually only publish 1.39 (±2.87) articles (p < 0.0001, d = 0.46). These differences persist in publication activities after graduation (45.5 vs. 29.7 %, p = 0.008, φ = 0.09). A structured doctorate seems to have positive influence on IFs (4.33 ± 2.91 vs. 3.37 ± 2.82, p = 0.006, d = 0.34). Further significant results concern the variables ‘final grade’ and ‘age’: An early doctoral graduation and an excellent or very good grade for the doctoral thesis positively influence publication productivity. Finally, joint publication activities between the graduate and his/her academic supervisor result in significantly higher IFs (3.64 ± 3.03 vs. 2.84 ± 2.25, p = 0.007, d = 0.28). The study’s results support the assumption about women’s underrepresentation in science as well as the relevance of structured doctoral study programs for preparing and recruiting young academics in medicine for scientific careers. Promoting women and further development of structured doctoral study programs are highly recommended.

Complementary medicine courses in Swiss medical schools: actual status and students' experience

In line with growing public popularity of complementary and alternative medicine (CAM), courses in CAM have been implemented in Medical Schools internationally, but as yet in an uncoordinated and heterogeneous way. In Switzerland, comprehensive data about CAM education at Medical Faculties are lacking.