Sustained impact of a short small group course with systematic feedback in addition to regular clinical clerkship activities on musculoskeletal examination skills-a controlled study.

BACKGROUND The discrepancy between the extensive impact of musculoskeletal complaints and the common deficiencies in musculoskeletal examination skills lead to increased emphasis on structured teaching and assessment. However, studies of single interventions are scarce and little is known about the time-dependent effect of assisted learning in addition to a standard curriculum. We therefore evaluated the immediate and long-term impact of a small group course on musculoskeletal examination skills. METHODS All 48 Year 4 medical students of a 6 year curriculum, attending their 8 week clerkship of internal medicine at one University department in Berne, participated in this controlled study. Twenty-seven students were assigned to the intervention of a 6×1 h practical course (4-7 students, interactive hands-on examination of real patients; systematic, detailed feedback to each student by teacher, peers and patients). Twenty-one students took part in the regular clerkship activities only and served as controls. In all students clinical skills (CS, 9 items) were assessed in an Objective Structured Clinical Examination (OSCE) station, including specific musculoskeletal examination skills (MSES, 7 items) and interpersonal skills (IPS, 2 items). Two raters assessed the skills on a 4-point Likert scale at the beginning (T0), the end (T1) and 4-12 months after (T2) the clerkship. Statistical analyses included Friedman test, Wilcoxon rank sum test and Mann-Whitney U test. RESULTS At T0 there were no significant differences between the intervention and control group. At T1 and T2 the control group showed no significant changes of CS, MSES and IPS compared to T0. In contrast, the intervention group significantly improved CS, MSES and IPS at T1 (p < 0.001). This enhancement was sustained for CS and MSES (p < 0.05), but not for IPS at T2. CONCLUSIONS Year 4 medical students were incapable of improving their musculoskeletal examination skills during regular clinical clerkship activities. However, an additional small group, interactive clinical skills course with feedback from various sources, improved these essential examination skills immediately after the teaching and several months later. We conclude that supplementary specific teaching activities are needed. Even a single, short-lasting targeted module can have a long lasting effect and is worth the additional effort.

The influence of students' prior clinical skills and context characteristics on mini-CEX scores in clerkships - a multilevel analysis.

BACKGROUND In contrast to objective structured clinical examinations (OSCEs), mini-clinical evaluation exercises (mini-CEXs) take place at the clinical workplace. As both mini-CEXs and OSCEs assess clinical skills, but within different contexts, this study aims at analyzing to which degree students' mini-CEX scores can be predicted by their recent OSCE scores and/or context characteristics. METHODS Medical students participated in an end of Year 3 OSCE and in 11 mini-CEXs during 5 different clerkships of Year 4. The students' mean scores of 9 clinical skills OSCE stations and mean 'overall' and 'domain' mini-CEX scores, averaged over all mini-CEXs of each student were computed. Linear regression analyses including random effects were used to predict mini-CEX scores by OSCE performance and characteristics of clinics, trainers, students and assessments. RESULTS A total of 512 trainers in 45 clinics provided 1783 mini-CEX ratings for 165 students; OSCE results were available for 144 students (87 %). Most influential for the prediction of 'overall' mini-CEX scores was the trainers' clinical position with a regression coefficient of 0.55 (95 %-CI: 0.26-0.84; p < .001) for residents compared to heads of department. Highly complex tasks and assessments taking place in large clinics significantly enhanced 'overall' mini-CEX scores, too. In contrast, high OSCE performance did not significantly increase 'overall' mini-CEX scores. CONCLUSION In our study, Mini-CEX scores depended rather on context characteristics than on students' clinical skills as demonstrated in an OSCE. Ways are discussed which focus on either to enhance the scores' validity or to use narrative comments only.

Cost-effectiveness of peer role play and standardized patients in undergraduate communication training.

BACKGROUND The few studies directly comparing the methodological approach of peer role play (RP) and standardized patients (SP) for the delivery of communication skills all suggest that both methods are effective. In this study we calculated the costs of both methods (given comparable outcomes) and are the first to generate a differential cost-effectiveness analysis of both methods. METHODS Medical students in their prefinal year were randomly assigned to one of two groups receiving communication training in Pediatrics either with RP (N = 34) or 19 individually trained SP (N = 35). In an OSCE with standardized patients using the Calgary-Cambridge Referenced Observation Guide both groups achieved comparable high scores (results published). In this study, corresponding costs were assessed as man-hours resulting from hours of work of SP and tutors. A cost-effectiveness analysis was performed. RESULTS Cost-effectiveness analysis revealed a major advantage for RP as compared to SP (112 vs. 172 man hours; cost effectiveness ratio .74 vs. .45) at comparable performance levels after training with both methods. CONCLUSIONS While both peer role play and training with standardized patients have their value in medical curricula, RP has a major advantage in terms of cost-effectiveness. This could be taken into account in future decisions.

Entwicklung und Implementierung von Modellen für ein Skills-Training-Parcours für internistische Assistenzärzte

„Entwicklung und Implementierung von Modellen für ein Skills-Training-Parcours für internistische Assistenzärzte “ V. Maier1 - K. Schnabel2 1 Universitätslinik für Allgemeine Innere Medizin, Inselspital, Bern 2 Berner interdisziplinäres Skills- und Schauspielpatientenzentrum (BiSS), Institut für Medizinische Lehre (IML), Abteilung für Unterricht und Medien (AUM) Einleitung: Im klinischen Alltag sind praktische Fertigkeiten gefordert, um Patienten sicher zu behandeln. Auch in der Schweizer Fachgesellschaft FMH kam es zu einer stärkeren Gewichtung der praktischen Fertigkeiten und müssen jetzt ein Logbuch über Art und Zeitpunkt der Intervention führen [1]. Am Inselspital Bern wurde dafür ein Skillsparcours etabliert, da in vielen Bereichen simulationsbasierte Ausbildungen traditionellen Methoden überlegen ist [2]. Der Skillsparcours besteht aus einem Nachmittag mit 4 nicht-invasiven Prozeduren und einem Nachmittag mit 5 invasiven Prozeduren. Eigens dafür wurden drei Modelle entwickelt und deren Tauglichkeit evaluiert. Fragestellung: Bilden die selbst gefertigten Modelle die Realität ausreichend ab? Material und Methoden: Innerhalb der 9 Posten (5 invasiv und 4 nichtinvasiv) wurden für die 5 invasiven Posten zwei Modelle aus dem Skillslab (BiSS) genutzt (Lumbalpunktion (LP) und Blasenkatheter (BK)) und drei Modelle neu entwickelt (Pleura-(PP), Aszites-(AP) und Knochenmarks-Punktion (KMP)). Die Modelle wurden mit Materialien aus dem Baumarkt entwickelt (Material ca. CHF 50/Stück). Der Aufbau der Modelle soll auf der Tagung demonstriert werden. Die Teilnehmer (N=12) und Dozenten (N=5) wurden zu der Qualität mittels Fragebogen befragt. Dabei wurde die individuelle Vorerfahrung und die Einschätzung der Teilnehmer erfragt. Die Frage zur Eignung des Modells war: „Das Modell war zum Üben geeignet“. Als Skala wurde eine Likert-Skala von 0 bis 5 (1=sehr ungeeignet, 5=sehr geeignet) benutzt. Ergebnisse: Die Assistenzärzte beurteilten die Modelleignung wie folgt (Median (Min;Max)): LP: 5 (4;5) KMP: 4.5 (3;5), PP: 4 (3;5), AP: 4.5 (2;5), BK-Einlage: 4.5 (4;6). Die Oberärzte, die jeweils nur das Modell bewerteten, an welchem sie den Kurs durchführten, beurteilten die Modelleignung wie folgt: LP 5.0, KMP: 5.0, PP 5.0, AP: 4.0, BK-Einlage: 3.0. Diskussion: Alle Modelle wurden sowohl von den Oberärzten als auch von den Assistenzärzten als zum Üben tauglich eingeschätzt. Zwischen den selbst hergestellten Low-Fidelity Modellen und den High-Fidelity Modellen gab es hierein keinen signifikanten Unterschied. Als am wenigsten tauglich wurde von den Oberärzten mit der Simulation der Blasenkatheter-Einlage ein High-Fidelity-Modell bewertet. Schlussfolgerungen: Alle Modelle für die Simulation der Punktionstechniken haben gut bis sehr gut funktioniert. Die selbst hergestellten Modelle bilden die Wirklich zum Üben der Techniken hinreichend gut und nicht schlechter als die High-Fidelity-Modelle ab. Selbst gebaute Modelle mit Materialien aus dem Baumarkt können das sonst sehr materialaufwändige Training mit Simulatoren genauso effektiv aber wesentlich effizienter durchführbar machen. Literatur bei den Autoren (1) Weiterbildungsordnung FMH 2014 (letzte Revision 4. September 2014). www.fmh.ch/files/pdf15/wbo_d.pdf (2) McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB (2011) Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011 Jun;86(6):706-11

Developing an alternative response format for the script concordance test

Introduction: Clinical reasoning is essential for the practice of medicine. In theory of development of medical expertise it is stated, that clinical reasoning starts from analytical processes namely the storage of isolated facts and the logical application of the ‘rules’ of diagnosis. Then the learners successively develop so called semantic networks and illness-scripts which finally are used in an intuitive non-analytic fashion [1], [2]. The script concordance test (SCT) is an example for assessing clinical reasoning [3]. However the aggregate scoring [3] of the SCT is recognized as problematic [4]. The SCT`s scoring leads to logical inconsistencies and is likely to reflect construct-irrelevant differences in examinees’ response styles [4]. Also the expert panel judgments might lead to an unintended error of measurement [4]. In this PhD project the following research questions will be addressed: 1. How does a format look like to assess clinical reasoning (similar to the SCT but) with multiple true-false questions or other formats with unambiguous correct answers, and by this address the above mentioned pitfalls in traditional scoring of the SCT? 2. How well does this format fulfill the Ottawa criteria for good assessment, with special regards to educational and catalytic effects [5]? Methods: 1. In a first study it shall be assessed whether designing a new format using multiple true-false items to assess clinical reasoning similar to the SCT-format is arguable in a theoretically and practically sound fashion. For this study focus groups or interviews with assessment experts and students will be undertaken. 2. In an study using focus groups and psychometric data Norcini`s and colleagues Criteria for Good Assessment [5] shall be determined for the new format in a real assessment. Furthermore the scoring method for this new format shall be optimized using real and simulated data.

Do multiple true-false items beat the commonly used one-best-answers questions regarding to the Ottawa Criteria for Good Assessment? Results of a literature review

Background: Multiple True-False-Items (MTF-Items) might offer some advantages compared to one-best-answer-questions (TypeA) as they allow more than one correct answer and may better represent clinical decisions. However, in medical education assessment MTF-Items are seldom used. Summary of Work: With this literature review existing findings on MTF-items and on TypeA were compared along the Ottawa Criteria for Good Assessment, i.e. (1) reproducibility, (2) feasibility, (3) validity, (4) acceptance, (5) educational effect, (6) catalytic effects, and (7) equivalence. We conducted a literature research on ERIC and Google Scholar including papers from the years 1935 to 2014. We used the search terms “multiple true-false”, “true-false”, “true/false”, and “Kprim” combined with “exam”, “test”, and “assessment”. Summary of Results: We included 29 out of 33 studies. Four of them were carried out in the medical field Compared to TypeA, MTF-Items are associated with (1) higher reproducibility (2) lower feasibility (3) similar validity (4) higher acceptance (5) higher educational effect (6) no studies on catalytic effects or (7) equivalence. Discussion and Conclusions: While studies show overall good characteristics of MTF items according to the Ottawa criteria, this type of question seems to be rather seldom used. One reason might be the reported lower feasibility. Overall the literature base is still weak. Furthermore, only 14 % of literature is from the medical domain. Further studies to better understand the characteristics of MTF-Items in the medical domain are warranted. Take-home messages: Overall the literature base is weak and therefore further studies are needed. Existing studies show that: MTF-Items show higher reliability, acceptance and educational effect; MTF-Items are more difficult to produce

Computer-based summative assessment of clinical reasoning in clerkships: A mixed-method comparison of key feature problems with case-based multiple choice questions

Background: It is yet unclear if there are differences between using electronic key feature problems (KFPs) or electronic case-based multiple choice questions (cbMCQ) for the assessment of clinical decision making. Summary of Work: Fifth year medical students were exposed to clerkships which ended with a summative exam. Assessment of knowledge per exam was done by 6-9 KFPs, 9-20 cbMCQ and 9-28 MC questions. Each KFP consisted of a case vignette and three key features (KF) using “long menu” as question format. We sought students’ perceptions of the KFPs and cbMCQs in focus groups (n of students=39). Furthermore statistical data of 11 exams (n of students=377) concerning the KFPs and (cb)MCQs were compared. Summary of Results: The analysis of the focus groups resulted in four themes reflecting students’ perceptions of KFPs and their comparison with (cb)MCQ: KFPs were perceived as (i) more realistic, (ii) more difficult, (iii) more motivating for the intense study of clinical reasoning than (cb)MCQ and (iv) showed an overall good acceptance when some preconditions are taken into account. The statistical analysis revealed that there was no difference in difficulty; however KFP showed a higher discrimination and reliability (G-coefficient) even when corrected for testing times. Correlation of the different exam parts was intermediate. Conclusions: Students perceived the KFPs as more motivating for the study of clinical reasoning. Statistically KFPs showed a higher discrimination and higher reliability than cbMCQs. Take-home messages: Including KFPs with long menu questions into summative clerkship exams seems to offer positive educational effects.

Improving residency training with the help of multisource feedback: Generation of a model of influencing factors

Background: Feedback is considered to be one of the most important drivers of learning. One form of structured feedback used in medical settings is multisource feedback (MSF). This feedback technique provides the opportunity to gain a differentiated view on a doctor’s performance from several perspectives using a questionnaire and a facilitating conversation, in which learning goals are formulated. While many studies have been conducted on the validity, reliability and feasibility of the instrument, little is known about the impact of factors that might influence the effects of MSF on clinical performance. Summary of Work: To study under which circumstances MSF is most effective, we performed a literature review on Google Scholar with focus on MSF and feedback in general. Main key-words were: MSF, multi-source-feedback, multi source feedback, and feedback each combined with influencing/ hindering/ facilitating factors, effective, effectiveness, doctors-intraining, and surgery. Summary of Results: Based on the literature, we developed a preliminary model of facilitating factors. This model includes five main factors influencing MSF: questionnaire, doctor-in-training, group of raters, facilitating supervisor, and facilitating conversation. Discussion and Conclusions: Especially the following points that might influence MSF have not yet been sufficiently studied: facilitating conversation with the supervisor, individual aspects of doctors-in-training, and the causal relations between influencing factors. Overall there are only very few studies focusing on the impact of MSF on actual and long-term performance. We developed a preliminary model of hindering and facilitating factors on MSF. Further studies are needed to better understand under which circumstances MSF is most effective. Take-home messages: The preliminary model might help to guide further studies on how to implement MSF to use it at its full potential.

Publication activities of German junior researchers in academic medicine: which factors impact impact factors?

Previous studies have shown medical students in Germany to have little interest in research while at the same time there is a lack of physician scientists. This study’s aim is to investigate factors influencing publication productivity of physicians during and after finishing their medical doctorate. We conducted a PubMed search for physicians having received their doctoral degree at Ludwig-Maxmilians-University Munich Faculty of Medicine between 2011 and 2013 (N = 924) and identified the appropriate impact factor (IF) for each journal the participants had published in. Gender, age, final grade of the doctorate, participation in a structured doctoral study program and joint publication activities between graduate and academic supervisor were defined as factors. For analyses we used nonparametric procedures. Men show significantly more publications than women. Before their doctoral graduation men publish 1.98 (SD ± 3.64) articles on average, women 1.15 (±2.67) (p < 0.0001, d = 0.27). After completion of the doctorate (up to 06/2015), 40 % of men still publish, while only 24.3 % of women (p < 0.0001, φ = 0.17) continue to publish. No differences were found concerning the value of IFs. Similar results were found regarding the variable ‘participation in a structured doctoral study program’. Until doctoral graduation, program participants publish 2.82 (±5.41) articles, whereas participants doing their doctorate individually only publish 1.39 (±2.87) articles (p < 0.0001, d = 0.46). These differences persist in publication activities after graduation (45.5 vs. 29.7 %, p = 0.008, φ = 0.09). A structured doctorate seems to have positive influence on IFs (4.33 ± 2.91 vs. 3.37 ± 2.82, p = 0.006, d = 0.34). Further significant results concern the variables ‘final grade’ and ‘age’: An early doctoral graduation and an excellent or very good grade for the doctoral thesis positively influence publication productivity. Finally, joint publication activities between the graduate and his/her academic supervisor result in significantly higher IFs (3.64 ± 3.03 vs. 2.84 ± 2.25, p = 0.007, d = 0.28). The study’s results support the assumption about women’s underrepresentation in science as well as the relevance of structured doctoral study programs for preparing and recruiting young academics in medicine for scientific careers. Promoting women and further development of structured doctoral study programs are highly recommended.