Individual Video Training iVT and Annotating Academic Videos AAV: two complementing technologies

Individual Video Training iVT and Annotating Academic Videos AAV: two complementing technologies 1. Recording communication skills training sessions and reviewing them by oneself, with peers, and with tutors has become standard in medical education. Increasing numbers of students paired with restrictions of financial and human resources create a big obstacle to this important teaching method. 2. Everybody who wants to increase efficiency and effectiveness of communication training can get new ideas from our technical solution. 3. Our goal was to increase the effectiveness of communication skills training by supporting self, peer and tutor assessment over the Internet. Two technologies of SWITCH, the national foundation to support IT solutions for Swiss universities, came handy for our project. The first is the authentication and authorization infrastructure providing all Swiss students with a nationwide single login. The second is SWITCHcast which allows automated recording, upload and publication of videos in the Internet. Students start the recording system by entering their single login. This automatically links the video with their password. Within a few hours, they find their video password protected on the Internet. They now can give access to peers and tutors. Additionally, an annotation interface was developed. This software has free text as well as checklist annotations capabilities. Tutors as well as students can create checklists. Tutor’s checklists are not editable by students. Annotations are linked to tracks. Tracks can be private or public. Public means visible to all who have access to the video. Annotation data can be exported for statistical evaluation. 4. The system was well received by students and tutors. Big numbers of videos were processed simultaneously without any problems. 5. iVT http://www.switch.ch/aaa/projects/detail/UNIBE.7 AAV http://www.switch.ch/aaa/projects/detail/ETHZ.9

Alignment between learning needs and learning goals of Mini-CEX in clerkships

Background: Defining learning goals (LG) in alignment with learning needs (LN) is one of the key purposes of formative workplace-based assessment, but studies about this topic are scarce. Summary of Work: We analysed quantitatively and qualitatively how often trainer-student pairs identified the same LN during Mini Clinical Evaluation Exercises (Mini-CEX) in clerkships and to what degree those LNs were in line with the recorded LGs. Multilevel logistic regression models were used to predict LGs by identified LNs, controlling for context variables. Summary of Results: 512 trainers and 165 students conducted 1783 Mini-CEX (98% completion rate). Concordantly, trainer-student pairs most often identified LNs in the domains ‘clinical reasoning’ (23% of 1167 complete forms), ‘organisation / efficiency’ (20%) and ‘physical examination’ (20%). At least one ‘defined’ LG was noted on 313 student forms (18% of 1710), with a total of 446 LGs. Of these, the most frequent LGs were ‘physical examination’ (49% of 446 LGs) and ‘history taking’ (21%); corresponding LNs as well as context variables (e.g. clinic size) were found to be predictors of these LGs. Discussion and Conclusions: Although trainer-student pairs often agreed in their identified LNs, many assessments did not result in an aligned LG or a LG at all. Interventions are needed to enhance the proportion of (aligned) LGs in Mini-CEX in order to tap into its full potential not only as a ‘diagnostic’ but also as an ‘educational tool’. Take-home messages: The sparseness of LGs, their dependency on context variables and their partial non-alignment with students’ LNs raise the question of how the effectiveness of Mini-CEX can be further enhanced.

Lernzielkatalog für die studentische Lehre im Fachbereich „Geriatrie“ an Fakultäten für Humanmedizin.

Zusammenfassung Die Betreuung geriatrischer Patientinnen und Patienten setzt, nebst einer entsprechenden Haltung, fundierte Kenntnisse in Diagnostik und Behandlung praktisch aller medizinischen Fachgebiete voraus. Daher ist es wichtig, dass die Kompetenz von Studierenden der Humanmedizin im Bereich Geriatrie entsprechend gefördert wird. Bis heute hat jedoch die studentische Ausbildung im Fach Geriatrie an vielen europäischen Universitäten einen unklaren oder untergeordneten Stellenwert. Als ersten Schritt zur Förderung der Lehre in der Geriatrie hat die Europäische Facharztvereinigung Geriatrie (UEMS-GMS) in einem Delphi-Prozess einen Lernzielkatalog entwickelt. Dieser Katalog enthält die Mindestanforderungen mit spezifischen Lernzielen (Wissen, Fertigkeiten und Haltungen), welche die Studierenden der Humanmedizin bezüglich Geriatrie bis zum Abschluss des Medizinstudiums erwerben sollen. Zur Förderung der Implementierung dieses neuen, kompetenzbasierten Lernzielkatalogs an den deutschsprachigen Universitäten wurde eine an den Sprachgebrauch des „DACH-Raums“ (Deutschland, Österreich und Schweiz) angepasste deutsche Version erstellt. Im vorliegenden Beitrag wird diese Übersetzung vorgestellt. Die Fachgesellschaften für Geriatrie aus Deutschland, Österreich und der Schweiz empfehlen den medizinischen Fakultäten der jeweiligen Länder, diesen Katalog umzusetzen.

Patient safety in dialysis access

Not only are dialysis access creation and maintenance prone to complications, but patients suffering from end-stage renal disease and its comorbidities generally have a high risk of adverse events during their continuous treatment. Preventive strategies are key to avoid harm and to improve the outcome of the treatment of the growing number of patients with chronic kidney failure, especially as doctors and nurses are not always aware of the consequences of unsafe behavior. This publication is intended for health care professionals – nurses as well as doctors – and aims to raise the awareness of patient safety aspects, combining medical education with evidence-based medicine. After a general overview of the topic, an international panel of authors provides a diversified insight into important concepts and technical tricks essential to create and maintain a functional dialysis access. Contributions to Nephrology, Vol. 184

Sustained impact of a short small group course with systematic feedback in addition to regular clinical clerkship activities on musculoskeletal examination skills-a controlled study.

BACKGROUND The discrepancy between the extensive impact of musculoskeletal complaints and the common deficiencies in musculoskeletal examination skills lead to increased emphasis on structured teaching and assessment. However, studies of single interventions are scarce and little is known about the time-dependent effect of assisted learning in addition to a standard curriculum. We therefore evaluated the immediate and long-term impact of a small group course on musculoskeletal examination skills. METHODS All 48 Year 4 medical students of a 6 year curriculum, attending their 8 week clerkship of internal medicine at one University department in Berne, participated in this controlled study. Twenty-seven students were assigned to the intervention of a 6×1 h practical course (4-7 students, interactive hands-on examination of real patients; systematic, detailed feedback to each student by teacher, peers and patients). Twenty-one students took part in the regular clerkship activities only and served as controls. In all students clinical skills (CS, 9 items) were assessed in an Objective Structured Clinical Examination (OSCE) station, including specific musculoskeletal examination skills (MSES, 7 items) and interpersonal skills (IPS, 2 items). Two raters assessed the skills on a 4-point Likert scale at the beginning (T0), the end (T1) and 4-12 months after (T2) the clerkship. Statistical analyses included Friedman test, Wilcoxon rank sum test and Mann-Whitney U test. RESULTS At T0 there were no significant differences between the intervention and control group. At T1 and T2 the control group showed no significant changes of CS, MSES and IPS compared to T0. In contrast, the intervention group significantly improved CS, MSES and IPS at T1 (p < 0.001). This enhancement was sustained for CS and MSES (p < 0.05), but not for IPS at T2. CONCLUSIONS Year 4 medical students were incapable of improving their musculoskeletal examination skills during regular clinical clerkship activities. However, an additional small group, interactive clinical skills course with feedback from various sources, improved these essential examination skills immediately after the teaching and several months later. We conclude that supplementary specific teaching activities are needed. Even a single, short-lasting targeted module can have a long lasting effect and is worth the additional effort.

The influence of students' prior clinical skills and context characteristics on mini-CEX scores in clerkships - a multilevel analysis.

BACKGROUND In contrast to objective structured clinical examinations (OSCEs), mini-clinical evaluation exercises (mini-CEXs) take place at the clinical workplace. As both mini-CEXs and OSCEs assess clinical skills, but within different contexts, this study aims at analyzing to which degree students' mini-CEX scores can be predicted by their recent OSCE scores and/or context characteristics. METHODS Medical students participated in an end of Year 3 OSCE and in 11 mini-CEXs during 5 different clerkships of Year 4. The students' mean scores of 9 clinical skills OSCE stations and mean 'overall' and 'domain' mini-CEX scores, averaged over all mini-CEXs of each student were computed. Linear regression analyses including random effects were used to predict mini-CEX scores by OSCE performance and characteristics of clinics, trainers, students and assessments. RESULTS A total of 512 trainers in 45 clinics provided 1783 mini-CEX ratings for 165 students; OSCE results were available for 144 students (87 %). Most influential for the prediction of 'overall' mini-CEX scores was the trainers' clinical position with a regression coefficient of 0.55 (95 %-CI: 0.26-0.84; p < .001) for residents compared to heads of department. Highly complex tasks and assessments taking place in large clinics significantly enhanced 'overall' mini-CEX scores, too. In contrast, high OSCE performance did not significantly increase 'overall' mini-CEX scores. CONCLUSION In our study, Mini-CEX scores depended rather on context characteristics than on students' clinical skills as demonstrated in an OSCE. Ways are discussed which focus on either to enhance the scores' validity or to use narrative comments only.

Cost-effectiveness of peer role play and standardized patients in undergraduate communication training.

BACKGROUND The few studies directly comparing the methodological approach of peer role play (RP) and standardized patients (SP) for the delivery of communication skills all suggest that both methods are effective. In this study we calculated the costs of both methods (given comparable outcomes) and are the first to generate a differential cost-effectiveness analysis of both methods. METHODS Medical students in their prefinal year were randomly assigned to one of two groups receiving communication training in Pediatrics either with RP (N = 34) or 19 individually trained SP (N = 35). In an OSCE with standardized patients using the Calgary-Cambridge Referenced Observation Guide both groups achieved comparable high scores (results published). In this study, corresponding costs were assessed as man-hours resulting from hours of work of SP and tutors. A cost-effectiveness analysis was performed. RESULTS Cost-effectiveness analysis revealed a major advantage for RP as compared to SP (112 vs. 172 man hours; cost effectiveness ratio .74 vs. .45) at comparable performance levels after training with both methods. CONCLUSIONS While both peer role play and training with standardized patients have their value in medical curricula, RP has a major advantage in terms of cost-effectiveness. This could be taken into account in future decisions.

Entwicklung und Implementierung von Modellen für ein Skills-Training-Parcours für internistische Assistenzärzte

„Entwicklung und Implementierung von Modellen für ein Skills-Training-Parcours für internistische Assistenzärzte “ V. Maier1 - K. Schnabel2 1 Universitätslinik für Allgemeine Innere Medizin, Inselspital, Bern 2 Berner interdisziplinäres Skills- und Schauspielpatientenzentrum (BiSS), Institut für Medizinische Lehre (IML), Abteilung für Unterricht und Medien (AUM) Einleitung: Im klinischen Alltag sind praktische Fertigkeiten gefordert, um Patienten sicher zu behandeln. Auch in der Schweizer Fachgesellschaft FMH kam es zu einer stärkeren Gewichtung der praktischen Fertigkeiten und müssen jetzt ein Logbuch über Art und Zeitpunkt der Intervention führen [1]. Am Inselspital Bern wurde dafür ein Skillsparcours etabliert, da in vielen Bereichen simulationsbasierte Ausbildungen traditionellen Methoden überlegen ist [2]. Der Skillsparcours besteht aus einem Nachmittag mit 4 nicht-invasiven Prozeduren und einem Nachmittag mit 5 invasiven Prozeduren. Eigens dafür wurden drei Modelle entwickelt und deren Tauglichkeit evaluiert. Fragestellung: Bilden die selbst gefertigten Modelle die Realität ausreichend ab? Material und Methoden: Innerhalb der 9 Posten (5 invasiv und 4 nichtinvasiv) wurden für die 5 invasiven Posten zwei Modelle aus dem Skillslab (BiSS) genutzt (Lumbalpunktion (LP) und Blasenkatheter (BK)) und drei Modelle neu entwickelt (Pleura-(PP), Aszites-(AP) und Knochenmarks-Punktion (KMP)). Die Modelle wurden mit Materialien aus dem Baumarkt entwickelt (Material ca. CHF 50/Stück). Der Aufbau der Modelle soll auf der Tagung demonstriert werden. Die Teilnehmer (N=12) und Dozenten (N=5) wurden zu der Qualität mittels Fragebogen befragt. Dabei wurde die individuelle Vorerfahrung und die Einschätzung der Teilnehmer erfragt. Die Frage zur Eignung des Modells war: „Das Modell war zum Üben geeignet“. Als Skala wurde eine Likert-Skala von 0 bis 5 (1=sehr ungeeignet, 5=sehr geeignet) benutzt. Ergebnisse: Die Assistenzärzte beurteilten die Modelleignung wie folgt (Median (Min;Max)): LP: 5 (4;5) KMP: 4.5 (3;5), PP: 4 (3;5), AP: 4.5 (2;5), BK-Einlage: 4.5 (4;6). Die Oberärzte, die jeweils nur das Modell bewerteten, an welchem sie den Kurs durchführten, beurteilten die Modelleignung wie folgt: LP 5.0, KMP: 5.0, PP 5.0, AP: 4.0, BK-Einlage: 3.0. Diskussion: Alle Modelle wurden sowohl von den Oberärzten als auch von den Assistenzärzten als zum Üben tauglich eingeschätzt. Zwischen den selbst hergestellten Low-Fidelity Modellen und den High-Fidelity Modellen gab es hierein keinen signifikanten Unterschied. Als am wenigsten tauglich wurde von den Oberärzten mit der Simulation der Blasenkatheter-Einlage ein High-Fidelity-Modell bewertet. Schlussfolgerungen: Alle Modelle für die Simulation der Punktionstechniken haben gut bis sehr gut funktioniert. Die selbst hergestellten Modelle bilden die Wirklich zum Üben der Techniken hinreichend gut und nicht schlechter als die High-Fidelity-Modelle ab. Selbst gebaute Modelle mit Materialien aus dem Baumarkt können das sonst sehr materialaufwändige Training mit Simulatoren genauso effektiv aber wesentlich effizienter durchführbar machen. Literatur bei den Autoren (1) Weiterbildungsordnung FMH 2014 (letzte Revision 4. September 2014). www.fmh.ch/files/pdf15/wbo_d.pdf (2) McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB (2011) Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011 Jun;86(6):706-11

Developing an alternative response format for the script concordance test

Introduction: Clinical reasoning is essential for the practice of medicine. In theory of development of medical expertise it is stated, that clinical reasoning starts from analytical processes namely the storage of isolated facts and the logical application of the ‘rules’ of diagnosis. Then the learners successively develop so called semantic networks and illness-scripts which finally are used in an intuitive non-analytic fashion [1], [2]. The script concordance test (SCT) is an example for assessing clinical reasoning [3]. However the aggregate scoring [3] of the SCT is recognized as problematic [4]. The SCT`s scoring leads to logical inconsistencies and is likely to reflect construct-irrelevant differences in examinees’ response styles [4]. Also the expert panel judgments might lead to an unintended error of measurement [4]. In this PhD project the following research questions will be addressed: 1. How does a format look like to assess clinical reasoning (similar to the SCT but) with multiple true-false questions or other formats with unambiguous correct answers, and by this address the above mentioned pitfalls in traditional scoring of the SCT? 2. How well does this format fulfill the Ottawa criteria for good assessment, with special regards to educational and catalytic effects [5]? Methods: 1. In a first study it shall be assessed whether designing a new format using multiple true-false items to assess clinical reasoning similar to the SCT-format is arguable in a theoretically and practically sound fashion. For this study focus groups or interviews with assessment experts and students will be undertaken. 2. In an study using focus groups and psychometric data Norcini`s and colleagues Criteria for Good Assessment [5] shall be determined for the new format in a real assessment. Furthermore the scoring method for this new format shall be optimized using real and simulated data.

Do multiple true-false items beat the commonly used one-best-answers questions regarding to the Ottawa Criteria for Good Assessment? Results of a literature review

Background: Multiple True-False-Items (MTF-Items) might offer some advantages compared to one-best-answer-questions (TypeA) as they allow more than one correct answer and may better represent clinical decisions. However, in medical education assessment MTF-Items are seldom used. Summary of Work: With this literature review existing findings on MTF-items and on TypeA were compared along the Ottawa Criteria for Good Assessment, i.e. (1) reproducibility, (2) feasibility, (3) validity, (4) acceptance, (5) educational effect, (6) catalytic effects, and (7) equivalence. We conducted a literature research on ERIC and Google Scholar including papers from the years 1935 to 2014. We used the search terms “multiple true-false”, “true-false”, “true/false”, and “Kprim” combined with “exam”, “test”, and “assessment”. Summary of Results: We included 29 out of 33 studies. Four of them were carried out in the medical field Compared to TypeA, MTF-Items are associated with (1) higher reproducibility (2) lower feasibility (3) similar validity (4) higher acceptance (5) higher educational effect (6) no studies on catalytic effects or (7) equivalence. Discussion and Conclusions: While studies show overall good characteristics of MTF items according to the Ottawa criteria, this type of question seems to be rather seldom used. One reason might be the reported lower feasibility. Overall the literature base is still weak. Furthermore, only 14 % of literature is from the medical domain. Further studies to better understand the characteristics of MTF-Items in the medical domain are warranted. Take-home messages: Overall the literature base is weak and therefore further studies are needed. Existing studies show that: MTF-Items show higher reliability, acceptance and educational effect; MTF-Items are more difficult to produce

Computer-based summative assessment of clinical reasoning in clerkships: A mixed-method comparison of key feature problems with case-based multiple choice questions

Background: It is yet unclear if there are differences between using electronic key feature problems (KFPs) or electronic case-based multiple choice questions (cbMCQ) for the assessment of clinical decision making. Summary of Work: Fifth year medical students were exposed to clerkships which ended with a summative exam. Assessment of knowledge per exam was done by 6-9 KFPs, 9-20 cbMCQ and 9-28 MC questions. Each KFP consisted of a case vignette and three key features (KF) using “long menu” as question format. We sought students’ perceptions of the KFPs and cbMCQs in focus groups (n of students=39). Furthermore statistical data of 11 exams (n of students=377) concerning the KFPs and (cb)MCQs were compared. Summary of Results: The analysis of the focus groups resulted in four themes reflecting students’ perceptions of KFPs and their comparison with (cb)MCQ: KFPs were perceived as (i) more realistic, (ii) more difficult, (iii) more motivating for the intense study of clinical reasoning than (cb)MCQ and (iv) showed an overall good acceptance when some preconditions are taken into account. The statistical analysis revealed that there was no difference in difficulty; however KFP showed a higher discrimination and reliability (G-coefficient) even when corrected for testing times. Correlation of the different exam parts was intermediate. Conclusions: Students perceived the KFPs as more motivating for the study of clinical reasoning. Statistically KFPs showed a higher discrimination and higher reliability than cbMCQs. Take-home messages: Including KFPs with long menu questions into summative clerkship exams seems to offer positive educational effects.