Self-assessment insufficiently predicts performance in emergency skills

Background: Residents demonstrate a broad range of performance levels for clinical skills, with some at an inadequate level. Adequate self-assessment is important for life long learning. However, its accuracy is questioned extensively. The aim of this study was to evaluate how far the residents’ self-assessment predicts their performance in an expert assessment of emergency skills. Summary of work: Twelve skills were identified as being relevant for the emergency duties of residents in smaller hospitals. Fifteen first-year residents from the departments of internal medicine and general surgery at a district hospital rated their performance on a questionnaire (self-assessment). This was followed by a structured, practical in vivo assessment by an anaesthesiologist (expert assessment). For both, a visual analogue scale from 0 to 10 was used, on which 0 stands for novice and 10 for expert. Predictive validity was described by Spearman’s correlation, which was significant in 3 out of 12 skills only. Median correlation (r) was 0.50 (range 0.16 to 0.93). Conclusion: At the beginning of postgraduate training, self-assessment alone is not sufficient to guide self-directed learning. Take-home message: At the beginning of their residency, physicians need structured feedback in emergency skills which can be offered by anaesthesiologists.

Computergestützte Lehre an Hochschulen (Blended Learning). Der Virtuelle Campus Erziehungswissenschaft der Universität Bern

Blended Learning-Angebote - Lehrveranstaltungen, die aus Präsenzanteilen und virtuellen Anteilen im Internet bestehen - halten zunehmend Einzug an Universitäten, Fachhochschulen und Pädagogischen Hochschulen. Diese neuen Lehrformen stehen im Spannungsfeld zwischen technischen Möglichkeiten, ökonomischen Erfordernissen und hochschuldidaktischen Anforderungen. Den Mittelpunkt des Buches bildet das computerunterstützte Lehrangebot des «Virtuellen Campus Erziehungswissenschaft» an der Universität Bern, das der Ausbildung zukünftiger Lehrpersonen an der Pädagogischen Hochschule Bern dient. Zum einen soll dieses in der Praxis bewährte Lehrangebot theoretisch analysiert werden. Zum anderen erfolgt ein Einblick in die Praxis des «Virtuellen Campus Erziehungswissenschaft», um anderen Bildungsinstitutionen Anregungen zur Einrichtung ähnlicher Angebote oder zur Modifizierung ihrer Blended-Learning-Kurse zu geben. Dabei werden die Bereiche (a) Planung und Entwicklung von Lehrangeboten, (b) Methoden der Vermittlung und Einsatz neuer Technologien, (c) Betreuung von Studierenden, (d) Assessment der Studierenden, (e) Qualitätssicherung der Lehre und der eigenen Lehrtätigkeit und (f) Selbstmanagement und Professionalität im Hochschulkontext abgedeckt. Schliesslich wird auch nach der hochschuldidaktischen Vernunft solcher Angebote gefragt.

Performance of experienced dentists in Switzerland after an e-learning program on ICDAS occlusal caries detection

This study aimed to evaluate the effect of an e-learning program on the validity and reproducibility of the International Caries Detection and Assessment System (ICDAS) in detecting occlusal caries. For the study, 170 permanent molars were selected. Four dentists in Switzerland who had no previous contact with ICDAS examined the teeth before and after the e-learning program and scored the sites according to ICDAS. Teeth were histologically prepared and assessed for caries extension. The significance level was set at 0.05. Sensitivity before and after the e-learning program was 0.80 and 0.77 (D1), 0.72 and 0.63 (D2), and 0.74 and 0.67 (D3,4), respectively. Specificity was 0.64 and 0.69 (D1), 0.70 and 0.81 (D2), and 0.81 and 0.87 (D3,4). A McNemar test did not show any difference between the values of sensitivity, specificity, accuracy, and area under the ROC curve (AUC) before and after the e-learning program. The averages of wK values for interexaminer reproducibility were 0.61 (before) and 0.66 (after). Correlation with histology presented wK values of 0.62 (before) and 0.63 (after). A Wilcoxon test showed a statistically significant difference between before and after the e-learning program. In conclusion, even though ICDAS performed well in detecting occlusal caries, the e-learning program did not have any statistically significant effect on its performance by these experienced dentists.

The mini-clinical evaluation exercise during medical clerkships: are learning needs and learning goals aligned?

OBJECTIVES The generation of learning goals (LGs) that are aligned with learning needs (LNs) is one of the main purposes of formative workplace-based assessment. In this study, we aimed to analyse how often trainer–student pairs identified corresponding LNs in mini-clinical evaluation exercise (mini-CEX) encounters and to what degree these LNs aligned with recorded LGs, taking into account the social environment (e.g. clinic size) in which the mini-CEX was conducted. METHODS Retrospective analyses of adapted mini-CEX forms (trainers’ and students’ assessments) completed by all Year 4 medical students during clerkships were performed. Learning needs were defined by the lowest score(s) assigned to one or more of the mini-CEX domains. Learning goals were categorised qualitatively according to their correspondence with the six mini-CEX domains (e.g. history taking, professionalism). Following descriptive analyses of LNs and LGs, multi-level logistic regression models were used to predict LGs by identified LNs and social context variables. RESULTS A total of 512 trainers and 165 students conducted 1783 mini-CEXs (98% completion rate). Concordantly, trainer–student pairs most often identified LNs in the domains of ‘clinical reasoning’ (23% of 1167 complete forms), ‘organisation/efficiency’ (20%) and ‘physical examination’ (20%). At least one ‘defined’ LG was noted on 313 student forms (18% of 1710). Of the 446 LGs noted in total, the most frequently noted were ‘physical examination’ (49%) and ‘history taking’ (21%). Corresponding LNs as well as social context factors (e.g. clinic size) were found to be predictors of these LGs. CONCLUSIONS Although trainer–student pairs often agreed in the LNs they identified, many assessments did not result in aligned LGs. The sparseness of LGs, their dependency on social context and their partial non-alignment with students’ LNs raise questions about how the full potential of the mini-CEX as not only a ‘diagnostic’ but also an ‘educational’ tool can be exploited.

Alignment between learning needs and learning goals of Mini-CEX in clerkships

Background: Defining learning goals (LG) in alignment with learning needs (LN) is one of the key purposes of formative workplace-based assessment, but studies about this topic are scarce. Summary of Work: We analysed quantitatively and qualitatively how often trainer-student pairs identified the same LN during Mini Clinical Evaluation Exercises (Mini-CEX) in clerkships and to what degree those LNs were in line with the recorded LGs. Multilevel logistic regression models were used to predict LGs by identified LNs, controlling for context variables. Summary of Results: 512 trainers and 165 students conducted 1783 Mini-CEX (98% completion rate). Concordantly, trainer-student pairs most often identified LNs in the domains ‘clinical reasoning’ (23% of 1167 complete forms), ‘organisation / efficiency’ (20%) and ‘physical examination’ (20%). At least one ‘defined’ LG was noted on 313 student forms (18% of 1710), with a total of 446 LGs. Of these, the most frequent LGs were ‘physical examination’ (49% of 446 LGs) and ‘history taking’ (21%); corresponding LNs as well as context variables (e.g. clinic size) were found to be predictors of these LGs. Discussion and Conclusions: Although trainer-student pairs often agreed in their identified LNs, many assessments did not result in an aligned LG or a LG at all. Interventions are needed to enhance the proportion of (aligned) LGs in Mini-CEX in order to tap into its full potential not only as a ‘diagnostic’ but also as an ‘educational tool’. Take-home messages: The sparseness of LGs, their dependency on context variables and their partial non-alignment with students’ LNs raise the question of how the effectiveness of Mini-CEX can be further enhanced.

Individual Video Training iVT - Development of a National Infrastructure for Video-Supported Learning

Introduction: Video‐Supported Learning is particularly effective when it comes to skills and behaviors. Video registration of patient‐physician interviews, class room instruction or practical skills allow it to learners themselves, their peers, and their tutors to assess the quality of the learner's performance, to give specific feedback, and to make suggestions for improvement. Methods: In Switzerland, four pedagogical universities and two medical faculties joined to initiate the development of a national infrastructure for Video Supported Learning. The goal was to have a system that is simple to use, has most steps automated, provides the videos over the Internet, and has a sophisticated access control. Together with SWITCH, the national IT‐Support‐Organisation for Swiss Universities, the program iVT (Individual Video Training) was developed by integrating two preexisting technologies. The first technology is SWITCHcast, a podcast system. With SWITCHcast, videos are automatically uploaded to a server as soon as the registration is over. There the videos are processed and converted to different formats. The second technology is the national Single Logon System AAI (Authentification and Authorization Infrastructure) that enables iVT to link each video with the corresponding learner. The learner starts the registration with his Single Logon. Thus, the video can unambiguously be assigned. Via his institution's Learning Management System (LMS), the learner can access his video and give access to his video to peers and tutors. Results: iVT is now used at all involved institutions. The system works flawlessly. In Bern, we use iVT for the communications skills training in the forth and sixth year. Since students meet with patient actors alone, iVT is also used to certify attendance. Students are encouraged to watch the videos of the interview and the feedback of the patient actor. The offer to discuss a video with a tutor was not used by the students. Discussion: We plan to expand the use of iVT by making peer assessment compulsory. To support this, annotation capabilities are currently added to iVT. We also want to use iVT in training of practical skills, again for self as well as for peer assessment.  At present, we use iVT for quality control of patient actor's performance.

Guidelines for WB3 Part II: Assessment of conservation strategies

These guidelines are a working instrument for the assessment and documentation of existing and potential strategies for land and water conservation (prevention and mitigation strategies) in DESIRE study sites. DESIRE (Desertification Mitigation and Remediation of Land) is a European Integrated Project. The DESIRE WB 3 methodology was developed by CDE and is based on experiences from Learning for Sustainability (LforS) and WOCAT.

Integrated Disaster Reduction Approach: LforS Learning and Simulation Game

Main objective of the game is to increase the coping capacity of players and familiarise them with the Integrated Disaster Reduction Approach. The game is intended to prepare for and introduce the players to a subsequent Learning for Sustainability capacity building workshop for community leaders. The game represents a typical emergency situation resulting from a natural disaster. Before and after the event, adequate measures help to prevent or minimise potential damages. Once a disaster has occurred, concerted actions and immediate measures need to be taken to rescue as much as possible (human lives, livestock, material) and safeguard the village against further damage and losses. In the course of the game, each playing team can proof its knowledge on adequate measures that have to be taken in order to avoid or reduce losses related to natural disasters. Such measures relate to assessment and monitoring of risks, prevention and mitigation measures, preparedness and response as well as recovery and reconstruction.

Call for uniform neuropsychological assessment after aneurysmal subarachnoid hemorrhage: Swiss recommendations.

BACKGROUND In a high proportion of patients with favorable outcome after aneurysmal subarachnoid hemorrhage (aSAH), neuropsychological deficits, depression, anxiety, and fatigue are responsible for the inability to return to their regular premorbid life and pursue their professional careers. These problems often remain unrecognized, as no recommendations concerning a standardized comprehensive assessment have yet found entry into clinical routines. METHODS To establish a nationwide standard concerning a comprehensive assessment after aSAH, representatives of all neuropsychological and neurosurgical departments of those eight Swiss centers treating acute aSAH have agreed on a common protocol. In addition, a battery of questionnaires and neuropsychological tests was selected, optimally suited to the deficits found most prevalent in aSAH patients that was available in different languages and standardized. RESULTS We propose a baseline inpatient neuropsychological screening using the Montreal Cognitive Assessment (MoCA) between days 14 and 28 after aSAH. In an outpatient setting at 3 and 12 months after bleeding, we recommend a neuropsychological examination, testing all relevant domains including attention, speed of information processing, executive functions, verbal and visual learning/memory, language, visuo-perceptual abilities, and premorbid intelligence. In addition, a detailed assessment capturing anxiety, depression, fatigue, symptoms of frontal lobe affection, and quality of life should be performed. CONCLUSIONS This standardized neuropsychological assessment will lead to a more comprehensive assessment of the patient, facilitate the detection and subsequent treatment of previously unrecognized but relevant impairments, and help to determine the incidence, characteristics, modifiable risk factors, and the clinical course of these impairments after aSAH.

Improving Pediatric Basic Life Support Performance Through Blended Learning With Web-Based Virtual Patients: Randomized Controlled Trial

BACKGROUND E-learning and blended learning approaches gain more and more popularity in emergency medicine curricula. So far, little data is available on the impact of such approaches on procedural learning and skill acquisition and their comparison with traditional approaches. OBJECTIVE This study investigated the impact of a blended learning approach, including Web-based virtual patients (VPs) and standard pediatric basic life support (PBLS) training, on procedural knowledge, objective performance, and self-assessment. METHODS A total of 57 medical students were randomly assigned to an intervention group (n=30) and a control group (n=27). Both groups received paper handouts in preparation of simulation-based PBLS training. The intervention group additionally completed two Web-based VPs with embedded video clips. Measurements were taken at randomization (t0), after the preparation period (t1), and after hands-on training (t2). Clinical decision-making skills and procedural knowledge were assessed at t0 and t1. PBLS performance was scored regarding adherence to the correct algorithm, conformance to temporal demands, and the quality of procedural steps at t1 and t2. Participants' self-assessments were recorded in all three measurements. RESULTS Procedural knowledge of the intervention group was significantly superior to that of the control group at t1. At t2, the intervention group showed significantly better adherence to the algorithm and temporal demands, and better procedural quality of PBLS in objective measures than did the control group. These aspects differed between the groups even at t1 (after VPs, prior to practical training). Self-assessments differed significantly only at t1 in favor of the intervention group. CONCLUSIONS Training with VPs combined with hands-on training improves PBLS performance as judged by objective measures.