Self-assessment insufficiently predicts performance in emergency skills

Background: Residents demonstrate a broad range of performance levels for clinical skills, with some at an inadequate level. Adequate self-assessment is important for life long learning. However, its accuracy is questioned extensively. The aim of this study was to evaluate how far the residents’ self-assessment predicts their performance in an expert assessment of emergency skills. Summary of work: Twelve skills were identified as being relevant for the emergency duties of residents in smaller hospitals. Fifteen first-year residents from the departments of internal medicine and general surgery at a district hospital rated their performance on a questionnaire (self-assessment). This was followed by a structured, practical in vivo assessment by an anaesthesiologist (expert assessment). For both, a visual analogue scale from 0 to 10 was used, on which 0 stands for novice and 10 for expert. Predictive validity was described by Spearman’s correlation, which was significant in 3 out of 12 skills only. Median correlation (r) was 0.50 (range 0.16 to 0.93). Conclusion: At the beginning of postgraduate training, self-assessment alone is not sufficient to guide self-directed learning. Take-home message: At the beginning of their residency, physicians need structured feedback in emergency skills which can be offered by anaesthesiologists.

Computergestützte Lehre an Hochschulen (Blended Learning). Der Virtuelle Campus Erziehungswissenschaft der Universität Bern

Blended Learning-Angebote - Lehrveranstaltungen, die aus Präsenzanteilen und virtuellen Anteilen im Internet bestehen - halten zunehmend Einzug an Universitäten, Fachhochschulen und Pädagogischen Hochschulen. Diese neuen Lehrformen stehen im Spannungsfeld zwischen technischen Möglichkeiten, ökonomischen Erfordernissen und hochschuldidaktischen Anforderungen. Den Mittelpunkt des Buches bildet das computerunterstützte Lehrangebot des «Virtuellen Campus Erziehungswissenschaft» an der Universität Bern, das der Ausbildung zukünftiger Lehrpersonen an der Pädagogischen Hochschule Bern dient. Zum einen soll dieses in der Praxis bewährte Lehrangebot theoretisch analysiert werden. Zum anderen erfolgt ein Einblick in die Praxis des «Virtuellen Campus Erziehungswissenschaft», um anderen Bildungsinstitutionen Anregungen zur Einrichtung ähnlicher Angebote oder zur Modifizierung ihrer Blended-Learning-Kurse zu geben. Dabei werden die Bereiche (a) Planung und Entwicklung von Lehrangeboten, (b) Methoden der Vermittlung und Einsatz neuer Technologien, (c) Betreuung von Studierenden, (d) Assessment der Studierenden, (e) Qualitätssicherung der Lehre und der eigenen Lehrtätigkeit und (f) Selbstmanagement und Professionalität im Hochschulkontext abgedeckt. Schliesslich wird auch nach der hochschuldidaktischen Vernunft solcher Angebote gefragt.

Learning Analytics: Challenges and Future Research Directions

In recent years, learning analytics (LA) has attracted a great deal of attention in technology-enhanced learning (TEL) research as practitioners, institutions, and researchers are increasingly seeing the potential that LA has to shape the future TEL landscape. Generally, LA deals with the development of methods that harness educational data sets to support the learning process. This paper provides a foundation for future research in LA. It provides a systematic overview on this emerging field and its key concepts through a reference model for LA based on four dimensions, namely data, environments, context (what?), stakeholders (who?), objectives (why?), and methods (how?). It further identifies various challenges and research opportunities in the area of LA in relation to each dimension.

(Lifelong) learning in your pocket?

Der Beitrag fokussiert die Entwicklung, den Einsatz und die Nutzung von innovativen Technologien zur Unterstützung von Bildungsszenarien in Schule, Hochschule und Weiterbildung. Ausgehend von den verschiedenen Phasen des Corporate Learning, Social Learning, Mobile Learning und Intelligent Learning wird in einem ersten Abschnitt das Nutzungsverhalten von Technologien durch Kinder, Jugendliche und (junge) Erwachsene in Schule, Studium und Lehre betrachtet. Es folgt die Darstellung technologischer Entwicklungen auf Basis des Technology Life Cycle und die Konsequenzen von unterschiedlichen Entwicklungszuständen und Reifegraden von Technologien wie Content Learning Management, sozialen Netzwerken, mobilen Endgeräten, multidimensionalen und -modalen Räumen bis hin zu Anwendungen augmentierter Realität und des Internets der Dinge, Dienste und Daten für den Einsatz und die Nutzung in Bildungsszenarien. Nach der Darstellung von Anforderungen an digitale Technologien hinsichtlich Inhalte, Didaktik und Methodik wie etwa hinsichtlich der Erstellung von Inhalten, deren Wiederverwendung, Digitalisierung und Auffindbarkeit sowie Standards werden methodische Hinweise zur Nutzung digitaler Technologien zur Interaktion von Lernenden, von Lehrenden, sozialer Interaktion, kollaborativem Autorieren, Kommentierung, Evaluation und Begutachtung gegeben. Abschließend werden - differenziert für Schule und Hochschule - Erkenntnisse zu Rahmenbedingungen, Einflussgrößen, hemmenden und fördernden Faktoren sowie Herausförderungen bei der Einführung und nachhaltigen Implementation digitaler Technologien im schulischen Unterricht, in Lehre, Studium und Weiterbildung im Überblick zusammengefasst.

Digital Systems for Open Access to Formal and Informal Learning

Today, Digital Systems and Services for Technology Supported Learning and Education are recognized as the key drivers to transform the way that individuals, groups and organizations “learn” and the way to “assess learning” in 21st Century. These transformations influence: Objectives - moving from acquiring new “knowledge” to developing new and relevant “competences”; Methods – moving from “classroom” based teaching to “context-aware” personalized learning; and Assessment – moving from “life-long” degrees and certifications to “on-demand” and “in-context” accreditation of qualifications. Within this context, promoting Open Access to Formal and Informal Learning, is currently a key issue in the public discourse and the global dialogue on Education, including Massive Open Online Courses (MOOCs) and Flipped School Classrooms. This volume on Digital Systems for Open Access to Formal and Informal Learning contributes to the international dialogue between researchers, technologists, practitioners and policy makers in Technology Supported Education and Learning. It addresses emerging issues related with both theory and practice, as well as, methods and technologies that can support Open Access to Formal and Informal Learning. In the twenty chapters contributed by international experts who are actively shaping the future of Educational Technology around the world, topics such as: - The evolution of University Open Courses in Transforming Learning - Supporting Open Access to Teaching and Learning of People with Disabilities - Assessing Student Learning in Online Courses - Digital Game-based Learning for School Education - Open Access to Virtual and Remote Labs for STEM Education - Teachers’ and Schools’ ICT Competence Profiling - Web-Based Education and Innovative Leadership in a K-12 International School Setting are presented. An in-depth blueprint of the promise, potential, and imminent future of the field, Digital Systems for Open Access to Formal and Informal Learning is necessary reading for researchers and practitioners, as well as, undergraduate and postgraduate students, in educational technology.

New Trends in Networking, Computing, E-learning, Systems Sciences, and Engineering

This book provides the latest in a series of books growing out of the International Joint Conferences on Computer, Information and Systems Sciences and Engineering. It includes chapters in the most advanced areas of Computing, Informatics, Systems Sciences and Engineering. It has accessible to a wide range of readership, including professors, researchers, practitioners and students. This book includes a set of rigorously reviewed world-class manuscripts addressing and detailing state-of-the-art research projects in the areas of Computer Science, Informatics, and Systems Sciences, and Engineering. It includes selected papers form the conference proceedings of the Ninth International Joint Conferences on Computer, Information, and Systems Sciences, and Engineering (CISSE 2013). Coverage includes topics in: Industrial Electronics, Technology & Automation, Telecommunications and Networking, Systems, Computing Sciences and Software Engineering, Engineering Education, Instructional Technology, Assessment, and E-learning.

Performance of experienced dentists in Switzerland after an e-learning program on ICDAS occlusal caries detection

This study aimed to evaluate the effect of an e-learning program on the validity and reproducibility of the International Caries Detection and Assessment System (ICDAS) in detecting occlusal caries. For the study, 170 permanent molars were selected. Four dentists in Switzerland who had no previous contact with ICDAS examined the teeth before and after the e-learning program and scored the sites according to ICDAS. Teeth were histologically prepared and assessed for caries extension. The significance level was set at 0.05. Sensitivity before and after the e-learning program was 0.80 and 0.77 (D1), 0.72 and 0.63 (D2), and 0.74 and 0.67 (D3,4), respectively. Specificity was 0.64 and 0.69 (D1), 0.70 and 0.81 (D2), and 0.81 and 0.87 (D3,4). A McNemar test did not show any difference between the values of sensitivity, specificity, accuracy, and area under the ROC curve (AUC) before and after the e-learning program. The averages of wK values for interexaminer reproducibility were 0.61 (before) and 0.66 (after). Correlation with histology presented wK values of 0.62 (before) and 0.63 (after). A Wilcoxon test showed a statistically significant difference between before and after the e-learning program. In conclusion, even though ICDAS performed well in detecting occlusal caries, the e-learning program did not have any statistically significant effect on its performance by these experienced dentists.

Assessing the Influence of Brain-based Instructional Methods and Its Impact on Systemic Change for Learning Communities

Background: As scholars who prepare future school leaders to be innovative instructional leaders for their learning communities, we are on the verge of a curriculum design revolution. The application of brain research findings promotes educational reform efforts to systemically change the way in which children experience school. However, most educators, school leaders, board members, and policy makers are ill prepared to reconsider the implications for assessment, pedagogy, school climate, daily schedules, and use of technology. This qualitative study asked future school leaders to reconsider how school leadership preparedness programs prepared them to become instructional leaders for the 21st century. The findings from this study will enhance the field of school leadership, challenging the current emphasis placed on standardized testing, traditional school calendars, assessments, monocultural instructional methods, and meeting the needs of diverse learning communities. [See PDF for complete abstract]

Assessment of anastomotic leakage: A Novel System for the Training of Surgeons to Perform a Tubular Anastomosis and Objectively Evaluate Anastomotic Leak

Introduction: Dehiscence of the suture line of an anastomosis can lead to reoperation, temporary or permanent stoma, and even sepsis or death. Few techniques for the laboratory training of tubular anastomosis use ex-vivo animal tissues. We describe a novel model that can be used in the laboratory for the training of anastomosis in tubular tissues and objectively assess any anastomotic leak. [See PDF for complete abstract]

Self-Assessment in a Competency-Based Curriculum: Preliminary Findings

Introduction: The introduction of the ACGME core competency framework brought challenges of developing appropriate evaluation tools (i.e. self assessment) to provide evidence of competency. Baylor College of Medicine has 43 competency goals organized within the 6 ACGME domains, each domain having 4-10 goals. [See PDF for complete abstract]

Use of instructional video to prepare parents for learning infant cardiopulmonary resuscitation.

Parents of premature infants often receive infant cardiopulmonary resuscitation (CPR) training prior to discharge from the hospital, but one study showed that 27.5% of parents could not demonstrate adequate CPR skills after completing an instructor-led class. We hypothesized that parents who viewed an instructional video on infant CPR before attending the class would perform better on a standardized skills test than parents who attended the class with no preparation. Parents randomized to the intervention (video) group viewed the video within 48 hours of the CPR class. Parents in the control group attended the class with no special preparation. All parents completed the CPR skills checklist test, usually within 7 days after class and before the infant's hospital discharge. The test rated subjects' skills in the areas of assessment, ventilation, and chest compressions; each section was rated as good, fair, or fail. In this pass/fail test, students had to be rated good or fair on all three sections to pass. All 10 subjects in the video group passed the test versus only 9 of 13 in the control group, but this difference was not significant (P = 0.08). However, 8 of 10 (80%) subjects in the video group were rated as good on all three sections, versus only 3 of 13 (18.7%) in the control group, and this was a significant difference (P = 0.012). We conclude that preparation of students using an instructional video prior to infant CPR class is associated with improvement in skills performance as measured by a standardized skills test. Video preparation is relatively inexpensive, eliminates the barrier of reading ability for preparation, and can be done at the convenience of the parent.

The mini-clinical evaluation exercise during medical clerkships: are learning needs and learning goals aligned?

OBJECTIVES The generation of learning goals (LGs) that are aligned with learning needs (LNs) is one of the main purposes of formative workplace-based assessment. In this study, we aimed to analyse how often trainer–student pairs identified corresponding LNs in mini-clinical evaluation exercise (mini-CEX) encounters and to what degree these LNs aligned with recorded LGs, taking into account the social environment (e.g. clinic size) in which the mini-CEX was conducted. METHODS Retrospective analyses of adapted mini-CEX forms (trainers’ and students’ assessments) completed by all Year 4 medical students during clerkships were performed. Learning needs were defined by the lowest score(s) assigned to one or more of the mini-CEX domains. Learning goals were categorised qualitatively according to their correspondence with the six mini-CEX domains (e.g. history taking, professionalism). Following descriptive analyses of LNs and LGs, multi-level logistic regression models were used to predict LGs by identified LNs and social context variables. RESULTS A total of 512 trainers and 165 students conducted 1783 mini-CEXs (98% completion rate). Concordantly, trainer–student pairs most often identified LNs in the domains of ‘clinical reasoning’ (23% of 1167 complete forms), ‘organisation/efficiency’ (20%) and ‘physical examination’ (20%). At least one ‘defined’ LG was noted on 313 student forms (18% of 1710). Of the 446 LGs noted in total, the most frequently noted were ‘physical examination’ (49%) and ‘history taking’ (21%). Corresponding LNs as well as social context factors (e.g. clinic size) were found to be predictors of these LGs. CONCLUSIONS Although trainer–student pairs often agreed in the LNs they identified, many assessments did not result in aligned LGs. The sparseness of LGs, their dependency on social context and their partial non-alignment with students’ LNs raise questions about how the full potential of the mini-CEX as not only a ‘diagnostic’ but also an ‘educational’ tool can be exploited.

Alignment between learning needs and learning goals of Mini-CEX in clerkships

Background: Defining learning goals (LG) in alignment with learning needs (LN) is one of the key purposes of formative workplace-based assessment, but studies about this topic are scarce. Summary of Work: We analysed quantitatively and qualitatively how often trainer-student pairs identified the same LN during Mini Clinical Evaluation Exercises (Mini-CEX) in clerkships and to what degree those LNs were in line with the recorded LGs. Multilevel logistic regression models were used to predict LGs by identified LNs, controlling for context variables. Summary of Results: 512 trainers and 165 students conducted 1783 Mini-CEX (98% completion rate). Concordantly, trainer-student pairs most often identified LNs in the domains ‘clinical reasoning’ (23% of 1167 complete forms), ‘organisation / efficiency’ (20%) and ‘physical examination’ (20%). At least one ‘defined’ LG was noted on 313 student forms (18% of 1710), with a total of 446 LGs. Of these, the most frequent LGs were ‘physical examination’ (49% of 446 LGs) and ‘history taking’ (21%); corresponding LNs as well as context variables (e.g. clinic size) were found to be predictors of these LGs. Discussion and Conclusions: Although trainer-student pairs often agreed in their identified LNs, many assessments did not result in an aligned LG or a LG at all. Interventions are needed to enhance the proportion of (aligned) LGs in Mini-CEX in order to tap into its full potential not only as a ‘diagnostic’ but also as an ‘educational tool’. Take-home messages: The sparseness of LGs, their dependency on context variables and their partial non-alignment with students’ LNs raise the question of how the effectiveness of Mini-CEX can be further enhanced.