An innovative blended learning approach using virtual patients as preparation for skills laboratory training: perceptions of students and tutors

BACKGROUND Currently only a few reports exist on how to prepare medical students for skills laboratory training. We investigated how students and tutors perceive a blended learning approach using virtual patients (VPs) as preparation for skills training. METHODS Fifth-year medical students (N=617) were invited to voluntarily participate in a paediatric skills laboratory with four specially designed VPs as preparation. The cases focused on procedures in the laboratory using interactive questions, static and interactive images, and video clips. All students were asked to assess the VP design. After participating in the skills laboratory 310 of the 617 students were additionally asked to assess the blended learning approach through established questionnaires. Tutors' perceptions (N=9) were assessed by semi-structured interviews. RESULTS From the 617 students 1,459 VP design questionnaires were returned (59.1%). Of the 310 students 213 chose to participate in the skills laboratory; 179 blended learning questionnaires were returned (84.0%). Students provided high overall acceptance ratings of the VP design and blended learning approach. By using VPs as preparation, skills laboratory time was felt to be used more effectively. Tutors perceived students as being well prepared for the skills laboratory with efficient uses of time. CONCLUSION The overall acceptance of the blended learning approach was high among students and tutors. VPs proved to be a convenient cognitive preparation tool for skills training.

Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: The design of Virtual Patients (VPs) is essential. So far there are no validated evaluation instruments for VP design published. Summary of work: We examined three sources of validity evidence of an instrument to be filled out by students aimed at measuring the quality of VPs with a special emphasis on fostering clinical reasoning: (1) Content was examined based on theory of clinical reasoning and an international VP expert team. (2) Response process was explored in think aloud pilot studies with students and content analysis of free text questions accompanying each item of the instrument. (3) Internal structure was assessed by confirmatory factor analysis (CFA) using 2547 student evaluations and reliability was examined utilizing generalizability analysis. Summary of results: Content analysis was supported by theory underlying Gruppen and Frohna’s clinical reasoning model on which the instrument is based and an international VP expert team. The pilot study and analysis of free text comments supported the validity of the instrument. The CFA indicated that a three factor model comprising 6 items showed a good fit with the data. Alpha coefficients per factor were 0,74 - 0,82. The findings of the generalizability studies indicated that 40-200 student responses are needed in order to obtain reliable data on one VP. Conclusions: The described instrument has the potential to provide faculty with reliable and valid information about VP design. Take-home messages: We present a short instrument which can be of help in evaluating the design of VPs.

Im Nebel der Präzision: die Bestimmung von GrenzfallkandidatInnen oder wie viel Reliabilität ist genug?

Hintergrund: Bei der Durchführung von summativen Prüfungen wird üblicherweise eine Mindestreliabilität von 0,8 gefordert. Bei praktischen Prüfungen wie OSCEs werden manchmal 0,7 akzeptiert (Downing 2004). Doch was kann man sich eigentlich unter der Präzision einer Messung mit einer Reliabilität von 0,7 oder 0,8 vorstellen? Methode: Mittels verschiedener statistischer Methoden wie dem Standardmessfehler oder der Generalisierbarkeitstheorie lässt sich die Reliabilität in ein Konfidenzintervall um eine festgestellte Kandidatenleistung übersetzen (Brennan 2003, Harvill 1991, McManus 2012). Hat ein Kandidat beispielsweise bei einer Prüfung 57 Punkte erreicht, schwankt seine wahre Leistung aufgrund der Messungenauigkeit der Prüfung um diesen Wert (z.B. zwischen 50 und 64 Punkte). Im Bereich der Bestehensgrenze ist die Messgenauigkeit aber besonders wichtig. Läge die Bestehensgrenze in unserem Beispiel bei 60 Punkten, wäre der Kandidat mit 57 Punkten zwar pro forma durchgefallen, allerdings könnte er aufgrund der Schwankungsbreite um seine gemessene Leistung in Wahrheit auch knapp bestanden haben. Überträgt man diese Erkenntnisse auf alle KandidatInnen einer Prüfung, kann man die Anzahl der Grenzfallkandidaten bestimmen, also all jene Kandidatinnen, die mit Ihrem Prüfungsergebnis so nahe an der Bestehensgrenze liegen, dass ihr jeweiliges Prüfungsresultate falsch positiv oder falsch negativ sein kann. Ergebnisse: Die Anzahl der GrenzfallkandidatInnen in einer Prüfung ist, nicht nur von der Reliabilität abhängig, sondern auch von der Leistung der KandidatInnen, der Varianz, dem Abstand der Bestehensgrenze zum Mittelwert und der Schiefe der Verteilung. Es wird anhand von Modelldaten und konkreten Prüfungsdaten der Zusammenhang zwischen der Reliabilität und der Anzahl der Grenzfallkandidaten auch für den Nichtstatistiker verständlich dargestellt. Es wird gezeigt, warum selbst eine Reliabilität von 0.8 in besonderen Situationen keine befriedigende Präzision der Messung bieten wird, während in manchen OSCEs die Reliabilität fast ignoriert werden kann. Schlussfolgerungen: Die Berechnung oder Schätzung der Grenzfallkandidaten anstatt der Reliabilität verbessert auf anschauliche Weise das Verständnis für die Präzision einer Prüfung. Wenn es darum geht, wie viele Stationen ein summativer OSCE benötigt oder wie lange eine MC-Prüfung dauern soll, sind Grenzfallkandidaten ein valideres Entscheidungskriterium als die Reliabilität. Brennan, R.L. (2003) Generalizability Theory. New York, Springer Downing, S.M. (2004) ‘Reliability: on the reproducibility of assessment data’, Medical Education 2004, 38, 1006–12 Harvill, L.M. (1991) ‘Standard Error of Measurement’, Educational Measurement: Issues and Practice, 33-41 McManus, I.C. (2012) ‘The misinterpretation of the standard error of measurement in medical education: A primer on the problems, pitfalls and peculiarities of the three different standard errors of measurement’ Medical teacher, 34, 569 - 76

Educational Technology-Benefits and Barriers

Statement of Problem: The second background paper for the Medical School Objective Project (MSOP), defined Educational Technology (ET) as the use of information technology to facilitate student’s learning.1 Medical schools as a group have made limited progress in accomplishing the recommended educational technology goals and there had been much greater use of such technology in basic sciences courses than in clinical clerkships. We will explore the positive and negative implications of incorporating ET into the educational experience of TMC schools. [See PDF for complete abstract]

Assessment of anastomotic leakage: A Novel System for the Training of Surgeons to Perform a Tubular Anastomosis and Objectively Evaluate Anastomotic Leak

Introduction: Dehiscence of the suture line of an anastomosis can lead to reoperation, temporary or permanent stoma, and even sepsis or death. Few techniques for the laboratory training of tubular anastomosis use ex-vivo animal tissues. We describe a novel model that can be used in the laboratory for the training of anastomosis in tubular tissues and objectively assess any anastomotic leak. [See PDF for complete abstract]

Streaming Video: Student Performance and Relative Use in the Preclinical Curriculum

Purpose: To assess the relationship between student utilization of learning resources, including streaming video (SV), and their performance in the pre-clinical curriculum. [See PDF for complete abstract]

The Critical Friends Group: An Innovative Way to Build Intercultural Competence Among Student and Faculty Groups

Introduction Few physicians involved in medical education are likely to have had formal training in teaching. One pedagogical method that can enhance relationships, thus improve teaching and learning is the Critical Friends Group (CFG). The CFG is a collegial support team that offers improved understanding of others. Unconditional high regard for team members frames the interactions in the CFG. These teams could be used to reduce bias and enhance intercultural competence among student CFGs and faculty CFGs. [See PDF for complete abstract]

Construct Validity of the Pig Intestine Model in the Simulation of Laparoscopic Urethrovesical Anastomosis

Introduction: Laparoscopic training models are increasingly important in urology to allow trainees to improve their laparoscopic skills prior to going to the operating room. For a training model to be valid, it must correlate with performance in a real case. The model must also discriminate between experienced and inexperienced subjects. [See PDF for complete abstract]

The new licencing examination for human medicine: from concept to implementation

A new Swiss federal licencing examination for human medicine (FLE) was developed and released in 2011. This paper describes the process from concept design to the first results obtained on implementation of the new examination. The development process was based on the Federal Act on University Medical Professions and involved all national stakeholders in this venture. During this process questions relating to the assessment aims, the assessment formats, the assessment dimensions, the examination content and necessary trade-offs were clarified. The aims were to create a feasible, fair, valid and psychometrically sound examination in accordance with international standards, thereby indicating the expected knowledge and skills level at the end of undergraduate medical education. Finally, a centrally managed and locally administered examination comprising a written multiple-choice element and a practical “clinical skills” test in the objective structured clinical examination (OSCE) format was developed. The first two administrations of the new FLE show that the examination concept could be implemented as intended. The anticipated psychometric indices were achieved and the results support the validity of the examination. Possible changes to the format or content in the future are discussed.

Communication Across Cultures - Use of Video to Promote Culturally Competent Patient Management

Diverse belief systems exist among dental and medical patients related to health, healing and wellness. Culturally competent health care may be defined as the ability to provide care to patients with diverse values, beliefs and behaviors, including modifying delivery of care to meet patients’ social and cultural needs. [See PDF for complete abstract]

COGNITIVE IMPACT OF INTERACTIVE MULTIMEDIA

Technology has been gradually introduced in heath education. One of the most attractive features of this technology-based education is the use of multimedia. In this article we explore the research evidence about the role that multimedia is playing in education. From that analysis we describe the most relevant features of this technology to prepare a common ground of discussion about the evaluation of its impact on educational outcomes. As part of this analysis, we organize current research evidence on the use of technology in medical education, distinguishing diverse variables involved in the process, like knowledge (declarative, procedural), learner characteristics, curricular scenario, etc. This article presents an overview of the Distributed Representations theory and its relationship with research on educational outcomes and multimedia. Next we discuss the relationship between media and diverse learning theories, proposing a theory based taxonomy for educational multimedia.

Narrated PowerPoints and Audio Podcasts in Clinical Teaching: One Clerkship's Experience

Introduction: Clinical medical education is increasingly utilizing novel technological approaches in order to supplement traditional lecture-based didactics. The Neurology Core Clerkship at Baylor College of Medicine is a four week required course taken by clinical medical students. Given the large amount of information to be disseminated in a short period of time, part of the didactic material has been provided online in the form of narrated PowerPoint files or lecture audio tracks along with stand-alone PowerPoint files. The narrated files are generated using the native PowerPoint narration function while the stand-alone audio files are created as MP3 format files using an inexpensive digital recording device. [See PDF for complete abstract]

Self-Assessment in a Competency-Based Curriculum: Preliminary Findings

Introduction: The introduction of the ACGME core competency framework brought challenges of developing appropriate evaluation tools (i.e. self assessment) to provide evidence of competency. Baylor College of Medicine has 43 competency goals organized within the 6 ACGME domains, each domain having 4-10 goals. [See PDF for complete abstract]

Simulator Network project report: a tool for improvement of teaching materials and targeted resource usage in Skills Labs

During the last decade, medical education in the German-speaking world has been striving to become more practice-oriented. This is currently being achieved in many schools through the implementation of simulation-based instruction in Skills Labs. Simulators are thus an essential part of this type of medical training, and their acquisition and operation by a Skills Lab require a large outlay of resources. Therefore, the Practical Skills Committee of the Medical Education Society (GMA) introduced a new project, which aims to improve the flow of information between the Skills Labs and enable a transparent assessment of the simulators via an online database (the Simulator Network).