Im Nebel der Präzision: die Bestimmung von GrenzfallkandidatInnen oder wie viel Reliabilität ist genug?

Hintergrund: Bei der Durchführung von summativen Prüfungen wird üblicherweise eine Mindestreliabilität von 0,8 gefordert. Bei praktischen Prüfungen wie OSCEs werden manchmal 0,7 akzeptiert (Downing 2004). Doch was kann man sich eigentlich unter der Präzision einer Messung mit einer Reliabilität von 0,7 oder 0,8 vorstellen? Methode: Mittels verschiedener statistischer Methoden wie dem Standardmessfehler oder der Generalisierbarkeitstheorie lässt sich die Reliabilität in ein Konfidenzintervall um eine festgestellte Kandidatenleistung übersetzen (Brennan 2003, Harvill 1991, McManus 2012). Hat ein Kandidat beispielsweise bei einer Prüfung 57 Punkte erreicht, schwankt seine wahre Leistung aufgrund der Messungenauigkeit der Prüfung um diesen Wert (z.B. zwischen 50 und 64 Punkte). Im Bereich der Bestehensgrenze ist die Messgenauigkeit aber besonders wichtig. Läge die Bestehensgrenze in unserem Beispiel bei 60 Punkten, wäre der Kandidat mit 57 Punkten zwar pro forma durchgefallen, allerdings könnte er aufgrund der Schwankungsbreite um seine gemessene Leistung in Wahrheit auch knapp bestanden haben. Überträgt man diese Erkenntnisse auf alle KandidatInnen einer Prüfung, kann man die Anzahl der Grenzfallkandidaten bestimmen, also all jene Kandidatinnen, die mit Ihrem Prüfungsergebnis so nahe an der Bestehensgrenze liegen, dass ihr jeweiliges Prüfungsresultate falsch positiv oder falsch negativ sein kann. Ergebnisse: Die Anzahl der GrenzfallkandidatInnen in einer Prüfung ist, nicht nur von der Reliabilität abhängig, sondern auch von der Leistung der KandidatInnen, der Varianz, dem Abstand der Bestehensgrenze zum Mittelwert und der Schiefe der Verteilung. Es wird anhand von Modelldaten und konkreten Prüfungsdaten der Zusammenhang zwischen der Reliabilität und der Anzahl der Grenzfallkandidaten auch für den Nichtstatistiker verständlich dargestellt. Es wird gezeigt, warum selbst eine Reliabilität von 0.8 in besonderen Situationen keine befriedigende Präzision der Messung bieten wird, während in manchen OSCEs die Reliabilität fast ignoriert werden kann. Schlussfolgerungen: Die Berechnung oder Schätzung der Grenzfallkandidaten anstatt der Reliabilität verbessert auf anschauliche Weise das Verständnis für die Präzision einer Prüfung. Wenn es darum geht, wie viele Stationen ein summativer OSCE benötigt oder wie lange eine MC-Prüfung dauern soll, sind Grenzfallkandidaten ein valideres Entscheidungskriterium als die Reliabilität. Brennan, R.L. (2003) Generalizability Theory. New York, Springer Downing, S.M. (2004) ‘Reliability: on the reproducibility of assessment data’, Medical Education 2004, 38, 1006–12 Harvill, L.M. (1991) ‘Standard Error of Measurement’, Educational Measurement: Issues and Practice, 33-41 McManus, I.C. (2012) ‘The misinterpretation of the standard error of measurement in medical education: A primer on the problems, pitfalls and peculiarities of the three different standard errors of measurement’ Medical teacher, 34, 569 - 76

Developing and Integrating the Management of Elder Abuse in Primary Practice: A Case Study Using A Web-Based CME Course Format

Introduction: As the population in the United States continues to age, more attention in primary practice settings is now devoted toward managing the care of the elderly. The occurrence of elder abuse is a growing problem. It is a condition many professionals in primary care may be ill prepared with the knowledge or resources to identify and manage. [See PDF for complete abstract]

Educational Technology-Benefits and Barriers

Statement of Problem: The second background paper for the Medical School Objective Project (MSOP), defined Educational Technology (ET) as the use of information technology to facilitate student’s learning.1 Medical schools as a group have made limited progress in accomplishing the recommended educational technology goals and there had been much greater use of such technology in basic sciences courses than in clinical clerkships. We will explore the positive and negative implications of incorporating ET into the educational experience of TMC schools. [See PDF for complete abstract]

Assessment of anastomotic leakage: A Novel System for the Training of Surgeons to Perform a Tubular Anastomosis and Objectively Evaluate Anastomotic Leak

Introduction: Dehiscence of the suture line of an anastomosis can lead to reoperation, temporary or permanent stoma, and even sepsis or death. Few techniques for the laboratory training of tubular anastomosis use ex-vivo animal tissues. We describe a novel model that can be used in the laboratory for the training of anastomosis in tubular tissues and objectively assess any anastomotic leak. [See PDF for complete abstract]

Streaming Video: Student Performance and Relative Use in the Preclinical Curriculum

Purpose: To assess the relationship between student utilization of learning resources, including streaming video (SV), and their performance in the pre-clinical curriculum. [See PDF for complete abstract]

The Critical Friends Group: An Innovative Way to Build Intercultural Competence Among Student and Faculty Groups

Introduction Few physicians involved in medical education are likely to have had formal training in teaching. One pedagogical method that can enhance relationships, thus improve teaching and learning is the Critical Friends Group (CFG). The CFG is a collegial support team that offers improved understanding of others. Unconditional high regard for team members frames the interactions in the CFG. These teams could be used to reduce bias and enhance intercultural competence among student CFGs and faculty CFGs. [See PDF for complete abstract]

Construct Validity of the Pig Intestine Model in the Simulation of Laparoscopic Urethrovesical Anastomosis

Introduction: Laparoscopic training models are increasingly important in urology to allow trainees to improve their laparoscopic skills prior to going to the operating room. For a training model to be valid, it must correlate with performance in a real case. The model must also discriminate between experienced and inexperienced subjects. [See PDF for complete abstract]

The new licencing examination for human medicine: from concept to implementation

A new Swiss federal licencing examination for human medicine (FLE) was developed and released in 2011. This paper describes the process from concept design to the first results obtained on implementation of the new examination. The development process was based on the Federal Act on University Medical Professions and involved all national stakeholders in this venture. During this process questions relating to the assessment aims, the assessment formats, the assessment dimensions, the examination content and necessary trade-offs were clarified. The aims were to create a feasible, fair, valid and psychometrically sound examination in accordance with international standards, thereby indicating the expected knowledge and skills level at the end of undergraduate medical education. Finally, a centrally managed and locally administered examination comprising a written multiple-choice element and a practical “clinical skills” test in the objective structured clinical examination (OSCE) format was developed. The first two administrations of the new FLE show that the examination concept could be implemented as intended. The anticipated psychometric indices were achieved and the results support the validity of the examination. Possible changes to the format or content in the future are discussed.

Communication Across Cultures - Use of Video to Promote Culturally Competent Patient Management

Diverse belief systems exist among dental and medical patients related to health, healing and wellness. Culturally competent health care may be defined as the ability to provide care to patients with diverse values, beliefs and behaviors, including modifying delivery of care to meet patients’ social and cultural needs. [See PDF for complete abstract]

COGNITIVE IMPACT OF INTERACTIVE MULTIMEDIA

Technology has been gradually introduced in heath education. One of the most attractive features of this technology-based education is the use of multimedia. In this article we explore the research evidence about the role that multimedia is playing in education. From that analysis we describe the most relevant features of this technology to prepare a common ground of discussion about the evaluation of its impact on educational outcomes. As part of this analysis, we organize current research evidence on the use of technology in medical education, distinguishing diverse variables involved in the process, like knowledge (declarative, procedural), learner characteristics, curricular scenario, etc. This article presents an overview of the Distributed Representations theory and its relationship with research on educational outcomes and multimedia. Next we discuss the relationship between media and diverse learning theories, proposing a theory based taxonomy for educational multimedia.

Narrated PowerPoints and Audio Podcasts in Clinical Teaching: One Clerkship's Experience

Introduction: Clinical medical education is increasingly utilizing novel technological approaches in order to supplement traditional lecture-based didactics. The Neurology Core Clerkship at Baylor College of Medicine is a four week required course taken by clinical medical students. Given the large amount of information to be disseminated in a short period of time, part of the didactic material has been provided online in the form of narrated PowerPoint files or lecture audio tracks along with stand-alone PowerPoint files. The narrated files are generated using the native PowerPoint narration function while the stand-alone audio files are created as MP3 format files using an inexpensive digital recording device. [See PDF for complete abstract]

Self-Assessment in a Competency-Based Curriculum: Preliminary Findings

Introduction: The introduction of the ACGME core competency framework brought challenges of developing appropriate evaluation tools (i.e. self assessment) to provide evidence of competency. Baylor College of Medicine has 43 competency goals organized within the 6 ACGME domains, each domain having 4-10 goals. [See PDF for complete abstract]

Simulator Network project report: a tool for improvement of teaching materials and targeted resource usage in Skills Labs

During the last decade, medical education in the German-speaking world has been striving to become more practice-oriented. This is currently being achieved in many schools through the implementation of simulation-based instruction in Skills Labs. Simulators are thus an essential part of this type of medical training, and their acquisition and operation by a Skills Lab require a large outlay of resources. Therefore, the Practical Skills Committee of the Medical Education Society (GMA) introduced a new project, which aims to improve the flow of information between the Skills Labs and enable a transparent assessment of the simulators via an online database (the Simulator Network).

Dermatology in the immunosuppressed patient

This anniversary book gives an animated description of the first one hundred years of the Swiss Society of Dermatology and Venereology (SSDV – SGDV). The approximately 60 authors write from the subjective perspective of the contemporary witness and thus create a vibrant picture of their field and the times in which we live. “Spirit and Soul of Swiss Dermatology and Venereology 1913 – 2013” is therefore an ideal companion to the medical history book “Dermatologie und Venerologie in der Schweiz – ein historischer Rückblick” (2002 Editions Alphil, ISBN 2-940235-08-2), published in 2003 by the SSDV – SGDV for its 90th anniversary. The anniversary edition for the centennial is written in English in order to make the history of the SSDV – SGDV accessible to a larger international public. The introductory chapter is written in all four national languages (German, French, Italian, and Romansh) and also translated into English. It is followed by chapters about the university and non-university public dermatology and venereology departments, the memoirs of those presidents still living, and the depiction of the numerous sub-disciplines of dermatology. Further important chapters include a large contribution on the beginnings of dermatological research in Switzerland, a series of pieces on medical education and continuing education, and finally an overview of healthcare politics in Switzerland. Our friends in Germany, Austria, France, Italy and the USA have provided the outside perspective on Swiss dermatology and venereology in their essays. All in all an informative and entertaining overview of a very diverse medical specialty has been created, which combines historical facts with dynamic insights into this topical field and the current political healthcare framework.

History of Medicine Schedule 2010-2011

William Osler (1849-1919): America’s Most Famous Physician (Robert E. Rakel) The Assassination of John F. Kennedy: A Neurosurgeon’s Eyewitness Account of the Medical Aspect of the Events of November 22, 1963 (Robert G. Grossman) Making Cancer History: Disease and Discovery at the University of Texas M.D. Anderson Cancer Center (James S. Olson) The History of Pathology as a Biological Science and Medical Specialty (L. Maximillian Buja) “Medicine in the Mid-19th Century America” (Student Essay Contest Winner) (David Hunter) The Achievements and Enduring Relevance of Rudolph Virchow (Nathan Grohmann) Medicine: Perspectives in History and Art (Robert E. Greenspan) What Every Physician Should Know: Lessons from the Past (Robert E. Greenspan) Medicine in Ancient Mesopotamia (Sajid Haque) The History of Texas Children’s Hospital (B. Lee Ligon) Visualizing Disease: Motion Pictures in the History of Medical Education (Kirsten Ostherr)