Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: A simulation and survey

Background Many medical exams use 5 options for multiple choice questions (MCQs), although the literature suggests that 3 options are optimal. Previous studies on this topic have often been based on non-medical examinations, so we sought to analyse rarely selected, 'non-functional' distractors (NF-D) in high stakes medical examinations, and their detection by item authors as well as psychometric changes resulting from a reduction in the number of options. Methods Based on Swiss Federal MCQ examinations from 2005-2007, the frequency of NF-D (selected by <1% or <5% of the candidates) was calculated. Distractors that were chosen the least or second least were identified and candidates who chose them were allocated to the remaining options using two extreme assumptions about their hypothetical behaviour: In case rarely selected distractors were eliminated, candidates could randomly choose another option - or purposively choose the correct answer, from which they had originally been distracted. In a second step, 37 experts were asked to mark the least plausible options. The consequences of a reduction from 4 to 3 or 2 distractors - based on item statistics or on the experts' ratings - with respect to difficulty, discrimination and reliability were modelled. Results About 70% of the 5-option-items had at least 1 NF-D selected by <1% of the candidates (97% for NF-Ds selected by <5%). Only a reduction to 2 distractors and assuming that candidates would switch to the correct answer in the absence of a 'non-functional' distractor led to relevant differences in reliability and difficulty (and to a lesser degree discrimination). The experts' ratings resulted in slightly greater changes compared to the statistical approach. Conclusions Based on item statistics and/or an expert panel's recommendation, the choice of a varying number of 3-4 (or partly 2) plausible distractors could be performed without marked deteriorations in psychometric characteristics.

Further concerns about self-assessment: are medical students aware of their progress in clinical skills during clerkship

Background: Medical students do not accurately self-assess their competence. However, little is known about the awareness of change of competence over time. The aim of this study was to evaluate if students are aware of their progress. Summary of work: Twenty-two fourth year medical students had self- and expert-assessments of their clinical skills in musculoskeletal medicine in an OSCE like station (4 point Likert scale) at the beginning (t0) and end (t1) of their eight weeks clerkship in internal medicine. Thirteen students were assigned to the intervention of a 6x1 hour practical examination course; nine took part in the regular clinical clerkship activities only and served as controls. Summary of results/Conclusions: The intervention students significantly improved their skills (from 2.78 ± 0.36 to 3.30 ± 0.36, p<0.05) in contrast to the control students (from 3.11 ± 0.58 to 2.83 ± 0.49, n.s.). At t0, 19 students, at t1 21 out of 22 students underestimated their competence. Correlations between the change of self- and expert-assessment were r=0.43, p<0.05 (all), r=0.47, n.s. (control) and r=-0.12, n.s. (intervention), respectively. Take-home message: Medical students improving their clinical skills by an interactive course in addition to their regular clerkship activities are not aware of their progress

Master of Education in Teaching Program with an Emphasis in the Health Sciences

Introduction: Concerns about the quality of physician education have changed current medical education practices. Learners must demonstrate competency in core areas, rather than solely participating in educational activities. Academic medical institutions are challenged with identifying leaders to direct curricular and evaluation reforms. An innovative partnership between the University of Houston College of Education and Baylor College of Medicine, the University of Texas Medical School at Houston, and the University of Texas Dental Branch at Houston offers a Masters of Education in Teaching degree with an emphasis in Health Sciences. Courses encompass fundamental areas including curriculum, instruction, technology, measurement, research design and statistics. [See PDF for complete abstract]

Waking Up to Sleep Medicine Education

Background: Sleep disorders are an important cause of morbidity among our population with billions of dollars spent on direct and indirect costs attributed to sleep disorders. In spite of raising prevalence and morbidity, surveys have shown inadequate education in sleep medicine at all levels at medical school. According to national sleep disorders research plan data, in 1990 about 37 % of medical schools did not offer any sleep education and of the schools which offered it, the average time devoted to sleep medicine was about 2 hours. Sleep disorders have found to be uniformly under diagnosed in primary care settings. [See PDF for complete abstract]

The physician's perception of medical error and its application to the development of an educational training tool.

It is becoming clear that if we are to impact the rate of medical errors it will have to be done at the practicing physician level. The purpose of this project was to survey the attitude of physicians in Alabama concerning their perception of medical error, and to obtain their thoughts and desires for medical education in the area of medical errors. The information will be used in the development of a physician education program.

Consequences of insecurity in emergency telephone consultations: an experimental study in medical students.

QUESTION UNDER STUDY Handling emergency telephone consultations (ETCs) is a challenging and very important task for doctors. The aims of the study were to document insecurity in medical students during ETCs and to identify the reasons for that insecurity. We hypothesised that insecurity is associated with advising more urgent action (e.g. advice to call for an ambulance) in ETCs. METHODS We used ETCs with simulated patients (SPs), with each student randomly allocated two of four possible cases. After the training, 137 students reported on any insecurity that they had in the various ETC phases. We analysed the reasons for insecurity using descriptive statistics. The association between the students' advice that urgent action was needed and their insecurity was analysed with Spearman rank correlation. RESULTS Overall, 95% of the students felt insecure in at least one phase of their ETC. History taking was the phase in which students felt most insecure (63.1%), followed by the phase of analysing the information given by the patient (44.9%). Perceived insecurity was associated with more urgent advice in one case scenario (abdominal pain; correlation r = 0.46; p <0.01). The other two cases (child with fever; chest pain) also had a positive, but not statistically significant, correlation trend (p <0.12; p <0.08). CONCLUSIONS Insecurity is highly prevalent among medical students in their ETC decision-making. ETC training in medical schools, with a focus on structured history taking and formulating discriminating questions, might help decrease insecurity in ETCs. Medical education should also teach management of insecurity.

Effects of geriatric clinical skills training on the attitudes of medical students.

BACKGROUND Physicians' attitudes, knowledge and skills are powerful determinants of quality of care for older patients. Previous studies found that using educational interventions to improve attitude is a difficult task. No previous study sought to determine if a skills-oriented educational intervention improved student attitudes towards elderly patients. METHODS This study evaluated the effect of a geriatric clinical skills training (CST) on attitudes of University of Bern medical students in their first year of clinical training. The geriatric CST consisted of four 2.5-hour teaching sessions that covered central domains of geriatric assessment (e.g., cognition, mobility), and a textbook used by students to self-prepare. Students' attitudes were the primary outcome, and were assessed with the 14-item University of California at Los Angeles Geriatrics Attitudes Scale (UCLA-GAS) in a quasi-randomized fashion, either before or after geriatric CST. RESULTS A total of 154 medical students participated. Students evaluated before the CST had a median UCLA-GAS overall scale of 49 (interquartile range 44-53). After the CST, the scores increased slightly, to 51 (interquartile range 47-54; median difference 2, 95% confidence interval 0-4, P = 0.062). Of the four validated UCLA-GAS subscales, only the resource distribution subscale was significantly higher in students evaluated after the geriatric CST (median difference 1, 95% confidence interval 0-2, P = 0.005). CONCLUSIONS Teaching that targets specific skills may improve the attitudes of medical students towards elderly patients, though the improvement was slight. The addition of attitude-building elements may improve the effectiveness of future skills-oriented educational interventions.

The mini-clinical evaluation exercise during medical clerkships: are learning needs and learning goals aligned?

OBJECTIVES The generation of learning goals (LGs) that are aligned with learning needs (LNs) is one of the main purposes of formative workplace-based assessment. In this study, we aimed to analyse how often trainer–student pairs identified corresponding LNs in mini-clinical evaluation exercise (mini-CEX) encounters and to what degree these LNs aligned with recorded LGs, taking into account the social environment (e.g. clinic size) in which the mini-CEX was conducted. METHODS Retrospective analyses of adapted mini-CEX forms (trainers’ and students’ assessments) completed by all Year 4 medical students during clerkships were performed. Learning needs were defined by the lowest score(s) assigned to one or more of the mini-CEX domains. Learning goals were categorised qualitatively according to their correspondence with the six mini-CEX domains (e.g. history taking, professionalism). Following descriptive analyses of LNs and LGs, multi-level logistic regression models were used to predict LGs by identified LNs and social context variables. RESULTS A total of 512 trainers and 165 students conducted 1783 mini-CEXs (98% completion rate). Concordantly, trainer–student pairs most often identified LNs in the domains of ‘clinical reasoning’ (23% of 1167 complete forms), ‘organisation/efficiency’ (20%) and ‘physical examination’ (20%). At least one ‘defined’ LG was noted on 313 student forms (18% of 1710). Of the 446 LGs noted in total, the most frequently noted were ‘physical examination’ (49%) and ‘history taking’ (21%). Corresponding LNs as well as social context factors (e.g. clinic size) were found to be predictors of these LGs. CONCLUSIONS Although trainer–student pairs often agreed in the LNs they identified, many assessments did not result in aligned LGs. The sparseness of LGs, their dependency on social context and their partial non-alignment with students’ LNs raise questions about how the full potential of the mini-CEX as not only a ‘diagnostic’ but also an ‘educational’ tool can be exploited.

An investigation into the factors that encourage learner participation in a large group medical classroom

BACKGROUND: Effective lectures often incorporate activities that encourage learner participation. A challenge for educators is how to facilitate this in the large group lecture setting. This study investigates the individual student characteristics involved in encouraging (or dissuading) learners to interact, ask questions, and make comments in class. METHODS: Students enrolled in a Doctor of Veterinary Medicine program at Ross University School of Veterinary Medicine, St Kitts, were invited to complete a questionnaire canvassing their participation in the large group classroom. Data from the questionnaire were analyzed using Excel (Microsoft, Redmond, WA, USA) and the R software environment (http://www.r-project.org/). RESULTS: One hundred and ninety-two students completed the questionnaire (response rate, 85.7%). The results showed statistically significant differences between male and female students when asked to self-report their level of participation (P=0.011) and their confidence to participate (P<0.001) in class. No statistically significant difference was identified between different age groups of students (P=0.594). Student responses reflected that an "aversion to public speaking" acted as the main deterrent to participating during a lecture. Female participants were 3.56 times more likely to report a fear of public speaking than male participants (odds ratio 3.56, 95% confidence interval 1.28-12.33, P=0.01). Students also reported "smaller sizes of class and small group activities" and "other students participating" as factors that made it easier for them to participate during a lecture. CONCLUSION: In this study, sex likely played a role in learner participation in the large group veterinary classroom. Male students were more likely to participate in class and reported feeling more confident to participate than female students. Female students in this study commonly identified aversion to public speaking as a factor which held them back from participating in the large group lecture setting. These are important findings for veterinary and medical educators aiming to improve learner participation in the classroom. Potential ways of addressing this challenge include addition of small group activities and audience response systems during lectures, and inclusion of training interventions in public speaking at an early stage of veterinary and medical curricula.

Medical educators: How they define themselves - Results of an international web survey.

BACKGROUND Little is known about medical educators' self-definition. AIMS The aim of this study is to survey an international community of medical educators focusing on the medical educators' self-definition. METHODS Within a comprehensive, web-based survey, an open question on the participants' views of how they would define a "medical educator" was sent to 2200 persons on the mailing list of the Association for Medical Education in Europe. The free text definitions were analysed using qualitative thematic analysis. RESULTS Of the, 200 medical educators invited to participate, 685 (31.1%) provided a definition of a "medical educator". The qualitative analysis of the free text definitions revealed that medical educators defined themselves in 13 roles, primarily as "Professional Expert", "Facilitator", "Information Provider", "Enthusiast", "Faculty Developer", "Mentor", "Undergraduate and Postgraduate Trainer", "Curriculum Developer", "Assessor and Assessment Creator", and "Researcher". CONCLUSIONS Our survey revealed that medical educators predominantly define themselves as "Professional Experts" and identified 12 further self-defined roles of a medical educator, several of which not to have been reported previously. The results can be used to further the understanding of our professional identity.

Undergraduate palliative care teaching in Swiss medical faculties: a nationwide survey and improved learning objectives

BACKGROUND In 2007, a first survey on undergraduate palliative care teaching in Switzerland has revealed major heterogeneity of palliative care content, allocation of hours and distribution throughout the 6 year curriculum in Swiss medical faculties. This second survey in 2012/13 has been initiated as part of the current Swiss national strategy in palliative care (2010 - 2015) to serve as a longitudinal monitoring instrument and as a basis for redefinition of palliative care learning objectives and curriculum planning in our country. METHODS As in 2007, a questionnaire was sent to the deans of all five medical faculties in Switzerland in 2012. It consisted of eight sections: basic background information, current content and hours in dedicated palliative care blocks, current palliative care content in other courses, topics related to palliative care presented in other courses, recent attempts at improving palliative care content, palliative care content in examinations, challenges, and overall summary. Content analysis was performed and the results matched with recommendations from the EAPC for undergraduate training in palliative medicine as well as with recommendations from overseas countries. RESULTS There is a considerable increase in palliative care content, academic teaching staff and hours in all medical faculties compared to 2007. No Swiss medical faculty reaches the range of 40 h dedicated specifically to palliative care as recommended by the EAPC. Topics, teaching methods, distribution throughout different years and compulsory attendance still differ widely. Based on these results, the official Swiss Catalogue of Learning Objectives (SCLO) was complemented with 12 new learning objectives for palliative and end of life care (2013), and a national basic script for palliative care was published (2015). CONCLUSION Performing periodic surveys of palliative care teaching at national medical faculties has proven to be a useful tool to adapt the national teaching framework and to improve the recognition of palliative medicine as an integral part of medical training.

The "Profiles" document: a modern revision of the objectives of undergraduate medical studies in Switzerland

The Joint Commission of the Swiss Medical Schools (SMIFK/CIMS) decided in 2000 to establish a Swiss Catalogue of Learning Objectives (SCLO) for undergraduate medical training, which was adapted from a similar Dutch blueprint. A second version of the SCLO was developed and launched in 2008. The catalogue is a prerequisite for the accreditation of the curricula of the six Swiss medical faculties and defines the contents of the Federal Licensing Examination (FLE). Given the evolution of the field of medicine and of medical education, the SMIFK/CIMS has decided to embark on a total revision of the SCLO. This article presents the proposed structure and content of Profiles, a new document which, in the future, will direct the format of undergraduate studies and of the FLE. Profiles stands for the Principal Relevant Objectives for Integrative Learning and Education in Switzerland. It is currently being developed by a group of experts from the six Swiss faculties as well as representatives of other institutions involved in these developments. The foundations of Profiles are grounded in the evolution of medical practice and of public health and are based on up-to-date teaching concepts, such as EPAs (entrustable professional activities). An introduction will cover the concepts and a tutorial will be displayed. Three main chapters will provide a description of the seven 2015 CanMEDS roles, a list of core EPAs and a series of ≈250 situations embracing the most frequent and current conditions affecting health. As Profiles is still a work in progress, it is hoped that this paper will attract the interest of all individuals involved in the training of medical students.

The population health performance of medical students at two medical schools

Public Health and medicine are complimentary disciplines dedicated to the health and well-being of humankind. Worldwide, medical school accreditation bodies require the inclusion of population health in medical education. In 2003, the Institutes of Medicine (IOM) recommended that all medical students receive basic public health training in population-based prevention. The purpose of this study was to (1) examine the public health clinical performance of third-year medical students at two independent medical schools, (2) compare the public health clinical practice performance of the schools, and (3) identify underlying predictors of high and low public health clinical performance at one of the medical schools. ^ This study is unique in its analysis and report of observed medical student public health clinical practices. The cohort consisted of 751 third-year medical students who completed a required clinical performance exam using trained standardized patients. Medical student performance scores on 24 consensus public health items derived from nine patient cases were analyzed.^ The analysis showed nearly identical results for both medical schools at the 60%, 65%, and 70% pass rate. Students performed poorly on items associated with prevention, behavioral science, and surveillance. Factors associated with high student performance included being from an underrepresented minority, matching to a primary care residency, and high class ranking. A review of medical school curriculum at both schools revealed a lack of training in four public health domains. Nationally, 32% of medical students reported inadequate training in public health in the year 2006.^ These findings suggest more dedicated teaching time for public health domains is needed at the medical schools represented in this study. Finally, more research is needed to assess attainment of public health knowledge and skills for medical students nationwide if we are to meet the recommendations of the IOM. ^