Further concerns about self-assessment: are medical students aware of their progress in clinical skills during clerkship

Background: Medical students do not accurately self-assess their competence. However, little is known about the awareness of change of competence over time. The aim of this study was to evaluate if students are aware of their progress. Summary of work: Twenty-two fourth year medical students had self- and expert-assessments of their clinical skills in musculoskeletal medicine in an OSCE like station (4 point Likert scale) at the beginning (t0) and end (t1) of their eight weeks clerkship in internal medicine. Thirteen students were assigned to the intervention of a 6x1 hour practical examination course; nine took part in the regular clinical clerkship activities only and served as controls. Summary of results/Conclusions: The intervention students significantly improved their skills (from 2.78 ± 0.36 to 3.30 ± 0.36, p<0.05) in contrast to the control students (from 3.11 ± 0.58 to 2.83 ± 0.49, n.s.). At t0, 19 students, at t1 21 out of 22 students underestimated their competence. Correlations between the change of self- and expert-assessment were r=0.43, p<0.05 (all), r=0.47, n.s. (control) and r=-0.12, n.s. (intervention), respectively. Take-home message: Medical students improving their clinical skills by an interactive course in addition to their regular clerkship activities are not aware of their progress

Effects of geriatric clinical skills training on the attitudes of medical students.

BACKGROUND Physicians' attitudes, knowledge and skills are powerful determinants of quality of care for older patients. Previous studies found that using educational interventions to improve attitude is a difficult task. No previous study sought to determine if a skills-oriented educational intervention improved student attitudes towards elderly patients. METHODS This study evaluated the effect of a geriatric clinical skills training (CST) on attitudes of University of Bern medical students in their first year of clinical training. The geriatric CST consisted of four 2.5-hour teaching sessions that covered central domains of geriatric assessment (e.g., cognition, mobility), and a textbook used by students to self-prepare. Students' attitudes were the primary outcome, and were assessed with the 14-item University of California at Los Angeles Geriatrics Attitudes Scale (UCLA-GAS) in a quasi-randomized fashion, either before or after geriatric CST. RESULTS A total of 154 medical students participated. Students evaluated before the CST had a median UCLA-GAS overall scale of 49 (interquartile range 44-53). After the CST, the scores increased slightly, to 51 (interquartile range 47-54; median difference 2, 95% confidence interval 0-4, P = 0.062). Of the four validated UCLA-GAS subscales, only the resource distribution subscale was significantly higher in students evaluated after the geriatric CST (median difference 1, 95% confidence interval 0-2, P = 0.005). CONCLUSIONS Teaching that targets specific skills may improve the attitudes of medical students towards elderly patients, though the improvement was slight. The addition of attitude-building elements may improve the effectiveness of future skills-oriented educational interventions.

The new final Clinical Skills examination in human medicine in Switzerland: Essential steps of exam development, implementation and evaluation, and central insights from the perspective of the national Working Group

Objective: Since 2011, the new national final examination in human medicine has been implemented in Switzerland, with a structured clinical-practical part in the OSCE format. From the perspective of the national Working Group, the current article describes the essential steps in the development, implementation and evaluation of the Federal Licensing Examination Clinical Skills (FLE CS) as well as the applied quality assurance measures. Finally, central insights gained from the last years are presented. Methods: Based on the principles of action research, the FLE CS is in a constant state of further development. On the foundation of systematically documented experiences from previous years, in the Working Group, unresolved questions are discussed and resulting solution approaches are substantiated (planning), implemented in the examination (implementation) and subsequently evaluated (reflection). The presented results are the product of this iterative procedure. Results: The FLE CS is created by experts from all faculties and subject areas in a multistage process. The examination is administered in German and French on a decentralised basis and consists of twelve interdisciplinary stations per candidate. As important quality assurance measures, the national Review Board (content validation) and the meetings of the standardised patient trainers (standardisation) have proven worthwhile. The statistical analyses show good measurement reliability and support the construct validity of the examination. Among the central insights of the past years, it has been established that the consistent implementation of the principles of action research contributes to the successful further development of the examination. Conclusion: The centrally coordinated, collaborative-iterative process, incorporating experts from all faculties, makes a fundamental contribution to the quality of the FLE CS. The processes and insights presented here can be useful for others planning a similar undertaking. Keywords: national final examination, licensing examination, summative assessment, OSCE, action research

The influence of students' prior clinical skills and context characteristics on mini-CEX scores in clerkships - a multilevel analysis.

BACKGROUND In contrast to objective structured clinical examinations (OSCEs), mini-clinical evaluation exercises (mini-CEXs) take place at the clinical workplace. As both mini-CEXs and OSCEs assess clinical skills, but within different contexts, this study aims at analyzing to which degree students' mini-CEX scores can be predicted by their recent OSCE scores and/or context characteristics. METHODS Medical students participated in an end of Year 3 OSCE and in 11 mini-CEXs during 5 different clerkships of Year 4. The students' mean scores of 9 clinical skills OSCE stations and mean 'overall' and 'domain' mini-CEX scores, averaged over all mini-CEXs of each student were computed. Linear regression analyses including random effects were used to predict mini-CEX scores by OSCE performance and characteristics of clinics, trainers, students and assessments. RESULTS A total of 512 trainers in 45 clinics provided 1783 mini-CEX ratings for 165 students; OSCE results were available for 144 students (87 %). Most influential for the prediction of 'overall' mini-CEX scores was the trainers' clinical position with a regression coefficient of 0.55 (95 %-CI: 0.26-0.84; p < .001) for residents compared to heads of department. Highly complex tasks and assessments taking place in large clinics significantly enhanced 'overall' mini-CEX scores, too. In contrast, high OSCE performance did not significantly increase 'overall' mini-CEX scores. CONCLUSION In our study, Mini-CEX scores depended rather on context characteristics than on students' clinical skills as demonstrated in an OSCE. Ways are discussed which focus on either to enhance the scores' validity or to use narrative comments only.

Sustained impact of a short small group course with systematic feedback in addition to regular clinical clerkship activities on musculoskeletal examination skills-a controlled study.

BACKGROUND The discrepancy between the extensive impact of musculoskeletal complaints and the common deficiencies in musculoskeletal examination skills lead to increased emphasis on structured teaching and assessment. However, studies of single interventions are scarce and little is known about the time-dependent effect of assisted learning in addition to a standard curriculum. We therefore evaluated the immediate and long-term impact of a small group course on musculoskeletal examination skills. METHODS All 48 Year 4 medical students of a 6 year curriculum, attending their 8 week clerkship of internal medicine at one University department in Berne, participated in this controlled study. Twenty-seven students were assigned to the intervention of a 6×1 h practical course (4-7 students, interactive hands-on examination of real patients; systematic, detailed feedback to each student by teacher, peers and patients). Twenty-one students took part in the regular clerkship activities only and served as controls. In all students clinical skills (CS, 9 items) were assessed in an Objective Structured Clinical Examination (OSCE) station, including specific musculoskeletal examination skills (MSES, 7 items) and interpersonal skills (IPS, 2 items). Two raters assessed the skills on a 4-point Likert scale at the beginning (T0), the end (T1) and 4-12 months after (T2) the clerkship. Statistical analyses included Friedman test, Wilcoxon rank sum test and Mann-Whitney U test. RESULTS At T0 there were no significant differences between the intervention and control group. At T1 and T2 the control group showed no significant changes of CS, MSES and IPS compared to T0. In contrast, the intervention group significantly improved CS, MSES and IPS at T1 (p < 0.001). This enhancement was sustained for CS and MSES (p < 0.05), but not for IPS at T2. CONCLUSIONS Year 4 medical students were incapable of improving their musculoskeletal examination skills during regular clinical clerkship activities. However, an additional small group, interactive clinical skills course with feedback from various sources, improved these essential examination skills immediately after the teaching and several months later. We conclude that supplementary specific teaching activities are needed. Even a single, short-lasting targeted module can have a long lasting effect and is worth the additional effort.

Self-assessment insufficiently predicts performance in emergency skills

Background: Residents demonstrate a broad range of performance levels for clinical skills, with some at an inadequate level. Adequate self-assessment is important for life long learning. However, its accuracy is questioned extensively. The aim of this study was to evaluate how far the residents’ self-assessment predicts their performance in an expert assessment of emergency skills. Summary of work: Twelve skills were identified as being relevant for the emergency duties of residents in smaller hospitals. Fifteen first-year residents from the departments of internal medicine and general surgery at a district hospital rated their performance on a questionnaire (self-assessment). This was followed by a structured, practical in vivo assessment by an anaesthesiologist (expert assessment). For both, a visual analogue scale from 0 to 10 was used, on which 0 stands for novice and 10 for expert. Predictive validity was described by Spearman’s correlation, which was significant in 3 out of 12 skills only. Median correlation (r) was 0.50 (range 0.16 to 0.93). Conclusion: At the beginning of postgraduate training, self-assessment alone is not sufficient to guide self-directed learning. Take-home message: At the beginning of their residency, physicians need structured feedback in emergency skills which can be offered by anaesthesiologists.

The new licencing examination for human medicine: from concept to implementation

A new Swiss federal licencing examination for human medicine (FLE) was developed and released in 2011. This paper describes the process from concept design to the first results obtained on implementation of the new examination. The development process was based on the Federal Act on University Medical Professions and involved all national stakeholders in this venture. During this process questions relating to the assessment aims, the assessment formats, the assessment dimensions, the examination content and necessary trade-offs were clarified. The aims were to create a feasible, fair, valid and psychometrically sound examination in accordance with international standards, thereby indicating the expected knowledge and skills level at the end of undergraduate medical education. Finally, a centrally managed and locally administered examination comprising a written multiple-choice element and a practical “clinical skills” test in the objective structured clinical examination (OSCE) format was developed. The first two administrations of the new FLE show that the examination concept could be implemented as intended. The anticipated psychometric indices were achieved and the results support the validity of the examination. Possible changes to the format or content in the future are discussed.

Stärken, Einflüsse und Weiterentwicklungsbedarf der Eidgenössischen Prüfung Humanmedizin zwei Jahre nach deren Implementierung: Eine Fokusgruppenstudie unter involvierten Experten und bildungspolitischen Entscheidungsträgern

Fragestellung/Einleitung: Die Eidgenössische Prüfung Humanmedizin (EP) wurde zwischenzeitlich dreimal erfolgreich durchgeführt. Daten zu Stärken, Schwächen und dem Weiterentwicklungsbedarf lagen bisher nur spärlich vor. Deshalb sollten diese anhand einer qualitativen Studie unter den involvierten Experten und bildungspolitischen Entscheidungsträgern erhoben werden. Methoden: Vier Fokus-Gruppen mit insgesamt 25 Teilnehmern wurden entsprechend internationaler Standards durchgeführt, um die Einschätzungen involvierter Experten und bildungspolitischer Entscheidungsträger bzgl. den erfahrenen Stärken, Einflüssen und dem Weiterentwicklungsbedarf der EP zu erhalten. Die Fokusgruppendiskussionen wurden wörtlich transkribiert und anhand von Inhaltsanalyse ausgewertet. Ergebnisse: Erfahrene Stärken waren v.a. die Kombination der beiden Prüfungs-Teile „Multiple Choice“ (MC) und „Clinical Skills“ (CS), die formatspezifischen Stärken der MC- und CS-Prüfung und die kollaborative Herangehensweise. Erfahrene Einflüsse der EP waren v.a. auf das studentische Lernverhalten, die Prüfer, den Lehrkörper, die Reform der Curricula, die Zusammenarbeit der Fakultäten und die erfahrene Wichtigkeit des Schweizer Lernzielkatalogs (SCLO). Bedarf zur Weiterentwicklung wurde v.a. in Folgendem gesehen: Dass Modifikationen nur angegangen werden, wenn diese wohlüberlegt und evidenzbasiert sind, in einer verbesserten Authentizität der CS-Prüfung, in weiteren Examensformaten, in einer verbesserten Kommunikationsstrategie, in der weiteren Revision des SCLO, in der Anerkennung der Limitationen eines „Single Shot Examens“ und im Aufbau einer Incentives-Struktur für die Kliniker, die aktiv die EP mitgestalten. Diskussion/Schlussfolgerung: Insgesamt wird die EP als geeignet für ihre Aufgaben angesehen. Diese Prüfung hat Einflüsse auf die Medizinstudierendenausbildung in der Schweiz auch über die direkten summativen Prüfungsaspekte hinaus. Es wurde ein Bedarf zur Weiterentwicklung gesehen, jedoch sollten die Veränderungen wohl begründet sein.

Communication about standard treatment options and clinical trials: can we teach doctors new skills to improve patient outcomes?

BACKGROUND: The International Breast Cancer Study Group conducted a phase III trial in Australian/New Zealand (ANZ) and Swiss/German/Austrian (SGA) centres on training doctors in clear and ethical information delivery about treatment options and strategies to encourage shared decision making. METHODS: Medical, surgical, gynaecological and radiation oncologists, and their patients for whom adjuvant breast cancer therapy was indicated, were eligible. Doctors were randomised to participate in a workshop with standardised teaching material and role playing. Patients were recruited in the experimental and control groups before and after the workshop. RESULTS: In ANZ centres, 21 eligible doctors recruited a total of 304 assessable patients. In SGA centres, 41 doctors recruited 390 patients. The training was well accepted. There was no overall effect on patient decisional conflict (primary endpoint) 2 weeks after the consultation. Overall, patients were satisfied with their treatment decision, their consultation and their doctors' consultation skills. Considerable variation was observed in patient outcomes between SGA and ANZ centres; the effect sizes of the intervention were marginal (<0.2). CONCLUSIONS: Shared decision making remains a challenge. A sustained training effect may require more intensive training tailored to the local setting. Cross-cultural differences need attention in conducting trials on communication interventions.

Maintenance of skills, competencies, and performance in allergy and clinical immunology: time to lay the foundation for a universal approach

Allergist/clinical immunologist maintenance of certification and training program reaccreditation are mandatory in some countries. The World Allergy Organization conducted surveys in 2009 and 2011 to assess where such programs were available and to promote the establishment of such programs on a global level. This was done with the presumption that after such an "inventory," World Allergy Organization could offer guidance to its Member Societies on the promotion of such programs to assure the highest standards of practice in the field of allergy and clinical immunology. This review draws on the experience of countries where successful programs are in place and makes recommendations for those wishing to implement such programs for the specialty.

Psychoendocrine validation of a short measure for assessment of perceived stress management skills in different non-clinical populations

BACKGROUND: We investigated the psychometric properties of a short questionnaire for combined assessment of different perceived stress management skills in the general population and tested whether scores relate to physiological stress reactivity. METHODS: For psychometric evaluation, we determined the factor structure of the questionnaire and investigated its measurement invariance in the participant groups and over time in three different independent samples representing the general population (total N=332). Reliability was tested by estimating test-retest reliability, internal consistency, and item reliabilities. We examined convergent and criterion validity using selected criterion variables. For endocrine validation, 35 healthy non-smoking and medication-free men in a laboratory study and 35 male and female employees in a workplace study underwent an acute standardized psychosocial stress task. We assessed stress management skills and measured salivary cortisol before and several times up to 60 min (workplace study) and 120 min (laboratory study) after stress. Potential confounders were controlled. RESULTS: The factor structure of the questionnaire consists of five scales reflecting acceptably distinct stress management skills such as cognitive strategies, use of social support, relaxation strategies, anger regulation, and perception of bodily tension. This factor structure was stable across participant groups and over time. Internal consistencies, item reliabilities, and test-retest reliabilities met established statistical requirements. Convergent and criterion validity were also established. In both endocrine validation studies, higher stress management skills were independently associated with lower cortisol stress reactivity (p's<.029). CONCLUSIONS: Our findings suggest that the questionnaire has good psychometric properties and that it relates to subjective psychological and objective physiological stress indicators. Therefore, the instrument seems a suitable measure for differential assessment of stress management skills in the general population.