Can Andean medicine coexist with biomedical healthcare? A comparison of two rural communities in Peru and Bolivia

Background It is commonly assumed that indigenous medical systems remain strong in developing countries because biomedicine is physically inaccessible or financially not affordable. This paper compares the health-seeking behavior of households from rural Andean communities at a Peruvian and a Bolivian study site. The main research question was whether the increased presence of biomedicine led to a displacement of Andean indigenous medical practices or to coexistence of the two healing traditions. Methodology Open-ended interviews and free listing exercises were conducted between June 2006 and December 2008 with 18 households at each study site. Qualitative identification of households’ therapeutic strategies and use of remedies was carried out by means of content analysis of interview transcriptions and inductive interference. Furthermore, a quantitative assessment of the incidence of culture-bound illnesses in local ethnobiological inventories was performed. Results Our findings indicate that the health-seeking behavior of the Andean households in this study is independent of the degree of availability of biomedical facilities in terms of quality of services provided, physical accessibility, and financial affordability, except for specific practices such as childbirth. Preference for natural remedies over pharmaceuticals coexists with biomedical healthcare that is both accessible and affordable. Furthermore, our results show that greater access to biomedicine does not lead to less prevalence of Andean indigenous medical knowledge, as represented by the levels of knowledge about culture-bound illnesses. Conclusions The take-home lesson for health policy-makers from this study is that the main obstacle to use of biomedicine in resource-poor rural areas might not be infrastructural or economic alone. Rather, it may lie in lack of sufficient recognition by biomedical practitioners of the value and importance of indigenous medical systems. We propose that the implementation of health care in indigenous communities be designed as a process of joint development of complementary knowledge and practices from indigenous and biomedical health traditions.

Further concerns about self-assessment: are medical students aware of their progress in clinical skills during clerkship

Background: Medical students do not accurately self-assess their competence. However, little is known about the awareness of change of competence over time. The aim of this study was to evaluate if students are aware of their progress. Summary of work: Twenty-two fourth year medical students had self- and expert-assessments of their clinical skills in musculoskeletal medicine in an OSCE like station (4 point Likert scale) at the beginning (t0) and end (t1) of their eight weeks clerkship in internal medicine. Thirteen students were assigned to the intervention of a 6x1 hour practical examination course; nine took part in the regular clinical clerkship activities only and served as controls. Summary of results/Conclusions: The intervention students significantly improved their skills (from 2.78 ± 0.36 to 3.30 ± 0.36, p<0.05) in contrast to the control students (from 3.11 ± 0.58 to 2.83 ± 0.49, n.s.). At t0, 19 students, at t1 21 out of 22 students underestimated their competence. Correlations between the change of self- and expert-assessment were r=0.43, p<0.05 (all), r=0.47, n.s. (control) and r=-0.12, n.s. (intervention), respectively. Take-home message: Medical students improving their clinical skills by an interactive course in addition to their regular clerkship activities are not aware of their progress

Self-assessment insufficiently predicts performance in emergency skills

Background: Residents demonstrate a broad range of performance levels for clinical skills, with some at an inadequate level. Adequate self-assessment is important for life long learning. However, its accuracy is questioned extensively. The aim of this study was to evaluate how far the residents’ self-assessment predicts their performance in an expert assessment of emergency skills. Summary of work: Twelve skills were identified as being relevant for the emergency duties of residents in smaller hospitals. Fifteen first-year residents from the departments of internal medicine and general surgery at a district hospital rated their performance on a questionnaire (self-assessment). This was followed by a structured, practical in vivo assessment by an anaesthesiologist (expert assessment). For both, a visual analogue scale from 0 to 10 was used, on which 0 stands for novice and 10 for expert. Predictive validity was described by Spearman’s correlation, which was significant in 3 out of 12 skills only. Median correlation (r) was 0.50 (range 0.16 to 0.93). Conclusion: At the beginning of postgraduate training, self-assessment alone is not sufficient to guide self-directed learning. Take-home message: At the beginning of their residency, physicians need structured feedback in emergency skills which can be offered by anaesthesiologists.

Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: The design of Virtual Patients (VPs) is essential. So far there are no validated evaluation instruments for VP design published. Summary of work: We examined three sources of validity evidence of an instrument to be filled out by students aimed at measuring the quality of VPs with a special emphasis on fostering clinical reasoning: (1) Content was examined based on theory of clinical reasoning and an international VP expert team. (2) Response process was explored in think aloud pilot studies with students and content analysis of free text questions accompanying each item of the instrument. (3) Internal structure was assessed by confirmatory factor analysis (CFA) using 2547 student evaluations and reliability was examined utilizing generalizability analysis. Summary of results: Content analysis was supported by theory underlying Gruppen and Frohna’s clinical reasoning model on which the instrument is based and an international VP expert team. The pilot study and analysis of free text comments supported the validity of the instrument. The CFA indicated that a three factor model comprising 6 items showed a good fit with the data. Alpha coefficients per factor were 0,74 - 0,82. The findings of the generalizability studies indicated that 40-200 student responses are needed in order to obtain reliable data on one VP. Conclusions: The described instrument has the potential to provide faculty with reliable and valid information about VP design. Take-home messages: We present a short instrument which can be of help in evaluating the design of VPs.

Alignment between learning needs and learning goals of Mini-CEX in clerkships

Background: Defining learning goals (LG) in alignment with learning needs (LN) is one of the key purposes of formative workplace-based assessment, but studies about this topic are scarce. Summary of Work: We analysed quantitatively and qualitatively how often trainer-student pairs identified the same LN during Mini Clinical Evaluation Exercises (Mini-CEX) in clerkships and to what degree those LNs were in line with the recorded LGs. Multilevel logistic regression models were used to predict LGs by identified LNs, controlling for context variables. Summary of Results: 512 trainers and 165 students conducted 1783 Mini-CEX (98% completion rate). Concordantly, trainer-student pairs most often identified LNs in the domains ‘clinical reasoning’ (23% of 1167 complete forms), ‘organisation / efficiency’ (20%) and ‘physical examination’ (20%). At least one ‘defined’ LG was noted on 313 student forms (18% of 1710), with a total of 446 LGs. Of these, the most frequent LGs were ‘physical examination’ (49% of 446 LGs) and ‘history taking’ (21%); corresponding LNs as well as context variables (e.g. clinic size) were found to be predictors of these LGs. Discussion and Conclusions: Although trainer-student pairs often agreed in their identified LNs, many assessments did not result in an aligned LG or a LG at all. Interventions are needed to enhance the proportion of (aligned) LGs in Mini-CEX in order to tap into its full potential not only as a ‘diagnostic’ but also as an ‘educational tool’. Take-home messages: The sparseness of LGs, their dependency on context variables and their partial non-alignment with students’ LNs raise the question of how the effectiveness of Mini-CEX can be further enhanced.

A Specific Mapping Study Using Fluorescence Sentinel Lymph Node Detection in Patients with Intermediate- and High-risk Prostate Cancer Undergoing Extended Pelvic Lymph Node Dissection.

Sentinel lymph node (SLN) detection techniques have the potential to change the standard of surgical care for patients with prostate cancer. We performed a lymphatic mapping study and determined the value of fluorescence SLN detection with indocyanine green (ICG) for the detection of lymph node metastases in intermediate- and high-risk patients undergoing radical prostatectomy and extended pelvic lymph node dissection. A total of 42 patients received systematic or specific ICG injections into the prostate base, the midportion, the apex, the left lobe, or the right lobe. We found (1) that external and internal iliac regions encompass the majority of SLNs, (2) that common iliac regions contain up to 22% of all SLNs, (3) that a prostatic lobe can drain into the contralateral group of pelvic lymph nodes, and (4) that the fossa of Marcille also receives significant drainage. Among the 12 patients who received systematic ICG injections, 5 (42%) had a total of 29 lymph node metastases. Of these, 16 nodes were ICG positive, yielding 55% sensitivity. The complex drainage pattern of the prostate and the low sensitivity of ICG for the detection of lymph node metastases reported in our study highlight the difficulties related to the implementation of SNL techniques in prostate cancer. PATIENT SUMMARY There is controversy about how extensive lymph node dissection (LND) should be during prostatectomy. We investigated the lymphatic drainage of the prostate and whether sentinel node fluorescence techniques would be useful to detect node metastases. We found that the drainage pattern is complex and that the sentinel node technique is not able to replace extended pelvic LND.

Enhancing Authenticity in Simulation Using Appropriate Techniques of Moulage

Workshop Overview The use of special effects (moulage) is a way to augment the authenticity of a scenario in simulation. This workshop will introduce different techniques of moulage (oil based cream colors, watercolors, transfer tattoos and 3D Prosthetics). The participants will have the opportunity to explore these techniques by applying various moulages. They will compare the techniques and discuss their advantages and disadvantages. Moreover, strategies for standardization and quality assurance will be discussed. Workshop Rationale Moulage supports the sensory perception in an scenario (1). It can provide evaluation clues (2) and help learners (and SPs) to engage in the simulation. However, it is of crucial importance that the simulated physical pathologies are represented accurate and reliable. Accuracy is achieved by using the appropriate technique, which requires knowledge and practice . With information about different moulage techniques, we hope to increases the knowledge of moulage during the workshop. By applying moulages in various techniques we will practice together. As standardization is critical for simulation scenarios in assessment (3, 4) strategies for standardization of moulage will be introduced and discussed. Workshop Objectives During the workshop participants will: - gain knowledge about different techniques of moulages - practice moulages in various techniques - discuss the advantages and disadvantages of moulage techniques - describe strategies for standardization and quality assurance of moulage Planned Format 5 min Introduction 15 min Overview – Background & Theory (presentation) 15 min Application of moulage for ankle sprain in 4 different techniques (oil based cream color, water color, temporary tatoo, 3D prosthetic) in small groups 5 min Comparing the results by interactive viewing of prepared moulages 15 min Application of moulages for burn in different techniques in small groups 5 min Comparing results the results by interactive viewing of prepared moulages 5 min Sharing experiences with different techniques in small groups 20 min Discussion of the techniques including standardization and quality assurance strategies (plenary discussion) 5 min Summary / Take home points

Do multiple true-false items beat the commonly used one-best-answers questions regarding to the Ottawa Criteria for Good Assessment? Results of a literature review

Background: Multiple True-False-Items (MTF-Items) might offer some advantages compared to one-best-answer-questions (TypeA) as they allow more than one correct answer and may better represent clinical decisions. However, in medical education assessment MTF-Items are seldom used. Summary of Work: With this literature review existing findings on MTF-items and on TypeA were compared along the Ottawa Criteria for Good Assessment, i.e. (1) reproducibility, (2) feasibility, (3) validity, (4) acceptance, (5) educational effect, (6) catalytic effects, and (7) equivalence. We conducted a literature research on ERIC and Google Scholar including papers from the years 1935 to 2014. We used the search terms “multiple true-false”, “true-false”, “true/false”, and “Kprim” combined with “exam”, “test”, and “assessment”. Summary of Results: We included 29 out of 33 studies. Four of them were carried out in the medical field Compared to TypeA, MTF-Items are associated with (1) higher reproducibility (2) lower feasibility (3) similar validity (4) higher acceptance (5) higher educational effect (6) no studies on catalytic effects or (7) equivalence. Discussion and Conclusions: While studies show overall good characteristics of MTF items according to the Ottawa criteria, this type of question seems to be rather seldom used. One reason might be the reported lower feasibility. Overall the literature base is still weak. Furthermore, only 14 % of literature is from the medical domain. Further studies to better understand the characteristics of MTF-Items in the medical domain are warranted. Take-home messages: Overall the literature base is weak and therefore further studies are needed. Existing studies show that: MTF-Items show higher reliability, acceptance and educational effect; MTF-Items are more difficult to produce

Comparability of Outcome Frameworks in Medical Education: Implications for framework development

Background: The demand for international harmonization in medical education increases with the growing mobility of students and health professionals. Many medical societies and governmental offices have issued outcome frameworks (OF), which describe aims and contents of medical education based on competencies. These national standards affect the development of curricula as well as assessment and licensing procedures. Comparing OF and identifying factors that limit their comparability may thus foster international harmonization of medical education. Summary of Work: We conducted a systematic search for national OF in MedLine, EmBase and the internet. We included all OF in German or English that resulted from a national consensus process and were published or endorsed by a national society or governmental body. We extracted information in five predetermined categories: history of origin, audience, formal structure, medical schooling system and key terms. Summary of Results: Out of 1816 results, 13 OF were included into further analyses. OF reference each other, often without addressing existing differences (e.g. in target audiences). The two most cited OF are “CanMEDs” and “Scottish Doctor”. OF differ especially in their level of detail as well as in the underlying educational system. Discussion and Conclusions: Based on our results we propose a two-step blueprint for OF, that may help to establish comparability for internationally aligned key features – so-called “core competencies” – while at the same time allowing for necessary regional adaptations in terms of “secondary competencies”. Take-home messages: Considerable differences in at least five categories of OF currently hinder the comparability of outcome frameworks.

Computer-based summative assessment of clinical reasoning in clerkships: A mixed-method comparison of key feature problems with case-based multiple choice questions

Background: It is yet unclear if there are differences between using electronic key feature problems (KFPs) or electronic case-based multiple choice questions (cbMCQ) for the assessment of clinical decision making. Summary of Work: Fifth year medical students were exposed to clerkships which ended with a summative exam. Assessment of knowledge per exam was done by 6-9 KFPs, 9-20 cbMCQ and 9-28 MC questions. Each KFP consisted of a case vignette and three key features (KF) using “long menu” as question format. We sought students’ perceptions of the KFPs and cbMCQs in focus groups (n of students=39). Furthermore statistical data of 11 exams (n of students=377) concerning the KFPs and (cb)MCQs were compared. Summary of Results: The analysis of the focus groups resulted in four themes reflecting students’ perceptions of KFPs and their comparison with (cb)MCQ: KFPs were perceived as (i) more realistic, (ii) more difficult, (iii) more motivating for the intense study of clinical reasoning than (cb)MCQ and (iv) showed an overall good acceptance when some preconditions are taken into account. The statistical analysis revealed that there was no difference in difficulty; however KFP showed a higher discrimination and reliability (G-coefficient) even when corrected for testing times. Correlation of the different exam parts was intermediate. Conclusions: Students perceived the KFPs as more motivating for the study of clinical reasoning. Statistically KFPs showed a higher discrimination and higher reliability than cbMCQs. Take-home messages: Including KFPs with long menu questions into summative clerkship exams seems to offer positive educational effects.

Improving residency training with the help of multisource feedback: Generation of a model of influencing factors

Background: Feedback is considered to be one of the most important drivers of learning. One form of structured feedback used in medical settings is multisource feedback (MSF). This feedback technique provides the opportunity to gain a differentiated view on a doctor’s performance from several perspectives using a questionnaire and a facilitating conversation, in which learning goals are formulated. While many studies have been conducted on the validity, reliability and feasibility of the instrument, little is known about the impact of factors that might influence the effects of MSF on clinical performance. Summary of Work: To study under which circumstances MSF is most effective, we performed a literature review on Google Scholar with focus on MSF and feedback in general. Main key-words were: MSF, multi-source-feedback, multi source feedback, and feedback each combined with influencing/ hindering/ facilitating factors, effective, effectiveness, doctors-intraining, and surgery. Summary of Results: Based on the literature, we developed a preliminary model of facilitating factors. This model includes five main factors influencing MSF: questionnaire, doctor-in-training, group of raters, facilitating supervisor, and facilitating conversation. Discussion and Conclusions: Especially the following points that might influence MSF have not yet been sufficiently studied: facilitating conversation with the supervisor, individual aspects of doctors-in-training, and the causal relations between influencing factors. Overall there are only very few studies focusing on the impact of MSF on actual and long-term performance. We developed a preliminary model of hindering and facilitating factors on MSF. Further studies are needed to better understand under which circumstances MSF is most effective. Take-home messages: The preliminary model might help to guide further studies on how to implement MSF to use it at its full potential.

Improving the Assessment of Communication Skills at the Swiss Federal Licensing Exam: Results of a Student's Needs Assessment

Introduction With a three year project the assessment of communication skills within the Swiss Federal Licensing Examinations (FLE) shall be improved. As a first step a needs assessment among communication experts and medical students of the Swiss Medical Faculties will be performed. In this presentation the results of the students’ needs assessment will be presented. Methods A bilingual student’s online questionnaire will be developed by an expert panel taking relevant literature, the Swiss Catalogue of Learning Objectives and other consensus statements for communication (e.g., the European and Basler consensus statements) into account. With a think aloud study response process validity evidence will be sought. The questionnaire will focus on the following topics related to communication skills: (1) What has been taught?, (2) What has been assessed in the faculty exams?, (3) What has been assessed in the FLE?, (4) What should have been assessed in the FLE and how should the assessment be improved? Results Results of the students’ needs assessment will be available by the end of 2015 and be presented. Conclusions/ Take-home message We hope for valuable input for improving the assessment of communications skills within the FLE also from the students’ side. Results of the needs assessment from the students and experts will be combined and taken as input for an international expert symposium on how to improve the communication skills assessment within the FLE.