Quality Control for SP Performance Regarding Patient Portrayal in a High-Stakes OSCE

Introduction Since the quality of patient portrayal of standardized patients (SPs) during an Objective Structured Clinical Exam (OSCE) has a major impact on the reliability and validity of the exam, quality control should be initiated. Literature about quality control of SP’s performance focuses on feedback [1, 2] or completion of checklists [3, 4]. Since we did not find a published instrument meeting our needs for the assessment of patient portrayal, we developed such an instrument after being inspired by others [5] and used it in our high-stakes exam. Methods SP trainers from all five Swiss medical faculties collected and prioritized quality criteria for patient portrayal. Items were revised with the partners twice, based on experiences during OSCEs. The final instrument contains 14 criteria for acting (i.e. adequate verbal and non-verbal expression) and standardization (i.e. verbatim delivery of the first sentence). All partners used the instrument during a high-stakes OSCE. Both, SPs and trainers were introduced to the instrument. The tool was used in training (more than 100 observations) and during the exam (more than 250 observations). FAIR_OSCE The list of items to assess the quality of the simulation by SPs was primarily developed and used to provide formative feedback to the SPs in order to help them to improve their performance. It was therefore named “Feedbackstruckture for the Assessment of Interactive Role play in Objective Structured Clinical Exams (FAIR_OSCE). It was also used to assess the quality of patient portrayal during the exam. The results were calculated for each of the five faculties individually. Formative evaluation was given to the five faculties with individual feedback without revealing results of other faculties other than overall results. Results High quality of patient portrayal during the exam was documented. More than 90% of SP performances were rated to be completely correct or sufficient. An increase in quality of performance between training and exam was noted. In example the rate of completely correct reaction in medical tests increased from 88% to 95%. 95% completely correct reactions together with 4% sufficient reactions add up to 99% of the reactions meeting the requirements of the exam. SP educators using the instrument reported an augmentation of SPs performance induced by the use of the instrument. Disadvantages mentioned were high concentration needed to explicitly observe all criteria and cumbersome handling of the paper-based forms. Conclusion We were able to document a very high quality of SP performance in our exam. The data also indicate that our training is effective. We believe that the high concentration needed using the instrument is well invested, considering the observed augmentation of performance. The development of an iPad based application for the form is planned to address the cumbersome handling of the paper.

Sensitive Questions in Online Surveys: An Experimental Evaluation of the Randomized Response Technique and the Crosswise Model

Self-administered online surveys provide a higher level of privacy protection to respondents than surveys administered by an interviewer. Yet, studies indicate that asking sensitive questions is problematic also in self-administered surveys. Because respondents might not be willing to reveal the truth and provide answers that are subject to social desirability bias, the validity of prevalence estimates of sensitive behaviors from online surveys can be challenged. A well-known method to overcome these problems is the Randomized Response Technique (RRT). However, convincing evidence that the RRT provides more valid estimates than direct questioning in online surveys is still lacking. A new variant of the RRT called the Crosswise Model has recently been proposed to overcome some of the deficiencies of existing RRT designs. We therefore conducted an experimental study in which different implementations of the RRT, including two implementations of the crosswise model, were tested and compared to direct questioning. Our study is a large-scale online survey (N = 6,037) on sensitive behaviors by students such as cheating in exams and plagiarism. Results indicate that the crosswise-model RRT---unlike the other variants of RRT we evaluated---yields higher prevalence estimates of sensitive behaviors than direct questioning. Whether higher estimates are a sufficient condition for more valid results, however, remains questionable.

Rigor mortis at the myocardium investigated by post-mortem magnetic resonance imaging

INTRODUCTION Post-mortem cardiac MR exams present with different contraction appearances of the left ventricle in cardiac short axis images. It was hypothesized that the grade of post-mortem contraction may be related to the post-mortem interval (PMI) or cause of death and a phenomenon caused by internal rigor mortis that may give further insights in the circumstances of death. METHOD AND MATERIALS The cardiac contraction grade was investigated in 71 post-mortem cardiac MR exams (mean age at death 52y, range 12-89y; 48 males, 23 females). In cardiac short axis images the left ventricular lumen volume as well as the left ventricular myocardial volume were assessed by manual segmentation. The quotient of both (LVQ) represents the grade of myocardial contraction. LVQ was correlated to the PMI, sex, age, cardiac weight, body mass and height, cause of death and pericardial tamponade when present. In cardiac causes of death a separate correlation was investigated for acute myocardial infarction cases and arrhythmic deaths. RESULTS LVQ values ranged from 1.99 (maximum dilatation) to 42.91 (maximum contraction) with a mean of 15.13. LVQ decreased slightly with increasing PMI, however without significant correlation. Pericardial tamponade positively correlated with higher LVQ values. Variables such as sex, age, body mass and height, cardiac weight and cause of death did not correlate with LVQ values. There was no difference in LVQ values for myocardial infarction without tamponade and arrhythmic deaths. CONCLUSION Based on the observation in our investigated cases, the phenomenon of post-mortem myocardial contraction cannot be explained by the influence of the investigated variables, except for pericardial tamponade cases. Further research addressing post-mortem myocardial contraction has to focus on other, less obvious factors, which may influence the early post-mortem phase too.

Computer-based summative assessment of clinical reasoning in clerkships: A mixed-method comparison of key feature problems with case-based multiple choice questions

Background: It is yet unclear if there are differences between using electronic key feature problems (KFPs) or electronic case-based multiple choice questions (cbMCQ) for the assessment of clinical decision making. Summary of Work: Fifth year medical students were exposed to clerkships which ended with a summative exam. Assessment of knowledge per exam was done by 6-9 KFPs, 9-20 cbMCQ and 9-28 MC questions. Each KFP consisted of a case vignette and three key features (KF) using “long menu” as question format. We sought students’ perceptions of the KFPs and cbMCQs in focus groups (n of students=39). Furthermore statistical data of 11 exams (n of students=377) concerning the KFPs and (cb)MCQs were compared. Summary of Results: The analysis of the focus groups resulted in four themes reflecting students’ perceptions of KFPs and their comparison with (cb)MCQ: KFPs were perceived as (i) more realistic, (ii) more difficult, (iii) more motivating for the intense study of clinical reasoning than (cb)MCQ and (iv) showed an overall good acceptance when some preconditions are taken into account. The statistical analysis revealed that there was no difference in difficulty; however KFP showed a higher discrimination and reliability (G-coefficient) even when corrected for testing times. Correlation of the different exam parts was intermediate. Conclusions: Students perceived the KFPs as more motivating for the study of clinical reasoning. Statistically KFPs showed a higher discrimination and higher reliability than cbMCQs. Take-home messages: Including KFPs with long menu questions into summative clerkship exams seems to offer positive educational effects.

Improving the Assessment of Communication Skills at the Swiss Federal Licensing Exam: Results of a Student's Needs Assessment

Introduction With a three year project the assessment of communication skills within the Swiss Federal Licensing Examinations (FLE) shall be improved. As a first step a needs assessment among communication experts and medical students of the Swiss Medical Faculties will be performed. In this presentation the results of the students’ needs assessment will be presented. Methods A bilingual student’s online questionnaire will be developed by an expert panel taking relevant literature, the Swiss Catalogue of Learning Objectives and other consensus statements for communication (e.g., the European and Basler consensus statements) into account. With a think aloud study response process validity evidence will be sought. The questionnaire will focus on the following topics related to communication skills: (1) What has been taught?, (2) What has been assessed in the faculty exams?, (3) What has been assessed in the FLE?, (4) What should have been assessed in the FLE and how should the assessment be improved? Results Results of the students’ needs assessment will be available by the end of 2015 and be presented. Conclusions/ Take-home message We hope for valuable input for improving the assessment of communications skills within the FLE also from the students’ side. Results of the needs assessment from the students and experts will be combined and taken as input for an international expert symposium on how to improve the communication skills assessment within the FLE.

Prevalence and severity of dental fluorosis among students from João Pessoa, PB, Brazil

The aim of this study was to determine the prevalence and severity of dental fluorosis among 12-15-year-old students from João Pessoa, PB, Brazil before starting a program of artificial fluoridation of drinking water. The use of fluoridated dentifrice was also surveyed. A sample of 1,402 students was randomly selected. However, 31 students refused to participate and 257 were not permanent residents in João Pessoa, thus leaving a final sample of 1,114 students. Clinical exams were carried out by two calibrated dentists (Kappa = 0.78) under natural indirect light. Upper and lower front teeth were cleaned with gauze and dried, and then examined using the TF index for fluorosis. A questionnaire on dentifrice ingestion and oral hygiene habits was applied to the students. The results revealed that fluorosis prevalence in this age group was higher than expected (29.2%). Most fluorosis cases were TF = 1 (66.8%), and the most severe cases were TF = 4 (2.2%). The majority of the students reported that they had been using fluoridated dentifrices since childhood; 95% of the participants preferred brands with a 1,500 ppm F concentration, and 40% remembered that they usually ingested or still ingest dentifrice during brushing. It was concluded that dental fluorosis prevalence among students in João Pessoa is higher than expected for an area with non-fluoridated water. However, although most students use fluoridated dentifrices, and almost half ingest slurry while brushing, the majority of cases had little aesthetic relevance from the professionals' point of view, thus suggesting that fluorosis is not a public health problem in the locality.