FAIR_OSCE – Feedback structure for assessment of interactive roleplay in Objective Structured Clinical Exams

Introduction Since the quality of patient portrayal of standardized patients (SPs) during an Objective Structured Clinical Exam (OSCE) has a major impact on the reliability and validity of the exam, quality control should be initiated. Literature about quality control of SPs’ performance focuses on feedback [1, 2] or completion of checklists [3, 4]. Since we did not find a published instrument meeting our needs for the assessment of patient portrayal, we developed such an instrument after being inspired by others [5] and used it in our high-stakes exam. Project description SP trainers from five medical faculties collected and prioritized quality criteria for patient portrayal. Items were revised twice, based on experiences during OSCEs. The final instrument contains 14 criteria for acting (i.e. adequate verbal and non-verbal expression) and standardization (i.e. verbatim delivery of the first sentence). All partners used the instrument during a high-stakes OSCE. SPs and trainers were introduced to the instrument. The tool was used in training (more than 100 observations) and during the exam (more than 250 observations). Outcome High quality of SPs’ patient portrayal during the exam was documented. More than 90% of SP performances were rated to be completely correct or sufficient. An increase in quality of performance between training and exam was noted. For example, the rate of completely correct reaction in medical tests increased from 88% to 95%. Together with 4% of sufficient performances these 95% add up to 99% of the reactions in medical tests meeting the standards of the exam. SP educators using the instrument reported an augmentation of SPs’ performance induced by the use of the instrument. Disadvantages mentioned were the high concentration needed to observe all criteria and the cumbersome handling of the paper-based forms. Discussion We were able to document a very high quality of SP performance in our exam. The data also indicates that our training is effective. We believe that the high concentration needed using the instrument is well invested, considering the observed enhancement of performance. The development of an iPad-based application for the form is planned to address the cumbersome handling of the paper.

CD62L (L-selectin) shedding for assessment of functional blockade of TNF alpha in anti-TNF treated IBD patients: clinical feasibility and perspectives (DOP060)

Background Tumor necrosis factor (TNF) inhibition is central to the therapy of inflammatory bowel diseases (IBD). However, loss of response (LOR) is frequent and additional tests to help decision making with costly anti-TNF Therapy are needed. Methods Consecutive IBD Patients receiving anti-TNF therapy (Infliximab (IFX) or Adalimumab (after IFX LOR) from Bern University Hospital were identified and followed prospectively. Patient whole blood was stimulated with a dose-titration of two triggers of TLR receptors human: TNF and LPS. Median fluorescence intensity of CD62L on the surface of granulocytes was quantified by surface staining with specific antibodies (CD33, CD62L) and flow cytometry and logistic curves to these data permits the calculation of EC50 or the half maximal effective concentration TNF concentration to induce shedding [1]. A shift in the concentration were CD62L shedding occurred was seen before and after the anti-TNF agent administraion which permits to predict the response to the drug. This predicted response was correlated to the clinical evolution of the patients in order to analyze the ability of this test to identify LOR to IFX. Results We collected prospective clinical data and blood samples, before and after anti-TNF agent administration, on 33 IBD patients, 25 Crohn's disease and 8 ulcerative colitis patients (45% females) between June 2012 and November 2013. The assay showed a functional blockade of IFX (PFR) for 22 patients (17 CD and 5 UC) whereas 11 (8 CD and 3 UC) had no functional response (NR) to IFX. Clinical characteristics (e.g. diagnosis, disease location, smoking status, BMI and number of infusions) were no significantly different between predicted PFR and NR. Among the 22 Patients with PRF, only 1 patient was a clinical non responder (LOR to IFX), based on clinical prospective evaluation by IBD gastroenterologists (PJ, AM), and among the 11 predicted NR, 3 had no clinical LOR. Sensitivity of this test was 95% and specificity 73% and AUC adjusted for age and gender was 0.81 (Figure 1). During follow up (median 10 mo, 3–15) 8 “hard” outcomes occured (3 medic. flares, 4 resections and 1 new fistula) 2 in the PFR and 6 in the NR group (25% vs. 75%; p < 0.01). Correlation with clinical response is presented in Figure 2. Figure 1. Figure 2. Correlation clinical response - log EC50 changes: 1 No, 2 partial, 3 complete clinical response. Conclusion CD62L (L-Selectin) shedding is the first validated test of functional blockade of TNF alpha in anti-TNF treated IBD patients and will be a useful tool to guide medical decision on the use of anti-TNF agents. Comparative studies with ATI and trough level of IFX are ongoing. 1. Nicola Patuto, Emma Slack, Frank Seibold and Andrew J. Macpherson, (2011), Quantitating Anti-TNF Functionality to Inform Dosing and Choice of Therapy, Gastroenterology, 140 (5, Suppl. I), S689.

New Clinical Indications for (18)F/(11)C-choline, New Tracers for Positron Emission Tomography and a Promising Hybrid Device for Prostate Cancer Staging: A Systematic Review of the Literature

CONTEXT Radiolabelled choline positron emission tomography has changed the management of prostate cancer patients. However, new emerging radiopharmaceutical agents, like radiolabelled prostate specific membrane antigen, and new promising hybrid imaging will begin new challenges in the diagnostic field. OBJECTIVE The continuous evolution in nuclear medicine has led to the improvement in the detection of recurrent prostate cancer (PCa), particularly distant metastases. New horizons have been opened for radiolabelled choline positron emission tomography (PET)/computed tomography (CT) as a guide for salvage therapy or for the assessment of systemic therapies. In addition, new tracers and imaging tools have been recently tested, providing important information for the management of PCa patients. Herein we discuss: (1) the available evidence in literature on radiolabelled choline PET and their recent indications, (2) the role of alternative radiopharmaceutical agents, and (3) the advantages of a recent hybrid imaging device (PET/magnetic resonance imaging) in PCa. EVIDENCE ACQUISITION Data from recently published (2010-2015), original articles concerning the role of choline PET/CT, new emerging radiotracers, and a new imaging device are analysed. This review is reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. EVIDENCE SYNTHESIS In the restaging phase, the detection rate of choline PET varies between 4% and 97%, mainly depending on the site of recurrence and prostate-specific antigen levels. Both 68gallium (68Ga)-prostate specific membrane antigen and 18F-fluciclovine are shown to be more accurate in the detection of recurrent disease as compared with radiolabelled choline PET/CT. Particularly, Ga68-PSMA has a detection rate of 50% and 68%, respectively for prostate-specific antigen levels < 0.5ng/ml and 0.5-2ng/ml. Moreover, 68Ga- PSMA PET/magnetic resonance imaging demonstrated a particularly higher accuracy in detecting PCa than PET/CT. New tracers, such as radiolabelled bombesin or urokinase-type plasminogen activator receptor, are promising, but few data in clinical practice are available today. CONCLUSIONS Some limitations emerge from the published papers, both for radiolabelled choline PET/CT and also for new radiopharmaceutical agents. Efforts are still needed to enhance the impact of published data in the world of oncology, in particular when new radiopharmaceuticals are introduced into the clinical arena. PATIENT SUMMARY In the present review, the authors summarise the last evidences in clinical practice for the assessment of prostate cancer, by using nuclear medicine modalities, like positron emission tomography/computed tomography and positron emission tomography/magnetic resonance imaging.

Computer-based summative assessment of clinical reasoning in clerkships: A mixed-method comparison of key feature problems with case-based multiple choice questions

Background: It is yet unclear if there are differences between using electronic key feature problems (KFPs) or electronic case-based multiple choice questions (cbMCQ) for the assessment of clinical decision making. Summary of Work: Fifth year medical students were exposed to clerkships which ended with a summative exam. Assessment of knowledge per exam was done by 6-9 KFPs, 9-20 cbMCQ and 9-28 MC questions. Each KFP consisted of a case vignette and three key features (KF) using “long menu” as question format. We sought students’ perceptions of the KFPs and cbMCQs in focus groups (n of students=39). Furthermore statistical data of 11 exams (n of students=377) concerning the KFPs and (cb)MCQs were compared. Summary of Results: The analysis of the focus groups resulted in four themes reflecting students’ perceptions of KFPs and their comparison with (cb)MCQ: KFPs were perceived as (i) more realistic, (ii) more difficult, (iii) more motivating for the intense study of clinical reasoning than (cb)MCQ and (iv) showed an overall good acceptance when some preconditions are taken into account. The statistical analysis revealed that there was no difference in difficulty; however KFP showed a higher discrimination and reliability (G-coefficient) even when corrected for testing times. Correlation of the different exam parts was intermediate. Conclusions: Students perceived the KFPs as more motivating for the study of clinical reasoning. Statistically KFPs showed a higher discrimination and higher reliability than cbMCQs. Take-home messages: Including KFPs with long menu questions into summative clerkship exams seems to offer positive educational effects.

A STUDY OF THE RELATIONSHIP BETWEEN SELF-ASSESSMENT AND CLINICAL EVALUATION OF HEARING STATUS

The relationship was explored between a subjective measure of hearing status, derived from a functional self-assessment expressed in terms of ability to hear and understand spoken words, and a comparable objective measure of hearing status, obtained from a speech reception test. The Augmentation Survey of the Health and Nutrition Examination Survey of the National Center for Health Statistics provided the necessary data for a sample of 3059 adults. Using chi-square tests for the subsample with the highest level of objectively assessed hearing status, favorable subjective assessments were found to be significantly associated with higher income, lower age group, higher level of educational attainment, greater psychological adjustment, fewer symptoms of depression, and higher self-ratings of overall health. In a linear regression with self-assessment of hearing status as the dependent variable, less than one-quarter of the variation could be explained by objective status and the six explanatory variables.^

Methods and Tools for Objective Assessment of Psychomotor Skills in Laparoscopic Surgery

Training and assessment paradigms for laparoscopic surgical skills are evolving from traditional mentor–trainee tutorship towards structured, more objective and safer programs. Accreditation of surgeons requires reaching a consensus on metrics and tasks used to assess surgeons’ psychomotor skills. Ongoing development of tracking systems and software solutions has allowed for the expansion of novel training and assessment means in laparoscopy. The current challenge is to adapt and include these systems within training programs, and to exploit their possibilities for evaluation purposes. This paper describes the state of the art in research on measuring and assessing psychomotor laparoscopic skills. It gives an overview on tracking systems as well as on metrics and advanced statistical and machine learning techniques employed for evaluation purposes. The later ones have a potential to be used as an aid in deciding on the surgical competence level, which is an important aspect when accreditation of the surgeons in particular, and patient safety in general, are considered. The prospective of these methods and tools make them complementary means for surgical assessment of motor skills, especially in the early stages of training. Successful examples such as the Fundamentals of Laparoscopic Surgery should help drive a paradigm change to structured curricula based on objective parameters. These may improve the accreditation of new surgeons, as well as optimize their already overloaded training schedules.

Tools in Spanish Universities to facilitate the transmission from High School to University and their assessment.

Incoming students in the University have education deficiencies, so universities studies require a sound basis of scientific knowledge. In this project are analysed instruments to reinforcing knowledge in those areas related to the studies that students are about to embark on public Spanish universities. There are important differences among universities and, in each university there are great differences among titles. Initial courses (cursos cero) are widespread (in 50% of universities) that selfevaluation instruments (14 % of universities). It is necessary to improve diffusion of those instruments because it is not possible to evaluate them. So are proposed the next actuations: to make regular standard surveys for professors and students; to publish results of surveys; public universities should institutionalize their basic training offer and improve the dissemination of this offer especially through the web. This paper presents a questionnaire to assess student opinion about these tools. To analyze the effectiveness, and make an initial estimate of the evaluation of these tools, we conducted a pilot test of the questionnaire with 68 students at the University of Extremadura. The results of preliminary statistical analysis conducted on the pilot test indicate that the survey results are reliable. A global evaluation of both tools, with a scale of 1 to 5, gave an average score of 3.29 for initial courses and 3.41 for selfevaluation. The 72.9% of the students consider the "self assessment" more effective than the "initial course"

Identifying the semantics of eligibility criteria of clinical trials based on relevant medical ontologies

An important objective of the INTEGRATE project1 is to build tools that support the efficient execution of post-genomic multi-centric clinical trials in breast cancer, which includes the automatic assessment of the eligibility of patients for available trials. The population suited to be enrolled in a trial is described by a set of free-text eligibility criteria that are both syntactically and semantically complex. At the same time, the assessment of the eligibility of a patient for a trial requires the (machineprocessable) understanding of the semantics of the eligibility criteria in order to further evaluate if the patient data available for example in the hospital EHR satisfies these criteria. This paper presents an analysis of the semantics of the clinical trial eligibility criteria based on relevant medical ontologies in the clinical research domain: SNOMED-CT, LOINC, MedDRA. We detect subsets of these widely-adopted ontologies that characterize the semantics of the eligibility criteria of trials in various clinical domains and compare these sets. Next, we evaluate the occurrence frequency of the concepts in the concrete case of breast cancer (which is our first application domain) in order to provide meaningful priorities for the task of binding/mapping these ontology concepts to the actual patient data. We further assess the effort required to extend our approach to new domains in terms of additional semantic mappings that need to be developed.

Assessment of irrigators’ preferences for different water supply risk management tools: option contract and insurance.

Irrigators face the risk of not having enough water to meet their crops’ demand. There are different mechanisms to cope with this risk, including water markets (option contracts) or insurance. A farmer will purchase them when the expected utility change derived from the tool is positive. This paper presents a theoretical assessment of the farmer’s expected utility under two different option contracts, a drought insurance and a combination of an option contract and the insurance. We analyze the conditions that determine farmer’s reference for one instrument or the other and perform a numerical application that is relevant for a Spanish region.

Lagrangian tools and the assessment of the predictive capacity of geophysical data sets

We examine, with recently developed Lagrangian tools, altimeter data and numerical simulations obtained from the HYCOM model in the Gulf of Mexico. Our data correspond to the months just after the Deepwater Horizon oil spill in the year 2010. Our Lagrangian analysis provides a skeleton that allows the interpretation of transport routes over the ocean surface. The transport routes are further verified by the simultaneous study of the evolution of several drifters launched during those months in the Gulf of Mexico. We find that there exist Lagrangian structures that justify the dynamics of the drifters, although the agreement depends on the quality of the data. We discuss the impact of the Lagrangian tools on the assessment of the predictive capacity of these data sets.

Peer Assessment and Self-assessment: Effective Learning Tools in Higher Education.

When used appropriately, self- and peer-assessment are very effective learning tools. In the present work, instructor formative assessment and feedback, self-assessment (SA), and peer-assessment (PA) have been compared. During the first part of a semester, the students followed a continuous formative assessment. Subsequently, they were divided into two subgroups based on similar performances. One subgroup performed SAs, and the other followedPAduring the last part of the course. The performances of the two groups in solving problems were compared. Results suggest that PA is a more effective learning tool than SA, and both are more effective than instructor formative assessment. However, a survey that was conducted at the end of the experiment showed higher student confidence in instructor assessment than in PA. The students recognized the usefulness of acting as peer assessors, but believed that SA helped them more than PA.

Feasibility assessment of the interactive use of a Monte Carlo algorithm in treatment planning for intraoperative electron radiation therapy

This work analysed the feasibility of using a fast, customized Monte Carlo (MC) method to perform accurate computation of dose distributions during pre- and intraplanning of intraoperative electron radiation therapy (IOERT) procedures. The MC method that was implemented, which has been integrated into a specific innovative simulation and planning tool, is able to simulate the fate of thousands of particles per second, and it was the aim of this work to determine the level of interactivity that could be achieved. The planning workflow enabled calibration of the imaging and treatment equipment, as well as manipulation of the surgical frame and insertion of the protection shields around the organs at risk and other beam modifiers. In this way, the multidisciplinary team involved in IOERT has all the tools necessary to perform complex MC dosage simulations adapted to their equipment in an efficient and transparent way. To assess the accuracy and reliability of this MC technique, dose distributions for a monoenergetic source were compared with those obtained using a general-purpose software package used widely in medical physics applications. Once accuracy of the underlying simulator was confirmed, a clinical accelerator was modelled and experimental measurements in water were conducted. A comparison was made with the output from the simulator to identify the conditions under which accurate dose estimations could be obtained in less than 3 min, which is the threshold imposed to allow for interactive use of the tool in treatment planning. Finally, a clinically relevant scenario, namely early-stage breast cancer treatment, was simulated with pre- and intraoperative volumes to verify that it was feasible to use the MC tool intraoperatively and to adjust dose delivery based on the simulation output, without compromising accuracy. The workflow provided a satisfactory model of the treatment head and the imaging system, enabling proper configuration of the treatment planning system and providing good accuracy in the dosage simulation.