Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: A simulation and survey

Background Many medical exams use 5 options for multiple choice questions (MCQs), although the literature suggests that 3 options are optimal. Previous studies on this topic have often been based on non-medical examinations, so we sought to analyse rarely selected, 'non-functional' distractors (NF-D) in high stakes medical examinations, and their detection by item authors as well as psychometric changes resulting from a reduction in the number of options. Methods Based on Swiss Federal MCQ examinations from 2005-2007, the frequency of NF-D (selected by <1% or <5% of the candidates) was calculated. Distractors that were chosen the least or second least were identified and candidates who chose them were allocated to the remaining options using two extreme assumptions about their hypothetical behaviour: In case rarely selected distractors were eliminated, candidates could randomly choose another option - or purposively choose the correct answer, from which they had originally been distracted. In a second step, 37 experts were asked to mark the least plausible options. The consequences of a reduction from 4 to 3 or 2 distractors - based on item statistics or on the experts' ratings - with respect to difficulty, discrimination and reliability were modelled. Results About 70% of the 5-option-items had at least 1 NF-D selected by <1% of the candidates (97% for NF-Ds selected by <5%). Only a reduction to 2 distractors and assuming that candidates would switch to the correct answer in the absence of a 'non-functional' distractor led to relevant differences in reliability and difficulty (and to a lesser degree discrimination). The experts' ratings resulted in slightly greater changes compared to the statistical approach. Conclusions Based on item statistics and/or an expert panel's recommendation, the choice of a varying number of 3-4 (or partly 2) plausible distractors could be performed without marked deteriorations in psychometric characteristics.

New Sites, New Challenges? Ethical and Methodological Questions in the Context of Research on Migration Exclusion

The challenges of research ethics and methodologies have been reflected on extensively, but – aside from the context of feminist methodologies – less so in relation to research on particular migration sites such as in transit, detention centres, at the borders or within migration administration. First attempts in this direction have been made (Düvell et al. 2010, Fresia et al. 2005, Riedner 2014, van Liempt/Bilger2009), however, more reflection and theorization is needed, considering the contested nature of these temporal and volatile sites. In this workshop, we thus aim at examining methodological as well as ethical questions that arise during field work: We attempt to reflect the power relations involved in the research process, the ethics of research design, the dissemination of research results, the question of gaining access to and – whenever necessary – staying in contact with our research subjects. How can we negotiate informed consent with subjects whose life is currently marked by transit and insecurity concerning their own future, and who are in an uncertain situation in which substantial information (legal, social, cultural etc.) is likely to be missing? How do we deal with the dilemma of possibly contributing to knowledge production that might facilitate removals and deportations in the future, considering that the reception of the results is not in the hands of the researchers? How do we deal with the anticipated as well as unexpected impacts of our research on social and political practice? Regarding fieldwork in state institutions, how do we negotiate the multiple loyalties we often find ourselves faced with as social researchers, both with the excluded migrants and with the authorities implementing the exclusions – two groupings considered to be opposite to each other (Lavanchy 2013)? Which different roles do researchers need to take on? The aim of our workshop is first and foremost to exchange experiences on fieldwork with others doing qualitative research on related topics and to consider its possible implications – including affective dimensions – for all participants involved in the research process: the migrants, the security staff of detention centres, its social workers, border police and bureaucrats and, last but not least, the researchers themselves. Furthermore, we generally wish to reflect upon the question of how best to conduct research in this contested field, applying an interdisciplinary perspective.