Moodle and multiple-choice tests

Multiple-Choice items are used in many different kinds of tests in several areas of knowledge. They can be considered an interesting tool to the self-assessing or as an alternative or complementary instrument to the traditional methods for assessing knowledge. The objectivity and accuracy of the multiple-choice tests is an important reason to think about. They are especially useful when the number of students to evaluate is too large. Moodle (Modular Object-Oriented Dynamic Learning Environment) is an Open Source course management system centered around learners' needs and designed to support collaborative approaches to teaching and learning. Moodle offers to the users a rich interface, context-specific help buttons, and a wide variety of tools such as discussion forums, wikis, chat, surveys, quizzes, glossaries, journals, grade books and more, that allow them to learn and collaborate in a truly interactive space. Come together the interactivity of the Moodle platform and the objectivity of this kind of tests one can easily build manifold random tests. The proposal of this paper is to relate our journey in the construction of these tests and share our experience in the use of the Moodle platform to create, take advantage and improve the multiple-choices tests in the Mathematic area.

Multiple-choice tests: a tool in assessing knowledge

The purpose of this paper is to analyse if Multiple-Choice Tests may be considered an interesting alternative for assessing knowledge, particularly in the Mathematics area, as opposed to the traditional methods, such as open questions exams. In this sense we illustrate some opinions of the researchers in this area. Often the perception of the people about the construction of this kind of exams is that they are easy to create. But it is not true! Construct well written tests it’s a hard work and needs writing ability from the teachers. Our proposal is analyse the construction difficulties of multiple - choice tests as well some advantages and limitations of this type of tests. We also show the frequent critics and worries, since the beginning of this objective format usage. Finally in this context some examples of Multiple-Choice Items in the Mathematics area are given, and we illustrate as how we can take advantage and improve this kind of tests.

Are multiple-choice tests a suitable method to evaluate students' knowledge? : against the extensive use of multiple-choice evaluation test. '¿Son los test de opción múltiple un método adecuado para evaluar el conocimiento de los estudiantes? : en contra del uso extensivo del test de evaluación de opción múltiple'.

Resumen basado en el de la publicación

Large-scale assessment of language proficiency: theoretical and pedagogical reflections on the use of multiple-choice tests

The new requirement placed on students in tertiary settings in Spain to demonstrate a B1 or a B2 proficiency level of English, in accordance with the Common European Framework of Reference for Languages (CEFRL), has led most Spanish universities to develop a program of certification or accreditation of the required level. The first part of this paper aims to provide a rationale for the type of test that has been developed at the Universidad Politécnica de Madrid for the accreditation of a B2 level, a multiple choice version, and to describe how it was constructed and validated. Then, in the second part of the paper, the results from its application to 924 students enrolled in different degree courses at a variety of schools and faculties at the university are analyzed based on a final test version item analysis. To conclude, some theoretical as well as practical conclusions about testing grammar that affect the teaching and learning process are drawn. RESUMEN. Las nuevas exigencias sobre niveles de competencia B1 y B2 en inglés según el Marco Común Europeo de Referencia para las Lenguas (MCERL) que se imponen sobre los estudiantes de grado y posgrado han llevado a la mayoría de las universidades españolas a desarrollar programas de acreditación o de certificación de estos niveles. La primera parte de este trabajo trata sobre las razones que fundamentan la elección de un tipo concreto de examen para la acreditación del nivel B2 de lengua inglesa en la Universidad Politécnica de Madrid. Se trata de un test de opción múltiple y en esta parte del trabajo se describe cómo fue diseñado y validado. En la segunda parte, se analizan los resultados de la aplicación del test a gran escala a un total de 924 estudiantes matriculados en varias escuelas y Facultades de la Universidad. Para terminar, se apuntan una serie de conclusiones teóricas y prácticas sobre la evaluación de la gramática y de qué modo influye en los procesos de enseñanza y aprendizaje.

Developing multiple - choice questions in mathematics

In this paper we wish to illustrate different perspectives used to create Multiple-Choice questions and we will show how we can improve these in the construction of math tests. As it is known, web technologies have a great influence on student’s behaviour. Based on an on-line project beginning at 2007 which has been contributing to help students on their individual work, we would like to share our experience and thoughts with colleagues who have a common concern when they have the task of constructing Multiple-Choice tests. We feel that Multiple-Choice tests play an important and a very useful supporting role in selfevaluation or self-examination of our students. Nonetheless, good Multiple–Choice Test Items are generally more complex and time-consuming to create than other types of tests. It requires a certain amount of skill. However, this skill maybe increases through study, practice and experience. This paper discusses a number of issues related to the use of Multiple-Choice questions, lists the advantages and disadvantages of this question format contrasting it with open questions. Some examples are given in this context.

E-assessment in mathematics courses with multiple-choice questions tests

With the implementation of the Bologna Process several challenges have been posed to higher education institution, particularly in Portugal. One of the main implications is related to the change of the paradigm of a teacher centered education, to a paradigm that is student centered. This change implies the change of the way to assess courses in higher education institutions. Continuous and formative assessments emerged as the focus, catalyzed by electronic assessment, or e-assessment. This paper presents a case of the implementation of an e-assessment strategy, implemented in order to allow continuous, formative assessment in numerous mathematics classes using multiple-choice questions tests implement in Moodle open-source learning management system. The implementation can be considered a success.

E-assessment in mathematics courses with multiple-choice questions tests

With the implementation of the Bologna Process several challenges have been posed to higher education institution, particularly in Portugal. One of the main implications is related to the change of the paradigm of a teacher centered education, to a paradigm that is student centered. This change implies the change of the way to assess courses in higher education institutions. Continuous and formative assessments emerged as the focus, catalyzed by electronic assessment, or e-assessment. This paper presents a case of the implementation of an e-assessment strategy, implemented in order to allow continuous, formative assessment in numerous mathematics classes using multiple-choice questions tests implement in Moodle open-source learning management system. The implementation can be considered a success.

English tests : graded multiple choice : A1. 'Test de Inglés : respuesta de múltiple elección : A1'.

Se trata de una serie de libros de test de respuesta de múltiple elección sobre el idioma inglés. La serie abarca los seis niveles del Marco Común Europeo, de A1 a C2, con un libro dedicado a cada nivel. Cada libro se compone de doce test de 30 minutos, cada uno de los cuales puede ser fácilmente realizado y corregido. Pero no son sólo pruebas para determinar los conocimientos de un estudiante en gramática y vocabulario, también están diseñados para ser utilizados como herramientas de enseñanza valiosas por derecho propio, permitiendo a profesores y estudiantes revisar las estructuras más comunes y las dificultades del lenguaje inglés, así como controlar su progreso a medida que avanzan.

English tests : graded multiple choice : A2. 'Test de Inglés : respuesta de múltiple elección : A2'.

Se trata de una serie de libros de test de respuesta de múltiple elección sobre el idioma inglés. La serie abarca los seis niveles del Marco Común Europeo, de A1 a C2, con un libro dedicado a cada nivel. Cada libro se compone de doce test de 30 minutos, cada uno de los cuales puede ser fácilmente realizado y corregido. Pero no son sólo pruebas para determinar los conocimientos de un estudiante en gramática y vocabulario, también están diseñados para ser utilizados como herramientas de enseñanza valiosas por derecho propio, permitiendo a profesores y estudiantes revisar las estructuras más comunes y las dificultades del lenguaje inglés, así como controlar su progreso a medida que avanzan.

English tests : graded multiple choice : B1. 'Test de Inglés : respuesta de múltiple elección : B1'.

Se trata de una serie de libros de test de respuesta de múltiple elección sobre el idioma inglés. La serie abarca los seis niveles del Marco Común Europeo, de A1 a C2, con un libro dedicado a cada nivel. Cada libro se compone de doce test de 30 minutos, cada uno de los cuales puede ser fácilmente realizado y corregido. Pero no son sólo pruebas para determinar los conocimientos de un estudiante en gramática y vocabulario, también están diseñados para ser utilizados como herramientas de enseñanza valiosas por derecho propio, permitiendo a profesores y estudiantes revisar las estructuras más comunes y las dificultades del lenguaje inglés, así como controlar su progreso a medida que avanzan.

English tests : graded multiple choice : C2. 'Test de Inglés : respuesta de múltiple elección : C2'.

Se trata de una serie de libros de test de respuesta de múltiple elección sobre el idioma inglés. La serie abarca los seis niveles del Marco Común Europeo, de A1 a C2, con un libro dedicado a cada nivel. Cada libro se compone de doce test de 30 minutos, cada uno de los cuales puede ser fácilmente realizado y corregido. Pero no son sólo pruebas para determinar los conocimientos de un estudiante en gramática y vocabulario, también están diseñados para ser utilizados como herramientas de enseñanza valiosas por derecho propio, permitiendo a profesores y estudiantes revisar las estructuras más comunes y las dificultades del lenguaje inglés, así como controlar su progreso a medida que avanzan.

English : multiple choice : 11 + practice papers.

Estos tests evalúan el nivel de inglés alcanzado por los niños durante la educación primaria, y les proporciona, además, la práctica para las pruebas de selección de la escuela secundaria.

Guidelines based on validity criteria for the development of multiple choice items

Resumen tomado de la publicaci??n

Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: A simulation and survey

Background Many medical exams use 5 options for multiple choice questions (MCQs), although the literature suggests that 3 options are optimal. Previous studies on this topic have often been based on non-medical examinations, so we sought to analyse rarely selected, 'non-functional' distractors (NF-D) in high stakes medical examinations, and their detection by item authors as well as psychometric changes resulting from a reduction in the number of options. Methods Based on Swiss Federal MCQ examinations from 2005-2007, the frequency of NF-D (selected by <1% or <5% of the candidates) was calculated. Distractors that were chosen the least or second least were identified and candidates who chose them were allocated to the remaining options using two extreme assumptions about their hypothetical behaviour: In case rarely selected distractors were eliminated, candidates could randomly choose another option - or purposively choose the correct answer, from which they had originally been distracted. In a second step, 37 experts were asked to mark the least plausible options. The consequences of a reduction from 4 to 3 or 2 distractors - based on item statistics or on the experts' ratings - with respect to difficulty, discrimination and reliability were modelled. Results About 70% of the 5-option-items had at least 1 NF-D selected by <1% of the candidates (97% for NF-Ds selected by <5%). Only a reduction to 2 distractors and assuming that candidates would switch to the correct answer in the absence of a 'non-functional' distractor led to relevant differences in reliability and difficulty (and to a lesser degree discrimination). The experts' ratings resulted in slightly greater changes compared to the statistical approach. Conclusions Based on item statistics and/or an expert panel's recommendation, the choice of a varying number of 3-4 (or partly 2) plausible distractors could be performed without marked deteriorations in psychometric characteristics.

Computer-based summative assessment of clinical reasoning in clerkships: A mixed-method comparison of key feature problems with case-based multiple choice questions

Background: It is yet unclear if there are differences between using electronic key feature problems (KFPs) or electronic case-based multiple choice questions (cbMCQ) for the assessment of clinical decision making. Summary of Work: Fifth year medical students were exposed to clerkships which ended with a summative exam. Assessment of knowledge per exam was done by 6-9 KFPs, 9-20 cbMCQ and 9-28 MC questions. Each KFP consisted of a case vignette and three key features (KF) using “long menu” as question format. We sought students’ perceptions of the KFPs and cbMCQs in focus groups (n of students=39). Furthermore statistical data of 11 exams (n of students=377) concerning the KFPs and (cb)MCQs were compared. Summary of Results: The analysis of the focus groups resulted in four themes reflecting students’ perceptions of KFPs and their comparison with (cb)MCQ: KFPs were perceived as (i) more realistic, (ii) more difficult, (iii) more motivating for the intense study of clinical reasoning than (cb)MCQ and (iv) showed an overall good acceptance when some preconditions are taken into account. The statistical analysis revealed that there was no difference in difficulty; however KFP showed a higher discrimination and reliability (G-coefficient) even when corrected for testing times. Correlation of the different exam parts was intermediate. Conclusions: Students perceived the KFPs as more motivating for the study of clinical reasoning. Statistically KFPs showed a higher discrimination and higher reliability than cbMCQs. Take-home messages: Including KFPs with long menu questions into summative clerkship exams seems to offer positive educational effects.