Computer-based summative assessment of clinical reasoning in clerkships: A mixed-method comparison of key feature problems with case-based multiple choice questions

Background: It is yet unclear if there are differences between using electronic key feature problems (KFPs) or electronic case-based multiple choice questions (cbMCQ) for the assessment of clinical decision making. Summary of Work: Fifth year medical students were exposed to clerkships which ended with a summative exam. Assessment of knowledge per exam was done by 6-9 KFPs, 9-20 cbMCQ and 9-28 MC questions. Each KFP consisted of a case vignette and three key features (KF) using “long menu” as question format. We sought students’ perceptions of the KFPs and cbMCQs in focus groups (n of students=39). Furthermore statistical data of 11 exams (n of students=377) concerning the KFPs and (cb)MCQs were compared. Summary of Results: The analysis of the focus groups resulted in four themes reflecting students’ perceptions of KFPs and their comparison with (cb)MCQ: KFPs were perceived as (i) more realistic, (ii) more difficult, (iii) more motivating for the intense study of clinical reasoning than (cb)MCQ and (iv) showed an overall good acceptance when some preconditions are taken into account. The statistical analysis revealed that there was no difference in difficulty; however KFP showed a higher discrimination and reliability (G-coefficient) even when corrected for testing times. Correlation of the different exam parts was intermediate. Conclusions: Students perceived the KFPs as more motivating for the study of clinical reasoning. Statistically KFPs showed a higher discrimination and higher reliability than cbMCQs. Take-home messages: Including KFPs with long menu questions into summative clerkship exams seems to offer positive educational effects.

Assessment in CLIL

The aim of this talk is to explore assessment in higher education and support for learners during their academic education. It investigates the assessment methods that provide success for universities and learners. Universities which implement CLIL should assess learners who would like to attend a programme in English without taking account of their English language qualifications. This assessment should be done in writing, listening and comprehension, speaking and reading and comprehension. In the literature, formative and summative assessments are considered. Formative assessment is ongoing, 'more complex as its intention is to be directly diagnostics with a view to immediately impacting on learner's next steps' (Coyle et al, 2010). Summative assessment occurs at the end of the term or course. McKay, 2006 divides assessment into three phases: design, operationalization and administration phase. If these three phases can't be embedded in the classroom as they are, they can be introduced as a set of questions; why?, how? and what?

A Teacher’s Formative Assessment Perceptions and Practices in Oral Intermediate English Courses at the Université de Montréal

Au cours des dernières années, les pratiques de l’évaluation comme aide à l’apprentissage (c’est-à-dire l’auto-évaluation, l’évaluation par les pairs, la rétroaction) dans la salle de classe ont été de plus en plus considérées comme des éléments essentiels (Black & Wiliam, 1998; Harlen & Winter, 2004; van de Watering & van der Rijt, 2006). Cependant, dans le domaine de l’apprentissage d’une langue seconde la recherche sur ce sujet est plutôt limitée. En nous fondant sur les études de Colby-Kelly et Turner (2007) et de Lyster et Ranta (1997), nous avons mené une recherche exploratoire visant à combler ce besoin. L’objectif général était de comprendre comment l’évaluation formative se réalise dans deux cours d’anglais intermédiaire à l’Université de Montréal, et de comparer comment ces pratiques sont perçues et vécues par la professeure et ses étudiants. Trois questions de recherche étaient posées: 1. Quelle est la nature de l’évaluation formative dans une salle de classe d’anglais langue seconde? 2. Est-ce que les pratiques de la professeure reflètent ce qu’elle pense de l’évaluation formative? 3. Quels sont les correspondances et différences entre les croyances de la professeure et les perceptions de ses étudiants quant aux bénéfices de l’évaluation formative pour l’apprentissage de l’anglais langue seconde? La collecte de données comprend des observations en classe, des entrevues auprès de la professeure et des questionnaires pour les étudiants. Pour répondre à notre première question de recherche, nous avons analysé la rétroaction donnée par la professeure pendant une production orale réalisée par ses étudiants à partir de la typologie de feedback et de uptake de l’étude de Lyster et Ranta (1997). En ce qui a trait à la deuxième question de recherche, nous avons fait des entrevues avec la professeure en vue d’expliciter ses perceptions relativement à l’évaluation formative. À la fin du trimestre, nous avons comparé ses réponses avec sa pratique à partir des enregistrements vidéo produits en classe. Finalement, pour répondre à notre dernière question de recherche nous avons comparé les réponses données par la professeure aux entrevues avec les réponses des étudiants à un questionnaire adapté à partir de celui de Colby-Kelly et Turner. Finalement, nous présentons et discutons les résultats les plus significatifs obtenus dans cette étude qualitative Nous concluons cette thèse en proposant de avenues pour des recherches futures.

Assessment and learning : is it time to rethink student activities and academic roles?

Resumen basado en el de la publicación

Use of the web to provide learning support for a large metabolism and nutrition class

To support student learning in a large Metabolism and Nutrition class, we have introduced a web-based package, using a commercially available program, WebCT. The package was developed at a minimal cost and with limited resources. In addition to downloadable (PDF) versions of lecture Powerpoint presentations, tutorial outlines and a practical class exercise, web-based self-directed learning exercises were included to reinforce and extend lecture material in an active learning environment. The web-site also contained a variety of formative and summative assessment tasks that examined both factual recall and higher order thinking Detailed course information, timetables and a bulletin board were also readily accessible. Student usage of the site was generally high, but varied widely between individual students. Students who achieved a high overall score for the course completed on average three times as many formative assessment items and achieved a higher score for all tests than students who did poorly. Student feedback about the site was very positive with the majority of students reporting that the course material and assessment items that were available were useful to their learning. Administration of the course was also facilitated. (C) 2001 IUBMB. Published by Elsevier Science Ltd. All rights reserved.

A incidência da avaliação sumativa externa (provas de aferição) nas práticas docentes do 1.º ciclo do Ensino Básico e nos projetos curriculares de turma

Dissertação de Mestrado, Educação (Desenvolvimento e Concepção de Projectos Educativos), 19 de Março de 2013, Universidade dos Açores.

Da docência de estagiário em história e geografia ao papel do erro na avaliação e na aprendizagem

Mestrado, Ensino de História e Geografia no 3.º Ciclo do Ensino Básico e no Ensino Secundário, 10 de Março de 2016, Universidade dos Açores (Relatório de Estágio).

Example uses of a learning object

A self study course for learning to program using the C programming language has been developed. A Learning Object approach was used in the design of the course. One of the benefits of the Learning Object approach is that the learning material can be reused for different purposes. 'Me course developed is designed so that learners can choose the pedagogical approach most suited to their personal learning requirements. For all learning approaches a set of common Assessment Learning Objects (ALOs or tests) have been created. The design of formative assessments with ALOs can be carried out by the Instructional Designer grouping ALOs to correspond to a specific assessment intention. The course is non-credit earning, so there is no summative assessment, all assessment is formative. In this paper examples of ALOs and their uses is presented together with their uses as decided by the Instructional Designer and learner. Personalisation of the formative assessment of skills can be decided by the Instructional Designer or the learner using a repository of pre-designed ALOs. The process of combining ALOs can be carried out manually or in a semi-automated way using metadata that describes the ALO and the skill it is designed to assess.

The new final Clinical Skills examination in human medicine in Switzerland: Essential steps of exam development, implementation and evaluation, and central insights from the perspective of the national Working Group

Objective: Since 2011, the new national final examination in human medicine has been implemented in Switzerland, with a structured clinical-practical part in the OSCE format. From the perspective of the national Working Group, the current article describes the essential steps in the development, implementation and evaluation of the Federal Licensing Examination Clinical Skills (FLE CS) as well as the applied quality assurance measures. Finally, central insights gained from the last years are presented. Methods: Based on the principles of action research, the FLE CS is in a constant state of further development. On the foundation of systematically documented experiences from previous years, in the Working Group, unresolved questions are discussed and resulting solution approaches are substantiated (planning), implemented in the examination (implementation) and subsequently evaluated (reflection). The presented results are the product of this iterative procedure. Results: The FLE CS is created by experts from all faculties and subject areas in a multistage process. The examination is administered in German and French on a decentralised basis and consists of twelve interdisciplinary stations per candidate. As important quality assurance measures, the national Review Board (content validation) and the meetings of the standardised patient trainers (standardisation) have proven worthwhile. The statistical analyses show good measurement reliability and support the construct validity of the examination. Among the central insights of the past years, it has been established that the consistent implementation of the principles of action research contributes to the successful further development of the examination. Conclusion: The centrally coordinated, collaborative-iterative process, incorporating experts from all faculties, makes a fundamental contribution to the quality of the FLE CS. The processes and insights presented here can be useful for others planning a similar undertaking. Keywords: national final examination, licensing examination, summative assessment, OSCE, action research

Cuestionarios online como herramienta de aprendizaje: comparación de resultados con diversos modelos de cuestionarios.

En este trabajo se comparan dos estrategias de evaluación a través de cuestionarios online: i) realizar un cuestionario de evaluación sumativa al final de cada bloque temático del curso o ii) permitir que, para cada bloque temático, el alumno pueda repetir varias veces un test (con realimentación de las respuestas correctas) y realizar un test similar después de esta fase de aprendizaje. Los resultados muestran varios beneficios de la segunda estrategia: una mayor participación de los alumnos hasta el final del semestre, una menor tasa de abandono de la asignatura y una mejor nota en los test. Sin embargo, para que el aprendizaje sea efectivo en el caso de repetir varias veces un test, se necesita un banco de preguntas suficientemente amplio para que exista una probabilidad baja de que las preguntas no se repitan. Este problema se ha resuelto utilizando la posibilidad que presentan algunas plataformas de gestión del aprendizaje de generar automáticamente variantes numéricas de una misma pregunta. Por último se observa un grado de acuerdo muy elevado por parte de los alumnos con el peso que los resultados de los tests tienen en la nota final del curso, en el caso bajo estudio. Abstract: This work compares two different assessment strategies through online tests: i) summative assessment tests at the end of each part of the course or ii) the students can do selfassessment tests (with feedback) prior to a summative assessment test at the end of each part of the course. Results show that the second strategy yields several benefits: an increment in student participation till the end of the semester, a reduction in the drop-out rate and an increase in the mean test marks. However, a large item bank is necessary to effectively implement this second strategy. This problem has been addressed by using the automatic item generation facility offered by several learning management systems. Finally, a high proportion of the students agree with the weight proposed for these tests in the final course mark.