Viewbrics: Formative Assessment of Complex Skills with Video-Enhanced Rubrics (VER) in Dutch Secondary Education

To learn complex skills, like collaboration, learners need to acquire a concrete and consistent mental model of what it means to master this skill. If learners know their current mastery level and know their targeted mastery level, they can better determine their subsequent learning activities. Rubrics support learners in judging their skill performance as they provide textual descriptions of skills’ mastery levels with performance indicators for all constituent subskills. However, text-based rubrics have a limited capacity to support the formation of mental models with contextualized, time-related and observable behavioral aspects of a complex skill. This paper outlines the design of a study that intends to investigate the effect of rubrics with video modelling examples compared to text-based rubrics on skills acquisition and feedback provisioning. The hypothesis is that video-enhanced rubrics, compared to text based rubrics, will improve mental model formation of a complex skill and improve the feedback quality a learner receives (from e.g. teachers, peers) while practicing a skill, hence positively effecting final mastery of a skill.

Machine translation and the disruption of foreign language learning activities

This study examines the question of how language teachers in a highly technology-friendly university environment view machine translation and the implications that this has for the personal learning environments of students. It brings an activity-theory perspective to the question, examining the ways that the introduction of new tools can disrupt the relationship between different elements in an activity system. This perspective opens up for an investigation of the ways that new tools have the potential to fundamentally alter traditional learning activities. In questionnaires and group discussions, respondents showed general agreement that although use of machine translation by students could be considered cheating, students are bound to use it anyway, and suggested that teachers focus on the kinds of skills students would need when using machine translation and design assignments and exams to practice and assess these skills. The results of the empirical study are used to reflect upon questions of what the roles of teachers and students are in a context where many of the skills that a person needs to be able to interact in a foreign language increasingly can be outsourced to laptops and smartphones.

A First Step Towards Video Enhanced Rubrics for the Formative Assessment of Complex Skills : A literature review and a plea

Paper presentation at the TEA2016 conference, Tallinn, Estonia.

Formative assessment and motivation in higher education: a successful approach in mechanical engineer thermodynamics

Between 2000/01 and 2006/07, the approval rate of a Thermodynamics course in a Mechanical Engineer graduation was 25%. However, a careful analysis of the results showed that 41% of the students chosen not to attend or dropped out, missing the final examination. Thus, a continuous assessment methodology was developed, whose purpose was to reduce drop out, motivating students to attend this course, believing that what was observed was due, not to the incapacity to pass, but to the anticipation of the inevitability of failure by the students. If, on one hand, motivation is defined as a broad construct pertaining to the conditions and processes that account for the arousal, direction, magnitude, and maintenance of effort, on the other hand, assessment is one of the most powerful tools to change the will that students have to learn, motivating them to learn in a quicker and permanent way. Some of the practices that were implemented, included: promoting learning goal orientation rather than performance goal orientation; cultivating intrinsic interest in the subject and put less emphasis on grades but make grading criteria explicit; emphasizing teaching approaches that encourage collaboration among students and cater for a range of teaching styles; explaining the reasons for, and the implications of, tests; providing feedback to students about their performance in a form that is non-egoinvolving and non-judgemental and helping students to interpret it; broadening the range of information used in assessing the attainment of individual students. The continuous assessment methodology developed was applied in 2007/08 and 2008/09, having found an increase in the approval from 25% to 55% (30%), accompanied by a decrease of the drop out from 41% to 23,5% (17,5%). Flunking with a numerical grade lowered from 34,4% to 22,0% (12,4%). The perception by the students of the continuous assessment relevance was evaluated with a questionnaire. 70% of the students that failed the course respond that, nevertheless, didn’t repent having done the continuous assessment.

The Lived Experience of Nursing Students With Formative Assessment Formally Embedded in Clinical Courses: A Transcendental Phenomenological Study

The potential of formative assessment (FA) for informing learning in classroom-based nursing courses is clearly established in the literature; however, research on FA in clinical courses remains scarce. This inquiry explored the lived experience of nursing students using transcendental phenomenology and described the phenomenon of being assessed in clinical courses. The research question guiding the study was: How is the phenomenon of assessment experienced by nursing students when FA is formally embedded in clinical courses? Inherent in this question were the following issues: (a) the meaning of clinical experiences for nursing students, (b) the meaning of being assessed through FA, and (c) what it is like to be assessed when FA is formally embedded within clinical experiences. The noematic themes that illuminated the whatness of the participants’ experience were (a) enabled cognitive activity, (b) useful feedback, (c) freedom to be, (d) enhanced focus, (e) stress moderator, and (f) respectful mentorship. The noetic themes associated with how the phenomenon was experienced were related to bodyhood, temporality, spatiality, and relationship to others. The results suggest a fundamental paradigm shift from traditional nursing education to a more pervasive integration of FA in clinical courses so that students have time to learn before being graded on their practice. Furthermore, this inquiry and the literature consulted provide evidence that using cognitive science theory to inform and reform clinical nursing education is a timely option to address the repeated calls from nursing leaders to modernize nursing education. This inquiry contributes to reduce our reliance on assumptions derived from research on FA in nursing classrooms and provides evidence based on the reality of using formative assessment in clinical courses. Recommendations for future research are presented.

TELMA: Technology enhanced learning environment for minimally invasive surgery

Background: Cognitive skills training for minimally invasive surgery has traditionally relied upon diverse tools, such as seminars or lectures. Web technologies for e-learning have been adopted to provide ubiquitous training and serve as structured repositories for the vast amount of laparoscopic video sources available. However, these technologies fail to offer such features as formative and summative evaluation, guided learning, or collaborative interaction between users. Methodology: The "TELMA" environment is presented as a new technology-enhanced learning platform that increases the user's experience using a four-pillared architecture: (1) an authoring tool for the creation of didactic contents; (2) a learning content and knowledge management system that incorporates a modular and scalable system to capture, catalogue, search, and retrieve multimedia content; (3) an evaluation module that provides learning feedback to users; and (4) a professional network for collaborative learning between users. Face validation of the environment and the authoring tool are presented. Results: Face validation of TELMA reveals the positive perception of surgeons regarding the implementation of TELMA and their willingness to use it as a cognitive skills training tool. Preliminary validation data also reflect the importance of providing an easy-to-use, functional authoring tool to create didactic content. Conclusion: The TELMA environment is currently installed and used at the Jesús Usón Minimally Invasive Surgery Centre and several other Spanish hospitals. Face validation results ascertain the acceptance and usefulness of this new minimally invasive surgery training environment.

A First Step Towards Synthesising Rubrics and Video for the Formative Assessment of Complex Skills

Abstract. The performance objectives used for the formative assessment of com- plex skills are generally set through text-based analytic rubrics[1]. Moreover, video modeling examples are a widely applied method of observational learning, providing students with context-rich modeling examples of complex skills that act as an analogy for problem solving [1]. The purpose of this theoretical paper is to synthesize the components of video modeling and rubrics to support the formative assessment of complex skills. Based on theory, we argue that application of the developed Video Enhanced Rubrics (VER) fosters learners’ development of mental models, quality of provided feedback by various actors and finally, the learners mastery of complex skills.

Use of web-based formative assessment to support student learning in metabolism/nutrition unit

In 2002, an integrated basic science course was introduced into the Bachelor of Dental Sciences programme at the University of Queensland, Australia. Learning activities for the Metabolism and Nutrition unit within this integrated course included lectures, problem-based learning tutorials, computer-based self-directed learning exercises and practicals. To support student learning and assist students to develop the skills necessary to become lifelong learners, an extensive bank of formative assessment questions was set up using the commercially available package, WebCT®. Questions included short-answer, multiple-choice and extended matching questions. As significant staff time was involved in setting up the question database, the extent to which students used the formative assessment and their perceptions of its usefulness to their learning were evaluated to determine whether formative assessment should be extended to other units within the course. More than 90% of the class completed formative assessment tasks associated with learning activities scheduled in the first two weeks of the block, but this declined to less than 50% by the fourth and final week of the block. Patterns of usage of the formative assessment were also compared in students who scored in the top 10% for all assessment for the semester with those who scored in the lowest 10%. High-performing students accessed the Web-based formative assessment about twice as often as those who scored in the lowest band. However, marks for the formative assessment tests did not differ significantly between the two groups. In a questionnaire that was administered at the completion of the block, students rated the formative assessment highly, with 80% regarding it as being helpful for their learning. In conclusion, although substantial staff time was required to set up the question database, this appeared to be justified by the positive responses of the students.

Technology-enhanced education for Millennials: how the Information Society is changing the way of learning

The Information Society has provided the context for the development of a new generation, known as the Millennials, who are characterized by their intensive use of technologies in everyday life. These features are changing the way of learning, prompting educational institutions to attempt to better adapt to youngneeds by incorporating technologies into education. Based on this premise, wehave reviewed the prominent reports of the integration of ICT into education atdifferent levels with the aim of evidencing how education is changing, and willchange, to meet the needs of Millennials with ICT support. The results show thatmost of the investments have simply resulted in an increase of computers andaccess to the Internet, with teachers reproducing traditional approaches to education and e-learning being seen as complementary to face-to-face education.While it would seem that the use of ICT is not revolutionizing learning, it isfacilitating the personalization, collaboration and ubiquity of learning.

Référentiel de pratiques professionnelles: soins et surveillance des accès veineux centraux de l'adulte pour la nutrition parentérale [Formative assessment in clinical nutrition: Care management of parenteral nutrition central venous access]

Les voies veineuses centrales (VVC) sont essentielles pour l'administration de la nutrition parentérale. Le risque de complications est dépendant de la qualité des soins apportés à la VVC qui influence de ce fait la qualité de vie des patients et le coût des soins. Beaucoup de complications des VVC, infectieuses ou non, peuvent être prévenues par l'existence de protocoles de soins appropriés et standardisés. L'information sur les soins des VVC et les éventuelles complications est essentielle pour le dépistage et le traitement précoce de ces complications ; elle doit faire l'objet de protocoles partagés entre les patients et les soignants. Cet article décrit une évaluation des pratiques professionnelles sous la forme d'un audit clinique destiné à améliorer la qualité de soins des patients en nutrition parentérale porteurs de VVC. Central venous access devices (CVAD) are essential for the administration of parenteral nutrition. The quality of the care of CVAD influences the risk of complications and so the quality of life of the patients and the costs of care. Numerous infectious or non-infectious complications of CVAD can be prevented by appropriate, standardized protocols of care. Information about the care of CVAD and complications is essential for the early recognition and treatment of complications and should be shared between patients and caregivers. This article describes an audit for CAVD care that can be used to improve quality of care in a professional practice evaluation program.

Investigating Formative Assessment: Exploring the Impact on the Self-Efficacy and Motivation of Mathematics Students

This is a study of the implementation and impact of formative assessment strategies on the motivation and self-efficacy of secondary school mathematics students. An explanatory sequential mixed methods design was implemented where quantitative and qualitative data were collected and analyzed sequentially in 2 different phases. The first phase involved quantitative data from student questionnaires and the second phase involved qualitative data from individual student and teacher interviews. The findings of the study suggest that formative assessment is implemented in practice in diverse ways and is a process where the strategies are interconnected. Teachers experience difficulty in incorporating peer and self-assessment and perceive a need for exemplars. Key factors described as influencing implementation include teaching philosophies, interpretation of ministry documents, teachers’ experiences, leadership in administration and department, teacher collaboration, misconceptions of teachers, and student understanding of formative assessment. Findings suggest that overall, formative assessment positively impacts student motivation and self-efficacy, because feedback is provided which offers encouragement and recognition by highlighting the progress that has been made and what steps need to be taken to improve. However, students are impacted differently with some considerations including how students perceive mistakes and if they fear judgement. Additionally, the impact of formative assessment is influenced by the connection between self-efficacy and motivation, namely how well a student is doing is a source of both concepts.

A Teacher’s Formative Assessment Perceptions and Practices in Oral Intermediate English Courses at the Université de Montréal

Au cours des dernières années, les pratiques de l’évaluation comme aide à l’apprentissage (c’est-à-dire l’auto-évaluation, l’évaluation par les pairs, la rétroaction) dans la salle de classe ont été de plus en plus considérées comme des éléments essentiels (Black & Wiliam, 1998; Harlen & Winter, 2004; van de Watering & van der Rijt, 2006). Cependant, dans le domaine de l’apprentissage d’une langue seconde la recherche sur ce sujet est plutôt limitée. En nous fondant sur les études de Colby-Kelly et Turner (2007) et de Lyster et Ranta (1997), nous avons mené une recherche exploratoire visant à combler ce besoin. L’objectif général était de comprendre comment l’évaluation formative se réalise dans deux cours d’anglais intermédiaire à l’Université de Montréal, et de comparer comment ces pratiques sont perçues et vécues par la professeure et ses étudiants. Trois questions de recherche étaient posées: 1. Quelle est la nature de l’évaluation formative dans une salle de classe d’anglais langue seconde? 2. Est-ce que les pratiques de la professeure reflètent ce qu’elle pense de l’évaluation formative? 3. Quels sont les correspondances et différences entre les croyances de la professeure et les perceptions de ses étudiants quant aux bénéfices de l’évaluation formative pour l’apprentissage de l’anglais langue seconde? La collecte de données comprend des observations en classe, des entrevues auprès de la professeure et des questionnaires pour les étudiants. Pour répondre à notre première question de recherche, nous avons analysé la rétroaction donnée par la professeure pendant une production orale réalisée par ses étudiants à partir de la typologie de feedback et de uptake de l’étude de Lyster et Ranta (1997). En ce qui a trait à la deuxième question de recherche, nous avons fait des entrevues avec la professeure en vue d’expliciter ses perceptions relativement à l’évaluation formative. À la fin du trimestre, nous avons comparé ses réponses avec sa pratique à partir des enregistrements vidéo produits en classe. Finalement, pour répondre à notre dernière question de recherche nous avons comparé les réponses données par la professeure aux entrevues avec les réponses des étudiants à un questionnaire adapté à partir de celui de Colby-Kelly et Turner. Finalement, nous présentons et discutons les résultats les plus significatifs obtenus dans cette étude qualitative Nous concluons cette thèse en proposant de avenues pour des recherches futures.