Rubrics in assessment

A 'rubric' is a way of assessing what an individual student has learned about a particular topic. A rubric uses a checklist made of the sub-tasks or components within the task, with graded descriptions of how well a student can DO or has LEARNED the task.

The use of scoring rubrics to determine clinical performance in the operating suite

Summary : This research evolved out of the need to examine the validity and inter-rater reliability of a set of performance-based scoring rubrics designed to measure competencies within the operating suite.

Method : Both holistic and analytical rubrics were developed aligned to the ACORN Standard [Australian College of Operating Room Nurses Standard NR4, 2004. ACORN Competency Standards for Perioperative Nurses: Standard NR4: The Instrument Nurse in the Perioperative Environment. Australian College of Operating Room Nurses Ltd, Adelaide] and underpinned by the Dreyfus model (1981). Three video clips that captured varying performance of nurses performing as instrument nurses in the operating suite were recorded and used as prompts by expert raters, who judged the performance using the rubrics.

Results : The study found that the holistic rubrics led to more consistent judgments than the analytical rubrics, yet the latter provided more diagnostic information for intervention purposes. Despite less consistency, the Analytical Observation Form had sufficient construct validity to satisfy the requirements of criterion referencing as determined by the Item Separation Index (Rasch, 1960), including high internal consistency and greater inter-rater reliability when average ratings were used.

Conclusion : The study was an empirical investigation of the use of concomitant Analytical and Holistic Rubrics to determine various levels of performance in the operating suite including inter-rater reliability. The methodology chosen was theoretically sound and sufficiently flexible to be used to develop other competencies within the operating suite.

Using rubrics in IT: experiences of assessment and feedback at Deakin University

 Students need to develop informed and realistic career aspirations to gain the most from their university studies towards their initial career development. However developing their aspirations, goals, and expectations is a complex process. In Information Technology (IT) no clear career development framework is evident in the literature. We present a pilot study which investigates the career aspirations of novice students studying IT at an Australian University. Through a series of career activities their aspirations were explored with the aim of improving support for career development. Results indicate that students have no clear short- or long- term aspirations, yet believe that programming skills are key to achieve a career in IT.

An historical inquiry into the true interpretation of the rubrics in the Book of Common Prayer : respecting the sermon and the communion service /

Includes bibliographical footnotes and index.

A history of the Book of common prayer and other books of authority : with an attempt to ascertain how the rubrics and canons have been understood and observed from the Reformation to the accession of George III : also an account of the state of religion and religious parties in England from 1640-1660 /

Mode of access: Internet.

The development of rubrics to measure undergraduate students' global awareness and global perspective: A validity study

Higher education institutions across the United States have developed global learning initiatives to support student achievement of global awareness and global perspective, but assessment options for these outcomes are extremely limited. A review of research for a global learning initiative at a large, Hispanic-serving, urban, public, research university in South Florida found a lack of instruments designed to measure global awareness and global perspective in the context of an authentic performance assessment. This quasi-experimental study explored the development of two rubrics for the global learning initiative and the extent to which evidence supported the rubrics' validity and reliability. One holistic rubric was developed to measure students' global awareness and the second to measure their global perspective. The study utilized a pretest/posttest nonequivalent group design. Multiple linear regression was used to ascertain the rubrics' ability to discern and compare average learning gains of undergraduate students enrolled in two global learning courses and students enrolled in two non-global learning courses. Parallel pretest/posttest forms of the performance task required students to respond to two open-ended questions, aligned with the learning outcomes, concerning a complex case narrative. Trained faculty raters read responses and used the rubrics to measure students' global awareness and perspective. Reliability was tested by calculating the rates of agreement among raters. Evidence supported the finding that the global awareness and global perspective rubrics yielded scores that were highly reliable measures of students' development of these learning outcomes. Chi-square tests of frequency found significant rates of inter-rater agreement exceeding the study's .80 minimum requirement. Evidence also supported the finding that the rubrics yielded scores that were valid measures of students' global awareness and global perspective. Regression analyses found little evidence of main effects; however, post hoc analyses revealed a significant interaction between global awareness pretest scores and the treatment, the global learning course. Significant interaction was also found between global perspective pretest scores and the treatment. These crossover interactions supported the finding that the global awareness and global perspective rubrics could be used to detect learning differences between the treatment and control groups as well as differences within the treatment group.

Viewbrics: Formatief evalueren van complexe vaardigheden middels ‘video-rubrics’ in het Voortgezet Onderwijs

Presentatie over de doelstelling en aanpak binnen het Viewbrics-project (www.viewbrics.nl), waarin gekeken wordt naar het effect van het gecombineerd gebruik van tekstuele rubrieken met videovoorbeelden op de kwaliteit van de feedback, mentale modellen en het aanleren van een aantal complexe (21e eeuwse) vaardigheden (samenwerken, presenteren, informatievaardigheden) in het Voortgezet Onderwijs.

A First Step Towards Synthesising Rubrics and Video for the Formative Assessment of Complex Skills

Abstract. The performance objectives used for the formative assessment of com- plex skills are generally set through text-based analytic rubrics[1]. Moreover, video modeling examples are a widely applied method of observational learning, providing students with context-rich modeling examples of complex skills that act as an analogy for problem solving [1]. The purpose of this theoretical paper is to synthesize the components of video modeling and rubrics to support the formative assessment of complex skills. Based on theory, we argue that application of the developed Video Enhanced Rubrics (VER) fosters learners’ development of mental models, quality of provided feedback by various actors and finally, the learners mastery of complex skills.

Het Effect van Formatief Gebruik van Rubrics op Transparantie, Zelfregulatie, Self-Efficacy en Prestatie bij Propedeusestudenten van Applied Science.

Boot-Groenink, A. (2016). Het Effect van Formatief Gebruik van Rubrics op Transparantie, Zelfregulatie, Self-Efficacy en Prestatie bij Propedeusestudenten van Applied Science. juli, 12, 2016, Heerlen, Nederland: Open Universiteit.

Viewbrics: Formative Assessment of Complex Skills with Video-Enhanced Rubrics (VER) in Dutch Secondary Education

To learn complex skills, like collaboration, learners need to acquire a concrete and consistent mental model of what it means to master this skill. If learners know their current mastery level and know their targeted mastery level, they can better determine their subsequent learning activities. Rubrics support learners in judging their skill performance as they provide textual descriptions of skills’ mastery levels with performance indicators for all constituent subskills. However, text-based rubrics have a limited capacity to support the formation of mental models with contextualized, time-related and observable behavioral aspects of a complex skill. This paper outlines the design of a study that intends to investigate the effect of rubrics with video modelling examples compared to text-based rubrics on skills acquisition and feedback provisioning. The hypothesis is that video-enhanced rubrics, compared to text based rubrics, will improve mental model formation of a complex skill and improve the feedback quality a learner receives (from e.g. teachers, peers) while practicing a skill, hence positively effecting final mastery of a skill.

A First Step Towards Video Enhanced Rubrics for the Formative Assessment of Complex Skills : A literature review and a plea

Paper presentation at the TEA2016 conference, Tallinn, Estonia.

Assessment rubrics: towards clearer and more replicable design, research and practice

‘Rubric’ is a term with a variety of meanings. As the use of rubrics has increased both in research and practice, the term has come to represent divergent practices. These range from secret scoring sheets held by teachers to holistic student-developed articulations of quality. Rubrics are evaluated, mandated, embraced and resisted based on often imprecise and inconsistent understandings of the term. This paper provides a synthesis of the diversity of rubrics, and a framework for researchers and practitioners to be clearer about what they mean when they say ‘rubric’. Fourteen design elements or decision points are identified that make one rubric different from another. This framework subsumes previous attempts to categorise rubrics, and should provide more precision to rubric discussions and debate, as well as supporting more replicable research and practice.

Authentic and conventional assessment in Singapore schools : an empirical study of teacher assignments and student work

This is an empirical examination of the quality of teacher assignments and student work in Singapore schools. Using a theoretical framework based on principles of authentic assessment and intellectual quality, two sets of criteria and scoring rubrics were developed for the training of expert teachers to judge the quality of assignments and student work. Following rigorous training, the inter-rater reliability of expert teacher scoring was high. Samples of teacher assignments and student work were collected in English, social studies, mathematics, and science subject areas from a random stratified sample of 30 elementary schools and 29 high schools. For both grade levels, there were significant differences for the authentic intellectual quality of teachers’ assignments by subject area. Likewise, the differences of authentic intellectual quality for student work were significant and varied by subject area. Subject area effect was large. The correlations between the quality of teachers’ assignment tasks and student work were strong and significant at both grade levels. Where teachers set more intellectually demanding tasks, students were more likely to generate work or artefacts judged to be of higher quality. The findings suggest that teacher professional development in authentic intellectual assessment task design can contribute to the improvement of student learning and performance. It is argued that this will be a key requisite of educational systems like Singapore that are seeking to expand pedagogy and student outcomes beyond a focus on factual and rote knowledge.