Engaging Students in the Implementation of Criterion Referenced Assessment in First Year Law

The Queensland University of Technology (QUT) University Academic Board approved a new QUT Assessment Policy in September 2003, which requires a criterion-referenced approach as opposed to a norm-referenced approach to assessment across the university(QUT,MOPP,2003). In 2004, the QUT Law School embarked upon a process of awareness raising about criterion-referenced assessment amongst staff and from 2004 – 2005 staggered the implementation of criterion-referenced assessment in all first year core undergraduate law units. This paper will briefly discuss the benefits and potential pitfalls of criterion referenced assessment and the context for implementing it in the first year law program, report on student’s feedback on the introduction of criterion referenced assessment and the strategies adopted in 2005 to engage students more fully in criterion referenced assessment processes to enhance their learning outcomes.

Is implementing criterion-referenced assessment worth the effort with gen Y?

Well implemented criterion-referenced assessment (CRA) requires dedicated time and effort, especially in describing realistic expectations of evidence of achievement to students in the form of criteria sheets (or grading rubrics). It is also takes time out of delivering content to teach students how to judge their own work using criteria sheets. In 2007, to engage third year Microbiology students in using criteria sheets for the first time in their degree, we devised an innovative assessment tutorial supported by online resources. We were sceptical of much of the literature that reported ‘agreed’ characteristics of our predominantly gen Y cohort, because of the older ages of the majority of authors. These authors claim gen Y has a propensity for digital media, overconfidence in their own abilities and a collaborative orientation. We rejected this stereotype when developing the tutorial. Evaluations by students were positive and there was no dramatic change to grades for the unit. These results are similar to those in the literature for non gen Y cohorts. This lends support to our claim that giving students control over their own learning, irrespective of their generational label, is worth the time and effort.

Assessing reflection skills in law using criterion-referenced assessment

This paper focuses on the assessment of reflective practice, an issue that has not been fully explored within legal education literature. While the issue of how reflective practice should be taught is one that requires careful consideration, it is beyond the scope of this paper to consider both the teaching and the assessment of reflective practice. Part II of this paper conceptualises reflective practice, and Part III explores the benefits of reflective practice in legal education and the use of reflective writing to assess experiential learning in a legal context. Part IV considers the diverse issues that arise in assessing reflective practice and whether there is an objective method for assessing reflection. Part V of the paper examines the assessment of reflective practice in the context of an exemplar undergraduate law subject that uses a reflective report to assess students’ experiential learning during a court visit.14 Finally, Part VI offers a rubric to facilitate criterion-referenced assessment of reflective practice and thereby provides a framework for assessing reflection skills. It is suggested that the rubric is transferable not only to other law subjects but also to subjects in other disciplines.

Does the summative assessment of real world learning using criterion-referenced assessment need to be discipline specific?

This paper synthesises the existing literature on the contemporary conception of ‘real world’ and compares it with similar notions such as ‘authentic’ and ‘work integrated learning’. While the term ‘real world’ may be partly dependent on the discipline, it does not necessarily follow that the criterion-referenced assessment of ‘real world’ assessment must involve criteria and performance descriptors that are discipline specific. Two examples of summative assessment (court report and trial process exercise) from a final year core subject at the Queensland University of Technology, LWB432 Evidence, emphasise real world learning, are authentic, innovative and better prepare students for the transition into the workplace than more generic forms of assessment such as tutorial participation or oral presentations. The court report requires students to attend a criminal trial in a Queensland Court and complete a two page report on what they saw in practice compared with what they learned in the classroom. The trial process exercise is a 50 minute written closed book activity conducted in tutorials, where students plan questions that they would ask their witness in examination-in-chief, plan questions that they would ask their opponent’s witness in cross-examination, plan questions that they would ask in reexamination given what their opponent asked in cross-examination, and prepare written objections to their opponent’s questions. The trial process exercise simulates the real world, whereas the court report involves observing the real world, and both assessment items are important to the role of counsel. The design of the criterion-referenced assessment rubrics for the court report and trial process exercise is justified by the literature. Notably, the criteria and performance descriptors are not necessarily law specific and this paper highlights the parts that may be easily transferred to other disciplines.

Towards a classification of criterion referenced assessment models in mathematics courses : student and academic perspectives

We present the findings of a study into the implementation of explicitly criterion- referenced assessment in undergraduate courses in mathematics. We discuss students' concepts of criterion referencing and also the various interpretations that this concept has among mathematics educators. Our primary goal was to move towards a classification of criterion referencing models in quantitative courses. A secondary goal was to investigate whether explicitly presenting assessment criteria to students was useful to them and guided them in responding to assessment tasks. The data and feedback from students indicates that while students found the criteria easy to understand and useful in informing them as to how they would be graded, it did not alter the way the actually approached the assessment activity.

An adaptable framework for the teaching and assessment of software development across year levels

Few frameworks exist for the teaching and assessment of programming subjects that are coherent and logical. Nor are they sufficiently generic and adaptable to be used outside the particular tertiary institutions in which they were developed. This paper presents the Teaching and Assessment of Software Development (TASD) frame-work. We describe its development and implementation at an Australian university and demonstrate, with examples, how it has been used, with supporting data. Extracts of criteria sheets (grading rubrics) for a variety of assessment tasks are included. The numerous advantages of this new framework are discussed with comparisons made to those reported in the published literature.

Assessment of online discussion forums for law students

This paper goes beyond the existing literature and explores the innovative topic of designing criterion-referenced assessment for online discussion forums. There are several benefits of embedding online discussion forums into subjects including engaging students in collaborative learning, and encouraging deeper analysis, critical thinking and reflection. Using the assessment principles of validity, reliability and transparency, this paper offers a range of practical strategies to tutors who plan to develop criterion-referenced assessment as opposed to norm-referenced assessment for online discussion forums, applies the assessment principles in the context of an undergraduate law subject, and exemplars a rubric for an online discussion forum in a work placement subject.

Using poster presentations as assessment of work integrated learning

Background / context: The ALTC WIL Scoping Study identified a need to develop innovative assessment methods for work integrated learning (WIL) that encourage reflection and integration of theory and practice within the constraints that result from the level of engagement of workplace supervisors and the ability of academic supervisors to become involved in the workplace. Aims: The aim of this paper is to examine how poster presentations can be used to authentically assess student learning during WIL. Method / Approach: The paper uses a case study approach to evaluate the use of poster presentations for assessment in two internship units at the Queensland University of Technology. The first is a unit in the Faculty of Business where students majoring in advertising, marketing and public relations are placed in a variety of organisations. The second unit is a law unit where students complete placements in government legal offices. Results / Discussion: While poster presentations are commonly used for assessment in the sciences, they are an innovative approach to assessment in the humanities. This paper argues that posters are one way that universities can overcome the substantial challenges of assessing work integrated learning. The two units involved in the case study adopt different approaches to the poster assessment; the Business unit is non-graded and the poster assessment task requires students to reflect on their learning during the internship. The Law unit is graded and requires students to present on a research topic that relates to their internship. In both units the posters were presented during a poster showcase which was attended by students, workplace supervisors and members of faculty. The paper evaluates the benefits of poster presentations for students, workplace supervisors and faculty and proposes some criteria for poster assessment in WIL. Conclusions / Implications: The paper concludes that posters can effectively and authentically assess various learning outcomes in WIL in different disciplines while at the same time offering a means to engage workplace supervisors with academic staff and other students and supervisors participating in the unit. Posters have the ability to demonstrate reflection in learning and are an excellent demonstration of experiential learning and assessing authentically. Keywords: Work integrated learning, assessment, poster presentations, industry engagement.

Communicating assessment criteria is not sufficient for influencing students' approaches to assessment tasks – perspectives from a differential equations class

This report presents the findings of an exploratory study into the perceptions held by students regarding the use of criterion-referenced assessment in an undergraduate differential equations class. Students in the class were largely unaware of the concept of criterion referencing and of the various interpretations that this concept has among mathematics educators. Our primary goal was to investigate whether explicitly presenting assessment criteria to students was useful to them and guided them in responding to assessment tasks. Quantitative data and qualitative feedback from students indicates that while students found the criteria easy to understand and useful in informing them as to how they would be graded, the manner in which they actually approached the assessment activity was not altered as a result of the use of explicitly communicated grading criteria.

Sustaining teacher professionalism in the context of standards referenced assessment reform

Standards referenced reform, tied to reporting, engages directly with assessment issues related to accountability. Assessment is the key to good education and is inseparable from curriculum. In an accountability context, standards are used as a lever to improve the reliability and consistency of teacher judgement; and classroom evidence is used by education systems for reporting and tracking achievement over time. Assessment is thus a powerful driver for change and is at the heart of the teaching-learning dynamic. The relationship between the learner, learning and assessment needs to be kept central and the idea of teacher empowerment is fundamental. This chapter is a call to honour and sustain teacher professionalism through educative forms of school-based and teacher-led evaluation, assessment and communities of judgement practice. It supports the argument for a central place for classroom assessment in the role of assessment in educational accountability...

Towards improving public understanding of judgement practice in standards-referenced assessment : an Australian perspective

Internationally there has been a move towards standards-referenced assessment with countries such as Australia developing a National Curriculum and Achievement Standards, New Zealand adopting National Standards for literacy and numeracy that involve schools making and reporting judgements about the reading, writing and mathematics achievement of children up to Year 8 (the end of primary school) and in Canada, classroom assessment standards aimed at the improvement of assessment practice of K-12 education are being formulated. Standards-driven reform has major implications for teachers’ work. The consequences of adopting a standards-driven approach to educational change by systems are often under-estimated with the unintended effects not fully understood by either the policy writers, and the public, including parents. It is for these reasons that the contention developed in this article relates to the teacher’s role, which it is argued remains central to policy focused on the improvement of the quality of education and educational standards.

Levelling expectations across different years in an undergraduate degree to support work-related learning : evidence for a multi-pronged approach.

One of the ways in which university departments and faculties can enhance the quality of learning and assessment is to develop a ‘well thought out criterion‐referenced assessment system’ (Biggs, 2003, p. 271). In designing undergraduate degrees (courses) this entails making decisions about the levelling of expectations across different years through devising objectives and their corresponding criteria and standards: a process of alignment analogous to what happens in unit (subject) design. These decisions about levelling have important repercussions in terms of supporting students’ work‐related learning, especially in relation to their ability to cope with the increasing cognitive and skill demands made on them as they progress through their studies. They also affect the accountability of teacher judgments of students’ responses to assessment tasks, achievement of unit objectives and, ultimately, whether students are awarded their degrees and are sufficiently prepared for the world of work. Research reveals that this decision‐making process is rarely underpinned by an explicit educational rationale (Morgan et al, 2002). The decision to implement criterion referenced assessment in an undergraduate microbiology degree was the impetus for developing such a rationale because of the implications for alignment, and therefore ‘levelling’ of expectations across different years of the degree. This paper provides supporting evidence for a multi‐pronged approach to levelling, through backward mapping of two revised units (foundation and exit year). This approach adheres to the principles of alignment while combining a work‐related approach (via industry input) with the blended disciplinary and learner‐centred approaches proposed by Morgan et al. (2002). It is suggested that this multi‐pronged approach has the potential for making expectations, especially work‐related ones across different year levels of degrees, more explicit to students and future employers.

Orals ain’t orals : the role of prepared oral presentations in the classroom and beyond. How do instruction and feedback practices guide delivery choices?

Despite an ostensibly technology-driven society, the ability to communicate orally continues to feature as an essential ability for students at school and university, as it is for graduates in the workplace. Pedagogically, one rationale is that the need to develop effective oral communication skills is tied to life-long learning which includes successful participation in future work-related tasks. One tangible way that educators have assessed proficiency in the area of communication is through prepared oral presentations. While much of the literature uses the terms 'oral communication' and 'oral presentation' interchangeably, some writers question the role more formal presentations play in the overall development of oral communication skills. However, such formal speaking tasks continue to be a recognised assessment practice in both the secondary school and academy, and, therefore, worthy of further investigation. Adding to the discussion, this thesis explores the knowledge and skills students bring into the academy from previous educational experiences. It examines some of the teaching and assessment methods used in secondary schools to develop oral communication skills through the use of formal oral presentations. Specifically, it investigates criterion-referenced assessment sheets and how these tools are used as a form of instruction, as well as their role and effectiveness in the evaluation of student ability. The focus is on the student's perspective and includes 12 semi-structured interviews with school students. The purpose of this thesis is to explore key thematics underpinning oral communication and to identify tensions between expectations and practice. While acknowledging the breadth and depth of material available under the heading of 'communication theory', this study specifically draws on an expanded view of the rhetorical tradition to fully interrogate the assumptions supporting the practice of assessing oral presentations. Finally, this thesis recommends reconnecting with an updated understanding of rhetoric as a way of assisting in the development of expressive, articulate and discerning communicators.

Higher order thinking in chemistry curriculum and its assessment

Higher-order thinking has featured persistently in the reform agenda for science education. The intended curriculum in various countries sets out aspirational statements for the levels of higher-order thinking to be attained by students. This study reports the extent to which chemistry examinations from four Australian states align and facilitate the intended higher-order thinking skills stipulated in curriculum documents. Through content analysis, the curriculum goals were identified for each state and compared to the nature of question items in the corresponding examinations. Categories of higher-order thinking were adapted from the OECD’s PISA Science test to analyze question items. There was considerable variation in the extent to which the examinations from the states supported the curriculum intent of developing and assessing higher-order thinking. Generally, examinations that used a marks-based system tended to emphasize lower-order thinking, with a greater distribution of marks allocated for lower-order thinking questions. Examinations associated with a criterion-referenced examination tended to award greater credit for higher-order thinking questions. The level of complexity of chemistry was another factor that limited the extent to which examination questions supported higher-order thinking. Implications from these findings are drawn for the authorities responsible for designing curriculum and assessment procedures and for teachers.