Is implementing criterion-referenced assessment worth the effort with gen Y?

Well implemented criterion-referenced assessment (CRA) requires dedicated time and effort, especially in describing realistic expectations of evidence of achievement to students in the form of criteria sheets (or grading rubrics). It is also takes time out of delivering content to teach students how to judge their own work using criteria sheets. In 2007, to engage third year Microbiology students in using criteria sheets for the first time in their degree, we devised an innovative assessment tutorial supported by online resources. We were sceptical of much of the literature that reported ‘agreed’ characteristics of our predominantly gen Y cohort, because of the older ages of the majority of authors. These authors claim gen Y has a propensity for digital media, overconfidence in their own abilities and a collaborative orientation. We rejected this stereotype when developing the tutorial. Evaluations by students were positive and there was no dramatic change to grades for the unit. These results are similar to those in the literature for non gen Y cohorts. This lends support to our claim that giving students control over their own learning, irrespective of their generational label, is worth the time and effort.

Does the summative assessment of real world learning using criterion-referenced assessment need to be discipline specific?

This paper synthesises the existing literature on the contemporary conception of ‘real world’ and compares it with similar notions such as ‘authentic’ and ‘work integrated learning’. While the term ‘real world’ may be partly dependent on the discipline, it does not necessarily follow that the criterion-referenced assessment of ‘real world’ assessment must involve criteria and performance descriptors that are discipline specific. Two examples of summative assessment (court report and trial process exercise) from a final year core subject at the Queensland University of Technology, LWB432 Evidence, emphasise real world learning, are authentic, innovative and better prepare students for the transition into the workplace than more generic forms of assessment such as tutorial participation or oral presentations. The court report requires students to attend a criminal trial in a Queensland Court and complete a two page report on what they saw in practice compared with what they learned in the classroom. The trial process exercise is a 50 minute written closed book activity conducted in tutorials, where students plan questions that they would ask their witness in examination-in-chief, plan questions that they would ask their opponent’s witness in cross-examination, plan questions that they would ask in reexamination given what their opponent asked in cross-examination, and prepare written objections to their opponent’s questions. The trial process exercise simulates the real world, whereas the court report involves observing the real world, and both assessment items are important to the role of counsel. The design of the criterion-referenced assessment rubrics for the court report and trial process exercise is justified by the literature. Notably, the criteria and performance descriptors are not necessarily law specific and this paper highlights the parts that may be easily transferred to other disciplines.

Towards a classification of criterion referenced assessment models in mathematics courses : student and academic perspectives

We present the findings of a study into the implementation of explicitly criterion- referenced assessment in undergraduate courses in mathematics. We discuss students' concepts of criterion referencing and also the various interpretations that this concept has among mathematics educators. Our primary goal was to move towards a classification of criterion referencing models in quantitative courses. A secondary goal was to investigate whether explicitly presenting assessment criteria to students was useful to them and guided them in responding to assessment tasks. The data and feedback from students indicates that while students found the criteria easy to understand and useful in informing them as to how they would be graded, it did not alter the way the actually approached the assessment activity.

Engaging Students in the Implementation of Criterion Referenced Assessment in First Year Law

The Queensland University of Technology (QUT) University Academic Board approved a new QUT Assessment Policy in September 2003, which requires a criterion-referenced approach as opposed to a norm-referenced approach to assessment across the university(QUT,MOPP,2003). In 2004, the QUT Law School embarked upon a process of awareness raising about criterion-referenced assessment amongst staff and from 2004 – 2005 staggered the implementation of criterion-referenced assessment in all first year core undergraduate law units. This paper will briefly discuss the benefits and potential pitfalls of criterion referenced assessment and the context for implementing it in the first year law program, report on student’s feedback on the introduction of criterion referenced assessment and the strategies adopted in 2005 to engage students more fully in criterion referenced assessment processes to enhance their learning outcomes.

Assessing reflection skills in law using criterion-referenced assessment

This paper focuses on the assessment of reflective practice, an issue that has not been fully explored within legal education literature. While the issue of how reflective practice should be taught is one that requires careful consideration, it is beyond the scope of this paper to consider both the teaching and the assessment of reflective practice. Part II of this paper conceptualises reflective practice, and Part III explores the benefits of reflective practice in legal education and the use of reflective writing to assess experiential learning in a legal context. Part IV considers the diverse issues that arise in assessing reflective practice and whether there is an objective method for assessing reflection. Part V of the paper examines the assessment of reflective practice in the context of an exemplar undergraduate law subject that uses a reflective report to assess students’ experiential learning during a court visit.14 Finally, Part VI offers a rubric to facilitate criterion-referenced assessment of reflective practice and thereby provides a framework for assessing reflection skills. It is suggested that the rubric is transferable not only to other law subjects but also to subjects in other disciplines.

Closing the loop 21st century style : providing feedback on written assessment via MP3 recordings

Feedback on student performance, whether in the classroom or on written assignments, enables them to reflect on their understandings and restructure their thinking in order to develop more powerful ideas and capabilities. Research has identified a number of broad principles of good feedback practice. These include the provision of feedback that facilitates the development of reflection in learning; helps clarify what good performance is in terms of goals, criteria and expected standards; provides opportunities to close the gap between current and desired performance; delivers high quality information to students about their learning; and encourages positive motivational beliefs and self-esteem. However, high staff–student ratios and time pressures often result in a gulf between this ideal and reality. Whilst greater use of criteria referenced assessment has enabled an improvement in the extent of feedback being provided to students, this measure alone does not go far enough to satisfy the requirements of good feedback practice. Technology offers an effective and efficient means by which personalised feedback may be provided to students. This paper presents the findings of a trial of the use of the freely available Audacity program to provide individual feedback via MP3 recordings to final year Media Law students at the Queensland University of Technology on their written assignments. The trial has yielded wide acclaim by students as an effective means of explaining the exact reasons why they received the marks they were awarded, the things they did well and the areas needing improvement. It also showed that good feedback practice can be achieved without the burden of an increase in staff workload.

Sustaining teacher professionalism in the context of standards referenced assessment reform

Standards referenced reform, tied to reporting, engages directly with assessment issues related to accountability. Assessment is the key to good education and is inseparable from curriculum. In an accountability context, standards are used as a lever to improve the reliability and consistency of teacher judgement; and classroom evidence is used by education systems for reporting and tracking achievement over time. Assessment is thus a powerful driver for change and is at the heart of the teaching-learning dynamic. The relationship between the learner, learning and assessment needs to be kept central and the idea of teacher empowerment is fundamental. This chapter is a call to honour and sustain teacher professionalism through educative forms of school-based and teacher-led evaluation, assessment and communities of judgement practice. It supports the argument for a central place for classroom assessment in the role of assessment in educational accountability...

Towards improving public understanding of judgement practice in standards-referenced assessment : an Australian perspective

Internationally there has been a move towards standards-referenced assessment with countries such as Australia developing a National Curriculum and Achievement Standards, New Zealand adopting National Standards for literacy and numeracy that involve schools making and reporting judgements about the reading, writing and mathematics achievement of children up to Year 8 (the end of primary school) and in Canada, classroom assessment standards aimed at the improvement of assessment practice of K-12 education are being formulated. Standards-driven reform has major implications for teachers’ work. The consequences of adopting a standards-driven approach to educational change by systems are often under-estimated with the unintended effects not fully understood by either the policy writers, and the public, including parents. It is for these reasons that the contention developed in this article relates to the teacher’s role, which it is argued remains central to policy focused on the improvement of the quality of education and educational standards.

Integrating ‘role play’ in assessment to strengthen professional conduct and accountability of students : a pilot study

BACKGROUND In a process engineering setting, graduates are frequently allocated reviews of existing operations or required to scope new production processes by their supervisors with a view to improving or expanding on operations and overall productivity. These tasks may be carried out in teams and in consultation with the process engineer’s immediate line manager or a more experienced engineer, such as the Production or Maintenance Manager; ultimately reporting to senior management, which is frequently a non-engineer. Although professional skills development is part of engineering curricula, ‘professional conduct’ and ‘accountability’ required for dealing with peers and superiors in industry is not very well addressed at university. Consequently, upon graduation, many students are, in terms of knowledge and experience in this area, underprepared to work effectively in industry settings. PURPOSE The purpose of this study was to develop and implement a role-play scenario within a core 2nd year process engineering unit, so that students could gain knowledge, skills and experience in different aspects (and nuances) of professional conduct and accountability. DESIGN/METHOD In the role-play scenario, students worked in ‘engineering production teams’ to design a process for an iconic Queensland fruitcake and to present their solution and recommendations (culminating in a poster presentation) to an assessment panel consisting of staff, role-playing as, ‘production and plant managers’. Students were assessed on several areas, including professionalism using a criteria referenced assessment guide by a 3-member cross-disciplinary staff panel consisting of a Business Faculty lecturer, an engineer from industry and the lecturer of the Process Engineering unit. Professional conduct and accountability was gauged through direct questioning by the panel. Feedback was also sought from students on various aspects through a survey questionnaire after the role play activity at the end of semester. RESULTS Overall, the role play was very well performed with students achieving an average score of 79.3/100 (distinction grade). Professional conduct as assessed by panel was on average better than scores given for professional accountability (4.0 compared with 3.6 out of 5). Feedback from students indicated that the learning activities had contributed to their overall understanding of the content and the role of process engineers. Industry involvement was rated very highly as contributing to their learning at 4.8 (on Likert scale from 1 – 5) and the poster presentation was rated at 3.6. CONCLUSIONS This pilot study was successful in implementing a new assessment task for modelling professional conduct and accountability within a 2nd year core unit. This task incorporated a role-play activity and there was evidence to suggest that this and associated learning tasks were successful in broadening students’ understanding and skills in this area required for engineering practice. Following feedback given by students and staff, improvements will be made to the nature of the problem, how it is defined, its assessment, and the approach taken in the role-play scenario when the unit is offered in 2014.

Enhancing fuzzy inference system based criterion-referenced assessment with an application

An important and difficult issue in designing a Fuzzy Inference System (FIS) is the specification of fuzzy sets, and fuzzy rules. The aim of this paper is to demonstrate how an additional qualitative information, i.e., monotonicity property, can be exploited and extended to be part of an FIS designing procedure (i.e., fuzzy sets and fuzzy rules design). In this paper, the FIS is employed as an alternative to the use of addition in aggregating the scores from test items/tasks in a Criterion-Referenced Assessment (CRA) model. In order to preserve the monotonicity property, the sufficient conditions of the FIS is proposed. Our proposed FIS based CRA procedure can be viewed as an enhancement for the FIS based CRA procedure, where monotonicity property is preserved. We demonstrate the applicability of the proposed approach with a case study related to a laboratory project assessment task at a university, and the results indicate the usefulness of the proposed approach in the CRA domain.

Cracking the code : a checklist to complement CRAs for first year Justice students

It is well recognised in the literature on first year higher education that there is a need for Universities to provide further support and development in student learning skills and engagement. Assessment and feedback is an area with differing expectations and understandings among academics and students (e.g. AUSSE, CEQ). Consistency and explicitness in academic feedback is fundamental in assisting students in their transition to university education and learning. This poster captures the progress of an 18 month funded by the Faculty of Law Teaching and Learning Grant scheme (QUT). The project sought to develop and trial an assessment checklist/diagnostic tool to accompany Criteria Referenced Assessment sheets for students within the School of Justice, Law Faculty, Queensland University of Technology (QUT).The checklist was trialled across four units in the School of Justice (Law faculty) amongst an estimated cohort of over 600 students undertaking single and dual degrees.