Changing assessment practices : the case for online moderation

Online moderation meetings have the potential to support the collaborative professional development of teachers, and the formation of a common understanding of what denotes quality in student work in a standards based assessment system. In doing so systemic calls for consistency across education systems are also being met. In this paper a case for employing online moderation meetings is developed through recourse to the demands of learning in the twenty-first century and the place of assessment within those discourses. It is argued that empirical data is needed on the efficacy of online moderation meetings to guide future practice as the use of information and communication technologies increases in education systems. Online moderation is one way of gathering teachers across vast distances to share their understandings and develop common meanings of assessment. While it is suggested that online moderation is one possible procedure to meet systemic requirements and support teachers’ professional collaboration, the implementation of such a system also introduces new challenges for schools and teachers. Meeting online to discuss professional understandings is a new way of operating for teachers and involves technology that has not yet been fully utilised within education departments. Issues such as the types of interactions that are afforded within such an environment, as well as technical operating problems that occur when using technology impact on the employment of online meetings. Online moderation meetings while potentially solving the issue of developing common understandings across an entire department also pose new issues to be resolved. There is a need for research into the efficacy of online moderation meetings so that future policy decisions may be based on sound empirical data. It is imperative that as new ways of knowing and acting are incorporated into school curriculum and pedagogy, assessment practices are also aligned. Online moderation meetings can support such practices by enabling teachers to communicate with a wider and more diverse group of teachers to establish common understandings.

An investigation into online moderation

The purpose of this paper is to understand how teachers’ identity as an assessor in a standards-referenced assessment system may be developed through their participation in online social moderation meetings. In these meetings teachers negotiate and share their understandings of assessment standards and judgement decisions. In particular, the paper focuses on the relationship between the technology, the moderation processes and teachers’ development in this assessment system. This paper draws on sociocultural theories of learning to analyse the qualitative data collected through observations of eleven online moderation meetings and interviews of the teachers involved in these meetings. The results provide insights into the mediating role of the technology with regard to teachers’ development of shared meanings and common practices within a standards-referenced assessment system.

Learning as identity and practice through involvement in online moderation

For teachers working in a standards-based assessment system, professional conversations through organised social moderation meetings are a vital element. This qualitative research investigated the learning that occurred as a result of online moderation discussions. Findings illustrate how participating in social moderation meetings in an online context can support teachers to understand themselves as assessors, and can provide opportunities for teachers to imagine possibilities for their teaching that move beyond the moderation practice.

The impact of high stakes testing: the Australian story

High stakes testing in Australia was introduced in 2008 by way of the National Assessment Program – Literacy and Numeracy (NAPLAN). Currently, every year all students in Years 3, 5, 7 and 9 are assessed on the same days using national tests in Reading, Writing, Language Conventions (Spelling, Grammar and Punctuation) and Numeracy. In 2010 the NAPLAN results were published on the Federal Government MySchool website. The impact of these high stakes tests on jurisdictions, school principals, parents and students is considered in this article. We draw on reported observations from the Australian Primary Principals Association during 2009–10 testing periods across the country and published Australian research on the impact of high stakes literacy and numeracy testing. We also examine alternative approaches that include the use of assessment evidence for learning improvement purposes and for accountability purposes. In considering alternatives to the current large-scale testing approach we draw on key insights from research on teacher judgement, achievement standards and social moderation in the context of national curriculum and assessment reform in support of the suggested directions forward.

Moderation practices in a faculty of education : the four discourse model

Moderation of student assessment is a critical component of teaching and learning in contemporary universities. In Australia, moderation is mandated through university policies and through the new national university accreditation authority, Tertiary Education Quality and Standards Agency which began operations in late January 2012 (TEQSA, 2012). The TEQSA requirement to declare details of moderation and any other arrangements used to support consistency and reliability of assessment and grading across each subject in the course of study is a radical step intended to move toward heightened accountability and greater transparency in the tertiary sector as well as entrenching evidence-based practice in the management of Australian academic programs. In light of this reform, the purpose of this project was to investigate and analyse current moderation practices operating within a faculty of education at a large urban university in Queensland, Australia. This qualitative study involved interviews with the unit coordinators (n=21) and tutors (n=8) of core undergraduate education units and graduate diploma units within the faculty. Four distinct discourses of moderation that academics drew on to discuss their practices were identified in the study. These were: equity, justification, community building, and accountability. These discourses, together with recommendations for changes to moderation practices are discussed in this paper.

What’s the point of moderation? A discussion of the purposes achieved through contemporary moderation practices

An increasingly regulated higher education sector is renewing its attention to those activities referred to as ‘moderation’ in its efforts to ensure that judgements of student achievement are based on appropriate standards. Moderation practices conducted throughout the assessment process can result in purposes identified as equity, justification, accountability and community building. This paper draws on the limited studies of moderation and wider relevant research on judgement, standards and professional learning to test commonly used moderation practices against these identified purposes. The paper concludes with recommendations for maximising the potential of moderation practices to establish and maintain achievement standards.

Portfolio assessment

At a time when global uncertainty is paramount and when a new form or re-form of curriculum is emerging – with content displaced by skills and knowledge acquisition by learning - assessment, too, begins to take on a new from or re-form. The focus for assessment has shifted to that which engages and promotes learning as s process rather than an assessment that focuses solely on measuring and reporting learning as product or score. The use of the portfolio for assessment offers the potential for the process and progress – integral to learning - to be included.

Integration of CENA practice standards for postgraduate emergency nursing assessment

The use of professional competency standards to assess postgraduate nursing student’s clinical performance has been in place since 2004, at the Queensland University of Technology, School of Nursing & Midwifery (SONAM) when the Graduate Certificate in Emergency Nursing degree commenced. Emergency nursing students were assessed in their workplace, using a Clinical Performance Appraisal Tool or CPAT which was based on the Australian College of Critical Care Nurses (ACCCN) Competency Standards. With the subsequent formation of a separate Emergency Nursing Course advisory group in 2007, there was a review of clinical assessment course component. The release of the 2008 CENA revised Practice Standards for the Emergency Nursing Specialist’s, led to the emergency nursing course advisory committee supporting the integration of the CENA practice standards for assessment of emergency nurses in preference to the less relevant ACCCN competency standards. The SONAM emergency nursing study area team commenced the phasing in and progression of the CENA practice standards across the two Graduate Certificate units, and Graduate Diploma and Master of Nursing (emergency) clinical major options in 2009. As some units undertaken in the degree are available to nurses in other disciplines a separate CPAT was devised for the clinical assessments according to speciality context. The team has had to carefully consider how the professional standards are integrated into the teaching and assessment of the unit and not just applied instead of the ACCCN competency standards. Professional standards for the emergency context has also helped tailor course content and learning outcomes to be relevant across a number of emergency nursing contexts in Australia. The assessment of the CPAT is undertaken at the workplace by QUT appointed clinical lecturers. Clinical lecturers need to apply and have suitable postgraduate qualification to undertake the position. The clinical lecturer support role is well established at QUT. The integration of the new CENA practice standards has necessitated a review of the postgraduate assessment of emergency nurses. A clinical lecturer workshop has been organised to review role, scope and how to utilise the new look CENA based CPAT, clinical assessment format.

Teacher professional standards : mirage or miracle cure - an archaeology of professionalism in education

This thesis is a problematisation of the development and implementation of professional standards as the mechanism to enhance professionalism and teacher quality in the teaching force within Australia and, more specifically, Queensland. Drawing on tools from Foucauldian archaeological analysis, the dominant discourses of professionalism from the academic literature, Australian federal and state policy documents and narratives from Queensland teachers are examined. These data sets are then cross referenced, analysing the intersections and divergences between the different texts. Findings suggest that through policy, political strategy and derisory statements from various authoritative voices, the managerial discourse of professionalism through professional standards documents has been unduly privileged as a means of regulating teachers, despite the fact that teachers themselves do not share this dominant notion of professionalism. The teachers in this study proffer ‘new classical-practical professionalism’ as a counter discourse, or discourse of resistance, to managerialism. However, an application of Foucault’s theorisations on power-knowledge reveals that their spoken discourses mean they are in fact yielding to the discourse of professional standards as docile bodies.

Development and implementation of the Australian universities radiation therapy student clinical assessment form

Purpose: Prior to 2009, one of the problems faced by radiation therapists who supervised and assessed students on placement in Australian clinical centres, was that each of the six Australian universities where Radiation Therapy (RT) programmes were conducted used different clinical assessment and reporting criteria. This paper describes the development of a unified national clinical assessment and reporting form that was implemented nationally by all six universities in 2009. Methods: A four phase methodology was used to develop the new assessment form and user guide. Phase 1 included university consensus around domains of student practice and assessment, and alignment with national competency standards; Phase 2 was a national consensus workshop attended by radiation therapists involved in student supervision and assessment; Phase 3 was an action research re-iterative Delphi technique involving two rounds of a mail-out to gain further expert consensus; and stage 4 was national piloting of the developed assessment form. Results: The new assessment form includes five main domains of practice and 19 sub-domain criteria which students are assessed against during placement. Feedback from the pilot centre participants was positive, with the new form being assessed to be comprehensive and complemented by the accompanying user guide. Conclusion: The new assessment form has improved both the formative and summative assessment of students on placement, as well as enhancing the quality of feedback to students and the universities. The new national form has high acceptance from the Australian universities and has been subject to wide review by the profession.

Using annotations to inform an understanding of achievement standards

The implementation of a new national curriculum and standards-referenced assessment in Australia has been an opportunity and a challenge for teacher assessment practices. In this case study of teachers in two Queensland schools, we explore how annotating student or exemplar assessment tasks could support teacher assessment practice. Three learning conversations between the researchers and the teacher teams are interpreted through the lens of Bernstein’s (1999) horizontal and vertical discourses to understand the complexities of coming to know an assessment standard. The study contributes to the literature on the use of annotations by exploring how teachers negotiated the purposes and processes of annotation, how annotating student work or exemplars before teaching commenced supported teachers to experience greater clarity about assessment standards and, finally, some of the tensions experienced by the teachers as they considered this practice within the practicalities of their daily work.

Teachers using annotations to engage students in assessment conversations : recontextualising knowledge

Assessment for Learning practices with students such as feedback, and self- and peer assessment are opportunities for teachers and students to develop a shared understanding of how to create quality learning performances. Quality is often represented through achievement standards. This paper explores how primary school teachers in Australia used the process of annotating work samples to develop shared understanding of achievement standards during their curriculum planning phase, and how this understanding informed their teaching so that their students also developed this understanding. Bernstein's concept of the pedagogic device is used to identify the ways teachers recontextualised their assessment knowledge into their pedagogic practices. Two researchers worked alongside seven primary school teachers in two schools over a year, gathering qualitative data through focus groups and interviews. Three general recontextualising approaches were identified in the case studies; recontextualising standards by reinterpreting the role of rubrics, recontextualising by replicating the annotation process with the students and recontextualising by reinterpreting practices with students. While each approach had strengths and limitations, all of the teachers concluded that annotating conversations in the planning phase enhanced their understanding, and informed their practices in helping students to understand expectations for quality.

Improving criteria and feedback in student assessment in law

Spurred on by both the 1987 Pearce Report1 and the general changes to higher education spawned by the “Dawkins revolution” from 1988, there has been much critical self-evaluation leading to profound improvements to the quality of teaching in Australian law schools.2 Despite the changes there are still areas of general law teaching practice which have lagged behind recent developments in our understanding of what constitutes high quality teaching. One such area is assessment criteria and feedback. The project Improving Feedback in Student Assessment in Law is an attempt to remedy this. It aims to produce a manual containing key principles for the design of assessment and the provision of feedback, with practical yet flexible ideas and illustrations which law teachers may adopt or modify. Most of the examples have been developed by teachers at the University of Melbourne Law School. The project was supported in 1996 by a Committee for the Advancement of University Teaching grant and the manual will be published late in 1997.3 This note summarises the core principles which are elaborated further in the manual.

Towards a theoretical framework for online professional discussions

Online interactions are becoming commonplace for a multitude of educational purposes. Each context presents a unique and dynamic mix of variables that combine to shape the practice and the identities of those involved. In this article, sociocultural theories of learning and sociocultural theories of technology are explored as a way to view and to map the complex interactions that can occur. The case of synchronous online moderation meetings are used as an example of the combination of variables that can impact on the development of shared understandings of a practice. Online moderation can involve teachers from geographically diverse areas discussing and negotiating their judgement decisions. These discussions represent an intersection of a national curriculum, standards-referenced assessment, moderation protocols, site-specific practices and understandings, and individual teachers’ knowledges and histories. It is suggested that the proposed theoretical combination addresses some of the limitations of each of the theories when investigating such a dynamic context. As higher education moves into increasing use of online modes of communication and a higher level of accountability the relevance of this discussion to higher education is evident.

Spatialised metaphors of practice: How teacher educators engage with professional standards for teachers

Preservice teacher educators, both nationally and internationally, must negotiate a plethora of expectations including using Professional Standards to enhance teacher quality. In Australia, the recent Teacher Education Ministerial Advisory Group (TEMAG) report highlighted weak application of Standards in Initial Teacher Education (ITE). However, recent findings suggest that education stakeholders feel positive about the implementation of the Australian Professional Standards for Teachers (APSTs). This study responds to these differing viewpoints by exploring how teacher educators in a large metropolitan university in Australia account for the use of Standards in their work. Discourse analytic techniques in conjunction with socio-spatial theory make visible particular metaphors of practice as teacher educators negotiate the real-and-imagined spaces of regulated teacher education programs. The findings highlight the importance of investigating the utility of Standards in the lived experiences of teacher educators, as they are responsible for preparing quality, classroom ready graduates.