Developing shared understandings of standards-based assessment : online moderation practices across geographically diverse contexts

This thesis investigates the place of online moderation in supporting teachers to work in a system of standards-based assessment. The participants of the study were fifty middle school teachers who met online with the aim of developing consistency in their judgement decisions. Data were gathered through observation of the online meetings, interviews, surveys and the collection of artefacts. The data were viewed and analysed through sociocultural theories of learning and sociocultural theories of technology, and demonstrates how utilising these theories can add depth to understanding the added complexity of developing shared meaning of standards in an online context. The findings contribute to current understanding of standards-based assessment by examining the social moderation process as it acts to increase the reliability of judgements that are made within a standards framework. Specifically, the study investigates the opportunities afforded by conducting social moderation practices in a synchronous online context. The study explicates how the technology affects the negotiation of judgements and the development of shared meanings of assessment standards, while demonstrating how involvement in online moderation discussions can support teachers to become and belong within a practice of standards-based assessment. This research responds to a growing international interest in standards-based assessment and the use of social moderation to develop consistency in judgement decisions. Online moderation is a new practice to address these concerns on a systemic basis.

Sustaining teacher professionalism in the context of standards referenced assessment reform

Standards referenced reform, tied to reporting, engages directly with assessment issues related to accountability. Assessment is the key to good education and is inseparable from curriculum. In an accountability context, standards are used as a lever to improve the reliability and consistency of teacher judgement; and classroom evidence is used by education systems for reporting and tracking achievement over time. Assessment is thus a powerful driver for change and is at the heart of the teaching-learning dynamic. The relationship between the learner, learning and assessment needs to be kept central and the idea of teacher empowerment is fundamental. This chapter is a call to honour and sustain teacher professionalism through educative forms of school-based and teacher-led evaluation, assessment and communities of judgement practice. It supports the argument for a central place for classroom assessment in the role of assessment in educational accountability...

Faculty mentoring, evidence-based assessment, and student learning : an Australian exploration of American initiatives

There have been many improvements in Australian engineering education since the 1990s. However, given the recent drive for assuring the achievement of identified academic standards, more progress needs to be made, particularly in the area of evidence-based assessment. This paper reports on initiatives gathered from the literature and engineering academics in the USA, through an Australian National Teaching Fellowship program. The program aims to establish a process to help academics in designing and implementing evidence-based assessments that meet the needs of not only students and the staff that teach them, but also industry as well as accreditation bodies. The paper also examines the kinds and levels of support necessary for engineering academics, especially early career ones, to help meet the expectations of the current drive for assured quality and standards of both research and teaching. Academics are experiencing competing demands on their time and energy with very high expectations in research performance and increased teaching responsibilities, although many are researchers who have not had much pedagogic training. Based on the literature and investigation of relevant initiatives in the USA, we conducted interviews with several identified experts and change agents who have wrought effective academic cultural change within their institutions and beyond. These reveal that assuring the standards and quality of student learning outcomes through evidence-based assessments cannot be appropriately addressed without also addressing the issue of pedagogic training for academic staff. To be sustainable, such training needs to be complemented by a culture of on-going mentoring support from senior academics, formalised through the university administration, so that mentors are afforded resources, time, and appropriate recognition.

Education for all : providing curriculum for students with disabilities in a standards-based environment : implications for Queensland schools

This report investigates lessons learned by educators in the United States when providing a standards-based curriculum for all students including Students with Disabilities (SWD). Assumptions about implementation of these lessons are then made to the Queensland school system. Queensland mainstream schools currently provide a standards-based curriculum for over sixteen thousand-four hundred students with mild-moderate disabilities and appear to be challenged by this new educational reform and its implications to school and teacher practices, beliefs and attitudes. The analysis of US research, literature and educational policy for this report, has provided some implications for Queensland schools in the areas of student participation, achievement and curriculum planning to provide an “education for all”. The analysis and comparison of legislation and policy, which demonstrates some significant similarities, provides greater validity for the application of lessons learned in the United States to the Queensland context. The key findings about lessons learned provides Queensland schools with some assumptions as to why and how they need to refocus school leader and teachers’ practices, beliefs and attitudes to provide an “education for all”. These lessons infer that school leaders and teachers to explicitly focus on equity, expectation, accountability, performance, alignment and collaboration so that effective curriculum is provided for SWD, indeed all students, in the Queensland standards-based curriculum environment.

Changing assessment practices : the case for online moderation

Online moderation meetings have the potential to support the collaborative professional development of teachers, and the formation of a common understanding of what denotes quality in student work in a standards based assessment system. In doing so systemic calls for consistency across education systems are also being met. In this paper a case for employing online moderation meetings is developed through recourse to the demands of learning in the twenty-first century and the place of assessment within those discourses. It is argued that empirical data is needed on the efficacy of online moderation meetings to guide future practice as the use of information and communication technologies increases in education systems. Online moderation is one way of gathering teachers across vast distances to share their understandings and develop common meanings of assessment. While it is suggested that online moderation is one possible procedure to meet systemic requirements and support teachers’ professional collaboration, the implementation of such a system also introduces new challenges for schools and teachers. Meeting online to discuss professional understandings is a new way of operating for teachers and involves technology that has not yet been fully utilised within education departments. Issues such as the types of interactions that are afforded within such an environment, as well as technical operating problems that occur when using technology impact on the employment of online meetings. Online moderation meetings while potentially solving the issue of developing common understandings across an entire department also pose new issues to be resolved. There is a need for research into the efficacy of online moderation meetings so that future policy decisions may be based on sound empirical data. It is imperative that as new ways of knowing and acting are incorporated into school curriculum and pedagogy, assessment practices are also aligned. Online moderation meetings can support such practices by enabling teachers to communicate with a wider and more diverse group of teachers to establish common understandings.

Open standards-based system integration for asset management decision support

Over the last decade, system integration has grown in popularity as it allows organisations to streamline business processes. Traditionally, system integration has been conducted through point-to-point solutions – as a new integration scenario requirement arises, a custom solution is built between the relevant systems. Bus-based solutions are now preferred, whereby all systems communicate via an intermediary system such as an enterprise service bus, using a common data exchange model. This research investigates the use of a common data exchange model based on open standards, specifically MIMOSA OSA-EAI, for asset management system integration. A case study is conducted that involves the integration of processes between a SCADA, maintenance decision support and work management system. A diverse number of software platforms are employed in developing the final solution, all tied together through MIMOSA OSA-EAI-based XML web services. The lessons learned from the exercise are presented throughout the paper.

Standards-driven reform years 1-10 : moderation an optional extra?

While externally moderated standards-based assessment has been practised in Queensland senior schooling for more than three decades, there has been no such practice in the middle years. With the introduction of standards at state and national levels in these years, teacher judgement as developed in moderation practices is now vital. This paper argues, that in this context of assessment reform, standards intended to inform teacher judgement and to build assessment capacity are necessary but not sufficient for maintaining teacher and public confidence in schooling. Teacher judgement is intrinsic to moderation, and to professional practice, and can no longer remain private. Moderation too is intrinsic to efforts by the profession to realise judgements that are defensible, dependable and open to scrutiny. Moderation can no longer be considered an optional extra and requires system-level support especially if, as intended, the standards are linked to system-wide efforts to improve student learning. In presenting this argument we draw on an Australian Research Council funded study with key industry partners (the Queensland Studies Authority and the National Council for Curriculum and Assessment of the Republic of Ireland). The data analysed included teacher interview data and additional teacher talk during moderation sessions. These were undertaken during the initial phase of policy development. The analysis identified those issues that emerge in moderation meetings that are designed to reach consistent, reliable judgements. Of interest are the different ways in which teachers talked through and interacted with one another to reach agreement about the quality of student work in the application of standards. There is evidence of differences in the way that teachers made compensations and trade-offs in their award of grades, dependent on the subject domain in which they teach. This article concludes with some empirically derived insights into moderation practices as policy and social events.

The development of teacher assessment identity through participation in online moderation

Teachers’ professional conversations regarding the qualities evidenced in student work provide opportunities to develop a shared understanding of achievement standards. This research investigates social moderation conducted in a synchronous online mode as a specific form of professional conversation. The discussion considers the different factors that influenced these conversations which included the technologic medium of the meeting. The focus of the discussion is how participation in online moderation can support teachers to develop an assessment identity as one who works within a standards-based assessment system. Qualitative data were gathered from middle school teachers from different year levels, in different curriculum areas, in diverse geographic locations, and in a range of sociocultural contexts within Queensland, Australia. Analysis of the data through a sociocultural lens of becoming suggests that participation in online moderation, while challenging for teachers, can also provide opportunities to construct and to negotiate an identity as an assessor of student work.

Towards improving public understanding of judgement practice in standards-referenced assessment : an Australian perspective

Internationally there has been a move towards standards-referenced assessment with countries such as Australia developing a National Curriculum and Achievement Standards, New Zealand adopting National Standards for literacy and numeracy that involve schools making and reporting judgements about the reading, writing and mathematics achievement of children up to Year 8 (the end of primary school) and in Canada, classroom assessment standards aimed at the improvement of assessment practice of K-12 education are being formulated. Standards-driven reform has major implications for teachers’ work. The consequences of adopting a standards-driven approach to educational change by systems are often under-estimated with the unintended effects not fully understood by either the policy writers, and the public, including parents. It is for these reasons that the contention developed in this article relates to the teacher’s role, which it is argued remains central to policy focused on the improvement of the quality of education and educational standards.

Work-based assessment : qualitative perspectives of novice nutrition and dietetics educators

Background The assessment of competence for health professionals including nutrition and dietetics professionals in work-based settings is challenging. The present study aimed to explore the experiences of educators involved in the assessment of nutrition and dietetics students in the practice setting and to identify barriers and enablers to effective assessment. Methods A qualitative research approach using in-depth interviews was employed with a convenience sample of inexperienced dietitian assessors. Interviews explored assessment practices and challenges. Data were analysed using a thematic approach within a phenomenological framework. Twelve relatively inexperienced practice educators were purposefully sampled to take part in the present study. Results Three themes emerged from these data. (i) Student learning and thus assessment is hindered by a number of barriers, including workload demands and case-mix. Some workplaces are challenged to provide appropriate learning opportunities and environment. Adequate support for placement educators from the university, managers and their peers and planning are enablers to effective assessment. (ii) The role of the assessor and their relationship with students impacts on competence assessment. (iii) There is a lack of clarity in the tasks and responsibilities of competency-based assessment. Conclusions The present study provides perspectives on barriers and enablers to effective assessment. It highlights the importance of reflective practice and feedback in assessment practices that are synonymous with evidence from other disciplines, which can be used to better support a work-based competency assessment of student performance.

An experimental based assessment of the deviation of the bearing characteristic frequencies

Slippage in the contact roller-races has always played a central role in the field of diagnostics of rolling element bearings. Due to this phenomenon, vibrations triggered by a localized damage are not strictly periodic and therefore not detectable by means of common spectral functions as power spectral density or discrete Fourier transform. Due to the strong second order cyclostationary component, characterizing these signals, techniques such as cyclic coherence, its integrated form and square envelope spectrum have proven to be effective in a wide range of applications. An expert user can easily identify a damage and its location within the bearing components by looking for particular patterns of peaks in the output of the selected cyclostationary tool. These peaks will be found in the neighborhood of specific frequencies, that can be calculated in advance as functions of the geometrical features of the bearing itself. Unfortunately the non-periodicity of the vibration signal is not the only consequence of the slippage: often it also involves a displacement of the damage characteristic peaks from the theoretically expected frequencies. This issue becomes particularly important in the attempt to develop highly automated algorithms for bearing damage recognition, and, in order to correctly set thresholds and tolerances, a quantitative description of the magnitude of the above mentioned deviations is needed. This paper is aimed at identifying the dependency of the deviations on the different operating conditions. This has been possible thanks to an extended experimental activity performed on a full scale bearing test rig, able to reproduce realistically the operating and environmental conditions typical of an industrial high power electric motor and gearbox. The importance of load will be investigated in detail for different bearing damages. Finally some guidelines on how to cope with such deviations will be given, accordingly to the expertise obtained in the experimental activity.