Standards, teacher judgement and moderation in contexts of national curriculum and assessment reform

This paper puts forward a proposal for reviewing the role and purpose of standards in the context of national curriculum and assessment reform more generally. It seeks to commence the much-needed conversation about standards in the work of teachers as distinct from large-scale testing companies and the policy personnel responsible for reporting. Four key conditions that relate to the effective use of standards to measure improvement and support learning are analysed: clarity about purpose and function; understanding of the representation of standards; moderation practice; and the assessment community. The Queensland experience of the use of standards, teacher judgement and moderation is offered to identify what is educationally preferable in terms of their use and their relationships to curriculum, improvement and accountability. The article illustrates how these practices have recently been challenged by emerging political constraints related to the Australian Government’s implementation of national testing and national partnership funding arrangements tied to the performance of students at or below minimum standards.

Role and purpose of standards in the context of national curriculum and assessment reform for accountability, improvement and equity in student learning

In this article our starting point is the current context of national curriculum change and intense speculation about the assessment, standards and reporting. It is written against a background of accountability measures and improvement imperatives, and focuses attention on standards as offering representations of quality. We understand standards to be constructs that aim to achieve public credibility and utility. Further, they can be examined for the purposes they seek to serve and also their expected functions. Fitness for purpose is therefore a useful notion in considering the nature of standards. Our interest in the discussion is the ‘fit’ between how standards are formulated and how they are used in practice, by whom and for what purposes. A related interest is in the matter of how standards can be harnessed to realise improvement.

Mild systemic hypoxia and photopic visual field sensitivity

Purpose: Flickering stimuli increase the metabolic demand of the retina,making it a sensitive perimetric stimulus to the early onset of retinal disease. We determine whether flickering stimuli are a sensitive indicator of vision deficits resulting from to acute, mild systemic hypoxia when compared to standard static perimetry. Methods: Static and flicker visual perimetry were performed in 14 healthy young participants while breathing 12% oxygen (hypoxia) under photopic illumination. The hypoxia visual field data were compared with the field data measured during normoxia. Absolute sensitivities (in dB) were analysed in seven concentric rings at 1°, 3°, 6°, 10°, 15°, 22° and 30° eccentricities as well as mean defect (MD) and pattern defect (PD) were calculated. Preliminary data are reported for mesopic light levels. Results: Under photopic illumination, flicker and static visual field sensitivities at all eccentricities were not significantly different between hypoxia and normoxia conditions. The mean defect and pattern defect were not significantly different for either test between the two oxygenation conditions. Conclusion: Although flicker stimulation increases cellular metabolism, flicker photopic visual field impairment is not detected during mild hypoxia. These findings contrast with electrophysiological flicker tests in young participants that show impairment at photopic illumination during the same levels of mild hypoxia. Potential mechanisms contributing to the difference between the visual fields and electrophysiological flicker tests including variability in perimetric data, neuronal adaptation and vascular autoregulation, are considered. The data have implications for the use of visual perimetry in the detection of ischaemic/hypoxic retinal disorders under photopic and mesopic light levels.

Standards-driven reform years 1-10 : moderation an optional extra?

While externally moderated standards-based assessment has been practised in Queensland senior schooling for more than three decades, there has been no such practice in the middle years. With the introduction of standards at state and national levels in these years, teacher judgement as developed in moderation practices is now vital. This paper argues, that in this context of assessment reform, standards intended to inform teacher judgement and to build assessment capacity are necessary but not sufficient for maintaining teacher and public confidence in schooling. Teacher judgement is intrinsic to moderation, and to professional practice, and can no longer remain private. Moderation too is intrinsic to efforts by the profession to realise judgements that are defensible, dependable and open to scrutiny. Moderation can no longer be considered an optional extra and requires system-level support especially if, as intended, the standards are linked to system-wide efforts to improve student learning. In presenting this argument we draw on an Australian Research Council funded study with key industry partners (the Queensland Studies Authority and the National Council for Curriculum and Assessment of the Republic of Ireland). The data analysed included teacher interview data and additional teacher talk during moderation sessions. These were undertaken during the initial phase of policy development. The analysis identified those issues that emerge in moderation meetings that are designed to reach consistent, reliable judgements. Of interest are the different ways in which teachers talked through and interacted with one another to reach agreement about the quality of student work in the application of standards. There is evidence of differences in the way that teachers made compensations and trade-offs in their award of grades, dependent on the subject domain in which they teach. This article concludes with some empirically derived insights into moderation practices as policy and social events.

Public regulatory reform and management earnings forecasts in a low private litigation environment

We examine the impact of continuous disclosure regulatory reform on the likelihood, frequency and qualitative characteristics of management earnings forecasts issued in New Zealand’s low private litigation environment. Using a sample of 720 earnings forecasts issued by 94 firms listed on the New Zealand Exchange before and after the reform (1999–2005), we provide strong evidence of significant changes in forecasting behaviour in the post-reform period. Specifically, firms were more likely to issue earnings forecasts to pre-empt earnings announcements and, in contrast to findings in other legal settings, those earnings forecasts exhibited higher frequency and improved qualitative characteristics (better precision and accuracy). An important implication of our findings is that public regulatory reforms may have a greater benefit in a low private litigation environment and thus add to the global debate about the effectiveness of alternative public regulatory reforms of corporate requirements.

Thrown in the deep end : newcomers and public sector reform

Triggered by the continuing global financial crisis, most public administration systems internationally are reviewing their ability to meet public expectations in more challenging strategic environments, while satisfying the pressure from their political masters to drive down the costs of public administration. Consequently public sector organizations are under constant pressure to reform to meet not only the global economic challenges, but the need for more responsive government (Brown et al 2003). Doyle et al (2000) propose that organizational change is seldom well managed, but that the public sector faces greater difficulty in implementing corporate change than the private sector because of its unique environment, e.g. the need to deliver bureaucratically impartial outcomes. The scale of the changes required, and the constraints imposed by the context within which these changes need to occur, have intensified the need for capable public sector leadership and management. The types of capability required now extend beyond those typically required in public organizations through the efficiency drive of new public management. Acquiring these capabilities remains a key issue for public organizations. One challenge for public management, then, is leadership and management quality, including the need to recruit externally to refresh, re-energize and change the sector and its individual organizations as well as develop advanced skills among existing senior executives.

Assessment reform and educational change in Australia

This chapter addresses the changing climate of assessment policy and practice in Australia in response to global trends in education and the mounting accountability demands of standards-driven reform. Queensland, a State of Australia, has a tradition of respecting and trusting teacher judgment through the practice of, and policy commitment to, externally moderated school-based assessment. This chapter outlines the global trends in curriculum and assessment reform, and then analyzes the impact of international comparisons on national policy. The creation of the Australian Curriculum, Assessment and Reporting Authority (ACARA) together with the intent of establishing a standards-referenced framework raises tensions and challenges for teachers’ practice. The argument for sustaining confidence in teacher-based assessment is developed with reference to research evidence pertaining to the use of more authentic assessments and moderation practices for the purposes of improving learning, equity and accountability. Evidence is drawn from local studies of teacher judgment practice and used to demonstrate these developments and in so doing illuminate the complex issues of engaging the demands of policy while sustaining confidence in teacher assessment.

Evaluation of financial incentives as a quality improvement strategy in the public hospital context : clinician attitudes, design variables, and economic costs

In 2008, a three-year pilot ‘pay for performance’ (P4P) program, known as ‘Clinical Practice Improvement Payment’ (CPIP) was introduced into Queensland Health (QHealth). QHealth is a large public health sector provider of acute, community, and public health services in Queensland, Australia. The organisation has recently embarked on a significant reform agenda including a review of existing funding arrangements (Duckett et al., 2008). Partly in response to this reform agenda, a casemix funding model has been implemented to reconnect health care funding with outcomes. CPIP was conceptualised as a performance-based scheme that rewarded quality with financial incentives. This is the first time such a scheme has been implemented into the public health sector in Australia with a focus on rewarding quality, and it is unique in that it has a large state-wide focus and includes 15 Districts. CPIP initially targeted five acute and community clinical areas including Mental Health, Discharge Medication, Emergency Department, Chronic Obstructive Pulmonary Disease, and Stroke. The CPIP scheme was designed around key concepts including the identification of clinical indicators that met the set criteria of: high disease burden, a well defined single diagnostic group or intervention, significant variations in clinical outcomes and/or practices, a good evidence, and clinician control and support (Ward, Daniels, Walker & Duckett, 2007). This evaluative research targeted Phase One of implementation of the CPIP scheme from January 2008 to March 2009. A formative evaluation utilising a mixed methodology and complementarity analysis was undertaken. The research involved three research questions and aimed to determine the knowledge, understanding, and attitudes of clinicians; identify improvements to the design, administration, and monitoring of CPIP; and determine the financial and economic costs of the scheme. Three key studies were undertaken to ascertain responses to the key research questions. Firstly, a survey of clinicians was undertaken to examine levels of knowledge and understanding and their attitudes to the scheme. Secondly, the study sought to apply Statistical Process Control (SPC) to the process indicators to assess if this enhanced the scheme and a third study examined a simple economic cost analysis. The CPIP Survey of clinicians elicited 192 clinician respondents. Over 70% of these respondents were supportive of the continuation of the CPIP scheme. This finding was also supported by the results of a quantitative altitude survey that identified positive attitudes in 6 of the 7 domains-including impact, awareness and understanding and clinical relevance, all being scored positive across the combined respondent group. SPC as a trending tool may play an important role in the early identification of indicator weakness for the CPIP scheme. This evaluative research study supports a previously identified need in the literature for a phased introduction of Pay for Performance (P4P) type programs. It further highlights the value of undertaking a formal risk assessment of clinician, management, and systemic levels of literacy and competency with measurement and monitoring of quality prior to a phased implementation. This phasing can then be guided by a P4P Design Variable Matrix which provides a selection of program design options such as indicator target and payment mechanisms. It became evident that a clear process is required to standardise how clinical indicators evolve over time and direct movement towards more rigorous ‘pay for performance’ targets and the development of an optimal funding model. Use of this matrix will enable the scheme to mature and build the literacy and competency of clinicians and the organisation as implementation progresses. Furthermore, the research identified that CPIP created a spotlight on clinical indicators and incentive payments of over five million from a potential ten million was secured across the five clinical areas in the first 15 months of the scheme. This indicates that quality was rewarded in the new QHealth funding model, and despite issues being identified with the payment mechanism, funding was distributed. The economic model used identified a relative low cost of reporting (under $8,000) as opposed to funds secured of over $300,000 for mental health as an example. Movement to a full cost effectiveness study of CPIP is supported. Overall the introduction of the CPIP scheme into QHealth has been a positive and effective strategy for engaging clinicians in quality and has been the catalyst for the identification and monitoring of valuable clinical process indicators. This research has highlighted that clinicians are supportive of the scheme in general; however, there are some significant risks that include the functioning of the CPIP payment mechanism. Given clinician support for the use of a pay–for-performance methodology in QHealth, the CPIP scheme has the potential to be a powerful addition to a multi-faceted suite of quality improvement initiatives within QHealth.

Inciting the social imagination to realising the dream : An unfolding story of reform in a low-socioeconomic school

This paper presentation addresses design-based research that became a catalyst for social change among a disadvantaged school community. The aim of the longitudinal research was to protoype an evidence-based model for whole school digital and print literacy pedagogy renewal among students from low socioeconomic, Indigenous, and migrant backgrounds. Applying Anthony Gidden’s principle of the “duality of structure”, the paper presentation interprets how the collective agency of researchers and the school community began to transform the structural properties of the institution in a two-way dynamism, so that the structural properties of the school were not outside of individual action, but were implicated in its reproduction and transformation.

Young children as regenerative tissue donors : considering the need for legal reform in light of divergent ethical approaches

In Australia, young children who lack decision-making capacity can have regenerative tissue removed to treat another person suffering from a severe or life-threatening disease. While great good can potentially result from this as the recipient’s life may be saved, ethical unease remains over the ‘use’ of young children in this way. This paper examines the ethical approaches that have featured in the debate over the acceptability and limits of this practice, and how these are reflected in Australia’s legal regime governing removal of tissue from young children. This analysis demonstrates a troubling dichotomy within the Australia’s laws that requires decision-makers to adopt inconsistent ethical approaches depending on where a donor child is situated. It is argued that this inconsistency in approach warrants legal reform of this ethically sensitive issue.