Challenging teachers’ assumptions in an era of curriculum and assessment change

Significant responsibility has been given to schools and sectors to interpret and plan for assessment within the Australian Curriculum. As schools take this opportunity to review and renew their school curriculum, it is important for teachers and school leaders to take the time to work out whether there are any assessment myths lurking in the conversations or assumptions that need to be challenged. Outdated myths or cultural narratives of learning can limit our thinking and student learning, without us being aware of it.

Patterns of resistance in managing assessment change

Achieving change in assessment practices in higher education is difficult. One of the reasons for this is resistance among those responsible for teaching and assessing. This paper seeks to explore this resistance through an analysis of staff dialogue during a major attempt to change the assessment practices at one institution. An institution-wide intervention to pilot new assessment practices was initiated, involving 35 academics across 12 departments. This paper reports on patterns emerging in dialogue about this among academics, and between academics and educational development personnel. Findings suggest that resistance is not a unitary concept, but that different stakeholders under varied circumstances express it differently; and that resistance to assessment change is particularly resilient. Implications are discussed in terms of relevance to current theories of higher education assessment and learning change management, as well as the practical considerations of attempts by institutions to engage in assessment change. © 2013 © 2013 Taylor & Francis.

Change of perspective: a measurable and desired outcome of chronic disease self-management intervention programs that violates the premise of preintervention/postintervention assessment

Objective: To determine if participants in chronic disease self-management courses have a change of perspective of their health status (a response shift), and if this is measurable with a paper-based questionnaire.

Methods: Nine items were developed to measure potential benefits of self-management courses. These were based on the constructs of a previous questionnaire, the Health Education Impact Questionnaire (HEI-Q). Cognitive interviews elicited spontaneous statements about the reasons for paper-based answers. Sensitivity, specificity, and overall accuracy of items were calculated using the interview as a relative gold standard. Response shift can be negative (i.e., after the course, participants realize that, before the course, they were worse than they thought they were), positive (i.e., participants now realize they were better than they thought they were), or absent (no change).

Results: Interviews (n = 39) reflected that true response shift occurred in approximately half the replies to questionnaire items. Of these, 31% were negative response shift, 20% were positive response shift. Response shift was absent in 32% of replies. Presence or absence of response shift could not be determined in 17% of replies across items. Significant concordance between questionnaires and cognitive interviews (average overall accuracy 0.79) indicated that the HEI-Q Perspective questionnaire detects response shift in participants of self-management courses. The questionnaire revealed that 87% of participants had response shift in at least 1 item.

Conclusion: This study suggests that preintervention/postintervention assessments of interventions such as self-management courses are confounded by a change in perspective of a large proportion of respondents. It also indicates response shift is a valuable outcome of self-management courses that can be measured with a paper-based questionnaire.

Assessment for learning : a sociocultural approach

Assessment for Learning (AfL) is a title given to classroom evaluative practices that share the purpose of diagnosing and informing teachers and students about learning progress, during the learning process. These practices also have the potential to develop learner autonomy by increasing student motivation and mastery through developing the learner's capacity to monitor and plan his or her own learning progress. Yet teacher adoption of the practices is not a straightforward implementation of techniques within an existing classroom repertoire. Recent research highlights a more complex interrelationship between teacher and student beliefs, identities, and traditions of power within assessment and learning in classroom contexts. These often hidden relationships can add layers of complexity for teachers implementing assessment change, and may act as barriers that frustrate efforts to realise the AfL goal of learner autonomy. By interpreting AfL practices from a sociocultural perspective, the social and cultural contexts that influence classroom assessment can be better understood. In turn teachers can thus be better supported in adopting AfL practices within the complexities of the social, cultural and policy contexts of schooling.

Marking time: leading and managing the development of assessment in higher education

This book highlights the University of New South Wales (UNSW) Assessment as Learning Project, a three year process of assessment change in faculty. The UNSW Vice-Chancellor, Professor Frederick Hilmer charged faculties with a learning and teaching agenda to investigate ways to make quality assessment practices more effective and efficient for both staff and students. The examples provided in this edited collection demonstrate exemplary case studies and theoretical perspectives on learning, teaching, and assessment. This book presents leading research in assessment as learning with a focus on changing practices as well as discipline specific practices.

Is your organizational culture fit for business process management?

All processes are modeled, all process metrics defined, all process support systems are set up; yet still, processes are not running smoothly and departmental silos are more present than ever. Both practitioners and academics tell the same story. A successful business process management (BPM) implementation goes beyond using the right methods and putting the right systems in place. In fact, an important success factor for BPM is establishing the right organizational culture, that is, a culture that supports the achievement and maintenance of efficient and effective business processes.

Developing a participatory impact assessment approach and action research culture within a communication for social change organisation in Nepal

Developing an effective impact evaluation framework, managing and conducting rigorous impact evaluations, and developing a strong research and evaluation culture within development communication organisations presents many challenges. This is especially so when both the community and organisational context is continually changing and the outcomes of programs are complex and difficult to clearly identify.----- This paper presents a case study from a research project being conducted from 2007-2010 that aims to address these challenges and issues, entitled Assessing Communication for Social Change: A New Agenda in Impact Assessment. Building on previous development communication projects which used ethnographic action research, this project is developing, trailing and rigorously evaluating a participatory impact assessment methodology for assessing the social change impacts of community radio programs in Nepal. This project is a collaboration between Equal Access – Nepal (EAN), Equal Access – International, local stakeholders and listeners, a network of trained community researchers, and a research team from two Australian universities. A key element of the project is the establishment of an organisational culture within EAN that values and supports the impact assessment process being developed, which is based on continuous action learning and improvement. The paper describes the situation related to monitoring and evaluation (M&E) and impact assessment before the project began, in which EAN was often reliant on time-bound studies and ‘success stories’ derived from listener letters and feedback. We then outline the various strategies used in an effort to develop stronger and more effective impact assessment and M&E systems, and the gradual changes that have occurred to date. These changes include a greater understanding of the value of adopting a participatory, holistic, evidence-based approach to impact assessment. We also critically review the many challenges experienced in this process, including:----- • Tension between the pressure from donors to ‘prove’ impacts and the adoption of a bottom-up, participatory approach based on ‘improving’ programs in ways that meet community needs and aspirations.----- • Resistance from the content teams to changing their existing M&E practices and to the perceived complexity of the approach.----- • Lack of meaningful connection between the M&E and content teams.----- • Human resource problems and lack of capacity in analysing qualitative data and reporting results.----- • The contextual challenges, including extreme poverty, wide cultural and linguistic diversity, poor transport and communications infrastructure, and political instability.----- • A general lack of acceptance of the importance of evaluation within Nepal due to accepting everything as fate or ‘natural’ rather than requiring investigation into a problem.

Assessment reform and educational change in Australia

This chapter addresses the changing climate of assessment policy and practice in Australia in response to global trends in education and the mounting accountability demands of standards-driven reform. Queensland, a State of Australia, has a tradition of respecting and trusting teacher judgment through the practice of, and policy commitment to, externally moderated school-based assessment. This chapter outlines the global trends in curriculum and assessment reform, and then analyzes the impact of international comparisons on national policy. The creation of the Australian Curriculum, Assessment and Reporting Authority (ACARA) together with the intent of establishing a standards-referenced framework raises tensions and challenges for teachers’ practice. The argument for sustaining confidence in teacher-based assessment is developed with reference to research evidence pertaining to the use of more authentic assessments and moderation practices for the purposes of improving learning, equity and accountability. Evidence is drawn from local studies of teacher judgment practice and used to demonstrate these developments and in so doing illuminate the complex issues of engaging the demands of policy while sustaining confidence in teacher assessment.

Assessment policy change in relation to English language teaching and learning in China : a study of perspectives from two universities

In response to concerns about the quality of English Language Learning (ELL) education at tertiary level, the Chinese Ministry of Education (CMoE) launched the College English Reform Program (CERP) in 2004. By means of a press release (CMoE, 2005) and a guideline document titled College English Curriculum Requirements (CECR) (CMoE, 2007), the CERP proposed two major changes to the College English assessment policy, which were: (1) the shift to optional status for the compulsory external test, the College English Test Band 4 (CET4); and (2) the incorporation of formative assessment into the existing summative assessment framework. This study investigated the interactions between the College English assessment policy change, the theoretical underpinnings, and the assessment practices within two Chinese universities (one Key University and one Non-Key University). It adopted a sociocultural theoretical perspective to examine the implementation process as experienced by local actors of institutional and classroom levels. Systematic data analysis using a constant comparative method (Merriam, 1998) revealed that contextual factors and implementation issues did not lead to significant differences in the two cases. Lack of training in assessment and the sociocultural factors such as the traditional emphasis on the product of learning and hierarchical teacher/students relationship are decisive and responsible for the limited effect of the reform.

Assessment of “change in flexibility method” in structural health monitoring systems

The modern structural diagnosis process is rely on vibration characteristics to assess safer serviceability level of the structure. This paper examines the potential of change in flexibility method to use in damage detection process and two main practical constraints associated with it. The first constraint addressed in this paper is reduction in number of data acquisition points due to limited number of sensors. Results conclude that accuracy of the change in flexibility method is influenced by the number of data acquisition points/sensor locations in real structures. Secondly, the effect of higher modes on damage detection process has been studied. This addresses the difficulty of extracting higher order modal data with available sensors. Four damage indices have been presented to identify their potential of damage detection with respect to different locations and severity of damage. A simply supported beam with two degrees of freedom at each node is considered only for a single damage cases throughout the paper.

Motivators and barriers to incorporating climate change-related health risks in environmental health impact assessment

Climate change presents risks to health that must be addressed by both decision-makers and public health researchers. Within the application of Environmental Health Impact Assessment (EHIA), there have been few attempts to incorporate climate change-related health risks as an input to the framework. This study used a focus group design to examine the perceptions of government, industry and academic specialists about the suitability of assessing the health consequences of climate change within an EHIA framework. Practitioners expressed concern over a number of factors relating to the current EHIA methodology and the inclusion of climate change-related health risks. These concerns related to the broad scope of issues that would need to be considered, problems with identifying appropriate health indicators, the lack of relevant qualitative information that is currently incorporated in assessment and persistent issues surrounding stakeholder participation. It was suggested that improvements are needed in data collection processes, particularly in terms of adequate communication between environmental and health practitioners. Concerns were raised surrounding data privacy and usage, and how these could impact on the assessment process. These findings may provide guidance for government and industry bodies to improve the assessment of climate change-related health risks.

Chinese EFL students' response to an assessment policy change

In light of its documented potential for enhancing learning, formative assessment has been adopted across a range of educational contexts to improve the quality of education. The assessment innovation that the Chinese Ministry of Education (CMoE) proposed to College English in 2007 via the College English Curriculum Requirements (CECR) (CMoE, 2007), is an initiative of this kind. Considering the acknowledged influence of assessment on students’ learning, it is instructive to explore the ways in which Chinese university students respond to an assessment policy change of this magnitude, particularly as it positions them as more active learners, having the potential for increased agency and engagement in their English language learning and assessment practices. In order to explore the response of students to this assessment initiative, a case study was conducted in the context of a College English classroom. Data included an interview with a College English teacher and four students from her classes, and classroom observations and a survey of her two classes of 100 students. Analysis of the data reveals that Chinese students’ responses to the assessment policy change are influenced by a variety of sociocultural factors, including their previous English language learning and assessment experience and the extent to which they are willing to play the ‘assessment game’. These findings have implications for policy and practice.

A life cycle assessment approach to quantifying greenhouse gas emissions from land-use change for beef production in eastern Australia

In life cycle assessment studies, greenhouse gas (GHG) emissions from direct land-use change have been estimated to make a significant contribution to the global warming potential of agricultural products. However, these estimates have a high uncertainty due to the complexity of data requirements and difficulty in attribution of land-use change. This paper presents estimates of GHG emissions from direct land-use change from native woodland to grazing land for two beef production regions in eastern Australia, which were the subject of a multi-impact life cycle assessment study for premium beef production. Spatially- and temporally consistent datasets were derived for areas of forest cover and biomass carbon stocks using published remotely sensed tree-cover data and regionally applicable allometric equations consistent with Australia's national GHG inventory report. Standard life cycle assessment methodology was used to estimate GHG emissions and removals from direct land-use change attributed to beef production. For the northern-central New South Wales region of Australia estimates ranged from a net emission of 0.03 t CO2-e ha-1 year-1 to net removal of 0.12 t CO2-e ha-1 year-1 using low and high scenarios, respectively, for sequestration in regrowing forests. For the same period (1990-2010), the study region in southern-central Queensland was estimated to have net emissions from land-use change in the range of 0.45-0.25 t CO2-e ha-1 year-1. The difference between regions reflects continuation of higher rates of deforestation in Queensland until strict regulation in 2006 whereas native vegetation protection laws were introduced earlier in New South Wales. On the basis of liveweight produced at the farm-gate, emissions from direct land-use change for 1990-2010 were comparable in magnitude to those from other on-farm sources, which were dominated by enteric methane. However, calculation of land-use change impacts for the Queensland region for a period starting 2006, gave a range from net emissions of 0.11 t CO2-e ha-1 year-1 to net removals of 0.07 t CO2-e ha-1 year-1. This study demonstrated a method for deriving spatially- and temporally consistent datasets to improve estimates for direct land-use change impacts in life cycle assessment. It identified areas of uncertainty, including rates of sequestration in woody regrowth and impacts of land-use change on soil carbon stocks in grazed woodlands, but also showed the potential for direct land-use change to represent a net sink for GHG.