A Systems-level approach to building sustainable assessment cultures : moderation, quality task design and dependability of judgement

This paper considers the conditions that are necessary at system and local levels for teacher assessment to be valid, reliable and rigorous. With sustainable assessment cultures as a goal, the paper examines how education systems can support local level efforts for quality learning and dependable teacher assessment. This is achieved through discussion of relevant research and consideration of a case study involving an evaluation of a cross-sectoral approach to promoting confidence in school-based assessment in Queensland, Australia. Building on the reported case study, essential characteristics for developing sustainable assessment cultures are presented, including: leadership in learning; alignment of curriculum, pedagogy and assessment; the design of quality assessment tasks and accompanying standards, and evidence-based judgement and moderation. Taken together, these elements constitute a new framework for building assessment capabilities and promoting quality assurance.

Evaluation of the quality of diverse teaching practice in higher education research methods

The purpose of this paper is to report on a methods research project investigating the evaluation of diverse teaching practice in Higher Education. The research method is a single site case study of an Australian university with data collected through published documents, surveys, interviews and focus groups. This project provides evidence of the wide variety of evaluation practice and diverse teaching practice across the university. This breadth identifies the need for greater flexibility of evaluation processes, tools and support to assist teaching staff to evaluate their diverse teaching practice. The employment opportunities for academics benchmark the university nationally and position the case study in the field. Finally this reaffirms the institutional responsibility for services to support teaching staff in an ongoing manner.

Reasonable adjustments in assessment : putting law and policy into practice in Australia

The Australian Disability Standards for Education 2005 (Cth) require education providers to make reasonable adjustments in educational assessment so that students with disability can participate on the same basis as other students and be able to demonstrate what they know and can do. Reasonableness is governed by a determination of the balance of interests, benefits and detriment to the parties involved. The Standards require providers to consult with students and associates on adjustments, although guidance on how consultation should occur and how the views of students and associates are to be taken into account is vague. In this article, we identify three principles to be considered in order to put appropriate and effective reasonable adjustments in assessment into practice. While Australian law and assessment contexts are used to examine intentions, expectations and practices in educational assessment for students with disability, we argue that these three principles must be considered in any national education system to ensure equitable assessment practices and achieve equitable educational inclusion for students with disability.

Learning and teaching reflection : developing capacities for lifelong learning

Reflection is an essential part of being an effective learner and working as a productive teacher. It enables the learner or teacher to deliberate about the factors that lead to successful learning and teaching for them and/or their students, in a particular place and for a specific purpose, so they can make reasoned and effective choices. This chapter introduces important frameworks that cover a century of thinking around reflection in education, and illustrates how preservice teachers can use these ideas across three phases. First, becoming a reflective learner as a university student to enhance learning and assessment outcomes; second, becoming a reflective teacher to improve classroom teaching and learning outcomes; and third, developing the reflective capacities of primary students so they can enhance their skills for lifelong learning.

Higher order thinking in chemistry curriculum and its assessment

Higher-order thinking has featured persistently in the reform agenda for science education. The intended curriculum in various countries sets out aspirational statements for the levels of higher-order thinking to be attained by students. This study reports the extent to which chemistry examinations from four Australian states align and facilitate the intended higher-order thinking skills stipulated in curriculum documents. Through content analysis, the curriculum goals were identified for each state and compared to the nature of question items in the corresponding examinations. Categories of higher-order thinking were adapted from the OECD’s PISA Science test to analyze question items. There was considerable variation in the extent to which the examinations from the states supported the curriculum intent of developing and assessing higher-order thinking. Generally, examinations that used a marks-based system tended to emphasize lower-order thinking, with a greater distribution of marks allocated for lower-order thinking questions. Examinations associated with a criterion-referenced examination tended to award greater credit for higher-order thinking questions. The level of complexity of chemistry was another factor that limited the extent to which examination questions supported higher-order thinking. Implications from these findings are drawn for the authorities responsible for designing curriculum and assessment procedures and for teachers.

Reconstructing places and spaces in blended work integrated learning

This paper considers the emergence and ongoing development of an embedded, studentnegotiated work placement model of Work Integrated Learning (WIL) in the engineering and built environment disciplines at an Australian metropolitan university. The characteristics of the model and a continuous improvement strategy are provided. The model is characterised by large student cohorts independently sourcing and negotiating relevant work placements and completing at least one, mandatory credit-bearing WIL unit. Through ongoing analyses and evaluation of the model more experiential and collaborative learning approaches have been adopted. This has included the creation of blended learning spaces using technology. The paper focuses on the five year journey travelled by the teaching team as they embarked on ways to improve curriculum, pedagogy, administrative processes and assessment - effectively relocating much of their interaction with students online. The insights derived from this rich, single case study should be of interest to others considering alternative ways of responding to increasing student enrolments in WIL and the impact of blended learning in this context.

Students' cultural capital : a study of assessment for learning as a field of exchange

This study was conducted within the context of a flexible education institution where conventional educational assessment practices and tests fail to recognise and assess the creativity and cultural capital of a cohort of marginalised young people. A new assessment model which included an electronic-portfolio-social-networking system (EPS) was developed and trialled to identify and exhibit evidence of students' learning. The study aimed to discern unique forms of cultural capital (Bourdieu, 1986) possessed by students who attend the Edmund Rice Education Australia Flexible Learning Centre Network (EREAFLCN). The EPS was trialled at the case study schools in an intervention and developed a space where students could make evident culturally specific forms of capital and funds of knowledge (Gonzalez, Moll, & Amanti, 2005). These resources were evaluated, modified and developed through dialogic processes utilising assessment for learning approaches (Qualifications and Curriculum Development Agency, 2009) in online and classroom settings. Students, peers and staff engaged in the recognition, judgement, revision and evaluation of students' cultural capital in a subfield of exchange (Bourdieu, 1990). The study developed the theory of assessment for learning as a field of exchange incorporating an online system as a teaching and assessment model. The term efield has been coined to describe this particular capital exchange model. A quasi-ethnographic approach was used to develop a collective case study (Stake, 1995). This case study involved an in-depth exploration of five students' forms of cultural capital and the ways in which this capital could be assessed and exchanged using the efield model. A comparative analysis of the five cases was conducted to identify the emergent issues of students' recognisable cultural capital resources and the processes of exchange that can be facilitated to acquire legitimate credentials for these students in the Australian field of education. The participants in the study were young people at two EREAFLC schools aged between 12 and 18 years. Data was collected through interviews, observations and examination of documents made available by the EREAFLCN. The data was coded and analysed using a theoretical framework based on Bourdieu's analytical tools and a sociocultural psychology theoretical perspective. Findings suggest that processes based on dialogic relationships can identify and recognise students' forms of cultural capital that are frequently misrecognised in mainstream school environments. The theory of assessment for learning as a field of exchange was developed into praxis and integrated in an intervention. The efield model was found to be an effective sociocultural tool in converting and exchanging students' capital resources for legitimated cultural and symbolic capital in the field of education.

Reconceptualizing Early Mathematics Learning : The Fundamental Role of Pattern and Structure

The Pattern and Structure Mathematics Awareness Program (PASMAP) was developed concurrently with the studies of AMPS and the development of the Pattern and Structure Assessment (PASA) interview. We summarize some early classroom-based teaching studies and describe the PASMAP that resulted. A large-scale two-year longitudinal study, Reconceptualizing Early Mathematics Learning (REML) resulted. We provide an overview of the REML study and discuss the consequences for our view of early mathematics learning. A purposive sample of four large primary schools, two in Sydney and two in Brisbane, representing 316 students from diverse socio-economic and cultural contexts, participated in an evaluation of the PASMAP intervention throughout the 2009 school year and a follow-up assessment in 2010. Two different mathematics programs were implemented: in each school, two Kindergarten teachers implemented the PASMAP and another two implemented their regular program. The study shows that both groups of students made substantial gains on the ‘I Can Do Maths’ standardized assessment and the PASA interview, but highly significant differences were found on the latter with PASMAP students outperforming the regular group on PASA scores. Qualitative analysis of students’ responses for structural development showed increased levels for the PASMAP students. Implications for pedagogy and curriculum are discussed.

Exploring the Possibilities and Methodological Challenges of Evaluation Practice : A Democratic Question

In our exploration of the methodological possibilities and challenges of evaluation practice that foregrounds a democratic approach, we argue that having sensitivity to participants’ different positions, ideological perspectives, and values, and an understanding of the existing power relationships are paramount for the evaluators if the evaluation is to achieve its goals. This chapter draws on our experiences as evaluators in Australia, where school leaders and teachers are currently experiencing significant curriculum and assessment changes, and now work in a context of heightened accountability. Our evaluation practices with schools are used to illustrate how democratic practices can be incorporated into evaluation in ways that provide opportunities and support for school self-evaluation. It is paramount to recognise that evaluation is a political and ideological practice, and therefore is not a neutral process. Our argument is that it remains the responsibility of evaluators to design spaces that build sustainable relationships with participants and the targets of the programmes being evaluated; ensure that dialogue and deliberation are valued between all stakeholders; represent a range of interests; and give a service back to the evaluated communities by offering understandings and promoting reflective practice and informed decision-making throughout programme implementation.

Designing Assessment for Quality Learning

- Considers broad-scale assessment approaches and how they impact on educational opportunity and outcomes. - Brings together internationally recognised scholars providing new insights into assessment for learning improvement and accountability. - Presents different theoretical and methodological perspectives influential in the field of assessment, learning and social change. - Contributes to the theorising of assessment in contexts characterised by heightened accountability requirements and constant change. This book brings together internationally recognised scholars with an interest in how to use the power of assessment to improve student learning and to engage with accountability priorities at both national and global levels. It includes distinguished writers who have worked together for some two decades to shift the assessment paradigm from a dominant focus on assessment as measurement towards assessment as central to efforts to improve learning. These writers have worked with the teaching profession and, in so doing, have researched and generated key insights into different ways of understanding assessment and its relationship to learning. The volume contributes to the theorising of assessment in contexts characterised by heightened accountability requirements and constant change. The book’s structure and content reflect already significant and growing international interest in assessment as contextualised practice, as well as theories of learning and teaching that underpin and drive particular assessment approaches. Learning theories and practices, assessment literacies, teachers’ responsibilities in assessment, the role of leadership, and assessment futures are the organisers within the book’s structure and content. The contributors to this book have in common the view that quality assessment, and quality learning and teaching are integrally related. Another shared view is that the alignment of assessment with curriculum, teaching and learning is linchpin to efforts to improve both learning opportunities and outcomes for all. Essentially, the book presents new perspectives on the enabling power of assessment. In so doing, the writers recognise that validity and reliability - the traditional canons of assessment – remain foundational and therefore necessary. However, they are not of themselves sufficient for quality education. The book argues that assessment needs to be radically reconsidered in the context of unprecedented societal change. Increasingly, communities are segregating more by wealth, with clear signs of social, political, economic and environmental instability. These changes raise important issues relating to ethics and equity, taken to be core dimensions in enabling the power of assessment to contribute to quality learning for all. This book offers readers new knowledge about how assessment can be used to re/engage learners across all phases of education.

Teaching and assessing ethics and social responsibility in undergraduate science : a position paper

Institutional graduate capabilities and discipline threshold learning outcomes require science students to demonstrate ethical conduct and social responsibility. However, neither the teaching nor the assessment of these concepts is straightforward. Australian chemistry academics participated in a workshop in 2013 to discuss and develop teaching and assessment in these areas and this paper reports on the outcomes of that workshop. Controversial issues discussed included: How broad is the mandate of the teacher, how should the boundaries between personal values and ethics be drawn, and how can ethics be assessed without moral judgement? In this position paper, I argue for a deep engagement with ethics and social justice, achieved through case studies and assessed against criteria that require discussion and debate. Strategies to effectively assess science students’ understanding of ethics and social responsibility are detailed.

Personal Epistemologies and Teaching

Over the last three decades, a growing body of research related to epistemic beliefs has been identified as a crucial for understanding teaching and learning (Yadav, Herron, & Samarapungavan, 2011). In particular, research related to beliefs about teaching, learning and knowledge has been considered important for understanding why teachers engage in certain approaches to pedagogy, curriculum and assessment in classrooms (Stromlo & Bråten, 2011). The beliefs held by teachers about knowledge and knowing, otherwise known as personal epistemology (Hofer, 2010), have been shown to influence other knowledge and beliefs (Schommer-Aikens, 2004). Five and Buehl (2012) indicated that because an individual’s understanding of reality is always seen through the lens of existing beliefs, the role of beliefs as a filter is particularly relevant in the context of teacher education. That is, if beliefs influence how individuals interpret new information and experiences, preservice and practicing teachers’ beliefs shape what and how they learn about teaching.” (p. 470-480). It is likely that such beliefs have an important relationship with teacher knowledge and practices...

Teaching and Assessing for Understanding at University in the South Pacific

Pacific people have their own unique ways of knowing that shape how they learn and this should be taken into account in planning curriculum and in teaching. Pacific people are more likely to want to learn by doing, seeing, collaborating and in a concrete environment whereas for Western students learning becomes formal quickly and depends more on words and theories. This assumed difference in learning preferences could present a problem for formal learning with the need to bridge the gap psychologically and epistemologically between concrete and formal modes of learning. It could be the reason why some students in the Pacific, even at the tertiary level, rely heavily on rote learning. This chapter is a discussion of learning and assessment practices that help to foster understanding as they might apply to teaching at university in the South Pacific.

Teaching and Assessing Reflective Writing in a Large Undergraduate Core Substantive Law Unit

The focus of higher education has shifted towards building students’ skills and self-awareness for future employment, in addition to developing substantive discipline knowledge. This means that there is an increasing need for embedding approaches to teaching and learning which provide a context for skills development and opportunities for students to prepare for the transition from legal education to professional practice. This chapter reports on a large (500-600 students) core undergraduate Equity law unit in an Australian University. ePortfolio has been embedded in Equity as a means of enabling students to document their reflections on their skill development in that unit. Students are taught, practice and are assessed on their teamwork and letter writing skills in the context of writing a letter of advice to a fictional client in response to a real world problem. Following submission of the team letter, students are asked to reflect on their skill development and document their reflections in ePortfolio. A scaffolded approach to teaching reflective writing is adopted using a blended model of delivery which combines face to face lectures and online resources, including an online module, facts sheets designed to guide students through the process of reflection by following the TARL model of reflection, and exemplars of reflective writing. Although students have engaged in the process of reflective writing in Equity for some years, in semester one 2011 assessment criteria were developed and the ePortfolio reflections were summatively assessed for the first time. The model of teaching and assessing reflective practice was evaluated in a range of ways by seeking feedback from students and academic staff responsible for implementing the model and asking them to reflect on their experiences. This chapter describes why skill development and reflective writing were embedded in the undergraduate law unit Equity; identify the teaching and learning approaches which were implemented to teach reflective writing to online and internal Equity students; explain the assessment processes; analyse the empirical evidence from evaluations; document the lessons learnt and discuss planned future improvements to the teaching and assessment strategies.

Illuminative evaluation as a method applied to Australian Government policy borrowing and implementation in higher education

Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.