Researching quality and equity in assessment for improved learner outcomes

Quality and equity issues as they relate to assessment practices and policy are becoming increasingly important as Australia introduces a national curriculum and achievement standards. In a context of high-stakes accountability, concerns regarding equity and quality have important implications for teachers‘ practice for the improvement of learner outcomes. This article is based on three research projects that were conducted over the past four years and were funded by the Australian Federal Government. The research focus, emergent questions, the educational contexts, and the rationale for the studies are discussed prior to the presentation of the analysis of the research findings and the implications for teachers‘ practice and policy reform.

Faculty mentoring, evidence-based assessment, and student learning : an Australian exploration of American initiatives

There have been many improvements in Australian engineering education since the 1990s. However, given the recent drive for assuring the achievement of identified academic standards, more progress needs to be made, particularly in the area of evidence-based assessment. This paper reports on initiatives gathered from the literature and engineering academics in the USA, through an Australian National Teaching Fellowship program. The program aims to establish a process to help academics in designing and implementing evidence-based assessments that meet the needs of not only students and the staff that teach them, but also industry as well as accreditation bodies. The paper also examines the kinds and levels of support necessary for engineering academics, especially early career ones, to help meet the expectations of the current drive for assured quality and standards of both research and teaching. Academics are experiencing competing demands on their time and energy with very high expectations in research performance and increased teaching responsibilities, although many are researchers who have not had much pedagogic training. Based on the literature and investigation of relevant initiatives in the USA, we conducted interviews with several identified experts and change agents who have wrought effective academic cultural change within their institutions and beyond. These reveal that assuring the standards and quality of student learning outcomes through evidence-based assessments cannot be appropriately addressed without also addressing the issue of pedagogic training for academic staff. To be sustainable, such training needs to be complemented by a culture of on-going mentoring support from senior academics, formalised through the university administration, so that mentors are afforded resources, time, and appropriate recognition.

A model for enhancing assessment and teaching practice at the coalface : Insights from a Fellow-In-Residence engagement program

Australian universities are currently engaging with new governmental policies and regulations that require them to demonstrate enhanced quality and accountability in teaching and research. The development of national academic standards for learning outcomes in higher education is one such instance of this drive for excellence. These discipline-specific standards articulate the minimum, or Threshold Learning Outcomes, to be addressed by higher education institutions so that graduating students can demonstrate their achievement to their institutions, accreditation agencies, and industry recruiters. This impacts not only on the design of Engineering courses (with particular emphasis on pedagogy and assessment), but also on the preparation of academics to engage with these standards and implement them in their day-to-day teaching practice on a micro level. This imperative for enhanced quality and accountability in teaching is also significant at a meso level, for according to the Australian Bureau of Statistics, about 25 per cent of teachers in Australian universities are aged 55 and above and more than 54 per cent are aged 45 and above (ABS, 2006). A number of institutions have undertaken recruitment drives to regenerate and enrich their academic workforce by appointing capacity-building research professors and increasing the numbers of early- and mid-career academics. This nationally driven agenda for quality and accountability in teaching permeates also the micro level of engineering education, since the demand for enhanced academic standards and learning outcomes requires both a strong advocacy for a shift to an authentic, collaborative, outcomes-focused education and the mechanisms to support academics in transforming their professional thinking and practice. Outcomes-focused education means giving greater attention to the ways in which the curriculum design, pedagogy, assessment approaches and teaching activities can most effectively make a positive, verifiable difference to students’ learning. Such education is authentic when it is couched firmly in the realities of learning environments, student and academic staff characteristics, and trustworthy educational research. That education will be richer and more efficient when staff works collaboratively, contributing their knowledge, experience and skills to achieve learning outcomes based on agreed objectives. We know that the school or departmental levels of universities are the most effective loci of changes in approaches to teaching and learning practices in higher education (Knight & Trowler, 2000). Heads of Schools are being increasingly entrusted with more responsibilities - in addition to setting strategic directions and managing the operational and sometimes financial aspects of their school, they are also expected to lead the development and delivery of the teaching, research and other academic activities. Guiding and mentoring individuals and groups of academics is one critical aspect of the Head of School’s role. Yet they do not always have the resources or support to help them mentor staff, especially the more junior academics. In summary, the international trend in undergraduate engineering course accreditation towards the demonstration of attainment of graduate attributes poses new challenges in addressing academic staff development needs and the assessment of learning. This paper will give some insights into the conceptual design, implementation and empirical effectiveness to date, of a Fellow-In-Residence Engagement (FIRE) program. The program is proposed as a model for achieving better engagement of academics with contemporary issues and effectively enhancing their teaching and assessment practices. It will also report on the program’s collaborative approach to working with Heads of Schools to better support academics, especially early-career ones, by utilizing formal and informal mentoring. Further, the paper will discuss possible factors that may assist the achievement of the intended outcomes of such a model, and will examine its contributions to engendering an outcomes-focussed thinking in engineering education.

The enactment of formative assessment in English language classrooms in two Chinese universities : teacher and student responses

The College English Curriculum Requirements (CECR), announced by the Chinese Ministry of Education in 2007, recommended the inclusion of formative assessment into the existing summative assessment framework of College English. This policy had the potential to fundamentally change the nature of assessment and its role in the teaching and learning of English in Chinese universities. In order to document and analyse these changes, case studies involving English language teachers and learners were undertaken in two Chinese Universities: one a Key university in the national capital; the other a non-Key university in a western province. The case study design incorporated classroom observations and interviews with English language teachers and their students. The type and focus of feedback and the engagement of students in assessment were analysed in the two contexts. Fundamental to the analysis was the concept of enactment, with the focus of this study on the ways that policy ideas and principles were enacted in the practices of the Chinese university classroom. Understandings of formative assessment as applied in contexts other than the predominantly Western, Anglophone contexts from where many of its principles derive, are offered.

Vocational education in science, technology, engineering and maths (STEM) : Curriculum innovation through school industry partnerships

Governments have recognised that the technological trades rely on knowledge embedded traditionally in science, technology, engineering and mathematics (STEM) disciplines. However, there is substantial evidence that students are turning away from these subjects in schools because the school curriculum is seen as irrelevant, with clear implications for not just vocational education but also higher education. In this paper, we report preliminary findings on the development of two curricula that attempt to integrate science and mathematics with workplace knowledge and practices. We argue that these curricula provide educational opportunities for students to pursue their preferred career pathways. These curricula were co-developed by industry and educational personnel across three industry sectors, namely, mining industry, aerospace and wine tourism. The aim was to provide knowledge appropriate for students moving from school to the workplace as trade apprentices in the respective industries. The analysis of curriculum and associated policy documents reveals that the curricula adopt applied learning orientations through teaching strategies and assessment practices which focus on practical skills. However, although key theoretical science and maths concepts have been well incorporated, the extent to which knowledge deriving from workplace practices is included varies across the curricula. The extent to which applications of concepts are included in the models depends on a number of factors not least the relevant expertise of the teacher as a practitioner in the industry. Our findings highlight the importance of teachers having substantial practical industry experience and the role that whole school policies play in attempts to align the range of learning experiences with the needs of industry.

Objectifying the Subjective : Assessment and Feedback in Creative Arts Studio Learning and Teaching

This paper examines the Assessment and Feedback aspects of Studio Teaching as Creative Arts pedagogy. Prompted by USQ’s newly offered Bachelor of Creative Arts (BCA), the author has developed an Assessment Matrix specifically designed to satisfy a number of imperatives, including: • ‘objectifying’ the subjective aspects of creative practice as assessable coursework/research • providing the means by which accurate, detailed, personalised and confidential feedback may be provided to students individually • providing consistent, accurate, meaningful assessment records for student, lecturer, and institution • ensuring consistency, continuity, and transparency of assessment processes and records to satisfy quality audits • minimising marking and assessment time, whilst maximising assessment integrity and depth • requiring only basic level skills and knowledge of a computer application already in common use (Microsoft Excel) • adaptability to a range of creative courses ‐ across disciplines This Assessment Matrix has been in development (and trialled) since January 2009.

Vocational education in science technology, engineering and maths (STEM) : Curriculum innovation through industry school partnerships

Governments have recognised that the technological trades rely on knowledge embedded traditionally in science, technology, engineering and mathematics (STEM) disciplines. In this paper, we report preliminary findings on the development of two curricula that attempt to integrate science and mathematics with workplace knowledge and practices. We argue that these curricula provide educational opportunities for students to pursue their preferred career pathways. These curricula were co-developed by industry and educational personnel across two industry sectors, namely, mining and aerospace. The aim was to provide knowledge appropriate for students moving from school to the workplace in the respective industries. The analysis of curriculum and associated policy documents reveals that the curricula adopt applied learning orientations through teaching strategies and assessment practices which focus on practical skills. However, although key theoretical science and maths concepts have been well incorporated, the extent to which knowledge deriving from workplace practices is included varies across the curricula. Our findings highlight the importance of teachers having substantial practical industry experience and the role that whole school policies play in attempts to align the range of learning experiences with the needs of industry.

Literacy assessment that counts : mediating, interpreting and contesting translocal policy in a primary school

In Australia, as in many western education systems over the last two decades, discourses of accountability and performativity have reshaped education policy that has in turn reorganised the work of school leaders and teachers. One of the effects of this reorganisation is increased attention to the production, analysis and display of student achievement data. In this paper we examine in detail a sequence of the production and reading of literacy assessment data in a small Catholic school. Our analysis uses institutional ethnography’s concept of the ‘active text’, the text as occurring in a specific place and time even as it is articulated to social relations beyond its immediate context. Through this process we learn from those involved how their everyday work brings into being formalised, textually authorised processes in a local site that ensure the school meets accountability requirements while enabling teachers to resist standardisation of literacy teaching and assessment.

Diversity or perversity?: Award-winning children's literature and the Australian curriculum

Two of the three cross-curriculum priorities for the national Australian Curriculum prescribed by the Australian Curriculum Assessment and Reporting Authority (ACARA) are focussed on what might be called diversity education: “Aboriginal and Torres Strait Islander histories and culture”, and "Asia and Australia's Engagement with Asia” (ACARA, “Cross”). One need not be versed in complex rhetorical theory to understand that, laudable and legitimate as such priorities are, their existence implies that mainstream education in Australia has been or is characterised by the marginalisation or erasure of Australia's history—the original Indigenous cultures are not only living and vibrant today, but also have tens of thousands of years’ “head start” on Australia’s settler cultures—and of its geography—Australia is, after all, located in some physical proximity to Asia. Some might even suggest that Australia is in Asia. These temporal and spatial “forgettings” constitute a kind of cultural perversity which the cross-curricular priorities both seek to address and serve to reinscribe. Even as ACARA requires Australian school students to engage with Aboriginal and Asian histories, cultures, societies, they imply that such histories, cultures, and societies are “diverse”, that they are not those of the students in Australian classrooms; producing them as objects of study rather than as lived experience. This should not necessarily be surprising. Michael W. Apple has provocatively argued that: “one of the perverse effects of a national curriculum actually will be to ‘legitimise inequality.’ It may in fact help create the illusion that whatever the massive differences in schools, they all have something in common” (18). In the Australian context, attempts to mitigate such perversity are articulated via the selection of literary texts. As educators move to resource ACARA’s cross-curricular priorities, ACARA notes that “Teachers and schools are best placed to make decisions about the selection of texts in their teaching and learning programs that address the content in the Australian Curriculum while also meeting the needs of the students in their classes” (ACARA, “Advice”). This assertion appears on a webpage called “Advice on selection of literary texts” which is notable first and foremost for its total lack of any literary texts being named, and its list of weblinks pointing to lists of texts compiled elsewhere, by other organisations, and in the main, compiled to serve agendas other than the Australian curriculum. One of the major resources referred to by ACARA for literary text selection is the Children’s Book Council of Australia (CBCA). Of course, the CBCA’s annual book awards do not share ACARA’s educational priorities, but do have a history of being drawn upon by schools as a curriculum resource. In this paper, I consider the literary texts which have been prized by the CBCA in recent years attending to their engagements with Aboriginal cultures.

Tensions between policy and practice : reconciliation agendas in the Australian curriculum English

In various parts of the world, Indigenous and non-Indigenous peoples are actively working towards Reconciliation. In Australia, the context in which we each undertake our work as educationalists and researchers, the Reconciliation agenda has been pushed into schools and English teachers have been called on to share responsibility for facilitating the move towards a new national order. The recently introduced Australian Curriculum mandates that Aboriginal and Torres Strait Islander Histories and Cultures be embedded with “a strong” but “varying presence” into each learning area (Australian Curriculum, Assessment and Reporting Authority, 2013). In this paper we consider the tensions between policy and practice, when discourses external to education are recontextualised into the discipline of English. We do so by applying an analytical framework based on Bernstein’s (1990, 1996,2000) sociological theories about the structure of instructional and regulative discourses. Our findings suggest that the space to exert Reconciliatory agendas in the Australian Curriculum English is ambiguous and thus holds the potential to not only marginalise Indigenous knowledges but also to create tensions between policy and practice for non-Indigenous teachers of English.

Improving criteria and feedback in student assessment in law

Spurred on by both the 1987 Pearce Report1 and the general changes to higher education spawned by the “Dawkins revolution” from 1988, there has been much critical self-evaluation leading to profound improvements to the quality of teaching in Australian law schools.2 Despite the changes there are still areas of general law teaching practice which have lagged behind recent developments in our understanding of what constitutes high quality teaching. One such area is assessment criteria and feedback. The project Improving Feedback in Student Assessment in Law is an attempt to remedy this. It aims to produce a manual containing key principles for the design of assessment and the provision of feedback, with practical yet flexible ideas and illustrations which law teachers may adopt or modify. Most of the examples have been developed by teachers at the University of Melbourne Law School. The project was supported in 1996 by a Committee for the Advancement of University Teaching grant and the manual will be published late in 1997.3 This note summarises the core principles which are elaborated further in the manual.

Geography and international assessment: Opportunity or distraction

Assessments of students in primary and secondary education are debated among practitioners, policy-makers, and parents. In some countries, assessment scores serve a criterion for passage between levels of education, for example, from secondary school to post-secondary education. Those practices are often traditions and while they come under criticism, they are a long-accepted part of the educational practices within a country. In those countries, the students’ assessment and examination scores are posted in public places or published in local news media. In other countries, assessments are used for the periodic checks on individual student progress. The results of assessments may be used for rating schools, and in some cases, they are used for evaluating the performance of teachers. Assessments are used less often to analyze student performance and make judgments regarding the performance of the curriculum. Even less often, assessments serve to critically establish strategies for the improvement of student learning and educational practices. The ends on the continuum of the assessment debate often focus on the opportunities that assessments present to improve education on one end. The other end is that assessments serve as a major distraction from the important work of teachers by removing classroom room time from instruction. The debate on those issues continues.

Enhanced student learning in accounting utilising web-based technology, peer-review feedback and reflective practices: A learning community approach to assessment

Higher education is becoming a major driver of economic competitiveness in an increasingly knowledge-driven global economy. Maintaining the competitive edge has seen an increase in public accountability of higher education institutions through the mechanism of ranking universities based on the quality of their teaching and learning outcomes. As a result, assessment processes are under scrutiny, creating tensions between standardisation and measurability and the development of creative and reflective learners. These tensions are further highlighted in the context of large undergraduate subjects, learner diversity and time-poor academics and students. Research suggests that high level and complex learning is best developed when assessment, combined with effective feedback practices, involves students as partners in these processes. This article reports on a four-phase, cross-institution and cross-discipline project designed to embed peer-review processes as part of the assessment in two large, undergraduate accounting classes. Using a social constructivist view of learning, which emphasises the role of both teacher and learner in the development of complex cognitive understandings, we undertook an iterative process of peer review. Successive phases built upon students’ feedback and achievements and input from language/learning and curriculum experts to improve the teaching and learning outcomes.

A preliminary analysis of teacher perceptions of the effects of NAPLAN on pedagogy and curriculum

This paper reports preliminary survey findings of Western Australian and South Australian teacher perceptions of the impact of NAPLAN on curriculum and pedagogy in their classroom and school. The paper examines how teachers perceive the effects of NAPLAN on curriculum and pedagogy and whether these perceptions mediated by the teacher’s gender, the socioeconomics of the school, the State and the school system in which the teacher works. Teachers report that they are either choosing or being instructed to teach to the test, that this results in less time being spent on other curriculum areas and that these effects contribute in a negative way to the class environment and the engagement of students. This largely agrees with a body of international research that suggests that high-stakes literacy and numeracy tests often results in unintended consequences such as a narrow curriculum focus, a return to teacher-centred instruction and a decrease in motivation. Analysis suggests there is a relationship between participant responses to the effect of NAPLAN on curriculum based on the characteristics of which State the teacher taught in, the socioeconomic status of the school and the school system in which they were employed (State, Catholic, and Independent).

The effects of NAPLAN: Teacher perceptions of the impact of NAPLAN on pedagogy and curriculum

This paper reports preliminary findings of a survey of in-service teachers in WA and SA conducted in 2012. Participants completed an online survey open to all teachers in WA and SA. The survey ran for three months from April to June 2012. One section of the survey asked teachers to report their perceptions of the impact that NAPLAN has had on the curriculum and pedagogy of their classroom and school. Two principal research questions were addressed in this preliminary analysis. First what are teacher perceptions of the effects on NAPLAN on curriculum and pedagogy? Second, are there any interaction effects between gender, socioeconomics status, location and school system on teachers perceptions? Statistical analyses examined one- and two-way MANOVA to assess main effects and interaction effects on teachers' global perceptions. These were followed by a series of exploratory one- and two-way ANOVA of specific survey items to suggest potential sources for differences among teachers from different socioeconomic regions, states and systems. Teachers report that they are either choosing or being instructed to teach to the test, that this results in less time being spent on other curriculum areas and that these effects contribute in a negative way on the engagement of students. This largely agrees with a body of international research that suggests that high-stakes literacy and numeracy tests often results in unintended consequences such as a narrow curriculum focus (Au, 2007), a return to teacher-centred instruction (Barret, 2009) and a decrease in motivation (Ryan & Wesinstein, 2009). Preliminary results from early survey respondents suggests there is a relationship between participant responses to the effect of NAPLAN on curriculum and pedagogy based on the characteristics of which State the teacher taught in, their perceptions of the socioeconomic status of the school and the school system in which they were employed (State, Catholic, and Independent).