Building sustainable and effective assurance of learning process in a changing higher education environment

In a study of assuring learning in Australian Business Schools, 25 Teaching and Learning Associate Deans were interviewed to identify current issues in developing and measuring the quality of teaching and learning outcomes. Results indicate that for most institutions developing a perspective on graduate attributes and mapping assessments to measure outcomes across an entire program required knowledge creation and the building of new inclusive processes. Common elements of effective practice, namely those which offered consistently superior outcomes, included: inclusive processes; embedded graduate attributes throughout a program; alongside consistent and appropriate assessment. Results indicate that assurance of learning processes are proliferating nationally while quality of teaching and learning outcomes and in the processes for assuring it is increasing as a result.

Governance of learning mechanisms : evidence from construction firms

The focus of knowledge management (KM) in the construction industry is moving towards capability building for value creation. The study reported by this paper is motivated by recent assertions about the genesis and evolution of knowledge management capability (KMC) in the strategic management field. It attempts to shed light on the governance of learning mechanisms that develop KMC within the context of construction firms. A questionnaire survey was administered to a sample of construction contractors operating in the very dynamic Hong Kong market to elicit opinions on the learning mechanisms and business outcomes of targeted firms. On the basis of a total of 149 usable responses, structural equation modeling (SEM) analysis identified relationships among knowledge-governance mechanisms, knowledge processes, and business performance, thereby supporting the existence of strategic learning loops. The study findings provide evidence from the construction context for capability assertions that knowledge-governance mechanisms and processes form learning mechanisms that carry out strategic learning to create value, effect performance outcomes, and ultimately drive the evolution of KMC. The findings imply that it is feasible for managing construction firms to govern learning mechanisms through managing the capability-based holistic KM system, thereby reconfiguring KMC to match needs in the dynamic market environment over time.

The centrality of teachers’ judgement practice in assessment : a study of standards in moderation

There is a strong quest in several countries including Australia for greater national consistency in education and intensifying interest in standards for reporting. Given this, it is important to make explicit the intended and unintended consequences of assessment reform strategies and the pressures to pervert and conform. In a policy context that values standardisation, the great danger is that the technical, rationalist approaches that generalise and make superficial assessment practices, will emerge. In this article, the authors contend that the centrality and complexity of teacher judgement practice in such a policy context need to be understood. To this end, we discuss and analyse recorded talk in teacher moderation meetings showing the processes that teachers use as they work with stated standards to award grades (A to E). We show how they move to and fro between (1) supplied textual artefacts, including stated standards and samples of student responses, (2) tacit knowledge of different types, drawing into the moderation, and (3) social processes of dialogue and negotiation. While the stated standards play a part in judgement processes, in and of themselves they are shown to be insufficient to account for how the teachers ascribe value and award a grade to student work in moderation. At issue is the nature of judgement as cognitive and social practice in moderation and the legitimacy (or otherwise) of the mix of factors that shape how judgement occurs.

Sustaining teacher professionalism in the context of standards referenced assessment reform

Standards referenced reform, tied to reporting, engages directly with assessment issues related to accountability. Assessment is the key to good education and is inseparable from curriculum. In an accountability context, standards are used as a lever to improve the reliability and consistency of teacher judgement; and classroom evidence is used by education systems for reporting and tracking achievement over time. Assessment is thus a powerful driver for change and is at the heart of the teaching-learning dynamic. The relationship between the learner, learning and assessment needs to be kept central and the idea of teacher empowerment is fundamental. This chapter is a call to honour and sustain teacher professionalism through educative forms of school-based and teacher-led evaluation, assessment and communities of judgement practice. It supports the argument for a central place for classroom assessment in the role of assessment in educational accountability...

Standards-driven reform years 1-10 : moderation an optional extra?

While externally moderated standards-based assessment has been practised in Queensland senior schooling for more than three decades, there has been no such practice in the middle years. With the introduction of standards at state and national levels in these years, teacher judgement as developed in moderation practices is now vital. This paper argues, that in this context of assessment reform, standards intended to inform teacher judgement and to build assessment capacity are necessary but not sufficient for maintaining teacher and public confidence in schooling. Teacher judgement is intrinsic to moderation, and to professional practice, and can no longer remain private. Moderation too is intrinsic to efforts by the profession to realise judgements that are defensible, dependable and open to scrutiny. Moderation can no longer be considered an optional extra and requires system-level support especially if, as intended, the standards are linked to system-wide efforts to improve student learning. In presenting this argument we draw on an Australian Research Council funded study with key industry partners (the Queensland Studies Authority and the National Council for Curriculum and Assessment of the Republic of Ireland). The data analysed included teacher interview data and additional teacher talk during moderation sessions. These were undertaken during the initial phase of policy development. The analysis identified those issues that emerge in moderation meetings that are designed to reach consistent, reliable judgements. Of interest are the different ways in which teachers talked through and interacted with one another to reach agreement about the quality of student work in the application of standards. There is evidence of differences in the way that teachers made compensations and trade-offs in their award of grades, dependent on the subject domain in which they teach. This article concludes with some empirically derived insights into moderation practices as policy and social events.

A Systems-level approach to building sustainable assessment cultures : moderation, quality task design and dependability of judgement

This paper considers the conditions that are necessary at system and local levels for teacher assessment to be valid, reliable and rigorous. With sustainable assessment cultures as a goal, the paper examines how education systems can support local level efforts for quality learning and dependable teacher assessment. This is achieved through discussion of relevant research and consideration of a case study involving an evaluation of a cross-sectoral approach to promoting confidence in school-based assessment in Queensland, Australia. Building on the reported case study, essential characteristics for developing sustainable assessment cultures are presented, including: leadership in learning; alignment of curriculum, pedagogy and assessment; the design of quality assessment tasks and accompanying standards, and evidence-based judgement and moderation. Taken together, these elements constitute a new framework for building assessment capabilities and promoting quality assurance.

The relationship of aptitude, course grades, and program length, to performance on a standards based test (NCLEX -RN)

The purpose of this study was threefold: first, to investigate variables associated with learning, and performance as measured by the National Council Licensure Examination for Registered Nurses (NCLEX-RN). The second purpose was to validate the predictive value of the Assessment Technologies Institute (ATI) achievement exit exam, and lastly, to provide a model that could be used to predict performance on the NCLEX-RN, with implications for admission and curriculum development. The study was based on school learning theory, which implies that acquisition in school learning is a function of aptitude (pre-admission measures), opportunity to learn, and quality of instruction (program measures). Data utilized were from 298 graduates of an associate degree nursing program in the Southeastern United States. Of the 298 graduates, 142 were Hispanic, 87 were Black, non-Hispanic, 54 White, non-Hispanic, and 15 reported as Others. The graduates took the NCLEX-RN for the first time during the years 2003–2005. This study was a predictive, correlational design that relied upon retrospective data. Point biserial correlations, and chi-square analyses were used to investigate relationships between 19 selected predictor variables and the dichotomous criterion variable, NCLEX-RN. The correlation and chi square findings indicated that men did better on the NCLEX-RN than women; Blacks had the highest failure rates, followed by Hispanics; older students were more likely to pass the exam than younger students; and students who passed the exam started and completed the nursing program with a higher grade point average, than those who failed the exam. Using logistic regression, five statistical models that used variables associated with learning and student performance on the NCLEX-RN were tested with a model adapted from Bloom's (1976) and Carroll's (1963) school learning theories. The derived model included: NCLEX-RNsuccess = f (Nurse Entrance Test and advanced medical-surgical nursing course grade achieved). The model demonstrates that student performance on the NCLEX-RN can be predicted by one pre-admission measure, and a program measure. The Assessment Technologies Institute achievement exit exam (an outcome measure) had no predictive value for student performance on the NCLEX-RN. The model developed accurately predicted 94% of the student's successful performance on the NCLEX-RN.

Changing assessment practices : the case for online moderation

Online moderation meetings have the potential to support the collaborative professional development of teachers, and the formation of a common understanding of what denotes quality in student work in a standards based assessment system. In doing so systemic calls for consistency across education systems are also being met. In this paper a case for employing online moderation meetings is developed through recourse to the demands of learning in the twenty-first century and the place of assessment within those discourses. It is argued that empirical data is needed on the efficacy of online moderation meetings to guide future practice as the use of information and communication technologies increases in education systems. Online moderation is one way of gathering teachers across vast distances to share their understandings and develop common meanings of assessment. While it is suggested that online moderation is one possible procedure to meet systemic requirements and support teachers’ professional collaboration, the implementation of such a system also introduces new challenges for schools and teachers. Meeting online to discuss professional understandings is a new way of operating for teachers and involves technology that has not yet been fully utilised within education departments. Issues such as the types of interactions that are afforded within such an environment, as well as technical operating problems that occur when using technology impact on the employment of online meetings. Online moderation meetings while potentially solving the issue of developing common understandings across an entire department also pose new issues to be resolved. There is a need for research into the efficacy of online moderation meetings so that future policy decisions may be based on sound empirical data. It is imperative that as new ways of knowing and acting are incorporated into school curriculum and pedagogy, assessment practices are also aligned. Online moderation meetings can support such practices by enabling teachers to communicate with a wider and more diverse group of teachers to establish common understandings.

Moderation and consistency of teacher judgement : teachers’ views

Major curriculum and assessment reforms in Australia have generated research interest in issues related to standards, teacher judgement and moderation. This article is based on one related inquiry of a large-scale Australian Research Council Linkage project conducted in Queensland. This qualitative study analysed interview data to identify teachers’ views on standards and moderation as a means to achieving consistency of teacher judgement. A complementary aspect of the research involved a blind review that was conducted to determine the degree of teacher consistency without the experience of moderation. Empirical evidence was gained that most teachers, of the total interviewed, articulated a positive attitude towards the use of standards in moderation and perceived that this process produces consistency in teachers’ judgements. Context was identified as an important influential factor in teachers’ judgements and it was concluded that teachers’ assessment beliefs, attitudes and practices impact on their perceptions of the value of moderation practice and the extent to which consistency can be achieved.

Building on the past for the future : assessment for learning as a field of exchange

Standardised testing does not recognise the creativity and skills of marginalised youth. This paper presents the development of an innovative approach to assessment designed for the re-engagement of at risk youth who have left formal schooling and are now in an alternative education institution. An electronic portfolio system (EPS) has been developed to capture, record and build on the broad range of students’ cultural and social capital. The assessment as a field of exchange model draws on categories from sociological fields of capital and reconceptualises an eportfolio and social networking hybrid system as a sociocultural zone of learning and development. The EPS, and assessment for learning more generally, are conceptualised as social fields for the exchange of capital (Bourdieu 1977, 1990). The research is underpinned by a sociocultural theoretical perspective that focuses on how students and teachers at the Flexible Learning Centre (FLC) develop and learn, within the zone of proximal development (Vygotsky, 1978). The EPS is seen to be highly effective in the engagement and social interaction between students, teachers and institutions. It is argued throughout this paper that the EPS provides a structurally identifiable space, an arena of social activity, or a field of exchange. The students, teachers and the FLC within this field are producing cultural capital exchanges. The term efield (exchange field) has been coined to refer to this constructed abstract space. Initial results from the trial show a general tendency towards engagement with the EPS and potential for the attainment of socially valued cultural capital in the form of school credentials.

An investigation into online moderation

The purpose of this paper is to understand how teachers’ identity as an assessor in a standards-referenced assessment system may be developed through their participation in online social moderation meetings. In these meetings teachers negotiate and share their understandings of assessment standards and judgement decisions. In particular, the paper focuses on the relationship between the technology, the moderation processes and teachers’ development in this assessment system. This paper draws on sociocultural theories of learning to analyse the qualitative data collected through observations of eleven online moderation meetings and interviews of the teachers involved in these meetings. The results provide insights into the mediating role of the technology with regard to teachers’ development of shared meanings and common practices within a standards-referenced assessment system.

Learning as identity and practice through involvement in online moderation

For teachers working in a standards-based assessment system, professional conversations through organised social moderation meetings are a vital element. This qualitative research investigated the learning that occurred as a result of online moderation discussions. Findings illustrate how participating in social moderation meetings in an online context can support teachers to understand themselves as assessors, and can provide opportunities for teachers to imagine possibilities for their teaching that move beyond the moderation practice.

Breaching the standardised assessment boundary : assessment for learning as a field of exchange

The introduction of the Australian curriculum, the use of standardised testing (e.g. NAPLAN) and the My School website are couched in a context of accountability. This circumstance has stimulated and in some cases renewed a range of boundaries in Australian Education. The consequences that arise from standardised testing have accentuated the boundaries produced by social reproduction in education which has led to an increase in the numbers of students disengaging from mainstream education and applying for enrolment at the Edmund Rice Education Australia Flexible Learning Centre Network (EREAFLCN). Boundaries are created for many young people who are denied access to credentials and certification as a result of being excluded from or in some way disengaging from standardised education and testing. Young people who participate at the EREAFLCN arrive with a variety of forms of cultural capital that are not valued in current education and employment fields. This is not to say that these young people’s different forms of cultural capital have no value, but rather that such funds of knowledge, repertoires and cultural capital are not valued by the majority of powerful agents in educational and employment fields. How then can the qualitative value of traditionally unorthodox - yet often intricate, ingenious, and astute - versions of cultural capital evident in the habitus of many young people be made to count, be recognised, be valuated? Can a process of educational assessment be a field of capital exchange and a space which breaches boundaries through a valuating process? This paper reports on the development of an innovative approach to assessment in an alternative education institution designed for the re-engagement of ‘at risk’ youth who have left formal schooling. A case study approach has been used to document the engagement of six young people, with an educational approach described as assessment for learning as a field of exchange across two sites in the EREAFLCN. In order to capture the broad range of students’ cultural and social capital, an electronic portfolio system (EPS) is under trial. The model draws on categories from sociological models of capital and reconceptualises the eportfolio as a sociocultural zone of learning and development. Results from the trial show a general tendency towards engagement with the EPS and potential for the attainment of socially valued cultural capital in the form of school credentials. In this way restrictive boundaries can be breached and a more equitable outcome achieved for many young Australians.

Towards general capabilities through technology : assessment for learning as a field of exchange

The introduction of the Australian curriculum, the use of standardised testing (e.g. NAPLAN) and the My School website have stimulated and in some cases renewed a range of boundaries for young people in Australian Education. Standardised testing has accentuated social reproduction in education with an increase in the numbers of students disengaging from mainstream education and applying for enrolment at the Edmund Rice Education Australia Flexible Learning Centre Network (EREAFLCN). Many young people are denied access to credentials and certification as they become excluded from standardised education and testing. The creativity and skills of marginalised youth are often evidence of general capabilities and yet do not appear to be recognised in mainstream educational institutions when standardised approaches are adopted. Young people who participate at the EREAFLCN arrive with a variety of forms of cultural capital, frequently utilising general capabilities, which are not able to be valued in current education and employment fields. This is not to say that these young people‟s different forms of cultural capital have no value, but rather that such funds of knowledge, repertoires and cultural capital are not valued by the majority of powerful agents in educational and employment fields. How then can the inherent value of traditionally unorthodox - yet often intricate, ingenious, and astute-versions of cultural capital evident in the habitus of many young people be made to count, be recognised, be valuated?Can a process of educational assessment be a field of capital exchange and a space which crosses boundaries through a valuating process? This paper reports on the development of an innovative approach to assessment in an alternative education institution designed for the re engagement of „at risk‟ youth who have left formal schooling. A case study approach has been used to document the engagement of six young people, with an educational approach described as assessment for learning as a field of exchange across two sites in the EREAFLCN. In order to capture the broad range of students‟ cultural and social capital, an electronic portfolio system (EPS) is under trial. The model draws on categories from sociological models of capital and reconceptualises the eportfolio as a sociocultural zone of learning and development. Results from the trial show a general tendency towards engagement with the EPS and potential for the attainment of socially valued cultural capital in the form of school credentials. In this way restrictive boundaries can be breached and a more equitable outcome achieved for many young Australians.