Moderation and consistency of teacher judgement : teachers’ views

Major curriculum and assessment reforms in Australia have generated research interest in issues related to standards, teacher judgement and moderation. This article is based on one related inquiry of a large-scale Australian Research Council Linkage project conducted in Queensland. This qualitative study analysed interview data to identify teachers’ views on standards and moderation as a means to achieving consistency of teacher judgement. A complementary aspect of the research involved a blind review that was conducted to determine the degree of teacher consistency without the experience of moderation. Empirical evidence was gained that most teachers, of the total interviewed, articulated a positive attitude towards the use of standards in moderation and perceived that this process produces consistency in teachers’ judgements. Context was identified as an important influential factor in teachers’ judgements and it was concluded that teachers’ assessment beliefs, attitudes and practices impact on their perceptions of the value of moderation practice and the extent to which consistency can be achieved.

Analysing mentoring dialogues for developing a preservice teacher’s classroom management practices

Mentors (cooperating classroom teachers) have a shared responsibility with universities for developing preservice teachers’ pedagogical practices, particularly towards becoming reflective practitioners. Preservice teachers need to participate actively in their own learning, by reflecting and acting on the mentor’s constructive feedback provided during planning and feedback dialogue sessions. This case study uses feedback practices outlined within a five-factor mentoring model to analyse dialogue between a mentor and her respective mentee during different stages in their school-based programs (first practicum). This investigation uses multiple data sources such as video and audio-recorded interviews, archival documents from participants such as lesson plans, reflections and reports to examine preservice teacher’s reflections and implementations of practice as a result of her mentor’s feedback (e.g., establish expectations, review lesson plans, observe teaching then provide oral and written feedback, and evaluate progress). Findings indicated that reflective thinking was more apparent when the mentor did not dominate conversations but instead asked astute pedagogical knowledge questions to facilitate the mentee’s reflections on practice.

Multimodal approaches to reflective teaching and assessment in higher education

Reflection is not a new concept in the teaching of higher education and is often an important component of many disciplinary courses. Despite this, past research shows that whilst there are examples of rich reflective strategies used in some areas of higher education, most approaches to and conceptualisations of reflective learning and assessment have been perfunctory and inconsistent. In many disciplinary areas reflection is often assessed as a written activity ‘tagged onto’ assessment practices. In creative disciplines however, reflective practice is an integral and cumulative form of learning and is often expressed in ways other than in the written form. This paper will present three case studies of reflective practice in the area of Creative Industries in higher education – Dance, Fashion and Music. It will discuss the ways in which higher education teachers and students use multi-modal approaches to expressing knowledge and reflective practice in context. The paper will argue that unless students are encouraged to participate in deep reflective disciplinary discourse via multi-modes then reflection will remain superficial in the higher education context.

Researching quality and equity in assessment for improved learner outcomes

Quality and equity issues as they relate to assessment practices and policy are becoming increasingly important as Australia introduces a national curriculum and achievement standards. In a context of high-stakes accountability, concerns regarding equity and quality have important implications for teachers‘ practice for the improvement of learner outcomes. This article is based on three research projects that were conducted over the past four years and were funded by the Australian Federal Government. The research focus, emergent questions, the educational contexts, and the rationale for the studies are discussed prior to the presentation of the analysis of the research findings and the implications for teachers‘ practice and policy reform.

A model for enhancing assessment and teaching practice at the coalface : Insights from a Fellow-In-Residence engagement program

Australian universities are currently engaging with new governmental policies and regulations that require them to demonstrate enhanced quality and accountability in teaching and research. The development of national academic standards for learning outcomes in higher education is one such instance of this drive for excellence. These discipline-specific standards articulate the minimum, or Threshold Learning Outcomes, to be addressed by higher education institutions so that graduating students can demonstrate their achievement to their institutions, accreditation agencies, and industry recruiters. This impacts not only on the design of Engineering courses (with particular emphasis on pedagogy and assessment), but also on the preparation of academics to engage with these standards and implement them in their day-to-day teaching practice on a micro level. This imperative for enhanced quality and accountability in teaching is also significant at a meso level, for according to the Australian Bureau of Statistics, about 25 per cent of teachers in Australian universities are aged 55 and above and more than 54 per cent are aged 45 and above (ABS, 2006). A number of institutions have undertaken recruitment drives to regenerate and enrich their academic workforce by appointing capacity-building research professors and increasing the numbers of early- and mid-career academics. This nationally driven agenda for quality and accountability in teaching permeates also the micro level of engineering education, since the demand for enhanced academic standards and learning outcomes requires both a strong advocacy for a shift to an authentic, collaborative, outcomes-focused education and the mechanisms to support academics in transforming their professional thinking and practice. Outcomes-focused education means giving greater attention to the ways in which the curriculum design, pedagogy, assessment approaches and teaching activities can most effectively make a positive, verifiable difference to students’ learning. Such education is authentic when it is couched firmly in the realities of learning environments, student and academic staff characteristics, and trustworthy educational research. That education will be richer and more efficient when staff works collaboratively, contributing their knowledge, experience and skills to achieve learning outcomes based on agreed objectives. We know that the school or departmental levels of universities are the most effective loci of changes in approaches to teaching and learning practices in higher education (Knight & Trowler, 2000). Heads of Schools are being increasingly entrusted with more responsibilities - in addition to setting strategic directions and managing the operational and sometimes financial aspects of their school, they are also expected to lead the development and delivery of the teaching, research and other academic activities. Guiding and mentoring individuals and groups of academics is one critical aspect of the Head of School’s role. Yet they do not always have the resources or support to help them mentor staff, especially the more junior academics. In summary, the international trend in undergraduate engineering course accreditation towards the demonstration of attainment of graduate attributes poses new challenges in addressing academic staff development needs and the assessment of learning. This paper will give some insights into the conceptual design, implementation and empirical effectiveness to date, of a Fellow-In-Residence Engagement (FIRE) program. The program is proposed as a model for achieving better engagement of academics with contemporary issues and effectively enhancing their teaching and assessment practices. It will also report on the program’s collaborative approach to working with Heads of Schools to better support academics, especially early-career ones, by utilizing formal and informal mentoring. Further, the paper will discuss possible factors that may assist the achievement of the intended outcomes of such a model, and will examine its contributions to engendering an outcomes-focussed thinking in engineering education.

Examining the Assessment Opportunities for Cultural Connectedness for Student Learning : A Sociocultural Analysis

The disparity that exists between the highest and lowest achievers together with deficit approaches to teaching, learning and assessment raise serious equity issues related to fairness, validity, culture and access, which were analysed in a recent Australian Research Council funded project. This chapter explores the potential that exists for teachers to work with Indigenous Teacher Assistants (ITAs) to secure cultural connectedness in teaching, learning and assessment of Indigenous students. The study was a design experiment, which was conducted in seven Catholic and Independent primary schools in northern Queensland and involved semi-structured focus group interviews with Year 4 and 6 Indigenous students, principals, teachers and Indigenous Teacher Assistants. Classroom observations and document analyses were also conducted. This corpus of data was analysed using a sociocultural theoretical lens. The use of a sociocultural analysis helped to identify cultural influences, Indigenous students’ funds of knowledge and values. The information from this analysis was made explicit to teachers to demonstrate how they could enhance their pedagogic and assessment practices by embracing and extending the cultural spaces for learning and teaching of Indigenous students. The way in which teachers construct their interactions for greater cultural connectedness and enhanced learning would appear to rely on relationship building with Indigenous staff, Indigenous students’ cultural knowledge, and improved understanding of assessment and related equity issues.

Towards fairer assessment

Drawing on the largest Australian collection and analysis of empirical data on multiple facets of Aboriginal and Torres Strait Islander education in state schools to date, this article critically analyses the systemic push for standardized testing and improved scores, and argues for a greater balance of assessment types by providing alternative, inclusive, participatory approaches to student assessment. The evidence for this article derives from a major evaluation of the Stronger Smarter Learning Communities. The first large-scale picture of what is occurring in classroom assessment and pedagogy for Indigenous students is reported in this evaluation yet the focus in this article remains on the issue of fairness in student assessment. The argument presented calls for “a good balance between formative and summative assessment” (OECD, Synergies for Better Learning An International Perspective on Evaluation and Assessment, Pointers for Policy Development, 2013) at a time of unrelenting high-stakes, standardized testing in Australia with a dominance of secondary as opposed to primary uses of NAPLAN data by systems, schools and principals. A case for more “intelligent accountability in education” (O’Neill, Oxford Review of Education 39(1):4–16, 2013) together with a framework for analyzing efforts toward social justice in education (Cazden, International Journal of Educational Psychology 1(3):178–198, 2012) and fairer assessment make the case for more alternative assessment practices in recognition of the need for teachers’ pedagogic practice to cater for increased diversity.

Elements of Better Assessment for the Improvement of Learning

Building a community of shared practice at the classroom level calls for clarity about the important assessment capabilities and dispositions of teachers, especially when teachers are expected to take a direct focus on learning. In this chapter, we present new ways of thinking about teachers’ assessment literacies, offering a formulation of better assessment for the improvement of learning, including three elements, namely (i) assessment criteria and standards; (ii) the teacher’s professional judgment; and (iii) social moderation. The potential of the first element lies in teachers’ classroom practices that deliberately embed assessment criteria and standards in pedagogy in productive ways. The second element involves the engagement of teachers and students in judgment practice, that develops the understanding that judgment involves more than the application of explicit or stated criteria. More fundamental is the matter of how teachers bring to bear stated features of quality and other intellectual and experiential resources in arriving at judgment. That is to say, they range across and orient to explicit (stated), tacit (unstated) and meta-criteria in judgment making. These insights have direct relevance to teachers’ efforts to develop students’ own evaluative experience, especially as this involves students working with stated features of quality for self-assessment and peer-assessment purposes. Further, practices for social moderation are discussed, giving examples of good practice in moderation, how teachers experience moderation and the potential benefits of various types.

Teachers’ understandings of their relationships with students : pedagogic connectedness

This paper reports on a doctoral study that explored the nature of pedagogic connectedness and revealed the ways in which teachers experience this phenomenon. Pedagogic connectedness is defined as the engagements between teacher and student that impact on student learning. In this study, twenty teachers in an independent college in South-East Queensland, Australia, were interviewed and the interview transcripts analysed iteratively. Five qualitatively different ways of experiencing pedagogic connectedness emerged from the data. The findings of this phenomenographic-related study are instructive in developing a framework for changes to teachers’ pedagogic practices.

The enactment of the New Basics Project in a special school

This study investigates the impact of the New Basics Project on teachers at a special school for students with intellectual impairments. The study is aimed at exploring the complex nature of the work of special educators as they enact the New Basics curriculum with a particular focus on the teachers’ opinions about challenges that arose for their curriculum, pedagogy and assessment practices. Attention is also paid to how the principal’s leadership supported the enactment of the New Basics in respect to what he did and why he used particular strategies. The nine teachers and their principal were involved in a series of in-depth, semi-structured interviews from one of only three special schools in phase one of the New Basics trial in Queensland, Australia. These interviews produced data from the special educators as they were confronted with a new curriculum that challenged their previous teaching practices. The enactment of the New Basics curriculum occurred within the context of a state-sanctioned mandate to provide alternative programs to those offered in mainstream schools, for students with special needs. This thesis explores these teachers' experiences using critical theory as a basis for analyzing their opinions on issues such as the role of the special educator, tensions between old and new curricula, pedagogical and assessment practices, and connections between the at-school learning experiences for intellectually impaired students and the realities of post-school life. The investigation also examines the leadership conduct of the principal in changing times at the school. The findings suggest that the New Basics has played a significant role in providing structures for developing communities of practice amongst teachers; in supporting special educators to focus more on the educational needs of the students (e.g., literacy, numeracy, financial planning) and less on their medical needs (e.g., toileting, feeding, personal hygiene); and supporting school leadership that empowers and listens critically to teachers as essential components of the successful enactment of curriculum reforms like the New Basics.

Identity and disciplinarity : Multi-disciplinary student cohorts in Higher Education [Abstract]

This paper will describe a research project that examines the implications of multidisciplinary student cohorts on teaching and learning within undergraduate and postgraduate units in higher education. Whist students generally specialise in one discipline, it is also common that, at some point during their degree, they will choose to undertake subjects that are outside their specialist area. Students may choose a multidisciplinary learning experience either out of interest or because the subject is seen as complementary to their core discipline. When the lens of identity is applied to the multi-disciplinary cohorts in undergraduate and postgraduate units, it assists in identifying learning needs. The nature of disciplinarity, and the impact it has on students’ academic identity, presents challenges to both students and teachers when they engage in teaching and learning, impacting on curriculum design, assessment practices and teaching delivery strategies (Winberg, 2008). This project aims to identify the barriers that exist to effective teaching and learning in units that have multidisciplinary student cohorts. It will identify the particular needs of students in multidisciplinary student cohorts and determine a teaching and learning model that meets the needs of such cohorts. References Becher, T. & Trowler, P.R. (2001). Academic tribes and territories: Intellectual enquiry and the culture of the discipline. Buckingham, UK: Open University Press. Light, G. & Cox, R. (2001). Learning and teaching in higher education: A reflective professional. Thousand Oaks, CA: Sage. Neumann, R. (2001). Disciplinary differences and university teaching. Studies in Higher Education, 26 (2), 135-46. Neumann, R., Parry, S. & Becher, T. (2002). Teaching and Learning in their disciplinary contexts: A conceptual analysis. Studies in Higher Education, 27(4), 405-417. Taylor, P.G. (1999) Making Sense of Academic Life: Academics, Universities and Change. Buckingham, UK: Open University Press. Winberg, C. (2008). Teaching engineering/engineering teaching: interdisciplinary collaboration and the construction of academic identities. Teaching in Higher Education, 13(3), 353 - 367.

Accounts of the visual art classroom : catering for artistically talented students

Inclusive education practices call for the diverse and individual needs of all students to be met satisfactorily. The needs and experiences of artistically talented students in Australian visual art classrooms are currently unknown. This study addresses this gap in research through an inquiry into the experiences of artistically talented students and their teachers in visual art classrooms, by examining the accounts of a group of students and teachers at one high school in South East Queensland. This study is significant as it provides teachers, parents and others involved in the education of artistically talented students with additional means to plan and cater for the educational needs of artistically talented students. Teacher and student accounts of the visual art classroom in this study indicated that identification processes for artistically talented students are unclear and contradictory. Furthermore, teacher and student accounts of their experiences presented a wide variety of conceptions of the visual art classroom and point towards an individualised approach to learning for artistically talented students. This study also discovered a mismatch between assessment practices in the subject visual art and assessment of art in the ‘real world’. Specifically, this study proposes a renewal of programs for artistically talented students, and recommends a revision of current procedures for the identification of artistically talented students in visual art classrooms.

The development and evaluation of an innovative nursing practice model to improve undergraduate nursing students' competence in paediatric physical assessment

Introduction The purpose of this study was to develop, implement and evaluate the impact of an educational intervention, comprising an innovative model of clinical decisionmaking and educational delivery strategy for facilitating nursing students‘ learning and development of competence in paediatric physical assessment practices. Background of the study Nursing students have an undergraduate education that aims to produce graduates of a generalist nature who demonstrate entry level competence for providing nursing care in a variety of health settings. Consistent with population morbidity and health care roles, paediatric nursing concepts typically form a comparatively small part of undergraduate curricula and students‘ exposure to paediatric physical assessment concepts and principles are brief. However, the nursing shortage has changed traditional nursing employment patterns and new graduates form the majority of the recruitment pool for paediatric nursing speciality staff. Paediatric nursing is a popular career choice for graduates and anecdotal evidence suggests that nursing students who select a clinical placement in their final year intend to seek employment in paediatrics upon graduation. Although concepts of paediatric nursing are included within undergraduate curriculum, students‘ ability to develop the required habits of mind to practice in what is still regarded as a speciality area of practice is somewhat limited. One of the areas of practice where this particularly impacts is in paediatric nursing physical assessment. Physical assessment is a fundamental component of nursing practice and competence in this area of practice is central to nursing students‘ development of clinical capability for practice as a registered nurse. Timely recognition of physiologic deterioration of patients is a key outcome of nurses‘ competent use of physical assessment strategies, regardless of the practice context. In paediatric nursing contexts children‘s physical assessment practices must specifically accommodate the child‘s different physiological composition, function and pattern of clinical deterioration (Hockenberry & Barrera, 2007). Thus, to effectively manage physical assessment of patients within the paediatric practice setting nursing students need to integrate paediatric nursing theory into their practice. This requires significant information processing and it is in this process where students are frequently challenged. The provision of rules or models can guide practice and assist novice-level nurses to develop their capabilities (Benner, 1984; Benner, Hooper-Kyriakidis & Stannard, 1999). Nursing practice models are cognitive tools that represent simplified patterns of expert analysis employing concepts that suit the limited reasoning of the inexperienced, and can represent the =rules‘ referred to by Benner (1984). Without a practice model of physical assessment students are likely to be uncertain about how to proceed with data collection, the interpretation of paediatric clinical findings and the appraisal of findings. These circumstances can result in ad hoc and unreliable nursing physical assessment that forms a poor basis for nursing decisions. The educational intervention developed as part of this study sought to resolve this problem and support nursing students‘ development of competence in paediatric physical assessment. Methods This study utilised the Context Input Process Product (CIPP) Model by Stufflebeam (2004) as the theoretical framework that underpinned the research design and evaluation methodology. Each of the four elements in the CIPP model were utilised to guide discrete stages of this study. The Context element informed design of the clinical decision-making process, the Paediatric Nursing Physical Assessment model. The Input element was utilised in appraising relevant literature, identifying an appropriate instructional methodology to facilitate learning and educational intervention delivery to undergraduate nursing students, and development of program content (the CD-ROM kit). Study One employed the Process element and used expert panel approaches to review and refine instructional methods, identifying potential barriers to obtaining an effective evaluation outcome. The Product element guided design and implementation of Study Two, which was conducted in two phases. Phase One employed a quasiexperimental between-subjects methodology to evaluate the impact of the educational intervention on nursing students‘ clinical performance and selfappraisal of practices in paediatric physical assessment. Phase Two employed a thematic analysis and explored the experiences and perspectives of a sample subgroup of nursing students who used the PNPA CD-ROM kit as preparation for paediatric clinical placement. Results Results from the Process review in Study One indicated that the prototype CDROM kit containing the PNPA model met the predetermined benchmarks for face validity and the impact evaluation instrumentation had adequate content validity in comparison with predetermined benchmarks. In the first phase of Study Two the educational intervention did not result in statistically significant differences in measures of student performance or self-appraisal of practice. However, in Phase Two qualitative commentary from students, and from the expert panel who reviewed the prototype CD-ROM kit (Study One, Phase One), strongly endorsed the quality of the intervention and its potential for supporting learning. This raises questions regarding transfer of learning and it is likely that, within this study, several factors have influenced students‘ transfer of learning from the educational intervention to the clinical practice environment, where outcomes were measured. Conclusion In summary, the educational intervention employed in this study provides insights into the potential e-learning approaches offer for delivering authentic learning experiences to undergraduate nursing students. Findings in this study raise important questions regarding possible pedagogical influences on learning outcomes, issues within the transfer of theory to practice and factors that may have influenced findings within the context of this study. This study makes a unique contribution to nursing education, specifically with respect to progressing an understanding of the challenges faced in employing instructive methods to impact upon nursing students‘ development of competence. The important contribution transfer of learning processes make to students‘ transition into the professional practice context and to their development of competence within the context of speciality practice is also highlighted. This study contributes to a greater awareness of the complexity of translating theoretical learning at undergraduate level into clinical practice, particularly within speciality contexts.

Evaluation of the year 1 literacy and numeracy checkpoints assessments trial - 2010

The draft Year 1 Literacy and Numeracy Checkpoints Assessments were in open and supported trial during Semester 2, 2010. The purpose of these trials was to evaluate the Year 1 Literacy and Numeracy Checkpoints Assessments (hereafter the Year 1 Checkpoints) that were designed in 2009 as a way to incorporate the use of the Year 1 Literacy and Numeracy Indicators as formative assessment in Year 1 in Queensland Schools. In these trials there were no mandated reporting requirements. The processes of assessment were related to future teaching decisions. As such the trials were trials of materials and the processes of using those materials to assess students, plan and teach in year 1 classrooms. In their current form the Year 1 Checkpoints provide assessment resources for teachers to use in February, June and October. They aim to support teachers in monitoring children's progress and making judgments about their achievement of the targeted P‐3 Literacy and Numeracy Indicators by the end of Year 1 (Queensland Studies Authority, 2010 p. 1). The Year 1 Checkpoints include support materials for teachers and administrators, an introductory statement on assessment, work samples, and a Data Analysis Assessment Record (DAAR) to record student performance. The Supported Trial participants were also supported with face‐to‐face and on‐line training sessions, involvement in a moderation process after the October Assessments, opportunities to participate in discussion forums as well as additional readings and materials. The assessment resources aim to use effective early years assessment practices in that the evidence is gathered from hands‐on teaching and learning experiences, rather than more formal assessment methods. They are based in a model of assessment for learning, and aim to support teachers in the “on‐going process of determining future learning directions” (Queensland Studies Authority, 2010 p. 1) for all students. Their aim is to focus teachers on interpreting and analysing evidence to make informed judgments about the achievement of all students, as a way to support subsequent planning for learning and teaching. The Evaluation of the Year 1 Literacy and Numeracy Checkpoints Assessments Supported Trial (hereafter the Evaluation) aimed to gather information about the appropriateness, effectiveness and utility of the Year 1 Checkpoints Assessments from early years’ teachers and leaders in up to one hundred Education Queensland schools who had volunteered to be part of the Supported Trial. These sample schools represent schools across a variety of Education Queensland regions and include schools with:  - A high Indigenous student population; - Urban, rural and remote school locations; - Single and multi‐age early phase classes; - A high proportion of students from low SES backgrounds. The purpose of the Evaluation was to: Evaluate the materials and report on the views of school‐based staff involved in the trial on the process, materials, and assessment practices utilised. The Evaluation has reviewed the materials, and used surveys, interviews, and observations of processes and procedures to collect relevant data to help present an informed opinion on the Year 1 Checkpoints as assessment for the early years of schooling. Student work samples and teacher planning and assessment documents were also collected. The evaluation has not evaluated the Year 1 Checkpoints in any other capacity than as a resource for Year 1 teachers and relevant support staff.

Assessment for learning in the accountability era : Queensland, Australia

Developments in school education in Australia over the past decade have witnessed the rise of national efforts to reform curriculum, assessment and reporting. Constitutionally the power to decide on curriculum matters still resides with the States. Higher stakes in assessment, brought about by national testing and international comparative analyses of student achievement data, have challenged State efforts to maintain the emphasis on assessment to promote learning while fulfilling accountability demands. In this article lessons from the Queensland experience indicate that it is important to build teachers' assessment capacity and their assessment literacy for the promotion of student learning. It is argued that teacher assessment can be a source of dependable results through moderation practice. The Queensland Studies Authority has recognised and supported the development of teacher assessment and moderation practice in the context of standards-driven, national reform. Recent research findings explain how the focus on learning can be maintained by avoiding an over-interpretation of test results in terms of innate ability and limitations and by encouraging teachers to adopt more tailored diagnosis of assessment data to address equity through focus on achievement for all. Such efforts are challenged as political pressures related to the Australian government’s implementation of national testing and national partnership funding arrangements tied to the performance of students at or below minimum standards become increasingly apparent.