A model for enhancing assessment and teaching practice at the coalface : Insights from a Fellow-In-Residence engagement program

Australian universities are currently engaging with new governmental policies and regulations that require them to demonstrate enhanced quality and accountability in teaching and research. The development of national academic standards for learning outcomes in higher education is one such instance of this drive for excellence. These discipline-specific standards articulate the minimum, or Threshold Learning Outcomes, to be addressed by higher education institutions so that graduating students can demonstrate their achievement to their institutions, accreditation agencies, and industry recruiters. This impacts not only on the design of Engineering courses (with particular emphasis on pedagogy and assessment), but also on the preparation of academics to engage with these standards and implement them in their day-to-day teaching practice on a micro level. This imperative for enhanced quality and accountability in teaching is also significant at a meso level, for according to the Australian Bureau of Statistics, about 25 per cent of teachers in Australian universities are aged 55 and above and more than 54 per cent are aged 45 and above (ABS, 2006). A number of institutions have undertaken recruitment drives to regenerate and enrich their academic workforce by appointing capacity-building research professors and increasing the numbers of early- and mid-career academics. This nationally driven agenda for quality and accountability in teaching permeates also the micro level of engineering education, since the demand for enhanced academic standards and learning outcomes requires both a strong advocacy for a shift to an authentic, collaborative, outcomes-focused education and the mechanisms to support academics in transforming their professional thinking and practice. Outcomes-focused education means giving greater attention to the ways in which the curriculum design, pedagogy, assessment approaches and teaching activities can most effectively make a positive, verifiable difference to students’ learning. Such education is authentic when it is couched firmly in the realities of learning environments, student and academic staff characteristics, and trustworthy educational research. That education will be richer and more efficient when staff works collaboratively, contributing their knowledge, experience and skills to achieve learning outcomes based on agreed objectives. We know that the school or departmental levels of universities are the most effective loci of changes in approaches to teaching and learning practices in higher education (Knight & Trowler, 2000). Heads of Schools are being increasingly entrusted with more responsibilities - in addition to setting strategic directions and managing the operational and sometimes financial aspects of their school, they are also expected to lead the development and delivery of the teaching, research and other academic activities. Guiding and mentoring individuals and groups of academics is one critical aspect of the Head of School’s role. Yet they do not always have the resources or support to help them mentor staff, especially the more junior academics. In summary, the international trend in undergraduate engineering course accreditation towards the demonstration of attainment of graduate attributes poses new challenges in addressing academic staff development needs and the assessment of learning. This paper will give some insights into the conceptual design, implementation and empirical effectiveness to date, of a Fellow-In-Residence Engagement (FIRE) program. The program is proposed as a model for achieving better engagement of academics with contemporary issues and effectively enhancing their teaching and assessment practices. It will also report on the program’s collaborative approach to working with Heads of Schools to better support academics, especially early-career ones, by utilizing formal and informal mentoring. Further, the paper will discuss possible factors that may assist the achievement of the intended outcomes of such a model, and will examine its contributions to engendering an outcomes-focussed thinking in engineering education.

Using real-time authentic online learning scenarios to teach criminal intelligence

Criminal intelligence is an area of expertise highly sought-after internationally and within a variety of justice-related professions; however, producing university graduates with the requisite professional knowledge, as well as analytical, organisational and technical skills presents a pedagogical and technical challenge to university educators. The situation becomes even more challenging when students are undertaking their studies by distance education. This best practice session showcases the design of an online undergraduate unit for final year justice students which uses an evolving real-time criminal scenario as the focus of authentic learning activities in order to prepare students for graduate roles within the criminal intelligence and justice professions. Within the unit, students take on the role of criminal intelligence analysts, applying relevant theories, models and strategies to solve a complex but realistic crime and complete briefings and documentation to industry standards as their major summative assessment task. The session will demonstrate how the design of the online unit corresponds to authentic learning principles, and will specifically map the elements of the unit design to Herrington & Oliver’s instructional design framework for authentic learning (2000; Herrington & Herrington 2006). The session will show how a range of technologies was used to create a rich learning experience for students that could be easily maintained over multiple unit iterations without specialist technical support. The session will also discuss the unique pedagogical affordances and challenges implicated in the location of the unit within an online learning environment, and will reflect on some of the lessons learned from the development which may be relevant to other authentic online learning contexts.

Using real-time authentic online learning scenarios to teach criminal intelligence : presentation

Criminal intelligence is an area of expertise highly sought-after internationally and within a variety of justice-related professions; however, producing university graduates with the requisite professional knowledge, as well as analytical, organisational and technical skills presents a pedagogical and technical challenge to university educators. The situation becomes even more challenging when students are undertaking their studies by distance education. This best practice session showcases the design of an online undergraduate unit for final year justice students which uses an evolving real-time criminal scenario as the focus of authentic learning activities in order to prepare students for graduate roles within the criminal intelligence and justice professions. Within the unit, students take on the role of criminal intelligence analysts, applying relevant theories, models and strategies to solve a complex but realistic crime and complete briefings and documentation to industry standards as their major summative assessment task. The session will demonstrate how the design of the online unit corresponds to authentic learning principles, and will specifically map the elements of the unit design to Herrington & Oliver’s instructional design framework for authentic learning (2000; Herrington & Herrington 2006). The session will show how a range of technologies was used to create a rich learning experience for students that could be easily maintained over multiple unit iterations without specialist technical support. The session will also discuss the unique pedagogical affordances and challenges implicated in the location of the unit within an online learning environment, and will reflect on some of the lessons learned from the development which may be relevant to other authentic online learning contexts.

Pre-service teachers’ perceptions of authentic learning in a capstone unit

Capstone units in higher education courses are learning experiences which are designed to bring reflection and focus to a whole course of study while, at the same time, leading students toward their entry into a new world of work (Humphrey, Brown, & Benson, 2005). The capstone experience described in this paper runs as a conference, called the Stepping Out Conference. The conference is delivered as one unit and is mandatory for all fourth year pre-service teachers. Participation in a capstone unit is an effective way for students to begin thinking of themselves as teaching professionals rather than as continuing students. Students engage in the assessment as part of their capstone experience. The assessment is designed to measure students’ knowledge and skills as they relate to ‘authentic’ real life situations (Darling-Hammond, 1991). This paper details pre-service teachers’ experiences of authentic learning through their participation in a capstone unit.

Pioneering use of narrative method in a first-year law course to provide an authentic, inquiry-based entry into the discipline of law

Law is saturated with stories. People tell their stories to lawyers; lawyers tell their client's stories to courts; and legislators develop regulation to respond to their constituent's stories of injustice or inequality. My approach to first-year legal education respects this narrative tradition. Both my curriculum design and assessment scheme in the compulsory first-year subject Australian Legal System deploy narrative methodology as the central teaching and learning device. Throughout the course, students work on resolving the problems of four hypothetical clients. Like a murder mystery, pieces of the puzzle come together as students learn more about legal institutions and the texts they produce, the process of legal research, the analysis and interpretation of primary legal sources, the steps in legal problem-solving, the genre conventions of legal writing style, the practical skills and ethical dimensions of professional practice, and critical inquiry into the normative underpinnings and impacts of the law. The assessment scheme mirrors this design. In their portfolio-based assignment, for example, students devise their own client profile, research the client's legal position and prepare a memorandum of advice.

Context-based assessment: Creating opportunities for resonance between classroom fields and societal fields

There is on-going international interest in the relationships between assessment instruments, students’ understanding of science concepts and context-based curriculum approaches. This study extends earlier research showing that students can develop connections between contexts and concepts – called fluid transitions – when studying context-based courses. We provide an in-depth investigation of one student’s experiences with multiple contextual assessment instruments that were associated with a context-based course. We analyzed the student’s responses to context-based assessment instruments to determine the extent to which contextual tests, reports of field investigations, and extended experimental investigations afforded her opportunities to make connections between contexts and concepts. A system of categorizing student responses was developed that can inform other educators when analyzing student responses to contextual assessment. We also refine the theoretical construct of fluid transitions that informed the study initially. Implications for curriculum and assessment design are provided in light of the findings.

Temporary tattoos:a novel OSCE assessment tool

Background: There are many issues regarding the use of real patients in objective structured clinical examinations (OSCEs). In dermatology OSCE stations, standardised patients (SPs) with clinical photographs are often used. Temporary transfer tattoos can potentially simulate skin lesions when applied to an SP. This study aims to appraise the use of temporary malignant melanoma tattoos within an OSCE framework. Method: Within an 11-station OSCE, a temporary malignant melanoma tattoo was developed and applied to SPs in a 'skin lesion' OSCE station. A questionnaire captured the opinions of the candidate, SP and examiners, and the degree of perceived realism of each station was determined. Standard post hoc OSCE analysis determined the psychometric reliability of the stations. Results: The response rates were 95.9 per cent of candidates and 100 per cent of the examiners and SPs. The 'skin lesion' station achieved the highest realism score compared with other stations: 89.0 per cent of candidates felt that the skin lesion appeared realistic; only 28 per cent of candidates had ever seen a melanoma before in training. The psychometric performance of the melanoma station was comparable with, and in many instances better than, other OSCE stations. Discussion: Transfer tattoo technology facilitates a realistic dermatology OSCE station encounter. Temporary tattoos, alongside trained SPs, provide an authentic, standardised and reliable experience, allowing the assessment of integrated dermatology clinical skills.

Enseñar a aprender a pensar en los centros educativos, incluso en las actividades de evaluación.= Teaching to learn how to think at schools, even within assessment activities

El auténtico protagonismo de los centros educativos tiene que dirigirse a ayudar a pensar a sus alumnos y a enseñarlos a aprender, es decir, el docente tiene que enseñar estrategias de aprendizaje y debe promover el esfuerzo del estudiante para facilitar la construcción de esquemas y el aprendizaje permanente. El profesor debe utilizar cualquier situación de aprendizaje para enseñar dichas estrategias de aprendizaje, incluso en las situaciones de evaluación; por lo tanto, en este trabajo se sugiere que en las evaluaciones de los alumnos y alumnas se tenga en cuenta la metacognición como factor fundamental en el aprendizaje y la enseñanza

Authenticity in assessment : reflecting undergraduate study and professional practice

The literature suggests that assessment is a powerful tool for influencing student study habits. It is also recognized that there is a tension between traditional forms of assessment and newer forms of assessment that offer a more authentic representation of practice, but are more complex and expensive to administer. The international trend in undergraduate engineering course accreditation to move to demonstration of attainment of graduate attributes poses new challenges in assessment of learning. A case study based on integrating assessment practices across the year levels of an engineering management studies stream in an undergraduate course is presented. Key features of the assessment portfolio include: the use of assessment in the first year as a foundational tool to establish student study habits and skills; the evolution of assessment tasks by the fourth year to reflect the world of professional practice and to allow students to demonstrate their integration of knowledge and skills; the weighting of assessment tasks to indicate the value attached to particular tasks; the structured inclusion of group work; a concern for student and staff workloads; the recognition of student diversity, in particular the needs of off-campus and mature-age students; and the matching of assessment tasks to professional accreditation requirements.

The impact of teaching models, group structures and assessment modes on cooperative learning in the student design studio

As a result of ever diminishing teaching resources, an increasing number of architectural educators are setting group design projects, rather then spreading their time thinly over a large number of individual projects. This allows them to co-ordinate longer and more in-depth review sessions on a smaller number of assignment submissions. However, while the group
model may offer an authentic learning model by reflecting design in practice, the approach is not without its obvious shortcomings as a teaching archetype for the assessment of the knowledge and skill competencies of individual students. Hence, what is clear is the need for a readily adoptable andragogy for the teaching and assessment of group design projects.
The following paper describes the background, methodology and findings of a Strategic Teaching and Learning Grant funded research project carried out in the year 2005 at the School of Architecture and Building at Deakin University. The project aimed to inform a change of classroom/studio practice governing the assemblage, teaching and assessment of student design teams. The development through these changes of cooperative and student centred learning principles focused on effective design collaboration and fair assessment should, it will be argued, lead to an enhanced group-learning experience in studio, which will subsequently and ultimately enhance professional practice.

Shifting to more authentic language and literacy activity in the middle years

This paper reports on the shifts in literacy teaching and learning that occurred at a Melbourne primary school, one of twelve schools that took part in a large research project undertaken by staff at Deakin University funded by the Victorian Department of Education, Training and Youth Affairs: Middle Years Literacy Research Project. The project focused on literacy teaching, learning, and assessment of students in the middle years of schooling. Through close collaboration between the researcher and teachers at the school, significant changes were made to the language and literacy program. These changes reflected current language theory and extended the school's focus on independent learning to the area of literacy. The development of more authentic ways of assessing student learning grew out of the work in the project as teachers sought assessment practices that were consistent with their philosophy of teaching and learning. With a focus on developing authentic literacy practices, teachers developed new ways of tracking and reporting student achievement.

Using online environments to promote assessment as a learning enhancement process

The 21st century presents new challenges relating to the need for assessment to be more innovative, more responsive to students' needs, and more relevant and authentic. The characteristics of good assessment are well known and resources abound to assist academic staff to change their practices. Yet, at a time when the imperative for reconsidering practices has never been greater, it appears to be very difficult to bring about and maintain substantive change in assessment. This paper presents case studies that illustrate the way some academic staff have responded to the challenges. In particular, how the online environment can enhance learning through formative assessment is illustrated with four case studies. The paper outlines specific challenges faced in each case and  discusses issues that arose during development and delivery. It concludes by identifying some of the factors that helped to facilitate changes in assessment practices in these specific cases.

The role of assessment in the development of judgement

This paper introduces an integrated assessment model developed within a project management discipline stream in a Construction Management course. Following Boud and Falchikov (2007) this model starts with practice, that is, the actual ‘doing’ of project management as the basis for shaping assessment that equips students to learn for the rest of their lives. Practice is understood as a holistic conception of what professionals do in particular contexts, and a theoretical construct that provides a method of framing ways in which we can investigate the world (Schatzki, 2001). This approach opens the way for considerations of assessment that engage with, and cultivate, certain kinds of professional learning and identity formation including the development of judgement. Integral to the model is the non-sequential nature of assessment activities, evolving team formation and ongoing self and communal reflection. The paper concludes that the use of an authentic and integrated assessment model creates a compelling learning environment that contributes meaningfully to the development of skills, knowledge and identities for future professional learning.

Teacher performance assessment in teacher education : an example in Malaysia

As part of a cross-cultural collaboration, a teacher performance assessment (TPA) was implemented during 2009 in three Malaysian institutes of teacher education. This paper reports on the TPA for graduating primary teachers in Malaysia. The investigation focused on the pre-service teachers’ perceptions about whether the TPA provided them with an opportunity to document successfully their professional learning and professional practice. Successful completion of the Malaysian TPA was closely aligned to successful relationships, support and collaboration between Malaysian lecturers and pre-service teachers, and between pre-service teachers and their classroom teachers. Overall, the TPA did provide pre-service teachers with an opportunity to focus on the connection between theory and professional learning during field-work, and to become reflective evidence-based practitioners. Recommendations for improving the assessment of pre-service teachers are discussed.

Initial teacher education and assessment of graduates in Australia

In this chapter we discuss the impact of recent standards-based legislation and implementation in Australia designed to improve the quality of teacher education and examine the ways in which teacher educators are assessing graduates in relation to beginning teacher standards, the evidence they are using within the context of accreditation of their programs, and the impact this is having on the teacher education curriculum. We report on research into the implementation and evaluation of an authentic teacher assessment approach being used in one Australian university and argue that such an approach not only empowers pre-service teachers but also teacher educators.