Sustainable assessment for large science classes : non-multiple choice, randomised assignments through a Learning Management System

This paper reports on the development of a tool that generates randomised, non-multiple choice assessment within the BlackBoard Learning Management System interface. An accepted weakness of multiple-choice assessment is that it cannot elicit learning outcomes from upper levels of Biggs’ SOLO taxonomy. However, written assessment items require extensive resources for marking, and are susceptible to copying as well as marking inconsistencies for large classes. This project developed an assessment tool which is valid, reliable and sustainable and that addresses the issues identified above. The tool provides each student with an assignment assessing the same learning outcomes, but containing different questions, with responses in the form of words or numbers. Practice questions are available, enabling students to obtain feedback on their approach before submitting their assignment. Thus, the tool incorporates automatic marking (essential for large classes), randomised tasks to each student (reducing copying), the capacity to give credit for working (feedback on the application of theory), and the capacity to target higher order learning outcomes by requiring students to derive their answers rather than choosing them. Results and feedback from students are presented, along with technical implementation details.

Development and evaluation of an integrated clinical learning model to inform continuing education for acute care nurses

Background Significant ongoing learning needs for nurses have occurred as a direct result of the continuous introduction of technological innovations and research developments in the healthcare environment. Despite an increased worldwide emphasis on the importance of continuing education, there continues to be an absence of empirical evidence of program and session effectiveness. Few studies determine whether continuing education enhances or develops practice and the relative cost benefits of health professionals’ participation in professional development. The implications for future clinical practice and associated educational approaches to meet the needs of an increasingly diverse multigenerational and multicultural workforce are also not well documented. There is minimal research confirming that continuing education programs contribute to improved patient outcomes, nurses’ earlier detection of patient deterioration or that standards of continuing competence are maintained. Crucially, evidence-based practice is demonstrated and international quality and safety benchmarks are adhered to. An integrated clinical learning model was developed to inform ongoing education for acute care nurses. Educational strategies included the use of integrated learning approaches, interactive teaching concepts and learner-centred pedagogies. A Respiratory Skills Update education (ReSKU) program was used as the content for the educational intervention to inform surgical nurses’ clinical practice in the area of respiratory assessment. The aim of the research was to evaluate the effectiveness of implementing the ReSKU program using teaching and learning strategies, in the context of organisational utility, on improving surgical nurses’ practice in the area of respiratory assessment. The education program aimed to facilitate better awareness, knowledge and understanding of respiratory dysfunction in the postoperative clinical environment. This research was guided by the work of Forneris (2004), who developed a theoretical framework to operationalise a critical thinking process incorporating the complexities of the clinical context. The framework used educational strategies that are learner-centred and participatory. These strategies aimed to engage the clinician in dynamic thinking processes in clinical practice situations guided by coaches and educators. Methods A quasi experimental pre test, post test non–equivalent control group design was used to evaluate the impact of the ReSKU program on the clinical practice of surgical nurses. The research tested the hypothesis that participation in the ReSKU program improves the reported beliefs and attitudes of surgical nurses, increases their knowledge and reported use of respiratory assessment skills. The study was conducted in a 400 bed regional referral public hospital, the central hub of three smaller hospitals, in a health district servicing the coastal and hinterland areas north of Brisbane. The sample included 90 nurses working in the three surgical wards eligible for inclusion in the study. The experimental group consisted of 36 surgical nurses who had chosen to attend the ReSKU program and consented to be part of the study intervention group. The comparison group included the 39 surgical nurses who elected not to attend the ReSKU program, but agreed to participate in the study. Findings One of the most notable findings was that nurses choosing not to participate were older, more experienced and less well educated. The data demonstrated that there was a barrier for training which impacted on educational strategies as this mature aged cohort was less likely to take up educational opportunities. The study demonstrated statistically significant differences between groups regarding reported use of respiratory skills, three months after ReSKU program attendance. Between group data analysis indicated that the intervention group’s reported beliefs and attitudes pertaining to subscale descriptors showed statistically significant differences in three of the six subscales following attendance at the ReSKU program. These subscales included influence on nursing care, educational preparation and clinical development. Findings suggest that the use of an integrated educational model underpinned by a robust theoretical framework is a strong factor in some perceptions of the ReSKU program relating to attitudes and behaviour. There were minimal differences in knowledge between groups across time. Conclusions This study was consistent with contemporary educational approaches using multi-modal, interactive teaching strategies and a robust overarching theoretical framework to support study concepts. The construct of critical thinking in the clinical context, combined with clinical reasoning and purposeful and collective reflection, was a powerful educational strategy to enhance competency and capability in clinicians.

Developing shared understandings of standards-based assessment : online moderation practices across geographically diverse contexts

This thesis investigates the place of online moderation in supporting teachers to work in a system of standards-based assessment. The participants of the study were fifty middle school teachers who met online with the aim of developing consistency in their judgement decisions. Data were gathered through observation of the online meetings, interviews, surveys and the collection of artefacts. The data were viewed and analysed through sociocultural theories of learning and sociocultural theories of technology, and demonstrates how utilising these theories can add depth to understanding the added complexity of developing shared meaning of standards in an online context. The findings contribute to current understanding of standards-based assessment by examining the social moderation process as it acts to increase the reliability of judgements that are made within a standards framework. Specifically, the study investigates the opportunities afforded by conducting social moderation practices in a synchronous online context. The study explicates how the technology affects the negotiation of judgements and the development of shared meanings of assessment standards, while demonstrating how involvement in online moderation discussions can support teachers to become and belong within a practice of standards-based assessment. This research responds to a growing international interest in standards-based assessment and the use of social moderation to develop consistency in judgement decisions. Online moderation is a new practice to address these concerns on a systemic basis.

Building on the past for the future : assessment for learning as a field of exchange

Standardised testing does not recognise the creativity and skills of marginalised youth. This paper presents the development of an innovative approach to assessment designed for the re-engagement of at risk youth who have left formal schooling and are now in an alternative education institution. An electronic portfolio system (EPS) has been developed to capture, record and build on the broad range of students’ cultural and social capital. The assessment as a field of exchange model draws on categories from sociological fields of capital and reconceptualises an eportfolio and social networking hybrid system as a sociocultural zone of learning and development. The EPS, and assessment for learning more generally, are conceptualised as social fields for the exchange of capital (Bourdieu 1977, 1990). The research is underpinned by a sociocultural theoretical perspective that focuses on how students and teachers at the Flexible Learning Centre (FLC) develop and learn, within the zone of proximal development (Vygotsky, 1978). The EPS is seen to be highly effective in the engagement and social interaction between students, teachers and institutions. It is argued throughout this paper that the EPS provides a structurally identifiable space, an arena of social activity, or a field of exchange. The students, teachers and the FLC within this field are producing cultural capital exchanges. The term efield (exchange field) has been coined to refer to this constructed abstract space. Initial results from the trial show a general tendency towards engagement with the EPS and potential for the attainment of socially valued cultural capital in the form of school credentials.

A case study of how primary school teachers in Papua New Guinea understand outcomes-based education

This case study explored how a group of primary school teachers in Papua New Guinea (PNG) understood Outcomes-based Education (OBE). OBE measures students. learning against specific outcomes. These outcomes are derived from a country.s vision of the kind of citizen that the education system should produce. While countries such as Australia, South Africa, New Zealand and the United States have abandoned OBE, others such as PNG have adopted it in various ways. How teachers understand OBE in PNG is important because such understandings are likely to influence how they implement the OBE curriculum. There has been no research to date which has investigated PNG primary school teachers. understandings and experiences with OBE. This study used a single exploratory case study design to investigate how twenty primary school teachers from the National Capital District (NCD) in PNG understood OBE. The study, underpinned by an intepretivist paradigm, explored the research question: How do primary school teachers understand outcomes-based education in PNG? The data comprised surveys, in-depth interviews and documents. Data were analysed thematically and using explanation building techniques. The findings revealed that OBE is viewed by teachers as a way to equip them with additional strategies for planning and programming, teaching and learning, and assessment. Teachers also described how OBE enabled both students and teachers to become more engaged and develop positive attitudes towards teaching and learning. There was also a perception that OBE enhanced students. future life skills through increased local community support. While some teachers commented on how the OBE reforms provided them with increased professional development opportunities, the greatest impediment to implementing OBE was perceived to be a lack of sufficient teaching and learning resources. The process of planning and programming classroom activities was also regarded as onerous. Some teachers indicated that they had been required to implement OBE without adequate in-service training support. The social constructivist theory of knowledge which underpins OBE.s student-centred pedagogy can cause tensions within PNG.s cultural contexts of teaching and learning. Teachers need to be aware of these tensions when conducting peer or group learning under OBE in PNG. By exploring how these PNG primary teachers understood OBE, the study highlighted how teachers engaged with OBE concepts when interpreting syllabus documents and how they applied these concepts to curriculum. Identifying differences in teacher understanding of OBE provides guidance for both the design of materials to support the implementation of OBE and for the design of in-service training. Thus, the outcomes of this study will inform educators about the implementation of OBE in PNG. In addition, the outcomes will provide much needed insight into how a mandated curriculum and pedagogical reform impacts teachers‟ practices in PNG.

Theorising a model for teaching and assessing reflective learning in higher education

The importance of reflection in higher education, and across disciplinary fields is widely recognised; it is generally included in university graduate attributes, professional standards and program objectives. Furthermore, reflection is commonly embedded into assessment requirements in higher education subjects, often without necessary scaffolding or clear expectations for students. Despite the rhetoric around the importance of reflection for ongoing learning, there is scant literature on any systematic, developmental approach to teaching reflective learning across higher education programs/courses. Given that professional or academic reflection is not intuitive, and requires specific pedagogic intervention to do well, a program/course-wide approach is essential. This paper draws on current literature to theorise a new, transferable and customisable model for teaching and assessing reflective learning across higher education, which foregrounds and explains the pedagogic field of higher education as a multi-dimensional space. We argue that explicit and strategic pedagogic intervention, supported by dynamic resources, is necessary for successful, broad-scale approaches to reflection in higher education.

Conceptualising and teaching discursive and performative reflection in higher education

The importance of reflection in higher education, and across disciplinary fields is widely recognised. It is generally embedded in university graduate attributes, professional standards and course objectives. Furthermore, reflection is commonly included in assessment requirements in higher education subjects, often without necessary scaffolding or clear expectations for students. It is essential that academic staff have substantive knowledge and clear expectations about the aims of reflective activities, the most effective mode of representation, and appropriate teaching strategies to support students in deep, critical reflection. The paper argues the case for reflection to be represented in different modes, using discursive (language) or performative (symbolic practice) forms of expression according to disciplinary context and individual communicative strengths. It introduces key discursive and expressive elements that constitute different modes of representation in reflective tasks. This functional analysis of textual elements provides explicit knowledge for teaching and assessing multiple modes of reflection in higher education.

Assessment policy change in relation to English language teaching and learning in China : a study of perspectives from two universities

In response to concerns about the quality of English Language Learning (ELL) education at tertiary level, the Chinese Ministry of Education (CMoE) launched the College English Reform Program (CERP) in 2004. By means of a press release (CMoE, 2005) and a guideline document titled College English Curriculum Requirements (CECR) (CMoE, 2007), the CERP proposed two major changes to the College English assessment policy, which were: (1) the shift to optional status for the compulsory external test, the College English Test Band 4 (CET4); and (2) the incorporation of formative assessment into the existing summative assessment framework. This study investigated the interactions between the College English assessment policy change, the theoretical underpinnings, and the assessment practices within two Chinese universities (one Key University and one Non-Key University). It adopted a sociocultural theoretical perspective to examine the implementation process as experienced by local actors of institutional and classroom levels. Systematic data analysis using a constant comparative method (Merriam, 1998) revealed that contextual factors and implementation issues did not lead to significant differences in the two cases. Lack of training in assessment and the sociocultural factors such as the traditional emphasis on the product of learning and hierarchical teacher/students relationship are decisive and responsible for the limited effect of the reform.

The student as customer model and its impact on the academic leadership role in higher education

This paper posits that the 'student as customer' model has a negative impact upon the academic leadership which in turn is responsible for the erosion of objectivity in the assessment process in the higher education sector. The paper draws on the existing literature to explore the relationship between the student as customer model, academic leadership, and student assessment. The existing research emanating from the literature provides the basis from which the short comings of the student as customer model are exposed. From a practical perspective the arguments made in this paper provide the groundwork for possible future research into the adverse affects of the student as customer model on academic leadership and job satisfaction in the academic work force. The concern for quality may benefit from empirical investigation of the relationship between the student as customer model and quality learning and assessment outcomes in the higher education sector. The paper raises awareness of the faults with the present reliance on the student as customer model and the negative impact on both students and academic staff. The issues explored have the potential to influence the future directions of the higher education sector with regard to the social implications of their quest for quality educational outcomes. The paper addresses a gap in the literature in regard to use of the student as customer model and the subsequent adverse affect on academic leadership and assessment in higher education.

Nurses driving change to improve access to education and quality in lcinical teaching : A partnership between Australia and Viet Nam

In Viet Nam, standards of nursing care fail to meet international competency standards. This increases risks to patient safety (eg. hospital acquired infection), consequently the Ministry of Health identified the need to strengthen nurse education in Viet Nam. This paper presents experiences of a piloted clinical teaching model developed in Ha Noi, to strengthen nurse led institutional capacity for in-service education and clinical teaching. Historically 90% of nursing education was conducted by physicians and professional development in hospitals for nurses was limited. There was minimal communication between hospitals and nursing schools about expectations of students and assessment and quality of the learning experience. As a result when students came to the clinical sites, no-one understood how to plan their learning objectives and utilise teaching and learning approaches appropriate to their level. Therefore student learning outcomes were variable. They focussed on procedures and techniques and “learning how to do” rather than learning how to plan, implement and evaluate patient care. This project is part of a multi-component capacity building program designed to improve nurse education in Viet Nam. The project was funded jointly by Queensland University of Technology (QUT) and the Australian Agency for International Development. Its aim was to develop a collaborative clinically-based model of teaching to create an environment that encourages evidence-based, student-centred clinical learning. Accordingly, strategies introduced promoted clinical teaching of competency based nursing practice utilising the regionally endorsed nurse core competency standards. Thirty nurse teachers from Viet Duc University Hospital and Hanoi Medical College participated in the program. These nurses and nurse teachers undertook face to face education in three workshops, and completed three assessment items. Assessment was applied, where participants integrated the concepts learned in each workshop and completed assessment tasks related to planning, implementing and evaluating teaching in the clinical area. Twenty of these participants were then selected to undertake a two week study tour in Brisbane, Australia where the clinical teaching model was refined and an action plan developed to integrate into both organisations with possible implementation across Viet Nam. Participants on this study tour also experienced clinical teaching and learning at QUT by attending classes held at the university, and were able to visit selected hospitals to experience clinical teaching in these settings as well. Effectiveness of the project was measured throughout the implementation phase and in follow up visits to the clinical site. To date changes have been noted on an individual and organisational level. There is also significant planning underway to incorporate the clinical teaching model developed across the organisation and how this may be implemented in other regions. Two participants have also been involved in disseminating aspects of this approach to clinical teaching in Ho Chi Minh, with further plans for more in-depth dissemination to occur throughout the country.

Integration of CENA practice standards for postgraduate emergency nursing assessment

The use of professional competency standards to assess postgraduate nursing student’s clinical performance has been in place since 2004, at the Queensland University of Technology, School of Nursing & Midwifery (SONAM) when the Graduate Certificate in Emergency Nursing degree commenced. Emergency nursing students were assessed in their workplace, using a Clinical Performance Appraisal Tool or CPAT which was based on the Australian College of Critical Care Nurses (ACCCN) Competency Standards. With the subsequent formation of a separate Emergency Nursing Course advisory group in 2007, there was a review of clinical assessment course component. The release of the 2008 CENA revised Practice Standards for the Emergency Nursing Specialist’s, led to the emergency nursing course advisory committee supporting the integration of the CENA practice standards for assessment of emergency nurses in preference to the less relevant ACCCN competency standards. The SONAM emergency nursing study area team commenced the phasing in and progression of the CENA practice standards across the two Graduate Certificate units, and Graduate Diploma and Master of Nursing (emergency) clinical major options in 2009. As some units undertaken in the degree are available to nurses in other disciplines a separate CPAT was devised for the clinical assessments according to speciality context. The team has had to carefully consider how the professional standards are integrated into the teaching and assessment of the unit and not just applied instead of the ACCCN competency standards. Professional standards for the emergency context has also helped tailor course content and learning outcomes to be relevant across a number of emergency nursing contexts in Australia. The assessment of the CPAT is undertaken at the workplace by QUT appointed clinical lecturers. Clinical lecturers need to apply and have suitable postgraduate qualification to undertake the position. The clinical lecturer support role is well established at QUT. The integration of the new CENA practice standards has necessitated a review of the postgraduate assessment of emergency nurses. A clinical lecturer workshop has been organised to review role, scope and how to utilise the new look CENA based CPAT, clinical assessment format.

Reasonable adjustment and the assessment of students with disabilities: Australian legal issues and trends

Student assessment is particularly important, and particularly controversial, because it is the means by which student achievement is determined. Reasonable adjustment to student assessment is of equal importance as the means of ensuring the mitigation, or even elimination, of disability related barriers to the demonstration of student achievement. The significance of reasonable adjustment is obvious in the later years of secondary school, and in the tertiary sector, because failure to adjust assessment may be asserted as the reason a student did not achieve as well as anticipated or as the reason a student was excluded from a course and, as a result, from future study and employment opportunities. Even in the early years of schooling, however, assessment and its management are a critical issue for staff and students, especially in an education system like Australia’s with an ever increasing emphasis on national benchmarks testing. This paper will explain the legislation which underpins the right to reasonable adjustment in education in Australian schools. It will give examples of the kinds of adjustment which may be made to promote equality of opportunity in the area of assessment. It will also consider some of the controversies which have confronted, or which, it may be speculated, are likely to confront Australian education institutions as they work towards compliance with reasonable adjustment laws.

Developing the Pattern and Structure Assessment (PASA) interview to inform early mathematics learning

The Pattern and Structure Mathematical Awareness Program(PASMAP) stems from a 2-year longitudinal study on students’ early mathematical development. The paper outlines the interview assessment the Pattern and Structure Assessment(PASA) designed to describe students’ awareness of mathematical pattern and structure across a range of concepts. An overview of students’ performance across items and descriptions of their structural development are described.

Closing the loop 21st century style : providing feedback on written assessment via MP3 recordings

Feedback on student performance, whether in the classroom or on written assignments, enables them to reflect on their understandings and restructure their thinking in order to develop more powerful ideas and capabilities. Research has identified a number of broad principles of good feedback practice. These include the provision of feedback that facilitates the development of reflection in learning; helps clarify what good performance is in terms of goals, criteria and expected standards; provides opportunities to close the gap between current and desired performance; delivers high quality information to students about their learning; and encourages positive motivational beliefs and self-esteem. However, high staff–student ratios and time pressures often result in a gulf between this ideal and reality. Whilst greater use of criteria referenced assessment has enabled an improvement in the extent of feedback being provided to students, this measure alone does not go far enough to satisfy the requirements of good feedback practice. Technology offers an effective and efficient means by which personalised feedback may be provided to students. This paper presents the findings of a trial of the use of the freely available Audacity program to provide individual feedback via MP3 recordings to final year Media Law students at the Queensland University of Technology on their written assignments. The trial has yielded wide acclaim by students as an effective means of explaining the exact reasons why they received the marks they were awarded, the things they did well and the areas needing improvement. It also showed that good feedback practice can be achieved without the burden of an increase in staff workload.