Does the summative assessment of real world learning using criterion-referenced assessment need to be discipline specific?

This paper synthesises the existing literature on the contemporary conception of ‘real world’ and compares it with similar notions such as ‘authentic’ and ‘work integrated learning’. While the term ‘real world’ may be partly dependent on the discipline, it does not necessarily follow that the criterion-referenced assessment of ‘real world’ assessment must involve criteria and performance descriptors that are discipline specific. Two examples of summative assessment (court report and trial process exercise) from a final year core subject at the Queensland University of Technology, LWB432 Evidence, emphasise real world learning, are authentic, innovative and better prepare students for the transition into the workplace than more generic forms of assessment such as tutorial participation or oral presentations. The court report requires students to attend a criminal trial in a Queensland Court and complete a two page report on what they saw in practice compared with what they learned in the classroom. The trial process exercise is a 50 minute written closed book activity conducted in tutorials, where students plan questions that they would ask their witness in examination-in-chief, plan questions that they would ask their opponent’s witness in cross-examination, plan questions that they would ask in reexamination given what their opponent asked in cross-examination, and prepare written objections to their opponent’s questions. The trial process exercise simulates the real world, whereas the court report involves observing the real world, and both assessment items are important to the role of counsel. The design of the criterion-referenced assessment rubrics for the court report and trial process exercise is justified by the literature. Notably, the criteria and performance descriptors are not necessarily law specific and this paper highlights the parts that may be easily transferred to other disciplines.

Development and evaluation of an integrated clinical learning model to inform continuing education for acute care nurses

Background Significant ongoing learning needs for nurses have occurred as a direct result of the continuous introduction of technological innovations and research developments in the healthcare environment. Despite an increased worldwide emphasis on the importance of continuing education, there continues to be an absence of empirical evidence of program and session effectiveness. Few studies determine whether continuing education enhances or develops practice and the relative cost benefits of health professionals’ participation in professional development. The implications for future clinical practice and associated educational approaches to meet the needs of an increasingly diverse multigenerational and multicultural workforce are also not well documented. There is minimal research confirming that continuing education programs contribute to improved patient outcomes, nurses’ earlier detection of patient deterioration or that standards of continuing competence are maintained. Crucially, evidence-based practice is demonstrated and international quality and safety benchmarks are adhered to. An integrated clinical learning model was developed to inform ongoing education for acute care nurses. Educational strategies included the use of integrated learning approaches, interactive teaching concepts and learner-centred pedagogies. A Respiratory Skills Update education (ReSKU) program was used as the content for the educational intervention to inform surgical nurses’ clinical practice in the area of respiratory assessment. The aim of the research was to evaluate the effectiveness of implementing the ReSKU program using teaching and learning strategies, in the context of organisational utility, on improving surgical nurses’ practice in the area of respiratory assessment. The education program aimed to facilitate better awareness, knowledge and understanding of respiratory dysfunction in the postoperative clinical environment. This research was guided by the work of Forneris (2004), who developed a theoretical framework to operationalise a critical thinking process incorporating the complexities of the clinical context. The framework used educational strategies that are learner-centred and participatory. These strategies aimed to engage the clinician in dynamic thinking processes in clinical practice situations guided by coaches and educators. Methods A quasi experimental pre test, post test non–equivalent control group design was used to evaluate the impact of the ReSKU program on the clinical practice of surgical nurses. The research tested the hypothesis that participation in the ReSKU program improves the reported beliefs and attitudes of surgical nurses, increases their knowledge and reported use of respiratory assessment skills. The study was conducted in a 400 bed regional referral public hospital, the central hub of three smaller hospitals, in a health district servicing the coastal and hinterland areas north of Brisbane. The sample included 90 nurses working in the three surgical wards eligible for inclusion in the study. The experimental group consisted of 36 surgical nurses who had chosen to attend the ReSKU program and consented to be part of the study intervention group. The comparison group included the 39 surgical nurses who elected not to attend the ReSKU program, but agreed to participate in the study. Findings One of the most notable findings was that nurses choosing not to participate were older, more experienced and less well educated. The data demonstrated that there was a barrier for training which impacted on educational strategies as this mature aged cohort was less likely to take up educational opportunities. The study demonstrated statistically significant differences between groups regarding reported use of respiratory skills, three months after ReSKU program attendance. Between group data analysis indicated that the intervention group’s reported beliefs and attitudes pertaining to subscale descriptors showed statistically significant differences in three of the six subscales following attendance at the ReSKU program. These subscales included influence on nursing care, educational preparation and clinical development. Findings suggest that the use of an integrated educational model underpinned by a robust theoretical framework is a strong factor in some perceptions of the ReSKU program relating to attitudes and behaviour. There were minimal differences in knowledge between groups across time. Conclusions This study was consistent with contemporary educational approaches using multi-modal, interactive teaching strategies and a robust overarching theoretical framework to support study concepts. The construct of critical thinking in the clinical context, combined with clinical reasoning and purposeful and collective reflection, was a powerful educational strategy to enhance competency and capability in clinicians.

An instrument to measure the approach to learning for assessment in economics

Economics education research studies conducted in the UK, USA and Australia to investigate the effects of learning inputs on academic performance have been dominated by the input-output model (Shanahan and Meyer, 2001). In the Student Experience of Learning framework, however, the link between learning inputs and outputs is mediated by students' learning approaches which in turn are influenced by their perceptions of the learning contexts (Evans, Kirby, & Fabrigar, 2003). Many learning inventories such as Biggs' Study Process Questionnaires and Entwistle and Ramsden' Approaches to Study Inventory have been designed to measure approaches to academic learning. However, there is a limitation to using generalised learning inventories in that they tend to aggregate different learning approaches utilised in different assessments. As a result, important relationships between learning approaches and learning outcomes that exist in specific assessment context(s) will be missed (Lizzio, Wilson, & Simons, 2002). This paper documents the construction of an assessment specific instrument to measure learning approaches in economics. The post-dictive validity of the instrument was evaluated by examining the association of learning approaches to students' perceived assessment demand in different assessment contexts.

A matrix for appropriate assessment of workplace learning in stem disciplines

With increasing interest shown by Universities in workplace learning, especially in STEM disciplines, an issue has arisen amongst educators and industry partners regarding authentic assessment tasks for work integrated learning (WIL) subjects. This paper describes the use of a matrix, which is also available as a decision-tree, based on the features of the WIL experience, in order to facilitate the selection of appropriate assessment strategies. The matrix divides the WIL experiences into seven categories, based on such factors as: the extent to which the experience is compulsory, required for membership of a professional body or elective; whether the student is undertaking a project, or embedding in a professional culture; and other key aspects of the WIL experience. One important variable is linked to the fundamental purpose of the assessment. This question revolves around the focus of the assessment: whether on the person (student development); the process (professional conduct/language); or the product (project, assignment, literature review, report, software). The matrix has been trialed at QUT in the Faculty of Science and Technology, and also at the University of Surrey, UK, and has proven to have good applicability in both universities.

What are students' understandings of how digital tools contribute to learning in design disciplines?

Building Information Modelling (BIM) is evolving in the Construction Industry as a successor to CAD. CAD is mostly a technical tool that conforms to existing industry practices, however BIM has the capacity to revolutionise industry practice. Rather than producing representations of design intent, BIM produces an exact Virtual Prototype of any building that in an ideal situation is centrally stored and freely exchanged between the project team, facilitating collaboration and allowing experimentation in design. Exposing design students to this technology through their formal studies allows them to engage with cutting edge industry practices and to help shape the industry upon their graduation. Since this technology is relatively new to the construction industry, there are no accepted models for how to “teach” BIM effectively at university level. Developing learning models to enable students to make the most out of their learning with BIM presents significant challenges to those teaching in the field of design. To date there are also no studies of students experiences of using this technology. This research reports on the introduction of Building Information Modeling (BIM) software into a second year Bachelor of Design course. This software has the potential to change industry standards through its ability to revolutionise the work practices of those involved in large scale design projects. Students’ understandings and experiences of using the software in order to complete design projects as part of their assessment are reported here. In depth semi-structured interviews with 6 students revealed that students had views that ranged from novice to sophisticate about the software. They had variations in understanding of how the software could be used to complete course requirements, to assist with the design process and in the workplace. They had engaged in limited exploration of the collaborative potential of the software as a design tool. Their understanding of the significance of BIM for the workplace was also variable. The results indicate that students are beginning to develop an appreciation for how BIM could aid or constrain the work of designers, but that this appreciation is highly varied and likely to be dependent on the students’ previous experiences of working in a design studio environment. Their range of understandings of the significance of the technology is a reflection of their level of development as designers (they are “novice” designers). The results also indicate that there is a need for subjects in later years of the course that allow students to specialise in the area of digital design and to develop more sophisticated views of the role of technology in the design process. There is also a need to capitalise on the collaborative potential inherent in the software in order to realise its capability to streamline some aspects of the design process. As students become more sophisticated designers we should explore their understanding of the role of technology as a design tool in more depth in order to make recommendations for improvements to teaching and learning practice related to BIM and other digital design tools.

Students’ conceptual knowledge about workplace pedagogies and applications to learning in the workplace

The development of effective workplace pedagogies is integral to work-integrated and work-based learning. The workplace pedagogies that facilitate and support learning include: activities in which individuals engage such as daily work practices, questioning; observing, and listening; interactions with more experienced workers through coaching and modelling; and referencing documented procedures. Each of these dimensions is significant in enhancing processes of workplace learning. Learning can be optimised when these pedagogies are appropriately embedded in the context of and process of participating in normal work activities. Yet learners need to understand the nature of these pedagogies, and how and for what purposes to use these to achieve a range of learning outcomes. This is because it is the worker-learners who play the key roles in the process and outcomes of learning through work. A pilot study was conducted on students’ conceptions of how each of these dimensions of workplace pedagogy help their learning, by providing examples of learning from these sources; and stating their preferences for learning in the workplace. A sample of seventeen students, enrolled in the second year of a Diploma in Nursing course at a Technical and Further Education institution, participated in a survey intended to capture these conceptions and the importance attached to each of them. The findings indicate that these students have basic understanding of how each of seven workplace pedagogic practices can contribute to their learning. They reported relying mostly on daily practices, observing and listening to others, modelling, coaching, and other workers. Their selection of these contributions emphasise significant opportunities for guided learning by others, yet suggest fewer student-initiated interactions, less intensity in interactions, and likelihood that learning is more passive. The data also suggests that these students rely mostly on using academic learning skill, and limited workplace learning skills. It is proposed, therefore, that the knowledge and understandings about workplace learning and pedagogies might be best embedded in the full curriculum and not become add-on shortly before students go on work placement. This approach will allow students to appreciate the significance and use of workplace pedagogies for learning.

High fidelity simulation; challenges encountered in developing and implementing student learning experiences, staff education and researching approaches to learning

High fidelity simulation as a teaching and learning approach is being embraced by many schools of nursing. Our school embarked on integrating high fidelity (HF) simulation into the undergraduate clinical education program in 2011. Low and medium fidelity simulation has been used for many years, but this did not simplify the integration of HF simulation. Alongside considerations of how and where HF simulation would be integrated, issues arose with: student consent and participation for observed activities; data management of video files; staff development, and conceptualising how methods for student learning could be researched. Simulation for undergraduate student nurses commenced as a formative learning activity, undertaken in groups of eight, where four students undertake the ‘doing’ role and four are structured observers, who then take a formal role in the simulation debrief. Challenges for integrating simulation into student learning included conceptualising and developing scenarios to trigger students’ decision making and application of skills, knowledge and attitudes explicit to solving clinical ‘problems’. Developing and planning scenarios for students to ‘try out’ skills and make decisions for problem solving lay beyond choosing pre-existing scenarios inbuilt with the software. The supplied scenarios were not concept based but rather knowledge, skills and technology (of the manikin) focussed. Challenges lay in using the technology for the purpose of building conceptual mastery rather than using technology simply because it was available. As we integrated use of HF simulation into the final year of the program, focus was on building skills, knowledge and attitudes that went beyond technical skill, and provided an opportunity to bridge the gap with theory-based knowledge that students often found difficult to link to clinical reality. We wished to provide opportunities to develop experiential knowledge based on application and clinical reasoning processes in team environments where problems are encountered, and to solve them, the nurse must show leadership and direction. Other challenges included students consenting for simulations to be videotaped and ethical considerations of this. For example if one student in a group of eight did not consent, did this mean they missed the opportunity to undertake simulation, or that others in the group may be disadvantaged by being unable to review their performance. This has implications for freely given consent but also for equity of access to learning opportunities for students who wished to be taped and those who did not. Alongside this issue were the details behind data management, storage and access. Developing staff with varying levels of computer skills to use software and undertake a different approach to being the ‘teacher’ required innovation where we took an experiential approach. Considering explicit learning approaches to be trialled for learning was not a difficult proposition, but considering how to enact this as research with issues of blinding, timetabling of blinded groups, and reducing bias for testing results of different learning approaches along with gaining ethical approval was problematic. This presentation presents examples of these challenges and how we overcame them.

Teaching and assessing critical thinking : the interaction of student approaches to learning and teaching approaches

This study explored the relationship among student approaches to learning and teaching methods on critical thinking in two business units. Key findings included differences in critical thinking scores between student approaches to learning and some evidence of an interaction between student approaches to learning and critical thinking teach method (immersion vs. infusion). Possible explanations for the results are examined and implications for developing critical thinking skills across a degree discussed. What is apparent is that as Universities move towards program-wide level assessment of critical thinking, further work is required in terms of the design of critical thinking teaching interventions and assessment at the unit, school, and degree level. The session will discuss the challenges in developing critical thinking programs in individual units and at the Faculty level.

Curriculum, assessment teaching and learning for all

This chapter explores how the culture of classrooms and schools can acknowledge diversity and meet all learning needs. Classroom and school culture can and should enhance the belonging and learning of all students. Understanding of learning, curriculum, pedagogy and assessment influences the ways teachers consider their expectations of student achievement and participation in school. We revisit the theory of social constructionism to emphasise the development of shared and valued curriculum, which meets all learner needs. Decisions about what to teach, how to teach and assess, and what supports student needs are important considerations discussed in this chapter. Key messages drawn from the Australian and New Zealand curriculum reinforce the need to ensure education responds to the diversity of students in classrooms. A range of models of pedagogy that have influenced education in Australia and New Zealand are presented, with a particular focus on meeting the needs of students who have disabilities. In addition, the issues related to student and teacher identity, the importance of respectful partnerships that acknowledge family knowledge, and respectful collaboration are discussed. Belonging to a community of learners is made possible through teachers forming authentic relationships with students and their families. In turn, these relationships support teachers to understand how the students in their classrooms learn, and to know their students’ strengths and interests.

Faculty mentoring, evidence-based assessment, and student learning : an Australian exploration of American initiatives

There have been many improvements in Australian engineering education since the 1990s. However, given the recent drive for assuring the achievement of identified academic standards, more progress needs to be made, particularly in the area of evidence-based assessment. This paper reports on initiatives gathered from the literature and engineering academics in the USA, through an Australian National Teaching Fellowship program. The program aims to establish a process to help academics in designing and implementing evidence-based assessments that meet the needs of not only students and the staff that teach them, but also industry as well as accreditation bodies. The paper also examines the kinds and levels of support necessary for engineering academics, especially early career ones, to help meet the expectations of the current drive for assured quality and standards of both research and teaching. Academics are experiencing competing demands on their time and energy with very high expectations in research performance and increased teaching responsibilities, although many are researchers who have not had much pedagogic training. Based on the literature and investigation of relevant initiatives in the USA, we conducted interviews with several identified experts and change agents who have wrought effective academic cultural change within their institutions and beyond. These reveal that assuring the standards and quality of student learning outcomes through evidence-based assessments cannot be appropriately addressed without also addressing the issue of pedagogic training for academic staff. To be sustainable, such training needs to be complemented by a culture of on-going mentoring support from senior academics, formalised through the university administration, so that mentors are afforded resources, time, and appropriate recognition.

Designing Assessment for Quality Learning

- Considers broad-scale assessment approaches and how they impact on educational opportunity and outcomes. - Brings together internationally recognised scholars providing new insights into assessment for learning improvement and accountability. - Presents different theoretical and methodological perspectives influential in the field of assessment, learning and social change. - Contributes to the theorising of assessment in contexts characterised by heightened accountability requirements and constant change. This book brings together internationally recognised scholars with an interest in how to use the power of assessment to improve student learning and to engage with accountability priorities at both national and global levels. It includes distinguished writers who have worked together for some two decades to shift the assessment paradigm from a dominant focus on assessment as measurement towards assessment as central to efforts to improve learning. These writers have worked with the teaching profession and, in so doing, have researched and generated key insights into different ways of understanding assessment and its relationship to learning. The volume contributes to the theorising of assessment in contexts characterised by heightened accountability requirements and constant change. The book’s structure and content reflect already significant and growing international interest in assessment as contextualised practice, as well as theories of learning and teaching that underpin and drive particular assessment approaches. Learning theories and practices, assessment literacies, teachers’ responsibilities in assessment, the role of leadership, and assessment futures are the organisers within the book’s structure and content. The contributors to this book have in common the view that quality assessment, and quality learning and teaching are integrally related. Another shared view is that the alignment of assessment with curriculum, teaching and learning is linchpin to efforts to improve both learning opportunities and outcomes for all. Essentially, the book presents new perspectives on the enabling power of assessment. In so doing, the writers recognise that validity and reliability - the traditional canons of assessment – remain foundational and therefore necessary. However, they are not of themselves sufficient for quality education. The book argues that assessment needs to be radically reconsidered in the context of unprecedented societal change. Increasingly, communities are segregating more by wealth, with clear signs of social, political, economic and environmental instability. These changes raise important issues relating to ethics and equity, taken to be core dimensions in enabling the power of assessment to contribute to quality learning for all. This book offers readers new knowledge about how assessment can be used to re/engage learners across all phases of education.

Psycho-educational assessment of specific learning disabilities: Views and practices of Australian psychologists and guidance counsellors

This paper reports an investigation of the views and practices of 203 Australian psychologists and guidance counsellors with respect to psycho-educational assessment of students with Specific Learning Disabilities (SLDs). Results from an online survey indicated that practitioners draw upon a wide-range of theoretical perspectives when conceptualising and identifying SLDs, including both response to intervention and IQ – achievement discrepancy models. Intelligence tests (particularly the Wechsler scales) are commonly employed, with the main stated reasons for their use being ‘traditional’ perspectives (including IQ-achievement discrepancy-based definitions of SLDs), to exclude a diagnosis of intellectual disability, and to guide further assessment and intervention. In contrast participants reported using measures of academic achievement and tests of specific cognitive deficits known to predict SLDs (e.g., phonological awareness) relatively infrequently.

The use of hand-held mobile devices as aids to learning within flexible learning spaces in tertiary Information Technology classes

Mobile devices are very popular among tertiary student populations. This study looks at student use of hand-held mobile devices within the context of a first year programming unit. This research sought for ways in which an educational app on these devices could be successfully integrated into such a class's learning.

The influences of conceptions of learning and learner self-concept on high school students' approaches to learning

This study investigated the mediating effect of learner selfconcept between conceptions of learning and students' approaches to learning using structural equation modelling. Data were collected using a modified version of Biggs' Learning Process Questionnaire, together with the recently developed 'What is Learning Survey' and 'Learner Self-Concept Scale'. A sample of 355 high school students participated in the study. Results indicate that learner self-concept does mediate between conceptions of meaning and approaches to learning. Students who adopted a deep approach liked learning new things and indirectly viewed learning as experiential, involving social interaction and directly viewed learning as personal development. Implications for teachers are discussed, with consideration given to appropriate classroom practice.

Elementary school students’ learner self-concept, academic self-concepts and approaches to learning

The present study investigated the relationships between academic selfconcepts, learner self-concept, and approaches to learning in elementary school students. A sample of 580 Australian Grade 6 and 7 school students with a mean age of 10.7 years participated in the study. Weak negative correlations between learner self-concepts and surface approaches to learning were identiŽ ed. In contrast, deep approaches for both boys and girls showed the highest positive correlations with school self-concept and learning self-concept. Only slight variations in these Ž gures were found between boys and girls.