REFRAME: a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

The potential of combining phenomenography, variation theory and threshold concepts to inform curriculum design in higher education

This chapter describes an innovative method of curriculum design that is based on combining phenomenographic research, and the associated variation theory of learning, with the notion of disciplinary threshold concepts to focus specialised design attention on the most significant and difficult parts of the curriculum. The method involves three primary stages: (i) identification of disciplinary concepts worthy of intensive curriculum design attention, using the criteria for threshold concepts; (ii) action research into variation in students’ understandings/misunderstandings of those concepts, using phenomenography as the research approach; (iii) design of learning activities to address the poorer understandings identified in the second stage, using variation theory as a guiding framework. The curriculum design method is inherently theory and evidence based. It was developed and trialed during a two-year project funded by the Australian Learning and Teaching Council, using physics and law disciplines as case studies. Disciplinary teachers’ perceptions of the impact of the method on their teaching and understanding of student learning were profound. Attempts to measure the impact on student learning were less conclusive; teachers often unintentionally deviated from the design when putting it into practice for the first time. Suggestions for improved implementation of the method are discussed.

Establishing a framework for transforming student engagement, success and retention in higher education institutions [Final Report]

The perennial issues of student engagement, success and retention (SESR) in higher education continue to attract attention as key indicators of learning and teaching quality. This project aimed to establish and provide a holistic framework that would allow higher education institutions (HEIs) manage and improve their student engagement and retention strategies and programs. The framework and main project deliverable is a Maturity Model (MM) for Student Engagement, Success and Retention (SESR-MM). The project involved three Australian universities with experience and reputations in SESR activities: Queensland University of Technology (lead institution), the University of Queensland and Griffith University, working cooperatively to develop and trial the project deliverables. Project findings suggest that the SESR-MM has the potential to positively transform the holistic—academic, social and personal—engagement experiences of students in Australian universities, and that the SESR-MM is a useful mechanism for sharing good practice and improving programs designed to enhance the student experience.

Leadership Enabling Effective Pedagogic Change in Higher Education

This chapter will consider pedagogic change in Higher Education from the perspective of an Assistant Dean (Teaching and Learning) and one member of their leadership team with particular focus on reflective writing in their courses. The discussion will focus on leadership for the development of teaching capability for reflective writing development and implications for quality assurance of teaching and learning across faculties of a leading comprehensive University. The authors will present and contrast the experiences and challenges of developing teaching approaches for reflective writing across the discipline of teacher education. The chapter will argue a position for the establishment of a framework of distributed leadership that supports effective pedagogical change management generally and with specific reference to reflective writing.

Enabling a positive first year experience in higher education through social media and mobile technologies

The first year experience for students within Higher Education institutions has become increasingly important as these institutions strive to improve student retention rates. With many universities also focusing on transforming teaching and learning in an effort to attract and retain students, there is a growing demand to understand and respond to individual student requirements, such as the need to feel a sense of belonging. The literature identifies a sense of belonging as being paramount to a students satisfaction with the institution and it is within this context that this paper reports on a three year study of how first year pre-service education students use social media and mobile technologies in their personal lives and their formal education. More specifically, the study identifies trends in the use of these technologies and the growing need for students to use digital media sharing tools to connect and engage with their peers. The paper contrasts the differences in use between these groups as it seeks to identify the role these technologies can play in their teaching and learning, as well as in promoting an overall positive first year experience.

Introduction: Reflective and Reflexive Approaches in Higher Education : A Warrant for Lifelong Learning?

In our complex and incongruous world, where variety produces more variety, and there is no blueprint for dealing with unprecedented change, it is imperative that individuals develop reflexive approaches to life and learning. Higher education has a role to play in guiding students to be self-analysts, with the ability to examine and mediate self and context for improved outcomes. This chapter elucidates the catchphrase of lifelong learning and its enactment in higher education. Theories of reflexivity and personal epistemology are utilised to provide the conceptual tools to understand the ways in which individuals manage competing influences and deliberate about action in their learning journey. The case is made for the integral role of higher education teachers in developing students’ capacities for reflective thinking and reflexive approaches to learning as a life project.

Illuminative evaluation as a method applied to Australian Government policy borrowing and implementation in higher education

Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.

Moderation in higher education: Four discourses

Across the globe, higher education institutions are working in environments of increasing accountability with little sign of this trend abating. This heightened focus on accountability has placed greater demands on institutions to provide evidence of quality and the achievement of standards that assure that quality. Moderation is one quality assurance process that plays a central role in the teaching, learning and assessment cycle in higher education institutions. While there is a growing body of research globally on teaching, learning and , to a lesser degree, assessment in higher education, the process of moderation has received even less attention (Watty, Freeman, Howieson, Hancock, O'Connell, et al. 2013). Until recently, moderation processes in Australian universities have been typically located within individual institutions, with universities given the responsibility for developing their own specific policies and practices. However, in 2009 the Australian Government announced that an independent national quality and regulatory body for higher education institutions would be established. With the introduction of the Tertiary Education Quality Standards Authority (TEQSA), more formalised requirements for moderation of assessment are being mandated. In light of these reforms, the purpose of this qualitative study was to identify and investigate current moderation practices operating within one faculty, the Faculty of Education, in a large urban university in eastern Australia. The findings of this study revealed four discourses of moderation: equity, justification, community building and accountability. These discourses provide a starting point for academics to engage in substantive conversations around assessment and to further critique the processes of moderation.

An investigation into students' motivation to learn English in higher education in Vietnam

This mixed-methods research explored students' motivation to learn English in an attempt to enhance the quality of teaching and learning English in Vietnamese higher education. An intricate picture of students' motivation was revealed: motivation to learn English to gain pragmatic benefits, to satisfy internal aspirations, and in response to external pressures. Students expressed that feeling connected with significant people (lecturers, peers and parents) enhanced their motivation to learn English. This research has significance at both practical and theoretical levels as it provides important insights into the teaching and learning English in Vietnamese higher education and the literature of second language learning motivation.

Social media in higher education: Strategy and support for staff and students

In 2013, social networking was the second most popular online activity after internet banking for Australians (ABS, 2014). The popularity and apparent ubiquity of social media is one of the most obvious and compelling arguments for integrating such technologies into higher education. Already, social media impacts a wide range of activities ranging in scope from marketing and communication to teaching and learning in higher education (Hrastinski & Dennen, 2012). Social media presents many exciting possibilities and opportunities for higher education. This session will focus on one staff focussed and one student focussed social media innovation currently underway at QUT. First, it will focus on the actions of QUT’s social media working group. The working group’s aim is to ensure an overarching social media policy for the university is developed and implemented that supports staff in the use of social media across a range of activities. Second, it will discuss the eResponsible and eProfessional Online resources for students project. The focus of this project is to develop a suite of online resources targeted at the devel opment of social media skills for undergraduate students at QUT. These initiatives are complementary and both aim to minimise risk while maximising opportuniti es for the university

Establishing a framework for transforming student engagement, success and retention in higher education institutions

This poster summarises the outcomes of a national project to develop and provide a holistic framework consisting of a series of sequential and increasingly sophisticated stages that will allow higher education institutions (HEIs) to manage and improve their student engagement and retention strategies/programs.

Perspectives on knowledge creating inquiry in higher education

Higher education is faced with the challenge of strengthening students competencies for the constantly evolving technology-mediated practices of knowledge work. The knowledge creation approach to learning (Paavola et al., 2004; Hakkarainen et al., 2004) provides a theoretical tool to address learning and teaching organized around complex problems and the development of shared knowledge objects, such as reports, products, and new practices. As in professional work practices, it appears necessary to design sufficient open-endedness and complexity for students teamwork in order to generate unpredictable and both practically and epistemologically challenging situations. The studies of the thesis examine what kinds of practices are observed when student teams engage in knowledge creating inquiry processes, how the students themselves perceive the process, and how to facilitate inquiry with technology-mediation, tutoring, and pedagogical models. Overall, 20 student teams collaboration processes and productions were investigated in detail. This collaboration took place in teams or small groups of 3-6 students from multiple domain backgrounds. Two pedagogical models were employed to provide heuristic guidance for the inquiry processes: the progressive inquiry model and the distributed project model. Design-based research methodology was employed in combination with case study as the research design. Database materials from the courses virtual learning environment constituted the main body of data, with additional data from students self-reflections and student and teacher interviews. Study I examined the role of technology mediation and tutoring in directing students knowledge production in a progressive inquiry process. The research investigated how the scale of scaffolding related to the nature of knowledge produced and the deepening of the question explanation process. In Study II, the metaskills of knowledge-creating inquiry were explored as a challenge for higher education: metaskills refers to the individual, collective, and object-centered aspects of monitoring collaborative inquiry. Study III examined the design of two courses and how the elaboration of shared objects unfolded based on the two pedagogical models. Study IV examined how the arranged concept-development project for external customers promoted practices of distributed, partially virtual, project work, and how the students coped with the knowledge creation challenge. Overall, important indicators of knowledge creating inquiry were the following: new versions of knowledge objects and artifacts demonstrated a deepening inquiry process; and the various productions were co-created through iterations of negotiations, drafting, and versioning by the team members. Students faced challenges of establishing a collective commitment, devising practices to co-author and advance their reports, dealing with confusion, and managing culturally diverse teams. The progressive inquiry model, together with tutoring and technology, facilitated asking questions, generating explanations, and refocusing lines of inquiry. The involvement of the customers was observed to provide a strong motivation for the teams. On the evidence, providing team-specific guidance, exposing students to models of scientific argumentation and expert work practices, and furnishing templates for the intended products appear to be fruitful ways to enhance inquiry processes. At the institutional level, educators do well to explore ways of developing collaboration with external customers, public organizations or companies, and between educational units in order to enhance educational practices of knowledge creating inquiry.

Cultivating creative learners in higher education: Engaging students in anatomy through art

Introduction
Nursing and midwifery students often struggle to engage with bioscience modules because they lack confidence in their ability to study science (Fell et al., 2012). Consequently many have difficulty applying anatomical and physiological information, essential to providing safe and effective patient care (Rogers, 2014; Rogers and Sterling, 2012); therefore a need exists for nurse educators to explore different methods of delivery of these important topics to enhance current curricula (Johnston, 2010). Inspired by the reported success of creative methods to enhance the teaching and learning of anatomy in medical education (Noel, 2013; Finn and McLachlan, 2010), this pilot study engaged nursing students in anatomy through the art of felt. The project was underpinned by the principles of good practice in undergraduate education, staff-student engagement, cooperation among students, active learning, prompt feedback, time on task, high expectations and respect for diverse learning styles (Chickering and Gamson, 1987).

Method
Undergraduate student nurses from Queen’s University, Belfast, enrolled in the year one ‘Health and Wellbeing’ model were invited to participate in the project. Over a six week period the student volunteers worked in partnership with teaching staff to construct individual, unique, three dimensional felt models of the upper body. Students researched the agreed topic for each week in terms of anatomical structure, location, tissue composition and vascular access. Creativity was encouraged in relation to the colour and texture of materials used. The evaluation of the project was based on the four level model detailed by Kirkpatrick and Kirkpatrick (2006) and included both quantitative and qualitative analysis:• pre and post knowledge scores• self-rated confidence• student reflections on the application of learning to practice.

Results
At the end of the project students had created felt pieces reflective of their learning throughout the project and ‘memorable’ three dimensional mental maps of the human anatomy. Evaluation revealed not only acquisition of anatomical knowledge, but the wider benefits of actively engaging in creative learning with other students and faculty teaching staff.

The project has enabled nurse educators to assess the impact of innovative methods for delivery of these important topics.

Engaging the Senses: Object-based learning in higher education

The use of museum collections as a path to learning for university students is fast becoming a new pedagogy for higher education. Despite a strong tradition of using lectures as a way of delivering the curriculum, the positive benefits of ‘active’ and ‘experiential learning’ are being recognised in universities at both a strategic level and in daily teaching practice. As museum artefacts, specimens and art works are used to evoke, provoke, and challenge students’ engagement with their subject, so transformational learning can take place. This unique book presents the first comprehensive exploration of ‘object-based learning’ as a pedagogy for higher education in a broad context. An international group of authors offer a spectrum of approaches at work in higher education today. They explore contemporary principles and practice of object-based learning in higher education, demonstrating the value of using collections in this context and considering the relationship between academic discipline and object-based learning as a teaching strategy.