Kaleidoscope : Reframing learning and teaching evaluation through a shared stakeholder lens

Sector wide interest in Reframe: QUT’s Evaluation Framework continues with a number of institutions requesting finer details as QUT embeds the new approach to evaluation across the university in 2013. This interest, both nationally and internationally has warranted QUT’s collegial response to draw upon its experiences from developing Reframe into distilling and offering Kaleidoscope back to the sector. The word Reframe is a relevant reference for QUT’s specific re-evaluation, reframing and adoption of a new approach to evaluation; whereas Kaleidoscope reflects the unique lens through which any other institution will need to view their own cultural specificity and local context through an extensive user-led stakeholder engagement approach when introducing new approaches to learning and teaching evaluation. Kaleidoscope’s objectives are for QUT to develop its research-based stakeholder approach to distil the successful experience exhibited in the Reframe Project into a transferable set of guidelines for use by other tertiary institutions across the sector. These guidelines will assist others to design, develop, and deploy, their own culturally specific widespread organisational change informed by stakeholder engagement and organisational buy-in. It is intended that these guidelines will promote, support and enable other tertiary institutions to embark on their own evaluation projects and maximise impact. Kaleidoscope offers an institutional case study of widespread organisational change underpinned by Reframe’s (i) evidence-based methodology; (ii) research including published environmental scan, literature review (Alderman, et al., 2012), development of a conceptual model (Alderman, et al., in press 2013), project management principles (Alderman & Melanie, 2012) and national conference peer reviews; and (iii) year-long strategic project with national outreach to collaboratively engage the development of a draft set of National Guidelines. Kaleidoscope’s aims are to inform Higher Education evaluation policy development through national stakeholder engagement, the finalisation of proposed National Guidelines. In correlation with the conference paper, the authors will present a Draft Guidelines and Framework ready for external peer review by evaluation practitioners from the Higher Education sector, as part of Kaleidoscope’s dissemination strategy (Hinton & Gannaway, 2011) applying illuminative evaluation theory (Parlett & Hamilton, 1976), through conference workshops and ongoing discussions (Shapiro, et al., 1983; Jacobs, 2000). The initial National Guidelines will be distilled from the Reframe: QUT’s Evaluation Framework’s Policy, Protocols, and incorporated Business Rules. It is intended that the outcomes of Kaleidoscope are owned by and reflect sectoral engagement, including iterative evaluation through multiple avenues of dissemination and collaboration including the Higher Education sector. The dissemination strategy with the inclusion of Illuminative Evaluation methodology provides an inclusive opportunity for other institutions and stakeholders across the Higher Education sector to give voice through the information-gathering component of evaluating the draft Guidelines, providing a comprehensive understanding of the complex realities experienced across the Higher Education sector, and thereby ‘illuminating’ both the shared and unique lenses and contexts. This process will enable any final guidelines developed to have broader applicability, greater acceptance, enhanced sustainability and additional relevance benefiting the Higher Education sector, and the adoption and adaption by any single institution for their local contexts.

Peer review of teaching practice and resources : exploring new spaces to embrace cultural change

The implementation of systematic peer review as a professional development activity, and as a support for educational design activities is under-utilised in many Australian higher education institutions. This case study reports on the first stages of planning and implementation of an institution-wide project to enhance teaching and learning quality at a remote and regional university, where one of the major strategies for improvement is peer review. Through a systematic process of staff engagement in peer review, within and from outside the organisation, a substantial change in flexible learning is envisaged. A mix of new and different learning spaces are to be used in the project, including blended learning spaces for academic development. This paper describes the research framework that will guide the peer review process and examines the early findings of the design-based research. Leadership, awareness raising and development of a supportive community of inquiry are seen as key components for successful implementation of peer review. In addition, unique contextual elements add to the complexity of designing for transformative change within such a relatively new organization.

Kaleidoscope : illuminating learning and teaching evaluation for a shared stakeholder lens

Within the Australian higher education sector, institutions are required to evaluate teaching, units and courses to assure the quality of the student learning experience, however with hardly any regulatory parameters guiding institutions, and with disparate practices, there are few opportunities to benchmakr across institutions or the sector. QUT has received interest and requests from national and international universities on accessing Reframe: QUT's Evaluation Framework.

Exploring the experiences of embedding Indigenous knowledge and perspectives on teaching practicum through interpretive phenomenology

In this paper, we discuss interpretive/hermeneutic phenomenology as a theoretical approach to explore the experiences of three stakeholder groups in embedding Indigenous knowledge and perspectives on teaching practicum, a project sponsored by ALTC. We begin by asking the phenomenological question ‘what is your experience of practice teaching?’ An open, explorative, phenomenological framework seeks the meanings of experiences, not truths, from the participants’ words themselves. Interpretive phenomenology is particularly suitable to explore educational experiences (Grumet, 1992; M. van Manen, 1990), as it provides rich ground for listening to the stakeholders’ lived experience and documenting it for interpretation. In an interpretive process, perspectives on lifeworlds, worldview and lenses get highlighted (Cunningham & Stanley, 2003). We establish how through various project stages, interpretive phenomenology gets to the essence of practice teaching experience creating a pedagogical ‘understanding’ of the essential nature of shared experience as lived by the participants (M van Manen, 2002). Thereby, it foregrounds voices of agency, dissent, acceptance and resistance. We consider how our research study focuses on the pedagogic voice of Indigenous pre-service teachers and the recognition of complex pedagogic fields in Indigenous education. We explain how this study seeks insights into their evaluation of pedagogic relations with two other education stakeholders – their practicum supervising teachers at schools and university staff involved practicum experience. As such, our study aims to support and develop long term, future-oriented opportunities for Indigenous pre-service teachers to embed Indigenous knowledge in the curricula. We conclude with some projections into the discourse on how Indigenous knowledge (IK) and perspectives might be diversely exemplified in pre-service teachers’ professional works (particularly E-portfolios). We speculate how this change could in turn maximise opportunities for Indigenous pre-service teachers, their supervising teachers and university staff to demonstrate leadership in their field through the creation of future tangible products such as units of work, resources, assessment and reflection tools. The processes contextualising the cultural interface of competing knowledge systems (Nakata, 2007) provide important analytical tools for understanding issues affecting student-teacher-mentor relationships occurring on practicum.

Teaching digital media to digital natives

The majority of today's undergraduate students are 'digital natives'; a generation born into a world shaped by digital technologies. It is important to understand the significance of this when considering how to teach Digital Media to digital natives. This paper examines the analogies to literacy that recur in digital native debates. It argues that if the concept of digital literacy is to be useful, educators must attend to the multiple layers and proficiencies that comprise literacy. Rather than completely dispose of old teaching methods, updated pedagogical practices should integrate analysis and critique with exploratory and creative modes of learning.

Could MOOCs answer the problems of teaching AQF required skills in Australian tertiary programmes?

From 2015, Australian universities will be required to demonstrate that their programmes explicitly teach, and assess achievement of, knowledge and skills and the application of both as specified by the Australian Qualifications Framework (AQF). Over the last twenty years, the sector has applied significant effort and resource to embedding the development of skills through tertiary programmes. Despite these national and institutional efforts, employer and industry concerns remain about the quality of graduate skills. The authors propose a ‘massive open online course’ (MOOC) approach to teaching and assessing AQF required skills. As an example the paper identifies the skills modules that would need to be developed by experts in each skill area for AQF level 9 master’s by coursework programmes. The proposed MOOC would include assessment tasks and rubrics allowing students to develop and demonstrate achievement of the AQF required skills. The assessment tasks could be used by institutions to provide evidence of attainment of coursework master’s standards.

Examining workload models in online and blended teaching

Over the past decade, most Australian universities have moved increasingly towards online course delivery for both undergraduate and graduate programs. In almost all cases, elements of online teaching are part of routine teaching loads. Yet detailed and accurate workload data are not readily available. As a result, institutional policies on academic staff workload are often guided more by untested assumptions about reduction of costs per student unit, rather than being evidence-based, with the result that implementation of new technologies for online teaching has resulted in poorly defined workload expectations. While the academics in this study often revealed a limited understanding of their institutional workload formulas, which in Australia are negotiated between management and the national union through their local branches, the costs of various types of teaching delivery have become a critical issue in a time of increasing student numbers, declining funding, pressures to increase quality and introduce minimum standards of teaching and curriculum, and substantial expenditure on technologies to support e-learning. There have been relatively few studies on the costs associated with workload for online teaching, and even fewer on the more ubiquitous ‘blended’, ‘hybrid’ or ‘flexible’ modes, in which face-to-face teaching is supplemented by online resources and activities. With this in mind the research reported here has attempted to answer the following question: What insights currently inform Australian universities about staff workload when teaching online?

A CLASS act : the teaching team approach to subject coordination

Advancing the development of good practice around the teaching team has been the focus of a recently completed, nationally funded Australian grant entitled Coordinators Leading Advancement of Sessional Staff (CLASS). The project focused on developing leadership capacity of subject coordinators to provide supportive contexts for sessional staff to enhance their knowledge of teaching practice and contribute to subject improvement through a team approach. An action learning approach and notions of distributed leadership underpinned the activities of the teaching teams in the program. This paper provides an overview of a practical approach, led by the subject coordinator, to engaging sessional staff through the facilitation of a supportive network within the teaching team. It addresses some of the gaps identified in the recent literature which includes lack of role clarity for all members of the team and provides some examples of initiatives that teams engaged with to address some of the challenges identified. Resources to support this approach were developed and are shared through the project website. Recommendations for future direction include improved policy and practice at the institutional level, better recognition and reward for subject coordinators and resourcing to support the participation and professional development needs of sessional staff.

A grand challenge : reflecting on university science curriculum design, collaborative partnerships and integration of information literacy skills

In 2012, Queensland University of Technology (QUT) committed to the massive project of revitalizing its Bachelor of Science (ST01) degree. Like most universities in Australia, QUT has begun work to align all courses by 2015 to the requirements of the updated Australian Qualifications Framework (AQF) which is regulated by the Tertiary Education Quality and Standards Agency (TEQSA). From the very start of the redesigned degree program, students approach scientific study with an exciting mix of theory and highly topical real world examples through their chosen “grand challenge.” These challenges, Fukushima and nuclear energy for example, are the lenses used to explore science and lead to 21st century learning outcomes for students. For the teaching and learning support staff, our grand challenge is to expose all science students to multidisciplinary content with a strong emphasis on embedding information literacies into the curriculum. With ST01, QUT is taking the initiative to rethink not only content but how units are delivered and even how we work together between the faculty, the library and learning and teaching support. This was the desired outcome but as we move from design to implementation, has this goal been achieved? A main component of the new degree is to ensure scaffolding of information literacy skills throughout the entirety of the three year course. However, with the strong focus on problem-based learning and group work skills, many issues arise both for students and lecturers. A move away from a traditional lecture style is necessary but impacts on academics’ workload and comfort levels. Therefore, academics in collaboration with librarians and other learning support staff must draw on each others’ expertise to work together to ensure pedagogy, assessments and targeted classroom activities are mapped within and between units. This partnership can counteract the tendency of isolated, unsupported academics to concentrate on day-to-day teaching at the expense of consistency between units and big picture objectives. Support staff may have a more holistic view of a course or degree than coordinators of individual units, making communication and truly collaborative planning even more critical. As well, due to staffing time pressures, design and delivery of new curriculum is generally done quickly with no option for the designers to stop and reflect on the experience and outcomes. It is vital we take this unique opportunity to closely examine what QUT has and hasn’t achieved to be able to recommend a better way forward. This presentation will discuss these important issues and stumbling blocks, to provide a set of best practice guidelines for QUT and other institutions. The aim is to help improve collaboration within the university, as well as to maximize students’ ability to put information literacy skills into action. As our students embark on their own grand challenges, we must challenge ourselves to honestly assess our own work.

Approaches to inquiry teaching : elementary teacher’s perspectives

Learning science through the process of inquiry is advocated in curriculum documents across many jurisdictions. However, a number of studies suggest that teachers struggle to help students engage in inquiry practices. This is not surprising as many teachers of science have not engaged in scientific inquiry and possibly hold naïve ideas about what constitutes scientific inquiry. This study investigates teachers’ self-reported approaches to teaching science through inquiry. Phenomenographic interviews undertaken with 20 elementary teachers revealed teachers identified six approaches to teaching for inquiry, clustered within three categories. These approaches were categorized as Free and Illustrated Inquiry as part of experience-centered category, Solution and Method Inquiry as part of problem-centered category, and Topic and Chaperoned Inquiry as part of a question-centered category. This study contributes to our theoretical understanding of how teachers approach Inquiry Teaching, and suggests fertile areas of future research into this valued and influential phenomenon broadly known as “Inquiry Teaching”.

The ABCs of teaching alphabet knowledge : affordances and challenges of ‘weaving’ visible and invisible pedagogies

As researchers interested in the pursuit of high quality/high equity literacy learning outcomes, we focus on the learning experiences of five early years French students, with a special regard for those who are already considered as being at-risk of educational failure. We narrow the empirical focus to a single lesson on a mechanical concept of print, that is matching lower and upper case alphabet letters. In doing so, we examine a deeply philosophical question: Which pedagogical practices dis/enable what sorts of early years students as literacy learners? We extend Cazden’s (2006) notion of ‘weaving’ knowledge across dimensions of knowing to describe how the case study teacher ‘weaves’ visible and invisible pedagogies over the four movements of a lesson. The findings reveal different pedagogical framings (Bernstein, 1996) have potentially different cognitive and social effects that constitute different kinds of literacy knowledge and oppressive subject positions for at-risk students (Young, 1990).

Playing with Grammar in the Early Years : Learning about Language in the Australian Curriculum : English

As teacher/researchers interested in the pursuit of socially-just outcomes in early childhood education, the form and function of language occupies a special position in our work. We believe that mastering a range of literacy competences includes not only the technical skills for learning, but also the resources for viewing and constructing the world (Freire and Macdeo, 1987). Rather than seeing knowledge about language as the accumulation of technical skills alone, the viewpoint to which we subscribe treats knowledge about language as a dialectic that evolves from, is situated in, and contributes to a social arena (Halliday, 1978). We do not shy away from this position just because children are in the early years of schooling. In ‘Playing with Grammar’, we focus on the Foundation to Year 2 grouping, in line with the Australian Curriculum, Assessment and Reporting Authority’s (hereafter ACARA) advice on the ‘nature of learners’ (ACARA, 2013). With our focus on the early years of schooling comes our acknowledgement of the importance and complexity of play. At a time where accountability in education has moved many teachers to a sense of urgency to prove language and literacy achievement (Genishi and Dyson, 2009), we encourage space to revisit what we know about literature choices and learning experiences and bring these together to facilitate language learning. We can neither ignore, nor overemphasise, the importance of play for the development of language through: the opportunities presented for creative use and practice; social interactions for real purposes; and, identifying and solving problems in the lives of young children (Marsh and Hallet, 2008). We argue that by engaging young children in opportunities to play with language we are ultimately empowering them to be active in their language learning and in the process fostering a love of language and the intricacies it holds. Our goal in this publication is to provide a range of highly practical strategies for scaffolding young children through some of the Content Descriptions from the Australian Curriculum English Version 5.0, hereafter AC:E V5.0 (ACARA, 2013). This recently released curriculum offers a new theoretical approach to building children’s knowledge about language. The AC:E V5.0 uses selected traditional terms through an approach developed in systemic functional linguistics (see Halliday and Matthiessen, 2004) to highlight the dynamic forms and functions of multimodal language in texts. For example, the following statement, taken from the ‘Language: Knowing about the English language’ strand states: English uses standard grammatical terminology within a contextual framework, in which language choices are seen to vary according to the topics at hand, the nature and proximity of the relationships between the language users, and the modalities or channels of communication available (ACARA, 2013). Put simply, traditional grammar terms are used within a functional framework made up of field, tenor, and mode. An understanding of genre is noted with the reference to a ‘contextual framework’. The ‘topics at hand’ concern the field or subject matter of the text. The ‘relationships between the language users’ is a description of tenor. There is reference to ‘modalities’, such as spoken, written or visual text. We posit that this innovative approach is necessary for working with contemporary multimodal and cross-cultural texts (see Exley and Mills, 2012). We believe there is enormous power in using literature to expose children to the richness of language and in turn develop language and literacy skills. Taking time to look at language patterns within actual literature is a pathway to ‘…capture interest, stir the imagination and absorb the [child]’ into the world of language and literacy (Saxby, 1993, p. 55). In the following three sections, we have tried to remain faithful to our interpretation of the AC:E V5.0 Content Descriptions without giving an exhaustive explanation of the grammatical terms. Other excellent tomes, such as Derewianka (2011), Humphrey, Droga and Feez (2012), and Rossbridge and Rushton (2011) provide these more comprehensive explanations as does the AC:E V5.0 Glossary. We’ve reproduced some of the AC:E V5.0 glossary at the end of this publication. Our focus is on the structure and unfolding of the learning experiences. We outline strategies for working with children in Foundation, Year 1 and Year 2 by providing some demonstration learning experiences based on texts we’ve selected, but maintain that the affordances of these strategies will only be realised when teaching and learning is purposively tied to authentic projects in local contexts. We strongly encourage you not to use only the resource texts we’ve selected, but to capitalise upon your skill for identifying the language features in the texts you and the children are studying and adapt some of the strategies we have outlined. Each learning experience is connected to one of the Content Descriptions from the AC:E V5.0 and contains an experience specific purpose, a suggested resource text and a sequence for the experience that always commences with an orientation to text followed by an examination of a particular grammatical resource. We expect that each of these learning experiences will take a couple if not a few teaching episodes to work through, especially if children are meeting a concept for the first time. We hope you use as much, or as little, of each experience as is needed. Our plans allow for focused discussion, shared exploration and opportunities to revisit the same text for the purpose of enhancing meaning making. We do not want the teaching of grammar to slip into a crisis of irrelevance or to be seen as a series of worksheet drills with finite answers. Strategies for effective practice, however, have much portability. We are both very keen to hear from teachers who are adopting and adapting these learning experiences in their classrooms. Please email us on b.exley@qut.edu.au or lkervin@uow.edu.au. We’d love to continue the conversation with you over time.

Creating efficiencies in teaching : where are the right resources when you need them?

Early career engineering academics are encouraged to join and contribute to established research groups at the leading edge of their discipline. This is often facilitated by various staff development and support programs. Given that academics are often appointed primarily on the basis of their research skills and outputs, such an approach is justified and is likely to result in advancing the individual academic’s career. It also enhances their capacity to attract competitive research funding, while contributing to the overall research performance of their institution, with further potential for an increased share of government funding. In contrast, there is much less clarity of direction or availability of support mechanisms for those academics in their role as teachers. Following a general induction to teaching and learning at their institution, they would commonly think about preparing some lecture materials, whether for delivery in a face-to-face or on-line modality. Typically they would look for new references and textbooks to act as a guide for preparing the content. They would probably find out how the course has been taught before, and what laboratory facilities and experiments have been used. In all of these and other related tasks, the majority of newly appointed academics are guided strongly by their own experiences as students, rather than any firm knowledge of pedagogical principles. At a time of increased demands on academics’ time, and high expectations of performance and productivity in both research and teaching, it is essential to examine possible actions to support academics in enhancing their teaching performance in effective and efficient ways. Many resources have been produced over the years in engineering schools around the world, with very high intellectual and monetary costs. In Australia, the last few years have seen a surge in the number of ALTC/OLT projects and fellowships addressing a range of engineering education issues and providing many resources. There are concerns however regarding the extent to which these resources are being effectively utilised. Why are academics still re-inventing the wheel and creating their own version of teaching resources and pedagogical practice? Why do they spend so much of their precious time in such an inefficient way? A symposium examining the above issues was conducted at the AAEE2012 conference, and some pointers to possible responses to the above questions were obtained. These are explored in this paper and supplemented by the responses to a survey of a group of engineering education leaders on some of the aspects of these research questions. The outcomes of the workshop and survey results have been analysed in view of the literature and the ALTC/OLT sponsored learning and teaching projects and resources. Other factors are discussed, including how such resources can be found, how their quality might be evaluated, and how assessment may be appropriately incorporated, again using readily available resources. This study found a strong resonance between resources reuse with work on technology acceptance (Davis, 1989), suggesting that technology adoption models could be used to encourage resource sharing. Efficient use of outstanding learning materials is an enabling approach. The paper provides some insights on the factors affecting the re-use of available resources, and makes some recommendations and suggestions on how the issue of resources re-use might be incorporated in the process of applying and completing engineering education projects.