Kaleidoscope : Reframing learning and teaching evaluation through a shared stakeholder lens

Sector wide interest in Reframe: QUT’s Evaluation Framework continues with a number of institutions requesting finer details as QUT embeds the new approach to evaluation across the university in 2013. This interest, both nationally and internationally has warranted QUT’s collegial response to draw upon its experiences from developing Reframe into distilling and offering Kaleidoscope back to the sector. The word Reframe is a relevant reference for QUT’s specific re-evaluation, reframing and adoption of a new approach to evaluation; whereas Kaleidoscope reflects the unique lens through which any other institution will need to view their own cultural specificity and local context through an extensive user-led stakeholder engagement approach when introducing new approaches to learning and teaching evaluation. Kaleidoscope’s objectives are for QUT to develop its research-based stakeholder approach to distil the successful experience exhibited in the Reframe Project into a transferable set of guidelines for use by other tertiary institutions across the sector. These guidelines will assist others to design, develop, and deploy, their own culturally specific widespread organisational change informed by stakeholder engagement and organisational buy-in. It is intended that these guidelines will promote, support and enable other tertiary institutions to embark on their own evaluation projects and maximise impact. Kaleidoscope offers an institutional case study of widespread organisational change underpinned by Reframe’s (i) evidence-based methodology; (ii) research including published environmental scan, literature review (Alderman, et al., 2012), development of a conceptual model (Alderman, et al., in press 2013), project management principles (Alderman & Melanie, 2012) and national conference peer reviews; and (iii) year-long strategic project with national outreach to collaboratively engage the development of a draft set of National Guidelines. Kaleidoscope’s aims are to inform Higher Education evaluation policy development through national stakeholder engagement, the finalisation of proposed National Guidelines. In correlation with the conference paper, the authors will present a Draft Guidelines and Framework ready for external peer review by evaluation practitioners from the Higher Education sector, as part of Kaleidoscope’s dissemination strategy (Hinton & Gannaway, 2011) applying illuminative evaluation theory (Parlett & Hamilton, 1976), through conference workshops and ongoing discussions (Shapiro, et al., 1983; Jacobs, 2000). The initial National Guidelines will be distilled from the Reframe: QUT’s Evaluation Framework’s Policy, Protocols, and incorporated Business Rules. It is intended that the outcomes of Kaleidoscope are owned by and reflect sectoral engagement, including iterative evaluation through multiple avenues of dissemination and collaboration including the Higher Education sector. The dissemination strategy with the inclusion of Illuminative Evaluation methodology provides an inclusive opportunity for other institutions and stakeholders across the Higher Education sector to give voice through the information-gathering component of evaluating the draft Guidelines, providing a comprehensive understanding of the complex realities experienced across the Higher Education sector, and thereby ‘illuminating’ both the shared and unique lenses and contexts. This process will enable any final guidelines developed to have broader applicability, greater acceptance, enhanced sustainability and additional relevance benefiting the Higher Education sector, and the adoption and adaption by any single institution for their local contexts.

Learning about being effective mentors : professional learning communities and mentoring

This study aimed to explore experienced mentors’ understandings about professional learning communities (PLCs), mentoring and leadership. This research analyses audio-taped transcripts and written responses from 27 experienced mentors who operate in varied roles (e.g., university academics, school executives, teachers, learning support personnel). Findings indicated that PLCs can provide professional renewal for existing teachers and that mentoring within PLCs can further advance knowledge about effective practices. PLCs can include other staff members and key stakeholders (e.g., preservice teachers, teacher aides) who can contribute to the learning within the group. Mentoring and PLCs can be cost-effective strategic levers for advancing professional knowledge.

Reconceptualizing Early Mathematics Learning : The Fundamental Role of Pattern and Structure

The Pattern and Structure Mathematics Awareness Program (PASMAP) was developed concurrently with the studies of AMPS and the development of the Pattern and Structure Assessment (PASA) interview. We summarize some early classroom-based teaching studies and describe the PASMAP that resulted. A large-scale two-year longitudinal study, Reconceptualizing Early Mathematics Learning (REML) resulted. We provide an overview of the REML study and discuss the consequences for our view of early mathematics learning. A purposive sample of four large primary schools, two in Sydney and two in Brisbane, representing 316 students from diverse socio-economic and cultural contexts, participated in an evaluation of the PASMAP intervention throughout the 2009 school year and a follow-up assessment in 2010. Two different mathematics programs were implemented: in each school, two Kindergarten teachers implemented the PASMAP and another two implemented their regular program. The study shows that both groups of students made substantial gains on the ‘I Can Do Maths’ standardized assessment and the PASA interview, but highly significant differences were found on the latter with PASMAP students outperforming the regular group on PASA scores. Qualitative analysis of students’ responses for structural development showed increased levels for the PASMAP students. Implications for pedagogy and curriculum are discussed.

Self-Regulated Learning : key strategies and their sources in a sample of adolescent males

This study identified the key self-regulated learning (SRL) strategies and their sources for nine school-aged adolescent males aged 15 to 17 years. The Self-Regulated Learning Interview Schedule (SRLIS) was used along with semi-structured interviews with the participants and their parents to elicit information on SRL strategies and contexts for the formation of self-regulatory habits. Early habit-forming experiences of the family home in relation to homework and study routines were found to form an important base for effective SRL. Teachers were identified as the most common source of SRL strategies with important formative experiences occurring during the first two years of high school.

Evaluating the use of online synchronous communication to enhance learning in statistics

According to social constructivists, learners are active participants in constructing new knowledge in a social process where they interact with others. In these social settings teachers or more knowledgeable peers provide support. This research study investigated the contribution that an online synchronous tutorial makes to support teaching and learning of undergraduate introductory statistics offered by an Australian regional university at a distance. The introductory statistics course which served as a research setting in this study was a requirement of a variety of programs at the University, including psychology, business and science. Often students in these programs perceive this course to be difficult and irrelevant to their programs of study. Negative attitudes and associated anxiety mean that students often struggle with the content. While asynchronous discussion forums have been shown to provide a level of interaction and support, it was anticipated that online synchronous tutorials would offer immediate feedback to move students forward through ―stuck places.‖ At the beginning of the semester the researcher offered distance students in this course the opportunity to participate in a weekly online synchronous tutorial which was an addition to the usual support offered by the teaching team. This tutorial was restricted to 12 volunteers to allow sufficient interaction to occur for each of the participants. The researcher, as participant-observer, conducted the weekly tutorials using the University's interactive online learning platform, Wimba Classroom, whereby participants interacted using audio, text chat and a virtual whiteboard. Prior to the start of semester, participants were surveyed about their previous mathematical experiences, their perceptions of the introductory statistics course and why they wanted to participate in the online tutorial. During the semester, they were regularly asked pertinent research questions related to their personal outcomes from the tutorial sessions. These sessions were recorded using screen capture software and the participants were interviewed about their experiences at the end of the semester. Analysis of these data indicated that the perceived value of online synchronous tutorial lies in the interaction with fellow students and a content expert and with the immediacy of feedback given. The collaborative learning environment offered the support required to maintain motivation, enhance confidence and develop problemsolving skills in these distance students of introductory statistics. Based on these findings a model of online synchronous learning is proposed.

Digital storytelling as a social work tool : learning from ethnographic research with women from refugee backgrounds

This paper reflects on the wider potential of digital narratives as a useful tool for social work practitioners. Despite the multiple points of connection between narrative approaches and social work, the influence of narratives on practice remains limited. A case study of a digital storytelling (DST) process employed in a research project with a small group of lone mothers from refugee backgrounds is used to trigger discussion of broader applications of DST as part of everyday social work practice. The use of DST acknowledged women’s capacities for self-representation and agency, in line with participatory and strengths-based approaches inherent in contemporary social work. The benefits of using DST with lone mothers from refugee backgrounds illustrate how this method can act as a pathway to produce counter-narratives, both at the individual and broader community levels. Documenting life stories digitally provides the opportunity to construct narratives about experiences of relocation and settlement as tools for social advocacy, which can assist social workers to ensure meaningful outcomes for service-users. These propositions can serve to inform social work practices with people from refugee backgrounds and address some of the intricacies of working in diverse and challenging contexts.

The use of Facebook in tertiary education : case study of a unit-related Facebook page in a university justice class

Purpose – This study aims to evaluate the usefulness of a university unit Facebook page, which was established to support a first-year university justice unit. The study pays particular regard to the Facebook page's impact on students learning outcomes and communications amongst students and between students and teaching staff. Design/methodology/approach – All students enrolled in the unit were asked to complete an online survey, which sought to determine whether they used the unit Facebook page and if so, the nature and extent of their use. Findings – The study found that the unit Facebook page was useful in achieving most learning objectives for the unit. This included enhancing students' knowledge and understanding of unit content, as well as their ability to critically analyse unit materials. Students also indicated that they found the Facebook page better than the university's central learning management system across a range of areas. It was particularly useful for facilitating unit-related discussions. Research limitations/implications – The survey results reported in this paper are based on a relatively small sample of students (n=67) from a first-year university justice unit. Future studies should seek to garner evidence from broader and larger samples that transcend specific unit populations. However, the findings of this study do indicate further support for the use of Facebook as a supplementary tool in university education. Originality/value – This study focuses on two aspects of social networking technologies that have not been previously researched and thus contributes to the growing literature on the uses and benefits of Facebook in tertiary education.

Scaffolding techniques : a teacher training for cooperative learning in Thailand primary education

Using cooperative learning in classrooms promotes academic achievement, communication skills, problem-solving, social skills and student motivation. Yet it is reported that cooperative learning as a Western educational concept may be ineffective in Asian cultural contexts. The study aims to investigate the utilisation of scaffolding techniques for cooperative learning in Thailand primary mathematics classes. A teacher training program was designed to foster Thai primary school teachers’ cooperative learning implementation. Two teachers participated in this experimental program for one and a half weeks and then implemented cooperative learning strategies in their mathematics classes for six weeks. The data collected from teacher interviews and classroom observations indicates that the difficulty or failure of implementing cooperative learning in Thailand education may not be directly derived from cultural differences. Instead, it does indicate that Thai culture can be constructively merged with cooperative learning through a teacher training program and practices of scaffolding techniques.

A conceptual framework for the cultural integration of cooperative learning : a Thai primary mathematics education perspective

The Thailand education reform adopted cooperative learning to improve the quality of education. However, it has been reported that the introduction and maintenance of cooperative learning has been difficult and uncertain because of the cultural differences. The study proposed a conceptual framework developed based on making a connection between Thai cultures and cooperative learning elements, and implemented a small-scale research project in a Thai primary mathematics class with a teacher and thirty-two Grade 4 students. The results uncovered that the three components including preparation of teachers, instructional strategies and preparation of students can be vehicles for the culture integration in cooperative learning.

Educating for digital futures : what the learning strategies of digital content professionals can teach higher education

Beginning in the second half of the 20th century, ICTs transformed many societies from industrial societies in which manufacturing was the central focus, into knowledge societies in which dealing effectively with data and information has become a central element of work (Anderson, 2008). To meet the needs of the knowledge society, universities must reinvent their structures and processes, their curricula and pedagogic practices. In addition to this, of course higher education is itself subject to the sweeping influence of ICTs. But what might effective higher education look like in the 21st century? In designing higher education systems and learning experiences which are responsive to the learning needs of the future and exploit the possibilities offered by ICTs, we can learn much from the existing professional development strategies of people who are already successful in 21st century fields, such as digital media. In this study, I ask: (1) what are the learning challenges faced by digital media professionals in the 21st century? (2) what are the various roles of formal and informal education in their professional learning strategies at present? (3) how do they prefer to acquire needed capabilities? In-depth interviews were undertaken with successful Australian digital media professionals working in micro businesses and SMEs to answer these questions. The strongest thematic grouping that emerged from the interviews related to the need for continual learning and relearning because of the sheer rate of change in the digital media industries. Four dialectical relationships became apparent from the interviewees’ commentaries around the learning imperatives arising out of the immense and continual changes occurring in the digital content industries: (1) currency vs best practice (2) diversification vs specialisation of products and services (3) creative outputs vs commercial outcomes (4) more learning opportunities vs less opportunity to learn. These findings point to the importance of ‘learning how to learn’ as a 21st century capability. The interviewees were ambivalent about university courses as preparation for professional life in their fields. Higher education was described by several interviewees as having relatively little value-add beyond what one described as “really expensive credentialling services.” For all interviewees in this study, informal learning strategies were the preferred methods of acquiring the majority of knowledge and skills, both for ongoing and initial professional development. Informal learning has no ‘curriculum’ per se, and tends to be opportunistic, unstructured, pedagogically agile and far more self-directed than formal learning (Eraut, 2004). In an industry impacted by constant change, informal learning is clearly both essential and ubiquitous. Inspired by the professional development strategies of the digital media professionals in this study, I propose a 21st century model of the university as a broad, open learning ecology, which also includes industry, professionals, users, and university researchers. If created and managed appropriately, the university learning network becomes the conduit and knowledge integrator for the latest research and industry trends, which students and professionals alike can access as needed.

Heritage language learning for Chinese Australians : the role of habitus

The relationship between Heritage Language and ethnic identity has gained significant research ground in social psychological and poststructural scholarship, with empirical evidence largely emerging from the North American settings. There is little pertinent sociological work conducted outside North America. To fill this gap, this sociological study sets its scene in an Australian context. Drawing on Bourdieu’s notion of habitus, the study examines the contribution of Chinese Australians’ Chineseness to their Chinese Heritage Language proficiency. Two hundred and thirty young Chinese Australians completed the online survey. Results from multiple regression indicate that habitus of Chineseness is one of the significant predictors for the Chinese Heritage Language proficiency of these young people. The study makes a theoretical contribution to investigate ethnic identity – Heritage Language link through the notion of habitus and makes a methodological contribution to quantify this habitus.

Distance education and the application of academagogy : a case study

The education sector has dramatically changed in the past half decade. In a time of globalisation of education and tightening budgets, various paradigm shifts and challenges have rapidly changed learning and teaching. These include: meeting student expectation for more engaging, more interactive learning experiences, the increased focus to deliver content online, and the complexities of fast-changing technologies. Rising to these challenges and responding to them is a complex and multi-faceted task. This paper discusses educational theories and issues and explores current educational practices in the context of teaching undergraduate students via distance education in the university context. A case study applies a framework drawn from engineering education using the learner-centric concept of academagogy. Results showed that academagogy actively empowers students to build effective learning, and engages facilitators in meaningful teaching and delivery methods.

Learning in 140 characters : microblogging for professional learning

This thesis investigated how microblogging, a form of online social networking, was being employed by educators to support their professional learning. The study found that educators who participate in microblogging engage in a wide range of behaviours, with certain behaviours and activities commonly exhibited. An advantage of microblogging as a professional learning tool is its ability to link educators globally to exchange ideas from different perspectives and to share resources and teaching practices. Educators who microblog have access to relevant and timely learning that is not constrained by time or distance and can be tailored to meet their individual needs.

Work integrated learning: Exposure to professional practice : expectations and challenges

Prior to graduation engineering students are expected to provide evidence of relevant experience in the workplace. This experience is expected to provide opportunities for exposure to the profession and to help students develop confidence, skills and capabilities as emerging professionals. This investigation considers the expectations and challenges in implementing WIL programs in different contexts. While this will inform the next iteration of engineering course development at QUT the issues and interventions described provide useful insights into options available and engineering curriculum design more broadly. This comparative analysis across three phases highlights expectations and challenges including stakeholder responsibilities, expectations, and assessment. The study draws on the findings of a 2005 investigation into the purpose and provision of WIL and findings of a 2012 Faculty review of the current WIL model. The enhancement of WIL through a series of developmental phases highlights strengths and weaknesses of various models. It is anticipated that this investigation will inform course development decisions on a whole-of-course approach to WIL that improves student engagement and learning experience. The importance of WIL is not disputed. However with industry expectations, increasing student numbers and cohort diversity the ways in which students and industry currently engage in WIL are not sustainable and more creative, flexible and engaging approaches are needed.

Cross-cultural learning experiences in architectural education

This qualitative study looks at the joint output of 20 architecture students from 2 different countries during their short respective Study Tours to each other’s country to discern the effect of cross-cultural experiences on their learning. This paper uses the students’ joint design efforts and reflective writings to investigate the outcome of this cross-cultural educational exchange. Their joint design efforts resulted in the making of small built structures, drawings and collaborative design proposals for an urban setting. In addition, a short questionnaire and personal interviews were also used as methods to gain insight into their experience and to use as a comparative study. The question is also raised in this paper of whether spontaneous friendship among students is integral to long term learning in a cross-cultural context in comparison to pre-designed learning objectives on the part of the educators. This paper also initiates the dialogue of the extent of cultural influences and universal ideas on collaborative architectural design. With increasing joint design ventures between architectural firms in different countries, there is interest in how collaborative design can be understood in a cross-cultural context. This paper examines short term cross cultural experiences and its contribution to architectural education.