Reframing evaluation of learning and teaching: An approach to change

Evaluation practices in the Higher Education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology (QUT) began a project which aims to reframe the organisation’s approach to the evaluation of learning and teaching. This paper describes the existing evaluation system; the emergence and early development of the project; and formulation of a conceptual framework identifying key dimensions of evaluation. It then compares the draft framework with other conceptualisations and models of evaluation identified in the literature, to determine its validity and suitability for supporting QUT’s plans for the future. Overall, the paper represents a structured evaluation of the REFRAME project at a particular point in its lifecycle. Given that the project follows an evidence based, practice-led process and applies an ongoing action research cycle, the findings are presented in the belief that QUT’s experience is broadly applicable to other institutions which may be contemplating change in relation to evaluation of learning and teaching.

Using information communication technologies to develop dynamic curriculum frameworks for diverse cohorts: a case study from event management

This article investigates the role of information communication technologies (ICTs) in establishing a well-aligned, authentic learning environment for a diverse cohort of non-cognate and cognate students studying event management in a higher education context. Based on a case study which examined the way ICTs assisted in accommodating diverse learning needs, styles and stages in an event management subject offered in the Creative Industries Faculty at Queensland University of Technology in Brisbane, Australia, the article uses an action research approach to generate grounded, empirical data on the effectiveness of the dynamic, individualised curriculum frameworks that the use of ICTs makes possible. The study provides insights into the way non-cognate and cognate students respond to different learning tools. It finds that whilst non-cognate and cognate students do respond to learning tools differently, due to a differing degree of emphasis on technical, task or theoretical competencies, the use of ICTs allows all students to improve their performance by providing multiple points of entry into the content. In this respect, whilst the article focuses on the way ICTs can be used to develop an authentic, well-aligned curriculum model that meets the needs of event management students in a higher education context, with findings relevant for event educators in Business, Hospitality, Tourism and Creative Industries, the strategies outlined may also be useful for educators in other fields who are faced with similar challenges when designing and developing curriculum for diverse cohorts.

Development and testing of an iPad application for teaching self-management to heart failure patients. (Nursing/Allied Health Professional Investigator Award)- Abstract

Purpose: Heart failure (HF) is the leading cause of hospitalization and significant burden to the health care system in Australia. To reduce hospitalizations, multidisciplinary approaches and enhance self-management programs have been strongly advocated for HF patients globally. HF patients who can effectively manage their symptoms and adhere to complex medicine regimes will experience fewer hospitalizations. Research indicates that information technologies (IT) have a significant role in providing support to promote patients' self-management skills. The iPad utilizes user-friendly interfaces and to date an application for HF patient education has not been developed. This project aimed to develop the HF iPad teaching application in the way that would be engaging, interactive and simple to follow and usable for patients' carers and health care workers within both the hospital and community setting. Methods: The design for the development and evaluation of the application consisted of two action research cycles. Each cycle included 3 phases of testing and feedback from three groups comprising IT team, HF experts and patients. All patient education materials of the application were derived from national and international evidence based practice guidelines and patient self-care recommendations. Results: The iPad application has animated anatomy and physiology that simply and clearly teaches the concepts of the normal heart and the heart in failure. Patient Avatars throughout the application can be changed to reflect the sex and culture of the patient. There is voice-over presenting a script developed by the heart failure expert panel. Additional engagement processes included points of interaction throughout the application with touch screen responses and the ability of the patient to enter their weight and this data is secured and transferred to the clinic nurse and/or research data set. The application has been used independently, for instance, at home or using headphones in a clinic waiting room or most commonly to aid a nurse-led HF consultation. Conclusion: This project utilized iPad as an educational tool to standardize HF education from nurses who are not always heart failure specialists. Furthermore, study is currently ongoing to evaluate of the effectiveness of this tool on patient outcomes and to develop several specifically designed cultural adaptations [Hispanic (USA), Aboriginal (Australia), and Maori (New Zealand)].

Solo life to Second Life : the design of physical and virtual learning spaces inspired by the drama classroom

This paper explores the design of virtual and physical learning spaces developed for students of drama and theatre studies. What can we learn from the traditional drama workshop that will inform the design of drama and theatre spaces created in technology-mediated learning environments? The authors examine four examples of spaces created for online, distance and on-campus students and discuss the relationship between the choice of technology, the learning and teaching methods, and the outcomes for student engagement. Combining insights from two previous action research projects, the discussion focuses on the physical space used for contemporary drama workshops, supplemented by Web 2.0 technologies; a modular online theatre studies course; the blogging space of students creating a group devised play; and the open and immersive world of Second Life, where students explore 3D simulations of historical theatre sites. The authors argue that the drama workshop can be used as inspiration for the design of successful online classrooms. This is achieved by focusing on students’ contributions to the learning as individuals and group members, the aesthetics and mise-en-scene of the learning space, and the role of mobile and networked technologies. Students in this environment increase their capacity to become co-creators of knowledge and to achieve creative outcomes. The drama workshop space in its physical and virtual forms is seen as a model for classrooms in other disciplines, where dynamic, creative and collaborative spaces are required.

Responding to national curriculum goals for English language learners : enhancing reading strategies in high school content areas

The official need for content teachers to teach the language features of their fields has never been greater in Australia than now. In 2012, the recently formed national curriculum board announced that all teachers are responsible for the English language development of students whose first language or dialect is not Standard Australian English (SAE). This formal endorsement is an important juncture regarding the way expertise might be developed, perceived and exchanged between content and language teachers through collaboration, in order for the goals of English language learners in content areas to be realised. To that end, we conducted an action research project to explore and extend the reading strategies pedagogy of one English language teacher who teaches English language learners in a parallel junior high school Geography program. Such pedagogy will be valuable for all teachers as they seek to contribute to English language development goals as outlined in national curricula.

Making sense of mental illness as a full human experience : Perspective of illness and recovery held by people with a mental illness living in the community.

Recovery is a highly contextualized concept amid divergent interpretations and unique experiences. There is substantial current interest in building evidence about recovery from mental illness in order to inform best practice founded in the ways people find to live productive and meaningful lives. This paper presents some accounts related to recovery and illness expressed by eight people through a Participatory Action Research project. The research facilitated entry to the subjective experiences of living in the community as an artist with a mental illness. The people in the research shared an integrated understanding of illness, recovery and identity. Their understanding provided insight into mental illness as an inseparable aspect of who they were. Further, specific issue was raised of recovery as a clinical term with a requirement to meet distinct conventions of recovery. This paper emphasizes that being ill and being well, for the person with a mental illness, is a dynamic and complex development not easily explained or transformed into uniform process or outcomes. Attempts to establish an integral or consensual approach to recovery has, to date, disregarded mental illness as a full human experience. This paper argues that broader frameworks for thinking and responding to the dynamic processes of mental illness and recovery are needed and require acknowledgment of competing and contradictory ideas.

Influencing innovation in SME’s : from designer to transitional engineer

The following paper presents insights found during an ongoing industry engagement with a family-owned manufacturing SME in Australia. The study seeks to unpack cultural, strategic, product opportunities and challenges available to the firm engaging in a design led approach to innovation. Design led innovation finds new market opportunities to innovate through a holistic perspective of both the internal business operations, focusing on the customer experience and external stakeholders. The action research based methodology consisted of 25 semi-structured interviews, a reflective journal and focus group, all of which were analysed thematically. This paper focuses on the resultant themes of the focus group data constituting a discussion around the best strategies to try and integrate design thinking as a fundamental skill across the company. The main thematic results included the participant’s limited views of themselves (in current roles) as being to create ‘buy-in’ to change initiatives, a preference to act as a group to gain traction in the firm, importance of better defining the problem before reaching a design solution and lastly the time required to implement versus the need for change. These findings are valuable in assisting other family owned firms in managing core growth activities with design and innovation initiatives, particularly in the area of new product development.

Exploring partnership : the experience of HCAP

This report summarises the participatory action research (PAR) undertaken as part of the Homelessness Community Action Planning (HCAP) project implemented across seven regions in Queensland in 2011 and 2012. The HCAP is a component of the Queensland strategy for the National Partnership Agreement on Homelessness, and is funded for three years (2010-2013). The report identifies and analyses factors which facilitated or constrained the development of Government- NGO partnerships at regional and state levels in HCAP. The study supports the view that the HCAP partnership between the Queensland Government and the Community Services Sector is working and likely to be productive.

A methodological approach to modelling design led innovation across secondary education : an Australian case study

Incorporating design thinking as a generic capability at a school level is needed to ensure future generations are empowered for business innovation and active citizenship. This paper describes the methodology of an investigation into modelling design led innovation approaches from the business sector to secondary education, as part of a larger study. It builds on a previously discussed research agenda by outlining the scope, significance and limitations of currently available research in this area, examining an action research methodology utilising an Australian design immersion program case study, and discussing implications and future work. It employs a triangulated approach encompassing thematic analysis of qualitative data collection from student focus groups, semi-structured convergent interviews with teachers and facilitators, and student journals. Eventual outcomes will be reviewed and analysed within the framework of a proposed innovation matrix model for educational growth, synthesising principles responding to 21st century student outcomes. It is anticipated this research will inform a successful design led secondary education innovation model, facilitating new engagement frameworks between tertiary and secondary education sectors, as well as providing new insight into the suitability of action research in prototyping social innovation in Australia.

Studios for the masses : can student colaboration replace the master-apprentice relationship in design instruction?

This action research project investigated the use of a collaborative learning approach for addressing issues associated with teaching urban design to large, diverse cohorts. As a case study, I observed two semesters of an urban design unit that I revised between 2011 and 2012 to incorporate collaborative learning activities. Data include instructional materials, participant observations, peer-reviews of collaborative learning activities, feedback from students and instructors and student projects. Themes that emerged through qualitative analysis include the challenge of removing inequalities inherent in the diverse cohort, the challenge of unifying project guidance and marking criteria, and the challenge of providing project guidance for a very large cohort. Most notably, the study revealed a need to clarify learning objectives relating to design principles in order to fully transition to and benefit from a collaborative learning model.

Successful outcomes in vocational education and training courses : how pedagogy and expectations influence achievement

This paper reports on a four year Australian Research Council funded Linkage Project titled Skilling Indigenous Queensland, conducted in regional areas of Queensland, Australia from 2009 to 2013. The project sought to investigate vocational education, training (VET) and teaching, Indigenous learners’ needs, employer cultural and expectations and community culture and expectations to identify best practice in numeracy teaching for Indigenous VET learners. Specifically it focused on ways to enhance the teaching and learning of courses and the associated mathematics in such courses to benefit learners and increase their future opportunities of employment. To date thirty-nine teachers/trainers/teacher aides and two hundred and thirty-one students consented to participate in the project. Nine VET courses were nominated to be the focus on the study. This paper focuses on questionnaire and interview responses from four trainers, two teacher aides and six students. In recent years a considerable amount of funding has been allocated to increasing Indigenous Peoples’ participation in education and employment. This increased funding is predicated on the assumption that it will make a difference and contribute to closing the education gap between Indigenous and non-Indigenous Australians (Council of Australia Governments, 2009). The central tenet is that access to education for Indigenous People will create substantial social and economic benefits for regional and remote Indigenous People. The project’s aim is to address some of the issues associated with the gap. To achieve the aims, the project adopted a mixed methods design aimed at benefitting research participants and included: participatory collaborative action research (Kemmis & McTaggart, 1988) and, community research (Smith, 1999). Participatory collaborative action research refers to a is a “collective, self-reflective enquiry undertaken by participants in social situations in order to improve the rationality and justice of their own social and educational practices” (Kemmis et al., 1988, p. 5). Community research is described as an approach that “conveys a much more intimate, human and self-defined space” (p. 127). Community research relies on and validates the community’s own definitions. As the project is informed by the social at a community level, it is described as “community action research or emancipatory research” (Smith, 1999, p. 127). It seeks to demonstrate benefit to the community, making positive differences in the lives of Indigenous People and communities. The data collection techniques included survey questionnaires, video recording of teaching and learning processes, teacher reflective video analysis of teaching, observations, semi-structured interviews and student numeracy testing. As a result of these processes, the findings indicate that VET course teachers work hard to adopt contextualising strategies to their teaching, however this process is not always straight forward because of the perceptions of how mathematics has been taught and learned historically. Further teachers, trainers and students have high expectations of one another with the view to successful outcomes from the courses.

Embedding diversity in a university justice curriculum : an exploration of the issues

This paper reports on an action-learning project conducted within the first year criminal justice curriculum in an Australian university. The project was initiated after an audit of first year units and student feedback revealed that there were gaps in the curriculum that possibly were disadvantaging certain groups of students, including mature, international, queer and disabled students, rendering them invisible. Official (university controlled student surveys and other feedback mechanisms) and anecdotal feedback found that at least some students in these groups felt disenfranchised; that is, unable to relate to either the subject mater, other students, or the university setting itself. As a school in which social justice provides the context for learning about criminal justice, first year subject coordinators as a group came to recognise the need for embedding diversity in the curriculum.

Second Skin 2012 Workshop

The Second Skin 2012 Workshop Program consisted of a full-day intensive design immersion workshop run on Saturday 14 July 2012, at the QUT Faculty of Creative Industries Fashion Studios at Kelvin Grove Brisbane, Australia, for 30? self-selected high-achieving junior and middle school (year 5-9) students, as part of the Queensland Academies ‘Young Scholars’ Program. Inspired by a scientist researching the impact of sun on skin, and mentored by tertiary fashion design and interior design educators, and six tertiary fashion design and interior design students, the workshop explored science and design-inspired prototype solutions for sun-safety. This action research study aimed to facilitate an acute awareness in young people of the sun safety message (alternative to a scare campaign), the role of design in society and the value of design thinking skills in solving complex challenges, and to inspire the generation of strategies to address a systemic health issue. It also aimed to investigate the value of collaboration between junior and middle school students, tertiary design educators and students and industry professionals in targeting youth sun safety, and inspiring post-secondary pathways and idea generation for education. During the workshop, students developed sketching, making, communication, presentation and collaboration skills to improve their design process, while considering social, cultural and environmental opportunities. Through a series of hands-on collaborative design experiments, participants explored in teams of five, ways in which a ‘second skin’ can mirror elements of our skin – the ability to protect, divide, enclose, stretch, scar, pattern, peel and reveal – inspiring both functional and aesthetic design solutions. Underpinned by the State Library of Queensland Design Minds Website ‘inquire, ideate and implement’ model of design thinking, the experiments culminated in the development of a detailed client brief, the design and fabrication of a fashionable sun safe clothing range and then a team presentation and modelling of prototypes in a fashion parade, viewed also by parents. The final collections were judged by three prominent judges: Louise Baldwin - Executive Manager Public Health QLD Cancer Council, Shane Thompson - Architect and 2012 Queensland Smart Design Fellow, and Leigh Buchanan – Fashion designer and Project Runway Australia finalist. The workshop was filmed for Queensland television program ‘Totally Wild’ for dissemination of the value of design, the Design Minds model and the sun safety message to a wider target youth audience.

Sociocultural animation

The emergence of global computer networks and the ubiquitous availability of advanced information and communication technology (ICT) since the mid Nineties has given rise to the hope that the traditional disadvantages faced by regional economies and regional communities could be elevated easily and swiftly. Yet, the experience of both community informatics and community development researchers and practitioners tells a different tale. Although the potential of ICT is in fact realised in some situations and locations and does provide means to ensure sustainability in some regional communities, elsewhere it has not been taken up or has not been able to elicit change for the promised better. Too many communities are still faced by a centralised structure in the context of commerce, service provision or governance and by various degrees of digital divides between connected and disconnected, between media literate and illiterate, between young and old, and between urban and rural. Many attempts to close or bridge the digital divide have been reported with various degrees of success (cf. Menou, 2001; Servon, 2002). Most of these accounts echo a common voice in that they report similar principles of action, and they reflect – in most cases unconsciously – practices of sociocultural animation. This article seeks to shed light onto the concept of sociocultural animation which is already commonplace in various forms in the arts, in education and professional development, youth work, sports, town planning, careers services, entrepreneurship and tourism. It starts by exploring the origins of sociocultural animation and draws parallels to the current state of research and practice. It unpacks the foundation of sociocultural animation and briefly describes underlying principles and how they can be applied in the context of community informatics and developing regional communities with ICT. Finally, further areas of investigation are being proposed.

ICT to enhance the triangle of life : the soil, the seed, and the life-giver farmer

This paper reports on the outcomes of an ICT enabled social sustainability project “Green Lanka1” trialled in the Wilgamuwa village, which is situated in the Dambulla district of Sri Lanka. The main goals of the project were focused towards the provision of information about market prices, transportation options, agricultural decision support and modern agriculture practices of the farmer communities to improve their livelihood with the effective use of technologies. The project used Web and Mobile (SMS) enabled systems. The Green Lanka project was sponsored by the Information Communication Technology Agency (ICTA) of Sri Lanka under the Institutional Capacity Building Programme (ICBP) grant scheme which was sponsored by the World Bank. Six hundred families in Wilgamuwa village participated in the project activities. The project was designed, executed and studied through an Action Research approach. The lessons learned through the project activities provide an important understanding of the complex interaction between different stakeholders in the process of implementation of ICT enabled solutions within digitally divided societies. The paper analyses the processes used to reduce the resistance to change and improved involvement of farmer communities in ICT enabled projects. It also analyses the interaction between stakeholders involved in design and implementation of the project activities to improve the chances of project success.