Reframing evaluation of learning and teaching: An approach to change

Evaluation practices in the Higher Education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology (QUT) began a project which aims to reframe the organisation’s approach to the evaluation of learning and teaching. This paper describes the existing evaluation system; the emergence and early development of the project; and formulation of a conceptual framework identifying key dimensions of evaluation. It then compares the draft framework with other conceptualisations and models of evaluation identified in the literature, to determine its validity and suitability for supporting QUT’s plans for the future. Overall, the paper represents a structured evaluation of the REFRAME project at a particular point in its lifecycle. Given that the project follows an evidence based, practice-led process and applies an ongoing action research cycle, the findings are presented in the belief that QUT’s experience is broadly applicable to other institutions which may be contemplating change in relation to evaluation of learning and teaching.

Reflection in Integrated Problem Based Learning : 20 years of continuing application to the teaching of architecture

In 1984 the School of Architecture and Built Environment within the University of Newcastle, Australia introduced an integrated program based on real design projects and using Integrated Problem Based Learning (IPBL) as the teaching method. Since 1984 there have been multiple changes arising from the expectations of the architectural fraternity, enrolling students, lecturers, available facilities, accreditation authorities and many others. These challenges have been successfully accommodated whilst maintaining the original purposes and principles of IPBL. The Architecture program has a combined two-degree structure consisting of a first degree, Bachelor of Science (Architecture), followed by a second degree, Bachelor of Architecture. The program is designed to simulate the problem-solving situations that face a working architect in every day practice. This paper will present the degree structure where each student is enrolled in a single course per semester incorporating design integration and study areas in design studies, professional studies, historical studies, technical studies, environmental studies and communication skills. Each year the design problems increase in complexity and duration set around an annual theme. With 20 years of successful delivery of any program there are highlights and challenges along the way and this paper will discuss some of the successes and barriers experienced within the School of Architecture and Built Environment in delivering IPBL. In addition, the reflective process investigates the currency of IPBL as an appropriate vehicle for delivering the curriculum in 2004 and any additional administrative or staff considerations required to enhance the continuing application of IPBL.

Learning to think spatially : what do students 'see' in numeracy test items?

Learning to think spatially in mathematics involves developing proficiency with graphics. This paper reports on 2 investigations of spatial thinking and graphics. The first investigation explored the importance of graphics as 1 of 3 communication systems (i.e. text, symbols, graphics) used to provide information in numeracy test items. The results showed that graphics were embedded in at least 50 % of test items across 3 year levels. The second investigation examined 11 – 12-year-olds’ performance on 2 mathematical tasks which required substantial interpretation of graphics and spatial thinking. The outcomes revealed that many students lacked proficiency in the basic spatial skills of visual memory and spatial perception and the more advanced skills of spatial orientation and spatial visualisation. This paper concludes with a reaffirmation of the importance of spatial thinking in mathematics and proposes ways to capitalize on graphics in learning to think spatially.

From rhetoric to practice : issues in teaching and learning touch keyboarding

Touch keyboarding as a vocational skill is disappearing at a time when students and educators across alleducational sectors are expected to use a computer keyboard on a regular basis. there is documentation surrounding the embedding of Information and Communication Technology (ICT) within the curricula and yet within the National Training Packages touch keyboarding, previously considered a core component, is now an elective in the Business Services framework. This situation is an odds with current practice overseas where touch keyboarding is a component of primary and secondary curricula. From Rhetoric to Practice explores the current issues and practice in teaching and learning touch keyboarding in primary, secondary and tertiary institutions. Through structured interview participants detailed current practice of teachers and their students. Further, tertiary students participated in a training program aimed at achquiring touch keyboarding as a skill to enhance their studies. The researcher's background experience of fifteen years teaching touch keyboarding and computer literacty to adults and 30 years in Business Services trade provides a strong basis for this project. The teaching experience is enhanced by industry experience in administration, course coordination in technical, community and tertiary institutions and a strong commitment to the efficient usage of a computer by all. The findings of this project identified coursework expectations requiring all students from kindergarten to tertiary to use a computer keyboard on a weekly basis and that neither teaching nor learning tough keyboarding appears in the primary, secondary and tertiary curricula in New South Wales. Further, teachers recognised tough keyboarding as the prefered style over 'hunt and peck' keyboarding while acknowledging the teaching and learning difficulties of time constraints, the need for qualified touch keyboarding teachers and issues arising when retraining students from existing poor habits. In conclusion, this project recommends that computer keyboarding be defined as a writing tool for education, vocation and life, with early instruction set in primary schooling area and embedding touch keyboarding with the secondary, technical and tertiary areas and finally to draw the attention of educational authorities to the Duty Of Care aspects associated with computer keyboarding in the classroom.

Collaboration and competition on a Wiki : the praxis of online social learning to improve academic writing and research in under-graduate students

The number of internet users in Australia has been steadily increasing, with over 10.9 million people currently subscribed to an internet provider (ABS, 2011). Over the past year, the most avid users of the Internet were 15 – 24 year olds, with approximately 95% accessing the internet on a regular basis (ABS, Social Trends, 2011). While the internet, in particularly Web 2.0, has been described as fundamental to higher education students, social and leisure internet tools are also increasingly being used by these students to generate and maintain their social and professional networks and interactions (Duffy & Bruns, 2006). Rapid technological advancements have enabled greater and faster access to information for learning and education (Hemmi et al, 2009; Glassman & Kang, 2011). As such, we sought to integrate interactive, online social media into the assessment profile of a Public Health undergraduate cohort at the Queensland University of Technology (QUT). The aim of this exercise was to engage undergraduate students to both develop and showcase their research on a range of complex, contemporary health issues within the online forum of Wikispaces for review and critique by their peers. We applied Bandura’s Social Learning Theory (SLT) to analyse the interactive processes from which students developed deeper and more sustained learning, and via which their overall academic writing standards were enriched. This paper outlines the assessment task, and the students’ feedback on their learning outcomes in relation to the Attentional, Retentional, Motor Reproduction, and Motivational Processes outlined by Bandura in SLT. We conceptualise the findings in a theoretical model, and discuss the implications for this approach within the broader tertiary environment.

Connecting Indigenous stories with geology : inquiry-based learning in a middle years classroom

One way to integrate indigenous perspectives in junior science is through links between indigenous stories of the local area and science concepts. Using local indigenous stories about landforms, a teacher of year 8 students designed a unit on geology that catered for the diverse student population in his class. This paper reports on the inquiry-based approach structured around the requirements of the Australian Curriculum highlighting the learning and engagement of students during the unit.

Supporting the learning of rapid application development in a database environment

Intelligent Tutoring Systems (ITSs) are computer systems designed to provide individualised help to students, learning in a problem solving context. The difference between an ITS and a Computer Assisted Instruction (CAI) system is that an ITS has a Student Model which allows it to provide a better educational environment. The Student Model contains information on what the student knows, and does not know, about the domain being learnt, as well as other personal characteristics such as preferred learning style. This research has resulted in the design and development of a new ITS: Personal Access Tutor (PAT). PAT is an ITS that helps students to learn Rapid Application Development in a database environment. More specifically, PAT focuses on helping students to learn how to create forms and reports in Microsoft Access. To provide an augmented learning environment, PAT’s architecture is different to most other ITSs. Instead of having a simulation, PAT uses a widelyused database development environment (Microsoft Access). This enables the students to ask for help, while developing real applications using real database software. As part of this research, I designed and created the knowledge base required for PAT. This contains four models: the domain, student, tutoring and exercises models. The Instructional Expert I created for PAT provides individualised help to the students to help them correctly finish each exercise, and also proposes the next exercise that a student should work on. PAT was evaluated by students enrolled in the Databases subject at QUT, and by staff members involved in teaching the subject. The results of the evaluation were positive and are discussed in the thesis.

Resilience by design: A participatory approach to designing an interactive digital application for promoting children's resilience

The promotion of resilience (the capacity of an individual or community to bounce back and recover from adversity) has become an important area of public health. In recent years it has expanded into the digital domain, and many online applications have been developed to promote children's resilience. In this study, it is argued that the majority of existing applications are limited because they take a didactic approach, and conceive of interaction as providing navigational choices. Because they simply provide information about resilience or replicate offline, scenario-based strategies, the understanding of resilience they provide is confined to a few, predetermined factors. In this study I propose a new, experiential approach to promoting resilience digitally. I define resilience as an emergent, situated and context-specific phenomenon. Using a Participatory Design model in combination with a salutogenic (strength-based) health methodology, this project has involved approximately 50 children as co-designers and co-researchers over two years. The children have contributed to the design of a new set of interactive resilience tools, which facilitate resilience promotion through dialogic and experiential learning. The major outcomes of this study include a new methodology for developing digital resilience tools, a new set of tools that have been developed and evaluated in collaboration with children and a set of design principles to guide future development. Beyond these initial and tangible outcomes, this study has also established that the benefits of introducing Participatory Design into a health promoting model rests primarily in the change of the role of children from "users" of technology and education to co-designers, where they assume a leadership role in both designing the tools and in directing their resilience learning.

Looking Yonder : Longitudinal Research and Evaluation of the Yonder Participative Arts-based Learning Model Report 2010-2012

Since 2007, KITE Arts Education Program @ QPAC has been engaged in a series of arts and drama-based experiences for students in selected primary schools on the edges of Brisbane and in regional Queensland. The in-school workshop experiences of the program have culminated in a performance by the children for their school community, parents and carers at the Queensland Performing Arts Centre or a regional cultural venue. In conducting an analysis of the Yonder project, the researcher aimed to provide evidence of outcomes brought about through participation by schools, school staff, students and their communities in the Yonder project. To develop longitudinal data project initiators, participants were interviewed at six-monthly intervals to establish patterns of engagement and participation. The report analyses arts-based workshops conducted by the teacher artist in edge-city Brisbane and a regional centre; interviews with teachers and school administrators from the participating schools; interviews with teacher artist and professional artists; interviews with community partners; teacher professional development workshops; community-based workshops; performance outcomes that were the culminating events of the workshop program; student work samples and student reflections on the program. This document covers data and project outputs from February 2010 to July 2012. There have been five iterations of the Yonder project since its commencement in mid-2009 — three in regional Queensland (February–April 2010; February–May 2011; February–May 2012) and two in edge-city1 Brisbane (July–September 2010; August–October 2011). This report is a result of a research partnership between Queensland Performing Arts Centre and Queensland University of Technology (QUT) Creative Industries Faculty(Drama).

Learning and assessment in Australia

Camp Kilda (CK) is regarded as being a quality early childhood center, and has many features you would typically expect to see in settings across Australia. The children are busily engaged in hands-on activity, playing indoors and outdoors, in the sandpit, under the shade of a big mango tree. The learning environment is planned to offer a variety of activities, including dramatic play, climbing equipment, balls, painting, drawing, clay, books, blocks, writing materials, scissors, manipulative materials. The children are free to access all the materials, and they play either individually or in small groups. The teachers encourage and stimulate the children’s learning, through interactions and thoughtful planning. Learning and assessment at CK is embedded within the cultural and social contexts of the children and their community. Children’s learning is made visible through a rich variety of strategies, including recorded observations, work samples, photographs, and other artifacts. Parents are actively encouraged to build on these “stories” of their children. Planning is based around the teachers’ analysis of the information they gather daily as they interact with the children and their families.

Imagination in school children's choice of their learning environment: An Australian study

A visual research project addressed school children's concepts of ideal learning environments. Drawings and accompanying narratives were collected from Year 5 and Year 6 children in nine Queensland primary schools. The 133 submissions were analysed and coded to develop themes, identify key features and consider the uses of imagination. The children's imagined schools echo ideas promoted by progressive educators. The results of this study suggest benefits for school designers can emerge from the imaginative contributions of children in creating engaging environments, while educational policy makers can benefit from children's ideas in the promotion of engaging, student-centred pedagogies.

International students using online information resources to learn : complex experience and learning needs

This paper reports the findings of a qualitative study which investigated 25 international students’ use of online information resources for study purposes at two Australian universities. Using an expanded critical incident approach, the study viewed international students through an information literacy lens, as information-using learners. The findings are presented in two complementary parts: as a word picture that describes their whole experience of using online information resources to learn; and as a tabulated set of critical findings that summarises their associated information literacy learning needs. The word picture shows international students’ resource use as a complex interplay of eight inter-related elements: students; information-learning environment; interactions (with online resources); strengths-challenges; learning-help; affective responses; reflective responses; cultural-linguistic dimensions. In using online resources, the international students experience an array of strengths and challenges, and an apparent information literacy imbalance between their more developed information skills and less developed critical information use. The critical findings about information literacy needs provide a framework for developing an inclusive informed learning approach that responds to international students’ complex information using experiences and needs. While the study is situated in Australia, the findings are of potential interest to educators, information professionals and researchers worldwide who seek to support learning in culturally diverse higher education contexts.

Enriching students’ intellectual diet through inquiry based learning

Traditionally, Science education has stressed the importance of teaching students to conduct ‘scientific inquiry’, with the main focus being the experimental model of inquiry used by real world scientists. Current educational approaches using constructivist pedagogy recognise the value of inquiry as a method for promoting the development of deep understanding of discipline content. A recent Information Learning Activity undertaken by a Grade Eight Science class was observed to discover how inquiry based learning is implemented in contemporary Science education. By analysing student responses to questionnaires and assessment task outcomes, the author was able to determine the level of inquiry inherent in the activity and how well the model supported student learning and the development of students’ information literacy skills. Although students achieved well overall, some recommendations are offered that may enable teachers to better exploit the learning opportunities provided by inquiry based learning. Planning interventions at key stages of the inquiry process can assist students to learn more effective strategies for dealing with cognitive and affective challenges. Allowing students greater input into the selection of topic or focus of the activity may encourage students to engage more deeply with the learning task. Students are likely to experience greater learning benefit from access to developmentally appropriate resources, increased time to explore topics and multiple opportunities to undertake information searches throughout the learning activity. Finally, increasing the cognitive challenge can enhance both the depth of students’ learning and their information literacy skills.

Learning processes in creative industries initiatives in developing countries : an examination of policy and practice

What does a dance group in Benin that mixes contemporary and ethnic dancing have in common with Mongolian felt producers that want to enter the design market in Europe? These are both examples of learning processes in Creative Industries initiatives in developing countries. Following the concept of sustainable development, I argue that the challenge for developing countries in contemporary society is to meet the very real need of people for economic development and opportunities for income generation, while at the same time avoiding unintended and unwanted consequences of economic development and globalisation. The concept of the Creative Industries may be a way to promote a development that is sustainable and avoids social exclusion of groups-at-risk. In line with this, I argue that the Creative Industries sector could, in fact, link economic development and the continuation and evolution of local traditions and cultural heritage. A pressing question then is: how can education and learning contribute to creating a context in which talent can flourish? This study aims to provide a comprehensive analysis of the research problem of this thesis: what elements are conducive for individual learning processes in creative development initiatives? In this, I argue that it is crucial to determine what ingredients and characteristics contribute to making these initiatives successful, that is, to meet their specific goals, in a developing context. This is explored through a staged analysis: an overview of quantitative data, an inventory and comparative case studies and, finally, the description and analysis of two in-depth case studies – felt design in Mongolia (Asia) and dance in Benin (Africa), in which I was an observer of the action phase of the local interventions. The analysis culminates in practice-related outcomes related to the operation of creative development initiatives, as well as the contribution to the academic debate on issues like the cultural gap between developed and developing countries, transformative learning and the connection of learning spaces.