Learner-focused evaluation cycles : facilitating learning using feedforward, concurrent and feedback evaluation

There is a growing trend to offer students learning opportunities that are flexible, innovative and engaging. As educators embrace student-centred agile teaching and learning methodologies, which require continuous reflection and adaptation, the need to evaluate students’ learning in a timely manner has become more pressing. Conventional evaluation surveys currently dominate the evaluation landscape internationally, despite recognition that they are insufficient to effectively evaluate curriculum and teaching quality. Surveys often: (1) fail to address the issues for which educators need feedback, (2) constrain student voice, (3) have low response rates and (4) occur too late to benefit current students. Consequently, this paper explores principles of effective feedback to propose a framework for learner-focused evaluation. We apply a three-stage control model, involving feedforward, concurrent and feedback evaluation, to investigate the intersection of assessment and evaluation in agile learning environments. We conclude that learner-focused evaluation cycles can be used to guide action so that evaluation is not undertaken simply for the benefit of future offerings, but rather to benefit current students by allowing ‘real-time’ learning activities to be adapted in the moment. As a result, students become co-producers of learning and evaluation becomes a meaningful, responsive dialogue between students and their instructors.

Sustainability of a successful health and nutrition program in a remote Aboriginal community

OBJECTIVE: To assess the long term effect of a nutrition program in a remote Aboriginal community (Minjilang). DESIGN: Evaluation of nutritional outcomes over the three years before and the three years after a health and nutrition program that ran from June 1989 to June 1990. Turnover of food items at the community store was used as a measure of dietary intake at Minjilang and a comparison community. SETTING: A community of about 150 Aboriginal people live at Minjilang on Croker Island, 240 km north-east of Darwin. A similar community of about 300 people on another island was used as the comparison. RESULTS: The program produced lasting improvements in dietary intake of most target foods (including fruit, vegetables and wholegrain bread) and nutrients (including folate, ascorbic acid and thiamine). Sugar intake fell in both communities before the program, but the additional decrease in sugar consumption during the program at Minjilang "rebounded" in the next year. Dietary improvements in the comparison community were delayed and smaller than at Minjilang. CONCLUSIONS: The success of the program at Minjilang was linked to an ongoing process of social change, which in turn provided a stimulus for dietary improvement in the comparison community. When Aboriginal people themselves control and maintain ownership of community-based intervention programs, nutritional improvements can be initiated and sustained.

Development of hospital disaster resilience : conceptual framework and potential measurement

Objective Despite ‘hospital resilience’ gaining prominence in recent years, it remains poorly defined. This article aims to define hospital resilience, build a preliminary conceptual framework and highlight possible approaches to measurement. Methods Searches were conducted of the commonly used health databases to identify relevant literature and reports. Search terms included ‘resilience and framework or model’ or ‘evaluation or assess or measure and hospital and disaster or emergency or mass casualty and resilience or capacity or preparedness or response or safety’. Articles were retrieved that focussed on disaster resilience frameworks and the evaluation of various hospital capacities. Result A total of 1480 potentially eligible publications were retrieved initially but the final analysis was conducted on 47 articles, which appeared to contribute to the study objectives. Four disaster resilience frameworks and 11 evaluation instruments of hospital disaster capacity were included. Discussion and conclusion Hospital resilience is a comprehensive concept derived from existing disaster resilience frameworks. It has four key domains: hospital safety; disaster preparedness and resources; continuity of essential medical services; recovery and adaptation. These domains were categorised according to four criteria, namely, robustness, redundancy, resourcefulness and rapidity. A conceptual understanding of hospital resilience is essential for an intellectual basis for an integrated approach to system development. This article (1) defines hospital resilience; (2) constructs conceptual framework (including key domains); (3) proposes comprehensive measures for possible inclusion in an evaluation instrument, and; (4) develops a matrix of critical issues to enhance hospital resilience to cope with future disasters.

Systematic evaluation of GoSoapBox in tertiary education : a student response system for improving learning experiences and outcomes

There is currently a wide range of research into the recent introduction of student response systems in higher education and tertiary settings (Banks 2006; Kay and Le Sange, 2009; Beatty and Gerace 2009; Lantz 2010; Sprague and Dahl 2009). However, most of this pedagogical literature has generated ‘how to’ approaches regarding the use of ‘clickers’, keypads, and similar response technologies. There are currently no systematic reviews on the effectiveness of ‘GoSoapBox’ – a more recent, and increasingly popular student response system – for its capacity to enhance critical thinking, and achieve sustained learning outcomes. With rapid developments in teaching and learning technologies across all undergraduate disciplines, there is a need to obtain comprehensive, evidence-based advice on these types of technologies, their uses, and overall efficacy. This paper addresses this current gap in knowledge. Our teaching team, in an undergraduate Sociology and Public Health unit at the Queensland University of Technology (QUT), introduced GoSoapBox as a mechanism for discussing controversial topics, such as sexuality, gender, economics, religion, and politics during lectures, and to take opinion polls on social and cultural issues affecting human health. We also used this new teaching technology to allow students to interact with each other during class – both on both social and academic topics – and to generate discussions and debates during lectures. The paper reports on a data-driven study into how this interactive online tool worked to improve engagement and the quality of academic work produced by students. This paper will firstly, cover the recent literature reviewing student response systems in tertiary settings. Secondly, it will outline the theoretical framework used to generate this pedagogical research. In keeping with the social and collaborative features of Web 2.0 technologies, Bandura’s Social Learning Theory (SLT) will be applied here to investigate the effectiveness of GoSoapBox as an online tool for improving learning experiences and the quality of academic output by students. Bandura has emphasised the Internet as a tool for ‘self-controlled learning’ (Bandura 2001), as it provides the education sector with an opportunity to reconceptualise the relationship between learning and thinking (Glassman & Kang 2011). Thirdly, we describe the methods used to implement the use of GoSoapBox in our lectures and tutorials, and which aspects of the technology we drew on for learning purposes, as well as the methods for obtaining feedback from the students about the effectiveness or otherwise of this tool. Fourthly, we report cover findings from an examination of all student/staff activity on GoSoapBox as well as reports from students about the benefits and limitations of it as a learning aid. We then display a theoretical model that is produced via an iterative analytical process between SLT and our data analysis for use by academics and teachers across the undergraduate curriculum. The model has implications for all teachers considering the use of student response systems to improve the learning experiences of their students. Finally, we consider some of the negative aspects of GoSoapBox as a learning aid.

Transformative approaches to sustainability

How to live sustainably is a topic of local, national and international importance. The Australian National Curriculum (ACARA, 2011) identifies sustainability as a cross-disciplinary strand, obligating teachers to build sustainability into their pedagogical practices. In early childhood education, the Early Years Learning Framework (2009) and more recently, the National Quality Framework (2011) provide impetus for early childhood education for sustainably (ECEfS). This article discusses ECEfS, but first, it addresses climate change putting this into a sustainability perspective.

Improved rural health through a collaborative model of care – qualitative findings

INTRODUCTION Health disparity between urban and rural regions in Australia is well-documented. In the Wheatbelt catchments of Western Australia there is higher incidence and rate of avoidable hospitalisation for chronic diseases. Structured care approach to chronic illnesses is not new but the focus has been on single disease state. A recent ARC Discovery Project on general practice nurse-led chronic disease management of diabetes, hypertension and stable ischaemic heart disease reported improved communication and better medical administration.[1] In our study we investigated the sustainability of such a multi-morbidities general practice –led collaborative model of care in rural Australia. METHODS A QUAN(qual) design was utilised. Eight pairs of rural general practices were matched. Inclusion criteria used were >18 years and capable of giving informed consent, at least one identified risk factor or diagnosed with chronic conditions. Patients were excluded if deemed medically unsuitable. A comprehensive care plan was formulated by the respective general practice nurse in consultation with the treating General Practitioner (GP) and patient based on the individual’s readiness to change, and was informed by available local resource. A case management approach was utilised. Shediaz-Rizkallah and Lee’s conceptual framework on sustainability informed our evaluation.[2] Our primary outcome on measures of sustainability was reduction in avoidable hospitalisation. Secondary outcomes were patients and practitioners acceptance and satisfaction, and changes to pre-determined interim clinical and process outcomes. RESULTS The qualitative interviews highlighted the community preference for a ‘sustainable’ local hospital in addition to general practice. Costs, ease of access, low prioritisation of self chronic care, workforce turnover and perception of losing another local resource if underutilised influenced the respondents’ decision to present at local hospital for avoidable chronic diseases regardless. CONCLUSIONS Despite the pragmatic nature of rural general practice in Australia, the sustainability of chronic multi-morbidities management in general practice require efficient integration of primary-secondary health care and consideration of other social determinants of health. What this study adds: What is already known on this subject: Structured approach to chronic disease management is not new and has been shown to be effective for reducing hospitalisation. However, the focus has been on single disease state. What does this study add: Sustainability of collaborative model of multi-morbidities care require better primary-secondary integration and consideration of social determinants of health.

The importance of agency in facilitating the transition to a multifunctional countryside

We appreciate Holmes' body of work relating to transitions within the Australian landscape, and welcome the opportunity to engage in a discussion on this topic. The paper to which Holmes refers (Bjørkhaug and Richards, 2008) examined the application of agricultural (rather than landscape) multifunctionality in both Norway and Australia. Of specific focus was how non-tradeable concerns, such as environmental sustainability, faired under these divergent systems. We argued that Norway's multifunctionality was strong, due to it being embraced at both the policy and actor level, whereas Australia's could be described as weak. This ‘weak multifunctionality’ that we observed in Australia was due to an emerging bi-lateral (state and federal) policy framework that advocated the importance of environmental values which was rarely embraced by landholders who found themselves trapped on the ‘agricultural treadmill’. The nature of the treadmill is that alternative forms of land use are unthinkable when on-farm investments have been made that support the status quo – to get bigger and/or more efficient. For many of the Australian landholders interviewed in relation to this study, efficiency in production was at odds with the values necessary to effect a transition toward multifunctionality. For instance, graziers in Central Queensland were unconvinced of the value of conserving native flora and fauna when economic viability can be better assured through clear felling native forests to increase the productive capacity of the land.

Establishing a framework for transforming student engagement, success and retention in higher education institutions [Final Report]

The perennial issues of student engagement, success and retention (SESR) in higher education continue to attract attention as key indicators of learning and teaching quality. This project aimed to establish and provide a holistic framework that would allow higher education institutions (HEIs) manage and improve their student engagement and retention strategies and programs. The framework and main project deliverable is a Maturity Model (MM) for Student Engagement, Success and Retention (SESR-MM). The project involved three Australian universities with experience and reputations in SESR activities: Queensland University of Technology (lead institution), the University of Queensland and Griffith University, working cooperatively to develop and trial the project deliverables. Project findings suggest that the SESR-MM has the potential to positively transform the holistic—academic, social and personal—engagement experiences of students in Australian universities, and that the SESR-MM is a useful mechanism for sharing good practice and improving programs designed to enhance the student experience.

Developing a culture to support peer review of teaching in higher education

This presentation addresses issues related to leadership, academic development and scholarship of teaching and learning, and highlights research funded by the Australian Office of Learning and Teaching (OLT) designed to embed and sustain peer review of teaching within the culture of 5 Australian universities: Queensland University of Technology, University of Technology, Sydney, University of Adelaide, Curtin University, and Charles Darwin University. Peer review of teaching in higher education will be emphasised as a professional process for providing feedback on teaching and learning practice, which if sustained, can become an effective ongoing strategy for academic development (Barnard et al, 2011; Bell, 2005; Bolt and Atkinson, 2010; McGill & Beaty 2001, 1992; Kemmis & McTaggart, 2000). The research affirms that using developmental peer review models (Barnard et al, 2011; D'Andrea, 2002; Hammersley-Fletcher & Orsmond, 2004) can bring about successful implementation, especially when implemented within a distributive leadership framework (Spillane & Healey, 2010). The project’s aims and objectives were to develop leadership capacity and integrate peer review as a cultural practice in higher education. The research design was a two stage inquiry process over 2 years. The project began in July 2011 and encompassed a development and pilot phase followed by a cascade phase with questionnaire and focus group evaluation processes to support ongoing improvement and measures of outcome. Leadership development activities included locally delivered workshops complemented by the identification and support of champions. To optimise long term sustainability, the project was implemented through existing learning and teaching structures and processes within the respective partner universities. Research outcomes highlight the fundamentals of peer review of teaching and the broader contextual elements of integration, leadership and development, expressed as a conceptual model for embedding peer review of teaching within higher education. The research opens a communicative space about introduction of peer review that goes further than simply espousing its worth and introduction. The conceptual model highlights the importance of development of distributive leadership capacity, integration of policies and processes, and understanding the values, beliefs, assumptions and behaviors embedded in an organizational culture. The presentation overviews empirical findings that demonstrate progress to advance peer review requires an ‘across-the-board’ commitment to embed change, and inherently demands a process that co-creates connection across colleagues, discipline groups, and the university sector. Progress toward peer review of teaching as a cultural phenomenon can be achieved and has advantages for academic staff, scholarship, teaching evaluation and an organisation, if attention is given to strategies that influence the contexts and cultures of teaching practice. Peer review as a strategy to develop excellence in teaching is considered from a holistic perspective that by necessity encompasses all elements of an educational environment and has a focus on scholarship of teaching. The work is ongoing and has implication for policy, research, teaching development and student outcomes, and has potential application world-wide.

An evaluation of the culture performance of two Macrobrachium species in Vanuatu: the exotic M. rosenbergii and the indigenous M. lar

Fisheries and aquaculture are important for food security, income generation and are critical to long term sustainability of many countries. Freshwater prawns have been harvested in the streams and creeks in Vanuatu, however due to over-exploitation catches have declined in recent years. To satisfy high demand for this product, Vanuatu government intends to establish economically viable small-scale aquaculture industries. The current project showed that wild Macrobrachium lar in Vanuatu constitute a single population for management purposes and that M. rosenbergii grows much faster than M. lar in simple pond grow-out systems, hence is a better species for culture in Vanuatu.

Cents and Sustainability : Securing Our Common Future by Decoupling Economic Growth from Environmental Pressures

The focus of Cents and Sustainability is to respond to the call by Dr Gro Brundtland in the seminal book Our Common Future to achieve, 'a new era of economic growth - growth that is forceful and at the same time socially and environmentally sustainable'. With the 20th anniversary of Our Common Future in 2007, it is clearly time to re-examine this important work in a modern global context. Using the framework of ‘Decoupling Economic Growth from Environmental Pressures’, Cents and Sustainability investigates a range of new evidence and research in order to develop a deeper understanding of how, and under what conditions, this 'forceful sustainable growth' is possible. With an introduction by Dr Jim MacNeill (former Secretary General to the Brundtland Commission, and former Director, OECD Environment Directorate 1978 -1984), the book will carry forewords from Dr Gro Brundtland (former Chair of the World Commission on Environment and Development), Dr Rajendra Pachauri (Chief, Intergovernmental Panel on Climate Change (IPCC), and joint recipient of the 2007 Nobel Peace Prize on behalf of the IPCC), and Dr Kenneth Ruffing (former Deputy Director and Chief Economist of the OECD Environment Directorate 2000 - 2005). Beginning with a detailed explanation of decoupling theory, along with investigation into a range of issues and barriers to its achievement, the book then focuses on informing national strategies for decoupling. Then putting this into action the book focuses on five key areas of decoupling, namely greenhouse gas emissions, biodiversity, freshwater extraction, waste production, and air pollution, and in each case showing compelling evidence for significant cost effective reductions in environmental pressures. The book concludes with a detailed case study of the groundbreaking application of public interest litigation to combat air pollution in Delhi, India.

Managing cost implications for highway infrastructure sustainability

Highway construction works have significant bearings on all aspects of sustainability. With the increasing level of public awareness and government regulatory measures, the construction industry is experiencing a cultural shift to recognise, embrace and pursue sustainability. Stakeholders are now keen to identify sustainable alternatives and the financial implications of including them on a lifecycle basis. They need tools that can aid the evaluation of investment options. To date, however, there have not been many financial assessments on the sustainability aspects of highway projects. This is because the existing life-cycle costing analysis (LCCA) models tend to focus on economic issues alone and are not able to deal with sustainability factors. This paper provides insights into the current practice of life-cycle cost analysis, and the identification and quantification of sustainability-related cost components in highway projects through literature review, questionnaire surveys and semi-structured interviews. The results can serve as a platform for highway project stakeholders to develop practical tools to evaluate highway investment decisions and reach an optimum balance between financial viability and sustainability deliverables.

An approach for adaptive and integrated environmental planning to deal with uncertainty in a Great Barrier Reef Catchment

This paper presents the Mossman Mill District Practices Framework. It was developed in the Wet Tropics region within the Great Barrier Reef in north-eastern Australia to describe the environmental benefits of agricultural management practices for the sugar cane industry. The framework translates complex, unclear and overlapping environmental plans, policy and legal arrangements into a simple framework of management practices that landholders can use to improve their management actions. Practices range from those that are old or outdated through to aspirational practices that have the potential to achieve desired resource condition targets. The framework has been applied by stakeholders at multiple scales to better coordinate and integrate a range of policy arrangements to improve natural resource management. It has been used to structure monitoring and evaluation in order to underpin a more adaptive approach to planning at mill district and property scale. Potentially, the framework and approach can be applied across fields of planning where adaptive management is needed. It has the potential to overcome many of the criticisms of property-scale and regional Natural Resource Management.

How public owners communicate the sustainability requirements of green design-build projects

The design-build (DB) system is regarded as an effective means of delivering sustainable buildings. Specifying clear sustainability requirements to potential contractors is of great importance to project success. This research investigates the current state-of-the-practice for the definition of sustainability requirements within the public sectors of the U.S. construction market using a robust content analysis of 49 DB requests for proposals (RFPs). The results reveal that owners predominantly communicate their desired level of sustainability through the LEED certification system. The sustainability requirement has become an important dimension for the best-value evaluation of DB contractors with specific importance weightings of up to 25%. Additionally, owners of larger projects and who provide less design information in their RFPs generally allocate significantly higher importance weightings to sustainability requirements. The primary knowledge contribution of this study to the construction industry is the reveal of current trend in DB procurement for green projects. The findings also provide owners, architects, engineers, and constructors with an effective means of communicating sustainability objectives in solicitation documents.

Applying threshold learning theory to teach sustainable business practice in post-graduate engineering education

This paper presents the results of a qualitative action-research inquiry into how a highly diverse cohort of post-graduate students could develop significant capacity in sustainable development within a single unit (course), in this case a compulsory component of four built environment masters programs. The method comprised applying threshold learning theory within the technical discipline of sustainable development, to transform student understanding of sustainable business practice in the built environment. This involved identifying a number of key threshold concepts, which once learned would provide a pathway to having a transformational learning experience. Curriculum was then revised, to focus on stepping through these targeted concepts using a scaffolded, problem-based-learning approach. Challenges included a large class size of 120 students, a majority of international students, and a wide span of disciplinary backgrounds across the spectrum of built environment professionals. Five ‘key’ threshold learning concepts were identified and the renewed curriculum was piloted in Semester 2 of 2011. The paper presents details of the study and findings from a mixed-method evaluation approach through the semester. The outcomes of this study will be used to inform further review of the course in 2012, including further consideration of the threshold concepts. In future, it is anticipated that this case study will inform a framework for rapidly embedding sustainability within curriculum.