Reframe : QUT's Evaluation Framework 2014 Communications Plan

2014 is the year for embedding the Reframe: QUT’s Evaluation Framework as core business within the University, with the following providing an outline of the Learning and Teaching Unit’s (LTU) planned activities to support this implementation. LTU is continuing its commitment to intensive support and engagement with the academic and teaching focused community within QUT. The Academic Quality and Standards team have extended the existing communication and dissemination activities into a comprehensive communication plan for 2014, with major initiatives defined within this document. It should be noted that these activities are aligned to the discussions from the 2013 Integrated Management of Feedback (IMF) Steering Group and as endorsed by the University Learning and Teaching Committee at their meeting in February, 2014.

Reframe QUT's evaluation framework : a case study of planning, policy and positioning leading to educational best practice

Every university in Australia has a set of policies that guide the institution in its educational practices, however, the policies are often developed in isolation to each other. Now imagine a space where policies are evidence-based, refined annually, cohesively interrelated, and meet stakeholders’ needs. Is this happenstance or the result of good planning? Culturally, Queensland University of Technology (QUT) is a risk-averse institution that takes pride in its financial solvency and is always keen to know “how are we going?” With a twenty-year history of annual reporting that assures the quality of course performance through multiple lines of evidence, QUT’s Learning and Teaching Unit went one step further and strategically aligned a suite of policies that take into consideration the needs of their stakeholders, collaborate with other areas across the institution and use multiple lines of evidence to inform curriculum decision-making. In QUT’s experience, strategic planning can lead to policy that is designed to meet stakeholders’ needs, not manage them; where decision-making is supported by evidence, not rhetoric; where all feedback is incorporated, not ignored; and where policies are cohesively interrelated, not isolated. While many may call this ‘policy nirvana’, QUT has positioned itself to demonstrate good educational practice through Reframe, its evaluation framework. In this case, best practice was achieved through the application of a theory of change and a design-led logic model that allows for transition to other institutions with different cultural specificity. The evaluation approach follows Seldin’s (2003) notion to offer depth and breadth to the evaluation framework along with Berk’s (2005) concept of multiple lines of evidence. In summary, this paper offers university executives, academics, planning and quality staff an opportunity to understand the critical steps that lead to strategic planning and design of evidence-based educational policy that positions a university for best practice in learning and teaching.

Evaluation framework to assess benefits and harms of bone-anchored prosthesis

Bone-anchored prostheses are increasingly acknowledged as viable alternative method of attachment of artificial limb compared to socket-suspended prostheses. To date, a few osseointegration fixations are commercially available. Several devices are at different stages of development particularly in Europe and the US. Clearly, the current momentum experienced worldwide is creating a need for a standardized evaluation framework to assess the benefits and safety of each procedure.

Outcome evaluation framework for the Queensland Alcohol Ignition Interlock Program. Report to Queensland Department of Transport and Main Roads (TMR 3814)

The Queensland Department of Transport and Main Roads (TMR) required an evaluation framework for the Queensland Alcohol Ignition Interlock Program (AIIP). The objective of this project was to develop a framework to evaluate the AIIP in terms of its effect on road safety outcomes.

Evaluation framework to assess orthopaedic procedures

Introduction & aims The demand for evidence of efficacy of treatments in general and orthopaedic surgical procedures in particular is ever increasing in Australia and worldwide. The aim of this study is to share the key elements of an evaluation framework recently implemented in Australia to determine the efficacy of bone-anchored prostheses. Method The proposed evaluation framework to determine the benefit and harms of bone-anchored prostheses for individuals with limb loss was extracted from a systematic review of the literature including seminal studies focusing on clinical benefits and safety of procedures involving screw-type implant (e.g., OPRA) and press-fit fixations (e.g., EEFT, ILP, OPL). [1-64] Results The literature review highlighted that a standard and replicable evaluation framework should focus on: • The clinical benefits with a systematic recording of health-related quality of life (e.g., SF-26, Q-TFA), mobility predictor (e.g., AMPRO), ambulation abilities (e.g., TUG, 6MWT), walking abilities (e.g., characteristic spatio-temporal) and actual activity level at baseline and follow-up post Stage 2 surgery, • The potential harms with systematic recording of residuum care, infection, implant stability, implant integrity, injuries (e.g., falls) after Stage 1 surgery. There was a general consensus around the instruments to monitor most of the benefits and harms. The benefits could be assessed using a wide spectrum of complementary assessments ranging from subjective patient self-reporting to objective measurements of physical activity. However, this latter was assessed using a broad range of measurements (e.g., pedometer, load cell, energy consumption). More importantly, the lack of consistent grading of infections was sufficiently noticeable to impede cross-fixation comparisons. Clearly, a more universal grading system is needed. Conclusions Investigators are encouraged to implement an evaluation framework featuring the domains and instruments proposed above using a single database to facilitate robust prospective studies about potential benefits and harms of their procedure. This work is also a milestone in the development of national and international clinical outcome registries.

The integrated land use and transportation indexing model : assessing the sustainability of the Gold Coast, Australia

Urban sustainability and sustainable urban development concepts have been identified as the ultimate goal of many contemporary planning endeavours and have become central concepts on which the urban development policies are formulated. In the confinement of these concepts, land use and transport integration has been highlighted as one of the most important policy objectives considering the interrelationship between them and available intervention means of planning. While its interpretation varies, in Australia, it has been embraced as integration of land use and transport planning/policies and been an integral part of regional and local plans. Accordingly, a number of principles have been defined to guide its implementation, to name a few, planning for compact and connected urban development, encouraging active transport modes, creation of mixed-use activity centres and public transport precincts, provision of high quality public transport services, and enhancing character and amenity of urban areas. However, there is lack of an evaluation framework to measure the extent of achievement of implementation of these principles. In pursuit of filling this gap, this study aims to devise an evaluation framework to measure the performance of urban settings according to the integration principles in South East Queensland, Australia context and to demarcate problematic areas which can be intervened by planning tools...

Input-orientated data envelopment analysis framework for measuring and decomposing economic, environmental and ecological efficiency : an application to OECD agriculture

This paper presents an input-orientated data envelopment analysis (DEA) framework which allows the measurement and decomposition of economic, environmental and ecological efficiency levels in agricultural production across different countries. Economic, environmental and ecological optimisations search for optimal input combinations that minimise total costs, total amount of nutrients, and total amount of cumulative exergy contained in inputs respectively. The application of the framework to an agricultural dataset of 30 OECD countries revealed that (i) there was significant scope to make their agricultural production systemsmore environmentally and ecologically sustainable; (ii) the improvement in the environmental and ecological sustainability could be achieved by being more technically efficient and, even more significantly, by changing the input combinations; (iii) the rankings of sustainability varied significantly across OECD countries within frontier-based environmental and ecological efficiency measures and between frontier-based measures and indicators.

Review: "Context : A framework for its Influence on Evaluation Practice (New Directions for Evaluation No. 135)" edited by Rog, D. J.; Fitzpatrick, J. L. and Conner, R. F.

This book offers a framework for the influence of context on evaluation practice and is applied to three case studies: environmental context; indigenous context and political context; and finishes with a process for implementation.

Kaleidoscope : Reframing learning and teaching evaluation through a shared stakeholder lens

Sector wide interest in Reframe: QUT’s Evaluation Framework continues with a number of institutions requesting finer details as QUT embeds the new approach to evaluation across the university in 2013. This interest, both nationally and internationally has warranted QUT’s collegial response to draw upon its experiences from developing Reframe into distilling and offering Kaleidoscope back to the sector. The word Reframe is a relevant reference for QUT’s specific re-evaluation, reframing and adoption of a new approach to evaluation; whereas Kaleidoscope reflects the unique lens through which any other institution will need to view their own cultural specificity and local context through an extensive user-led stakeholder engagement approach when introducing new approaches to learning and teaching evaluation. Kaleidoscope’s objectives are for QUT to develop its research-based stakeholder approach to distil the successful experience exhibited in the Reframe Project into a transferable set of guidelines for use by other tertiary institutions across the sector. These guidelines will assist others to design, develop, and deploy, their own culturally specific widespread organisational change informed by stakeholder engagement and organisational buy-in. It is intended that these guidelines will promote, support and enable other tertiary institutions to embark on their own evaluation projects and maximise impact. Kaleidoscope offers an institutional case study of widespread organisational change underpinned by Reframe’s (i) evidence-based methodology; (ii) research including published environmental scan, literature review (Alderman, et al., 2012), development of a conceptual model (Alderman, et al., in press 2013), project management principles (Alderman & Melanie, 2012) and national conference peer reviews; and (iii) year-long strategic project with national outreach to collaboratively engage the development of a draft set of National Guidelines. Kaleidoscope’s aims are to inform Higher Education evaluation policy development through national stakeholder engagement, the finalisation of proposed National Guidelines. In correlation with the conference paper, the authors will present a Draft Guidelines and Framework ready for external peer review by evaluation practitioners from the Higher Education sector, as part of Kaleidoscope’s dissemination strategy (Hinton & Gannaway, 2011) applying illuminative evaluation theory (Parlett & Hamilton, 1976), through conference workshops and ongoing discussions (Shapiro, et al., 1983; Jacobs, 2000). The initial National Guidelines will be distilled from the Reframe: QUT’s Evaluation Framework’s Policy, Protocols, and incorporated Business Rules. It is intended that the outcomes of Kaleidoscope are owned by and reflect sectoral engagement, including iterative evaluation through multiple avenues of dissemination and collaboration including the Higher Education sector. The dissemination strategy with the inclusion of Illuminative Evaluation methodology provides an inclusive opportunity for other institutions and stakeholders across the Higher Education sector to give voice through the information-gathering component of evaluating the draft Guidelines, providing a comprehensive understanding of the complex realities experienced across the Higher Education sector, and thereby ‘illuminating’ both the shared and unique lenses and contexts. This process will enable any final guidelines developed to have broader applicability, greater acceptance, enhanced sustainability and additional relevance benefiting the Higher Education sector, and the adoption and adaption by any single institution for their local contexts.

Sustainability outcomes of infrastructure sustainability rating schemes for road projects

The construction and operation of infrastructure assets can have significant impact on society and the region. Using a sustainability assessment framework can be an effective means to build sustainability aspects into the design, construction and operation of infrastructure assets. The conventional evaluation processes and procedures for infrastructure projects do not necessarily measure the qualitative/quantitative effectiveness of all aspects of sustainability: environment, social wellbeing and economy. As a result, a few infrastructure sustainability rating schemes have been developed with a view to assess the level of sustainability attained in the infrastructure projects. These include: Infrastructure Sustainability (Australia); CEEQUAL (UK); and Envision (USA). In addition, road sector specific sustainability rating schemes such as Greenroads (USA) and Invest (Australia) have also been developed. These schemes address several aspects of sustainability with varying emphasis (weightings) on areas such as: use of resources; emission, pollution and waste; ecology; people and place; management and governance; and innovation. The attainment of sustainability of an infrastructure project depends largely on addressing the whole-of-life environmental issues. This study has analysed the rating schemes’ coverage of different environmental components for the road infrastructure under the five phases of a project: material, construction, use, maintenance and end-of-life. This is based on a comprehensive life cycle assessment (LCA) system boundary. The findings indicate that there is a need for the schemes to consider key (high impact) life cycle environmental components such as traffic congestion during construction, rolling resistance due to surface roughness and structural stiffness of the pavement, albedo, lighting, and end-of-life management (recycling) to deliver sustainable road projects.

Developing a sustainability assessment model : the sustainable infrastructure, land-use, environment and transport model

Measuring the comparative sustainability levels of cities, regions, institutions and projects is an essential procedure in creating sustainable urban futures. This paper introduces a new urban sustainability assessment model: “The Sustainable Infrastructure, Land-use, Environment and Transport Model (SILENT)”. The SILENT Model is an advanced geographic information system and indicator-based comparative urban sustainability indexing model. The model aims to assist planners and policy makers in their daily tasks in sustainable urban planning and development by providing an integrated sustainability assessment framework. The paper gives an overview of the conceptual framework and components of the model and discusses the theoretical constructs, methodological procedures, and future development of this promising urban sustainability assessment model.

Sustainability : driver for decision making in infrastructure

With the growing importance of sustainability assessment in the construction industry, many green building rating schemes have been adopted in the building sector of Australia. However, there is an abnormal delay in the similar adoption in the infrastructure sector. This prolonged delay in practice poses a challenge in mapping the project objectives with sustainability outcomes. Responding to the challenge of sustainable development in infrastructure, it is critical to create a set of decision indicators for sustainability in infrastructure, which to be used in conjunction with the emerging infrastructure sustainability assessment framework of the Australian Green Infrastructure Council. The various literature sources confirm the lack of correlation between sustainability and infrastructure. This theoretical missing link signifies the crucial validation of the interrelationship and interdependency in sustainability, decision making and infrastructure. This validation is vital for the development of decision indicators for sustainability in infrastructure. Admittedly, underpinned by the serious socio-environmental vulnerability, the traditional focus on economic emphasis in infrastructure development needs to be drifted towards the appropriate decisions for sustainability enhancing the positive social and environmental outcomes. Moreover, the research findings suggest sustainability being observed as powerful socio-political and influential socio-environmental driver in deciding the infrastructure needs and its development. These newly developed sustainability decision indicators create the impetus for change leading to sustainability in infrastructure by integrating the societal cares, environmental concerns into the holistic financial consideration. Radically, this development seeks to transform principles into actions for infrastructure sustainability. Lastly, the thesis concludes with knowledge contribution in five significant areas and future research opportunities. The consolidated research outcomes suggest that the development of decision indicators has demonstrated sustainability as a pivotal driver for decision making in infrastructure.

Evaluation of texture and geometry for dimensional facial expression recognition

Facial expression recognition (FER) algorithms mainly focus on classification into a small discrete set of emotions or representation of emotions using facial action units (AUs). Dimensional representation of emotions as continuous values in an arousal-valence space is relatively less investigated. It is not fully known whether fusion of geometric and texture features will result in better dimensional representation of spontaneous emotions. Moreover, the performance of many previously proposed approaches to dimensional representation has not been evaluated thoroughly on publicly available databases. To address these limitations, this paper presents an evaluation framework for dimensional representation of spontaneous facial expressions using texture and geometric features. SIFT, Gabor and LBP features are extracted around facial fiducial points and fused with FAP distance features. The CFS algorithm is adopted for discriminative texture feature selection. Experimental results evaluated on the publicly accessible NVIE database demonstrate that fusion of texture and geometry does not lead to a much better performance than using texture alone, but does result in a significant performance improvement over geometry alone. LBP features perform the best when fused with geometric features. Distributions of arousal and valence for different emotions obtained via the feature extraction process are compared with those obtained from subjective ground truth values assigned by viewers. Predicted valence is found to have a more similar distribution to ground truth than arousal in terms of covariance or Bhattacharya distance, but it shows a greater distance between the means.