Evaluation methodology for work integrated learning - placements : a discussion paper

Background: This paper will address the evaluation of WIL (placements) in order to provide a strategy to improve performance in universities' WIL as benchmarked in the AUSSE, GDS and placement unit feedback.
Overview of issue: Although WIL placements are important and valuable for student engagement, learning, graduate employability and industry partnerships, there are few empirical studies or reviews that inform evaluation methodology for them. The assessment of placement outcomes and the student experience is typically more complex than evaluation of a standard university unit because of the wide variation that occurs with placements. Students are likely to be working in different organisations, and working on different projects within their various disciplines. Adding to this complexity, the organisation supervisor is an additional stakeholder critical to the placement experience, and who ultimately makes the judgment of student performance. Although an organisation supervisor may complete an individual feedback form for their students, generally this information is not strategically aligned within a broader university evaluation process. Initial examination of available literature revealed that the multi-dimensional perspective (organisational supervisor, placement co-ordinator and student) is not usually incorporated into evaluation of placements to inform continuous improvement for example. There are gaps in the evaluation process which could be addressed through more  comprehensive evaluation that could be utilised across Faculties and Institutions. In 2010, we will conduct an inter-faculty project to develop and trial an evaluation methodology for WIL placements. It will incorporate a triangulated approach including student, organisation and university supervisor feedback. Preliminary results gained from this project will be discussed.
Argument: An evaluation methodology that is inclusive of a triangulated approach, would provide university stakeholders with comprehensive feedback that could be used to strategically inform continuous improvement efforts in Institutions in the areas of WIL placements.
Implications: The importance of triangulated data gathering leading to a comprehensive evaluation and subsequent improvement strategies will be concluded. Suggested evaluation process and potential tools will be presented.

Some reflections on Pawson’s science of evaluation: A realist manifesto

This article examines the evolution of Ray Pawson’s realist theory of evaluation, with a particular focus on his most recent book, Science of Evaluation: A Realist Manifesto. It is not a substitute for reading the original text. Reviews always say less about a book than the book in question. The goal is to provide a broader context for interpretation and an invitation to consider critically the practical import of Pawson’s grand ambitions for a new evaluation science. Like previous writings, this latest call-to-arms will appeal to some quarters of the evaluation community and dismay others. Regardless, evaluators should not remain indifferent. Pawson presents a vigorous, if at times irreverent, argument for advancing evaluation as a realist enterprise. Science of Evaluation deserves to be read and discussed widely.

A virtuous circle: the positive evaluation phenomenon in arts audience research

Arts impact evaluation is the subject of widespread criticism, ranging from a detailed critique of methodology to a wholesale rejection of the very purpose of the endeavour. In particular, it is often identified that audience evaluations are almost always positive. Yet whatever the critique, arts impact evaluation is becoming more and not less prevalent as a condition of public and philanthropic funding. This article explores both the methodologicaland conceptual problems that contribute to the perceived positive characterof social impact research, in two parts: (1) an investigation of the critical literature on audience evaluation, particularly in relation to the argument that evaluation is inevitably positive and as such leads to a confusion between the goals of evaluation and arts advocacy; (2) a reflection on our practices as audience researchers in the performing arts and the practical factors that contribute to the dominance of the positive in audience evaluation.