Future directions in course quality assurance

The Course Quality Assurance System at Queensland University of Technology (QUT) has as its centrepiece an exemplar of data visualisation known as the Individual Course Report (ICR). This report provides every course coordinator with an annual snapshot of their performance data evaluated against QUT and national benchmarks. In this article, the impact of the ICR is explored through the case study of one undergraduate course identified as underperforming. The case study features an innovative, ethnographic approach to working with course teams and highlights the importance of context, collaboration and appropriate support in creating evidence-based action plans for course improvement.

The Mirror Room Project: A Critical Ethnographic Program Evaluation of a Teaching Personal and Social Responsibility Based Youth Development Program

Amongst a host of other benefits, proper physical education has the possibility to create a safe place where responsibility can be transferred from the teacher/facilitator, to the student. This is especially true with an underserved population. This critical program evaluation of the program CHARM was done for the purpose of program improvement. This research was a place for participants to share their experiences of the program. The participants were 5 underserved youth, 5 undergraduate students, 3 teachers and 1 graduate student. Observations, interviews, and document analysis were used to gather data. Data was analyzed using a first level read-through, and two second-level analyses. Summaries were written, and cross-case analyses were completed. The main finding of the research was the development of a Handbook, which is a guide to running the program. Secondary findings include issues of program structure, goal setting, meaningful relationships, roles, SNAP, and an outlier in the data.

“Filling out the forms was a nightmare” : project evaluation and the reflective practitioner in community theatre in contemporary Northern Ireland

Since the Good Friday Agreement of 1998, large sums have been invested in community theatre projects in Northern Ireland, in the interests of conflict transformation and peace building. While this injection of funds has resulted in an unprecedented level of applied theatre activity, opportunities to maximise learning from this activity are being missed. It is generally assumed that project evaluation is undertaken at least partly to assess the degree of success of projects against important social objectives, with a view to learning what works, what does not, and what might work in the future. However, three ethnographic case studies of organisations delivering applied theatre projects in Northern Ireland indicate that current processes used to evaluate such projects are both flawed and inadequate for this purpose. Practitioners report that the administrative work involved in applying for and justifying funding is onerous, burdensome, and occurs at the expense of artistic activity. This is a very real concern when the time and effort devoted to ‘filling out the forms’ does not ultimately result in useful evaluative information. There are strong disincentives for organisations to report honestly on their experiences of difficulties, or undesirable impacts of projects, and this problem is not transcended by the use of external evaluators. Current evaluation processes provide little opportunity to capture unexpected benefits of projects, and small but significant successes which occur in the context of over-ambitious objectives. Little or no attempt is made to assess long-term impacts of projects on communities. Finally, official evaluation mechanisms fail to capture the reflective practice and dialogic analysis of practitioners, which would richly inform future projects. The authors argue that there is a need for clearer lines of communication, and more opportunities for mutual learning, among stakeholders involved in community development. In particular, greater involvement of the higher education sector in partnership with government and non-government agencies could yield significant benefits in terms of optimizing learning from applied theatre project evaluations.

Critical reflections on the use of participatory methodologies to build evaluation capacities in international development organisations

There has been a greater focus on strengthening evaluation capacity building (ECB) within development organisations in recent years. This can be attributed in part to the growing appreciation of the value of participatory and collaborative forms of evaluation. Evaluation is increasingly seen as an ongoing learning process and an important means of strengthening capacity and improving organisational performance (Horton et al., 2003:7). While there are many benefits of using participatory methodologies in ECB projects, our experiences and a review of the literature in this area highlight the many challenges, issues and contradictions that can affect the success of such ECB efforts. We discuss these issues, drawing on our learnings from the ongoing participatory action research (PAR) project 'Assessing Communication for Social Change’ (AC4SC). This four year project, which began in 2007, is a collaboration between communication and development academics and evaluation specialists from two Australian universities and communication for development practitioners and monitoring and evaluation (M&E) staff in the NGO Equal Access Nepal (EAN). The aim is to develop, implement, and evaluate a participatory methodology for assessing the social change impacts of community radio programs produced by EAN. It builds on previous projects that used ethnographic action research (EAR) methodology (Tacchi et al., 2007).

Ethnographic methodologies for construction research: knowing, practice and interventions

Ethnographic methodologies developed in social anthropology and sociology hold considerable promise for addressing practical, problem-based research concerned with the construction site. The extended researcher-engagement characteristic of ethnography reveals rich insights, yet is infrequently used to understand how workplace realities are lived out on construction sites. Moreover, studies that do employ these methods are rarely reported within construction research journals. This paper argues that recent innovations in ethnographic methodologies offer new routes to: posing questions; understanding workplace socialities (i.e. the qualities of the social relationships that develop on construction sites); learning about forms, uses and communication of knowledge on construction sites; and turning these into meaningful recommendations. This argument is supported by examples from an interdisciplinary ethnography concerning migrant workers and communications on UK construction sites. The presented research seeks to understand how construction workers communicate with managers and each other and how they stay safe on site, with the objective of informing site health-and-safety strategies and the production and evaluation of training and other materials.

An ethnographic investigation into gender and language in the Northern Ireland Assembly

This paper investigates the extent to which the negative evaluation of one of the women Ministers in the Northern Ireland Assembly can be attributed to gender. Interviews with politicians as well as the Minister herself illuminate this discussion by identifying the ‘gendered discourses’ that are drawn upon when describing the Minister’s communicative style in debates. Close analyses of transcripts of debates offer a description of some elements of this style, and find that while the Minister is confrontational in debates and ‘stands her ground’, she does not take part in illegal interventions that disrupt the debate floor and are characteristic of the Assembly as a whole. Although the construction of the Minister’s unpopularity can be attributed to a complex interplay of factors, it can be concluded that it is partly the way she draws on gendered linguistic resources that leads her to be negatively judged by her peers.