Developing a participatory impact assessment approach and action research culture within a communication for social change organisation in Nepal

Developing an effective impact evaluation framework, managing and conducting rigorous impact evaluations, and developing a strong research and evaluation culture within development communication organisations presents many challenges. This is especially so when both the community and organisational context is continually changing and the outcomes of programs are complex and difficult to clearly identify.----- This paper presents a case study from a research project being conducted from 2007-2010 that aims to address these challenges and issues, entitled Assessing Communication for Social Change: A New Agenda in Impact Assessment. Building on previous development communication projects which used ethnographic action research, this project is developing, trailing and rigorously evaluating a participatory impact assessment methodology for assessing the social change impacts of community radio programs in Nepal. This project is a collaboration between Equal Access – Nepal (EAN), Equal Access – International, local stakeholders and listeners, a network of trained community researchers, and a research team from two Australian universities. A key element of the project is the establishment of an organisational culture within EAN that values and supports the impact assessment process being developed, which is based on continuous action learning and improvement. The paper describes the situation related to monitoring and evaluation (M&E) and impact assessment before the project began, in which EAN was often reliant on time-bound studies and ‘success stories’ derived from listener letters and feedback. We then outline the various strategies used in an effort to develop stronger and more effective impact assessment and M&E systems, and the gradual changes that have occurred to date. These changes include a greater understanding of the value of adopting a participatory, holistic, evidence-based approach to impact assessment. We also critically review the many challenges experienced in this process, including:----- • Tension between the pressure from donors to ‘prove’ impacts and the adoption of a bottom-up, participatory approach based on ‘improving’ programs in ways that meet community needs and aspirations.----- • Resistance from the content teams to changing their existing M&E practices and to the perceived complexity of the approach.----- • Lack of meaningful connection between the M&E and content teams.----- • Human resource problems and lack of capacity in analysing qualitative data and reporting results.----- • The contextual challenges, including extreme poverty, wide cultural and linguistic diversity, poor transport and communications infrastructure, and political instability.----- • A general lack of acceptance of the importance of evaluation within Nepal due to accepting everything as fate or ‘natural’ rather than requiring investigation into a problem.

New horizons, new homes

Working with 12 journalism students plus a research assistant, producer/director Romano conducted five community focus groups and discussions with 80 people on the street. These provided the themes and concepts and the creative approaches for each program. Each was structured around one of the emergent themes; all programs offered different voices rather than coming to a single conclusion. New Horizons, New Homes aired over three weeks n Radio 4EB and was entered into the 2005 UN Media Peace Award where it won the Best Radio Category ahead of ABC and SBS. The UN commended the way in which the programs brought together a wide base of research to create a better understanding in the community on this issue. This project did not just improve the accuracy and social inclusiveness of reporting. It applied principles of deliberative democracy in the creation of journalism that enhances citizens’ deliberative potential on complex social issues

Critical reflections on the use of participatory methodologies to build evaluation capacities in international development organisations

There has been a greater focus on strengthening evaluation capacity building (ECB) within development organisations in recent years. This can be attributed in part to the growing appreciation of the value of participatory and collaborative forms of evaluation. Evaluation is increasingly seen as an ongoing learning process and an important means of strengthening capacity and improving organisational performance (Horton et al., 2003:7). While there are many benefits of using participatory methodologies in ECB projects, our experiences and a review of the literature in this area highlight the many challenges, issues and contradictions that can affect the success of such ECB efforts. We discuss these issues, drawing on our learnings from the ongoing participatory action research (PAR) project 'Assessing Communication for Social Change’ (AC4SC). This four year project, which began in 2007, is a collaboration between communication and development academics and evaluation specialists from two Australian universities and communication for development practitioners and monitoring and evaluation (M&E) staff in the NGO Equal Access Nepal (EAN). The aim is to develop, implement, and evaluate a participatory methodology for assessing the social change impacts of community radio programs produced by EAN. It builds on previous projects that used ethnographic action research (EAR) methodology (Tacchi et al., 2007).

Commercial vehicle pre-clearance programs : current issues and recommendations for potential implementation

The regulation of overweight trucks is of increasing importance. Quickly growing heavy vehicle volumes over-proportionally contribute to roadway damage. Raising maintenance costs and compromised road safety are also becoming a major concern to managing agencies. Minimizing pavement wear is done by regulating overloaded trucks on major highways at weigh stations. However, due to lengthy inspections and insufficient capacities, weigh stations tend to be inefficient. New practices, using Radio Frequency Identification (RFID) transponders and weigh-in-motion technologies, called preclearance programs, have been set up in a number of countries. The primary aim of this study is to investigate the current issues with regard to the implementation and operation of the preclearance program. The State of Queensland, Australia, is used as a case study. The investigation focuses on three aspects; the first emphasizes on identifying the need for improvement of the current regulation programs in Queensland. Second, the operators of existing preclearance programs are interviewed for their lessons-learned and the marketing strategies used for promoting their programs. The trucking companies in Queensland are interviewed for their experiences with the current weighing practices and attitudes toward the potential preclearance system. Finally, the estimated benefit of the preclearance program deployment in Queensland is analyzed. The penultimate part brings the former four parts together and provides the study findings and recommendations. The framework and study findings could be valuable inputs for other roadway agencies considering a similar preclearance program or looking to promote their existing ones.