Developing a participatory impact assessment approach and action research culture within a communication for social change organisation in Nepal

Developing an effective impact evaluation framework, managing and conducting rigorous impact evaluations, and developing a strong research and evaluation culture within development communication organisations presents many challenges. This is especially so when both the community and organisational context is continually changing and the outcomes of programs are complex and difficult to clearly identify.----- This paper presents a case study from a research project being conducted from 2007-2010 that aims to address these challenges and issues, entitled Assessing Communication for Social Change: A New Agenda in Impact Assessment. Building on previous development communication projects which used ethnographic action research, this project is developing, trailing and rigorously evaluating a participatory impact assessment methodology for assessing the social change impacts of community radio programs in Nepal. This project is a collaboration between Equal Access – Nepal (EAN), Equal Access – International, local stakeholders and listeners, a network of trained community researchers, and a research team from two Australian universities. A key element of the project is the establishment of an organisational culture within EAN that values and supports the impact assessment process being developed, which is based on continuous action learning and improvement. The paper describes the situation related to monitoring and evaluation (M&E) and impact assessment before the project began, in which EAN was often reliant on time-bound studies and ‘success stories’ derived from listener letters and feedback. We then outline the various strategies used in an effort to develop stronger and more effective impact assessment and M&E systems, and the gradual changes that have occurred to date. These changes include a greater understanding of the value of adopting a participatory, holistic, evidence-based approach to impact assessment. We also critically review the many challenges experienced in this process, including:----- • Tension between the pressure from donors to ‘prove’ impacts and the adoption of a bottom-up, participatory approach based on ‘improving’ programs in ways that meet community needs and aspirations.----- • Resistance from the content teams to changing their existing M&E practices and to the perceived complexity of the approach.----- • Lack of meaningful connection between the M&E and content teams.----- • Human resource problems and lack of capacity in analysing qualitative data and reporting results.----- • The contextual challenges, including extreme poverty, wide cultural and linguistic diversity, poor transport and communications infrastructure, and political instability.----- • A general lack of acceptance of the importance of evaluation within Nepal due to accepting everything as fate or ‘natural’ rather than requiring investigation into a problem.

Finding order in chaos : a problem based collaborative assignment for 1300 students across 19 disciplines

The use of collaborative assignments for assessment is a risky undertaking for students and course designers. Yet the benefits, in terms of core learning outcomes, competencies, collaborative sense making and student involvement, suggest that the effort is worthwhile. Formal descriptions and rules do little to ameliorate the perception of risk and increased anxiety by students. (Ryan, 2007). BEB100 Introducing Professional Learning is a faculty-wide foundation unit with over 1300 students from 19 disciplines across the Faculty of the Built Environment and Engineering (“BEE”) at the Queensland University of Technology (“QUT”), Brisbane, Australia. Finding order in chaos outlines the approach and justification, assessment criteria, learning resources, teamwork tools, tutorial management, communication strategies, 2007-09 Student Learning Experience Survey results, annual improvements, findings and outcomes.

Research Monograph: Building Online Participatory Systems: Towards the Community Based Interactive Environmental Decision Support Systems

The importance of broadening community participation in environmental decision-making is widely recognized and lack of participation in this process appears to be a perennial problem. In this context, there have been calls from some academics for the more extensive use of geographic information systems (GIS) and distance learning technologies, accessible via the Internet, as a possible means to inform and empower communities. However, a number of problems exist. For instance, at present the scope for online interaction between policy-makers and citizens is currently limited. Contemporary web-based environmental information systems suffer from this lack of interactivity on the one hand and on the other hand from the apparent complexity for the lay user. This paper explores the issue of online community participation at the local level and attempts to construct a framework for a new (and potentially more effective) model of online participatory decision-making. The key components, system architecture and stages of such a model are introduced. This model, referred to as a ‘Community Based Interactive Environmental Decision Support System’, incorporates advanced information technologies, distance learning and community involvement tools which will be applied and evaluated in the field through a pilot project in Tokyo in the summer of 2002.

Low-complexity estimation of CFO and frequency independent I/Q mismatch for OFDM systems

CFO and I/Q mismatch could cause significant performance degradation to OFDM systems. Their estimation and compensation are generally difficult as they are entangled in the received signal. In this paper, we propose some low-complexity estimation and compensation schemes in the receiver, which are robust to various CFO and I/Q mismatch values although the performance is slightly degraded for very small CFO. These schemes consist of three steps: forming a cosine estimator free of I/Q mismatch interference, estimating I/Q mismatch using the estimated cosine value, and forming a sine estimator using samples after I/Q mismatch compensation. These estimators are based on the perception that an estimate of cosine serves much better as the basis for I/Q mismatch estimation than the estimate of CFO derived from the cosine function. Simulation results show that the proposed schemes can improve system performance significantly, and they are robust to CFO and I/Q mismatch.

Experience of problem-based learning (PBL) in virtual space : a phenomenographical study

This thesis reports the outcomes of an investigation into students’ experience of Problem-based learning (PBL) in virtual space. PBL is increasingly being used in many fields including engineering education. At the same time many engineering education providers are turning to online distance education. Unfortunately there is a dearth of research into what constitutes an effective learning experience for adult learners who undertake PBL instruction through online distance education. Research was therefore focussed on discovering the qualitatively different ways that students experience PBL in virtual space. Data was collected in an electronic environment from a course, which adopted the PBL strategy and was delivered entirely in virtual space. Students in this course were asked to respond to open-ended questions designed to elicit their learning experience in the course. Data was analysed using the phenomenographical approach. This interpretative research method concentrated on mapping the qualitative differences in students’ interpretations of their experience in the course. Five qualitatively different ways of experiencing were discovered: Conception 1: ‘A necessary evil for program progression’; Conception 2: ‘Developing skills to understand, evaluate, and solve technical Engineering and Surveying problems’; Conception 3: ‘Developing skills to work effectively in teams in virtual space’; Conception 4: ‘A unique approach to learning how to learn’; Conception 5: ‘Enhancing personal growth’. Each conception reveals variation in how students attend to learning by PBL in virtual space. Results indicate that the design of students’ online learning experience was responsible for making students aware of deeper ways of experiencing PBL in virtual space. Results also suggest that the quality and quantity of interaction with the team facilitator may have a significant impact on the student experience in virtual PBL courses. The outcomes imply pedagogical strategies can be devised for shifting students’ focus as they engage in the virtual PBL experience to effectively manage the student learning experience and thereby ensure that they gain maximum benefit. The results from this research hold important ramifications for graduates with respect to their ease of transition into professional work as well as their later professional competence in terms of problem solving, ability to transfer basic knowledge to real-life engineering scenarios, ability to adapt to changes and apply knowledge in unusual situations, ability to think critically and creatively, and a commitment to continuous life-long learning and self-improvement.

Nation & internation practices in decision support tools in road asset management

This document provides a review of international and national practices in investment decision support tools in road asset management. Efforts were concentrated on identifying analytic frameworks, evaluation methodologies and criteria adopted by current tools. Emphasis was also given to how current approaches support Triple Bottom Line decision-making. Benefit Cost Analysis and Multiple Criteria Analysis are principle methodologies in supporting decision-making in Road Asset Management. The complexity of the applications shows significant differences in international practices. There is continuing discussion amongst practitioners and researchers regarding to which one is more appropriate in supporting decision-making. It is suggested that the two approaches should be regarded as complementary instead of competitive means. Multiple Criteria Analysis may be particularly helpful in early stages of project development, say strategic planning. Benefit Cost Analysis is used most widely for project prioritisation and selecting the final project from amongst a set of alternatives. Benefit Cost Analysis approach is useful tool for investment decision-making from an economic perspective. An extension of the approach, which includes social and environmental externalities, is currently used in supporting Triple Bottom Line decision-making in the road sector. However, efforts should be given to several issues in the applications. First of all, there is a need to reach a degree of commonality on considering social and environmental externalities, which may be achieved by aggregating the best practices. At different decision-making level, the detail of consideration of the externalities should be different. It is intended to develop a generic framework to coordinate the range of existing practices. The standard framework will also be helpful in reducing double counting, which appears in some current practices. Cautions should also be given to the methods of determining the value of social and environmental externalities. A number of methods, such as market price, resource costs and Willingness to Pay, are found in the review. The use of unreasonable monetisation methods in some cases has discredited Benefit Cost Analysis in the eyes of decision makers and the public. Some social externalities, such as employment and regional economic impacts, are generally omitted in current practices. This is due to the lack of information and credible models. It may be appropriate to consider these externalities in qualitative forms in a Multiple Criteria Analysis. Consensus has been reached in considering noise and air pollution in international practices. However, Australia practices generally omitted these externalities. Equity is an important consideration in Road Asset Management. The considerations are either between regions, or social groups, such as income, age, gender, disable, etc. In current practice, there is not a well developed quantitative measure for equity issues. More research is needed to target this issue. Although Multiple Criteria Analysis has been used for decades, there is not a generally accepted framework in the choice of modelling methods and various externalities. The result is that different analysts are unlikely to reach consistent conclusions about a policy measure. In current practices, some favour using methods which are able to prioritise alternatives, such as Goal Programming, Goal Achievement Matrix, Analytic Hierarchy Process. The others just present various impacts to decision-makers to characterise the projects. Weighting and scoring system are critical in most Multiple Criteria Analysis. However, the processes of assessing weights and scores were criticised as highly arbitrary and subjective. It is essential that the process should be as transparent as possible. Obtaining weights and scores by consulting local communities is a common practice, but is likely to result in bias towards local interests. Interactive approach has the advantage in helping decision-makers elaborating their preferences. However, computation burden may result in lose of interests of decision-makers during the solution process of a large-scale problem, say a large state road network. Current practices tend to use cardinal or ordinal scales in measure in non-monetised externalities. Distorted valuations can occur where variables measured in physical units, are converted to scales. For example, decibels of noise converts to a scale of -4 to +4 with a linear transformation, the difference between 3 and 4 represents a far greater increase in discomfort to people than the increase from 0 to 1. It is suggested to assign different weights to individual score. Due to overlapped goals, the problem of double counting also appears in some of Multiple Criteria Analysis. The situation can be improved by carefully selecting and defining investment goals and criteria. Other issues, such as the treatment of time effect, incorporating risk and uncertainty, have been given scant attention in current practices. This report suggested establishing a common analytic framework to deal with these issues.

International practices in investment decision making in road sector

This document provides the findings of an international review of investment decision-making practices in road asset management. Efforts were concentrated on identifying the strategic objectives of agencies in road asset management, establishing and understanding criteria different organisations adopted and ascertaining the exact methodologies used by different countries and international organisations. Road assets are powerful drivers of economic development and social equity. They also have significant impacts on the natural and man-made environment. The traditional definition of asset management is “A systematic process of maintaining, upgrading and operating physical assets cost effectively. It combines engineering principles with sound business practices and economic theory and it provides tools to facilitate a more organised, logical approach to decision-making” (US Dept. of Transportation, 1999). In recent years, the concept has been broadened to cover the complexity of decision making, based on a wider variety of policy considerations as well as social and environmental issues rather than is covered by Benefit-Cost analysis and pure technical considerations. Current international practices are summarised in table 2. It was evident that Engineering-economic analysis methods are well advanced to support decision-making. A range of tools available supports performance predicting of road assets and associated cost/benefit in technical context. The need for considering triple plus one bottom line of social, environmental and economic as well as political factors in decision-making is well understood by road agencies around the world. The techniques used to incorporate these however, are limited. Most countries adopt a scoring method, a goal achievement matrix or information collected from surveys. The greater uncertainty associated with these non-quantitative factors has generally not been taken into consideration. There is a gap between the capacities of the decision-making support systems and the requirements from decision-makers to make more rational and transparent decisions. The challenges faced in developing an integrated decision making framework are both procedural and conceptual. In operational terms, the framework should be easy to be understood and employed. In philosophical terms, the framework should be able to deal with challenging issues, such as uncertainty, time frame, network effects, model changes, while integrating cost and non-cost values into the evaluation. The choice of evaluation techniques depends on the feature of the problem at hand, on the aims of the analysis, and on the underlying information base At different management levels, the complexity in considering social, environmental, economic and political factor in decision-making is different. At higher the strategic planning level, more non-cost factors are involved. The complexity also varies based on the scope of the investment proposals. Road agencies traditionally place less emphasis on evaluation of maintenance works. In some cases, social equity, safety, environmental issues have been used in maintenance project selection. However, there is not a common base for the applications.

How environmental and organizational complexity affects opportunity recognition in development projects

New product development projects are experiencing increasing internal and external project complexity. Complexity leadership theory proposes that external complexity requires adaptive and enabling leadership, which facilitates opportunity recognition (OR). We ask whether internal complexity also requires OR for increased adaptability. We extend a model of EO and OR to conclude that internal complexity may require more careful OR. This means that leaders of technically or structurally complex projects need to evaluate opportunities more carefully than those in projects with external or technological complexity.