Developing a participatory impact assessment approach and action research culture within a communication for social change organisation in Nepal

Developing an effective impact evaluation framework, managing and conducting rigorous impact evaluations, and developing a strong research and evaluation culture within development communication organisations presents many challenges. This is especially so when both the community and organisational context is continually changing and the outcomes of programs are complex and difficult to clearly identify.----- This paper presents a case study from a research project being conducted from 2007-2010 that aims to address these challenges and issues, entitled Assessing Communication for Social Change: A New Agenda in Impact Assessment. Building on previous development communication projects which used ethnographic action research, this project is developing, trailing and rigorously evaluating a participatory impact assessment methodology for assessing the social change impacts of community radio programs in Nepal. This project is a collaboration between Equal Access – Nepal (EAN), Equal Access – International, local stakeholders and listeners, a network of trained community researchers, and a research team from two Australian universities. A key element of the project is the establishment of an organisational culture within EAN that values and supports the impact assessment process being developed, which is based on continuous action learning and improvement. The paper describes the situation related to monitoring and evaluation (M&E) and impact assessment before the project began, in which EAN was often reliant on time-bound studies and ‘success stories’ derived from listener letters and feedback. We then outline the various strategies used in an effort to develop stronger and more effective impact assessment and M&E systems, and the gradual changes that have occurred to date. These changes include a greater understanding of the value of adopting a participatory, holistic, evidence-based approach to impact assessment. We also critically review the many challenges experienced in this process, including:----- • Tension between the pressure from donors to ‘prove’ impacts and the adoption of a bottom-up, participatory approach based on ‘improving’ programs in ways that meet community needs and aspirations.----- • Resistance from the content teams to changing their existing M&E practices and to the perceived complexity of the approach.----- • Lack of meaningful connection between the M&E and content teams.----- • Human resource problems and lack of capacity in analysing qualitative data and reporting results.----- • The contextual challenges, including extreme poverty, wide cultural and linguistic diversity, poor transport and communications infrastructure, and political instability.----- • A general lack of acceptance of the importance of evaluation within Nepal due to accepting everything as fate or ‘natural’ rather than requiring investigation into a problem.

Finding order in chaos : a problem based collaborative assignment for 1300 students across 19 disciplines

The use of collaborative assignments for assessment is a risky undertaking for students and course designers. Yet the benefits, in terms of core learning outcomes, competencies, collaborative sense making and student involvement, suggest that the effort is worthwhile. Formal descriptions and rules do little to ameliorate the perception of risk and increased anxiety by students. (Ryan, 2007). BEB100 Introducing Professional Learning is a faculty-wide foundation unit with over 1300 students from 19 disciplines across the Faculty of the Built Environment and Engineering (“BEE”) at the Queensland University of Technology (“QUT”), Brisbane, Australia. Finding order in chaos outlines the approach and justification, assessment criteria, learning resources, teamwork tools, tutorial management, communication strategies, 2007-09 Student Learning Experience Survey results, annual improvements, findings and outcomes.

Experience of problem-based learning (PBL) in virtual space : a phenomenographical study

This thesis reports the outcomes of an investigation into students’ experience of Problem-based learning (PBL) in virtual space. PBL is increasingly being used in many fields including engineering education. At the same time many engineering education providers are turning to online distance education. Unfortunately there is a dearth of research into what constitutes an effective learning experience for adult learners who undertake PBL instruction through online distance education. Research was therefore focussed on discovering the qualitatively different ways that students experience PBL in virtual space. Data was collected in an electronic environment from a course, which adopted the PBL strategy and was delivered entirely in virtual space. Students in this course were asked to respond to open-ended questions designed to elicit their learning experience in the course. Data was analysed using the phenomenographical approach. This interpretative research method concentrated on mapping the qualitative differences in students’ interpretations of their experience in the course. Five qualitatively different ways of experiencing were discovered: Conception 1: ‘A necessary evil for program progression’; Conception 2: ‘Developing skills to understand, evaluate, and solve technical Engineering and Surveying problems’; Conception 3: ‘Developing skills to work effectively in teams in virtual space’; Conception 4: ‘A unique approach to learning how to learn’; Conception 5: ‘Enhancing personal growth’. Each conception reveals variation in how students attend to learning by PBL in virtual space. Results indicate that the design of students’ online learning experience was responsible for making students aware of deeper ways of experiencing PBL in virtual space. Results also suggest that the quality and quantity of interaction with the team facilitator may have a significant impact on the student experience in virtual PBL courses. The outcomes imply pedagogical strategies can be devised for shifting students’ focus as they engage in the virtual PBL experience to effectively manage the student learning experience and thereby ensure that they gain maximum benefit. The results from this research hold important ramifications for graduates with respect to their ease of transition into professional work as well as their later professional competence in terms of problem solving, ability to transfer basic knowledge to real-life engineering scenarios, ability to adapt to changes and apply knowledge in unusual situations, ability to think critically and creatively, and a commitment to continuous life-long learning and self-improvement.

Nation & internation practices in decision support tools in road asset management

This document provides a review of international and national practices in investment decision support tools in road asset management. Efforts were concentrated on identifying analytic frameworks, evaluation methodologies and criteria adopted by current tools. Emphasis was also given to how current approaches support Triple Bottom Line decision-making. Benefit Cost Analysis and Multiple Criteria Analysis are principle methodologies in supporting decision-making in Road Asset Management. The complexity of the applications shows significant differences in international practices. There is continuing discussion amongst practitioners and researchers regarding to which one is more appropriate in supporting decision-making. It is suggested that the two approaches should be regarded as complementary instead of competitive means. Multiple Criteria Analysis may be particularly helpful in early stages of project development, say strategic planning. Benefit Cost Analysis is used most widely for project prioritisation and selecting the final project from amongst a set of alternatives. Benefit Cost Analysis approach is useful tool for investment decision-making from an economic perspective. An extension of the approach, which includes social and environmental externalities, is currently used in supporting Triple Bottom Line decision-making in the road sector. However, efforts should be given to several issues in the applications. First of all, there is a need to reach a degree of commonality on considering social and environmental externalities, which may be achieved by aggregating the best practices. At different decision-making level, the detail of consideration of the externalities should be different. It is intended to develop a generic framework to coordinate the range of existing practices. The standard framework will also be helpful in reducing double counting, which appears in some current practices. Cautions should also be given to the methods of determining the value of social and environmental externalities. A number of methods, such as market price, resource costs and Willingness to Pay, are found in the review. The use of unreasonable monetisation methods in some cases has discredited Benefit Cost Analysis in the eyes of decision makers and the public. Some social externalities, such as employment and regional economic impacts, are generally omitted in current practices. This is due to the lack of information and credible models. It may be appropriate to consider these externalities in qualitative forms in a Multiple Criteria Analysis. Consensus has been reached in considering noise and air pollution in international practices. However, Australia practices generally omitted these externalities. Equity is an important consideration in Road Asset Management. The considerations are either between regions, or social groups, such as income, age, gender, disable, etc. In current practice, there is not a well developed quantitative measure for equity issues. More research is needed to target this issue. Although Multiple Criteria Analysis has been used for decades, there is not a generally accepted framework in the choice of modelling methods and various externalities. The result is that different analysts are unlikely to reach consistent conclusions about a policy measure. In current practices, some favour using methods which are able to prioritise alternatives, such as Goal Programming, Goal Achievement Matrix, Analytic Hierarchy Process. The others just present various impacts to decision-makers to characterise the projects. Weighting and scoring system are critical in most Multiple Criteria Analysis. However, the processes of assessing weights and scores were criticised as highly arbitrary and subjective. It is essential that the process should be as transparent as possible. Obtaining weights and scores by consulting local communities is a common practice, but is likely to result in bias towards local interests. Interactive approach has the advantage in helping decision-makers elaborating their preferences. However, computation burden may result in lose of interests of decision-makers during the solution process of a large-scale problem, say a large state road network. Current practices tend to use cardinal or ordinal scales in measure in non-monetised externalities. Distorted valuations can occur where variables measured in physical units, are converted to scales. For example, decibels of noise converts to a scale of -4 to +4 with a linear transformation, the difference between 3 and 4 represents a far greater increase in discomfort to people than the increase from 0 to 1. It is suggested to assign different weights to individual score. Due to overlapped goals, the problem of double counting also appears in some of Multiple Criteria Analysis. The situation can be improved by carefully selecting and defining investment goals and criteria. Other issues, such as the treatment of time effect, incorporating risk and uncertainty, have been given scant attention in current practices. This report suggested establishing a common analytic framework to deal with these issues.

A user driven data mining process model and learning system

This paper deals with the problem of using the data mining models in a real-world situation where the user can not provide all the inputs with which the predictive model is built. A learning system framework, Query Based Learning System (QBLS), is developed for improving the performance of the predictive models in practice where not all inputs are available for querying to the system. The automatic feature selection algorithm called Query Based Feature Selection (QBFS) is developed for selecting features to obtain a balance between the relative minimum subset of features and the relative maximum classification accuracy. Performance of the QBLS system and the QBFS algorithm is successfully demonstrated with a real-world application

Defining reference service life : an open innovation approach

The endeavour to obtain estimates of durability of components for use in lifecycle assessment or costing and infrastructure and maintenance planning systems is large. The factor method and the reference service life concept provide a very valuable structure, but do not resolve the central dilemma of the need to derive an extensive database of service life. Traditional methods of estimating service life, such as dose functions or degradation models, can play a role in developing this database, however the scale of the problem clearly indicates that individual dose functions cannot be derived for each component in each different local and geographic setting. Thus, a wider range of techniques is required in order to devise reference service life. This paper outlines the approaches being taken in the Cooperative Research Centre for Construction Innovation project to predict reference service life. Approaches include the development of fundamental degradation and microclimate models, the development of a situation-based reasoning ‘engine’ to vary the ‘estimator’ of service life, and the development of a database on expert performance (Delphi study). These methods should be viewed as complementary rather than as discrete alternatives. As discussed in the paper, the situation-based reasoning approach in fact has the possibility of encompassing all other methods.

A soft approach to solving hard problems in construction project management

Construction projects are faced with a challenge that must not be underestimated. These projects are increasingly becoming highly competitive, more complex, and difficult to manage. They become ‘wicked problems’, which are difficult to solve using traditional approaches. Soft Systems Methodology (SSM) is a systems approach that is used for analysis and problem solving in such complex and messy situations. SSM uses “systems thinking” in a cycle of action research, learning and reflection to help understand the various perceptions that exist in the minds of the different people involved in the situation. This paper examines the benefits of applying SSM to wicked problems in construction project management, especially those situations that are challenging to understand and difficult to act upon. It includes relevant examples of its use in dealing with the confusing situations that incorporate human, organizational and technical aspects.

Five case studies applying soft systems methodology to knowledge management

Construction projects are faced with a challenge that must not be underestimated. These projects are increasingly becoming highly competitive, more complex, and difficult to manage. They become problems that are difficult to solve using traditional approaches. Soft Systems Methodology (SSM) is a systems approach that is used for analysis and problem solving in such complex and messy situations. SSM uses “systems thinking” in a cycle of action research, learning and reflection to help understand the various perceptions that exist in the minds of the different people involved in the situation. This paper examines the benefits of applying SSM to problems of knowledge management in construction project management, especially those situations that are challenging to understand and difficult to act upon. It includes five case studies of its use in dealing with the confusing situations that incorporate human, organizational and technical aspects.