Meta-evaluation to improve learning, evaluation capacity development and sustainability : Findings from a participatory evaluation project in Nepal

Recent studies suggest that meta-evaluation can be valuable in developing new approaches to evaluation, building evaluation capacities, and enhancing organizational learning. These new extensions of the concept of meta-evaluation are significant, given the growing emphasis on improving the quality and effectiveness of evaluation practices in the South Asian region. Following a review of the literature, this paper presents a case study of the use of concurrent meta-evaluation in the four-year project Assessing Communication for Social Change which developed and trialled a participatory impact assessment methodology in collaboration with a development communication Non-government organization (NGO) in Nepal. Key objectives of the meta-evaluation included to: continuously develop, adapt and improve the impact assessment methodology, Monitoring and Evaluation (M&E) systems and process and other project activities; identify impacts of the project; and build capacities in critical reflection and review. Our analysis indicates that this meta-evaluation was essential to understanding various constraints related to the organizational context that affected the success of the project and the development of improved M&E systems and capacities within the NGO. We identified several limitations of our meta-evaluation methods, which were balanced by the strengths of other methods. Our case study suggests that as well as assessing the quality, credibility and value of evaluation practices, meta-evaluations need to focus on important contextual issues that can have significant impacts on the outcomes of participatory evaluation projects. They include hierarchical organizational cultures, communication barriers, power/knowledge relations, and the time and resources available. Meta-evaluations also need to consider wider issues such as the sustainability of evaluation systems and approaches.

Higher education for capacity development or revenue maximisation? Lessons from transnational higher education programs at two Indonesian universities

HE has been changing rapidly due to globalisation that has increased the interconnectedness between nations and people throughout the world (Mok, 2012). As HE has manifested into different forms and governed by competing rationales in recent years, this paper focuses on transnational HE, which is an example of the interconnectedness of universities beyond the national borders. Indonesia is also influenced by the above changes. It took part in free-trade agreements that include HE as a sector to be liberated and accessed by international providers (Nizam, 2006). Indonesian universities found themselves bracing for the global competition for students and simultaneously having to improve their quality in order to survive amidst the growing competition. This competition gave birth to joint transnational HE programs with overseas partners among many Indonesian universities (Macaranas, 2010).

A framework for predicting the residual load carrying capacity of concrete structures exposed to aggressive environments

Reinforced concrete structures are susceptible to a variety of deterioration mechanisms due to creep and shrinkage, alkali-silica reaction (ASR), carbonation, and corrosion of the reinforcement. The deterioration problems can affect the integrity and load carrying capacity of the structure. Substantial research has been dedicated to these various mechanisms aiming to identify the causes, reactions, accelerants, retardants and consequences. This has improved our understanding of the long-term behaviour of reinforced concrete structures. However, the strengthening of reinforced concrete structures for durability has to date been mainly undertaken after expert assessment of field data followed by the development of a scheme to both terminate continuing degradation, by separating the structure from the environment, and strengthening the structure. The process does not include any significant consideration of the residual load-bearing capacity of the structure and the highly variable nature of estimates of such remaining capacity. Development of performance curves for deteriorating bridge structures has not been attempted due to the difficulty in developing a model when the input parameters have an extremely large variability. This paper presents a framework developed for an asset management system which assesses residual capacity and identifies the most appropriate rehabilitation method for a given reinforced concrete structure exposed to aggressive environments. In developing the framework, several industry consultation sessions have been conducted to identify input data required, research methodology and output knowledge base. Capturing expert opinion in a useable knowledge base requires development of a rule based formulation, which can subsequently be used to model the reliability of the performance curve of a reinforced concrete structure exposed to a given environment.

An evaluation framework for sustaining the impact of educational development

Notwithstanding significant efforts by international aid agencies, aid ineffectiveness became apparent in 1990s as the impact of continued development intervention did not endure the expected outcomes. Conventional monitoring and evaluation by those agencies is critiqued for focusing on measuring project outcomes and giving little attention to aspects of sustainability. As a result, devising a rigorous evaluation framework for educational development has been sought in light of recent paradigm shifts in international development. This paper reports on a case study of an Egyptian educational development project highlighting the importance of transforming the evaluation procedures to process evaluation so as to enhance project impact and longevity. This requires building evaluation capacity of the aid recipient country.

Evaluating Communication for Development : A Framework for Social Change

Evaluating Communication for Development presents a comprehensive framework for evaluating communication for development (C4D). This framework combines the latest thinking from a number of fields in new ways. It critiques dominant instrumental, accountability-based approaches to development and evaluation and offers an alternative holistic, participatory, mixed methods approach based on systems and complexity thinking and other key concepts. It maintains a focus on power, gender and other differences and social norms. The authors have designed the framework as a way to focus on achieving sustainable social change and to continually improve and develop C4D initiatives. The benefits and rigour of this approach are supported by examples and case studies from a number of action research and evaluation capacity development projects undertaken by the authors over the past fifteen years. Building on current arguments within the fields of C4D and development, the authors reinforce the case for effective communication being a central and vital component of participatory forms of development, something that needs to be appreciated by decision makers. They also consider ways of increasing the effectiveness of evaluation capacity development from grassroots to management level in the development context, an issue of growing importance to improving the quality, effectiveness and utilisation of monitoring and evaluation studies in this field. The book includes a critical review of the key approaches, methodologies and methods that are considered effective for planning evaluation, assessing the outcomes of C4D, and engaging in continuous learning. This rigorous book is of immense theoretical and practical value to students, scholars, and professionals researching or working in development, communication and media, applied anthropology, and evaluation and program planning.

Research Monograph: Building Online Participatory Systems: Towards the Community Based Interactive Environmental Decision Support Systems

The importance of broadening community participation in environmental decision-making is widely recognized and lack of participation in this process appears to be a perennial problem. In this context, there have been calls from some academics for the more extensive use of geographic information systems (GIS) and distance learning technologies, accessible via the Internet, as a possible means to inform and empower communities. However, a number of problems exist. For instance, at present the scope for online interaction between policy-makers and citizens is currently limited. Contemporary web-based environmental information systems suffer from this lack of interactivity on the one hand and on the other hand from the apparent complexity for the lay user. This paper explores the issue of online community participation at the local level and attempts to construct a framework for a new (and potentially more effective) model of online participatory decision-making. The key components, system architecture and stages of such a model are introduced. This model, referred to as a ‘Community Based Interactive Environmental Decision Support System’, incorporates advanced information technologies, distance learning and community involvement tools which will be applied and evaluated in the field through a pilot project in Tokyo in the summer of 2002.

An evaluation framework for educational reform projects for teacher quality improvements in developing countries : a case study of Egyptian education reform

The role of the evaluation for Official Development Assistance (ODA) enterprises including educational development has become critical after increasing “aid fatigue” experienced by the international community in the 1990s. To date, however, monitoring and evaluating outcomes of the projects has been limited to the project life. Consequently these have been mainly through the international aid agencies. Furthermore, the monitoring and evaluation led by international aid agencies have paid little attention to aspects of the sustainability of technical cooperation in educational development. To sustain the impact of technical cooperation, the reinforcement of evaluation has drawn increasing attention in light of the emerging modalities in international development. Therefore this research was inspired to investigate alternative evaluation frameworks for an educational reform project for teacher quality improvement that may increase possibilities for long term sustainability. Importantly, the new modalities in international development and educational issues provide new options. In addition, the research reviewed theoretical and practical issues surrounding evaluation in general, and highlighted the evaluation of education reform projects. The research reported explored via case studies, the evaluation processes employed by the Egyptian education reform projects implemented by the Japan International Cooperation Agency (JICA) and the United Nations Children’s Fund (UNICEF). The case studies used three data sources (archival and relevant documents, a survey questionnaire and interviews) to illuminate the contextually-embedded evaluation processes. The research found that process evaluation is a potential alternative method since it is likely to be locally institutionalised, which may yield long-term sustainability of the projects.

A comparative review of quantitative VFM methodology for PPP infrastructure project in Indonesia and Australia

The Government of Indonesia (GoI) increasingly relies on the private sector financing to build and operate infrastructures through public private partnership (PPP) schemes. However, PPP does not automatically provide the solution for the financing scheme due to value for money (VFM) issues. The procurement authority must show whether a PPP proposal is the optimal solution that provides best VFM outcome. The paper presents a literature review of comparing quantitative VFM methodology for PPP infrastructure project procurement in Indonesia and Australia. Public Sector Comparator (PSC) is used to assess the potential project VFM quantitatively in Australia. In Indonesia, the PSC has not been applied, where the PPP procurement authority tends to utilize a common project evaluation method that ignores the issues of risk. Unlike the conventional price bid evaluation, the PSC enables a financial comparison including costs/gains and risks. Since the construction of PSC is primarily on risk management approach, it can facilitate risk negotiation processes between the involved parties. The study indicates that the quantitative VFM methodology of PSC is potentially applicable in Indonesia for water supply sector. Various supporting regulations are available that emphasize the importance of VFM and risk management in infrastructure investment. However, the study also reveals a number of challenges that need to be anticipated, such as the need of a more comprehensive PPP policy at both central and local government level, a more specific legal instrument for bidding evaluation method and the issue of institutional capacity development in PPP Units at the local level.

Investigating knowledge transfer through transnational programs between Indonesian and Australian universities : A conceptual framework

Globalization stimulated demand for transnational higher education. However, little is known about how transnational programs can facilitate knowledge transfer between the partner universities—something that is often implied in the partnership arrangement. For the purpose of this paper transnational programs will be limited to dual degree programs between Indonesian and Australian universities, which, in recent years have been rising. Nevertheless, the discussion and synthesis may have value to other transnational programs. Situating the knowledge transfer in the wider context of globalization of higher education and the current reforms in Indonesian education, the paper acknowledges the tension between the need to generate revenue and the ideals of facilitating knowledge transfer and capacity development for Indonesian universities. Drawing on a detailed review of research findings in the business and education sector literature, this paper presents a conceptual framework that may be used to analyze knowledge transfer between Indonesian and Australian universities. Such investigations delineate the significant drivers underlying transnational programs for supporting knowledge transfer. The proposed framework incorporates the types of knowledge, knowledge transfer mechanisms, and the processes. It also considers how the inter-university antecedents may affect the potential for knowledge transfer between the universities. A discussion of how the framework may be used to implemented knowledge transfer concludes this paper.

Knowledge transfer through dual degree programs between Australian and Indonesian universities : a case study

While dual degree programs (DDPs) between Australian and Indonesian universities are expected to facilitate knowledge transfer (KT) between the partnering universities, little is known about how and what KT process taking place within DDP partnerships. Using an inter-organisational KT framework, this study investigated Indonesian universities’ rationales and outcomes of establishing DDPs and mechanisms facilitating knowledge transfer between Australian and Indonesian universities. Two Indonesian universities along with their common Australian partner university participated in this case study. Semi-structured interviews were conducted with 27 key university officers and pertinent university documents provided the main data. Both data sources were thematically analysed to identify emerging patterns. The findings suggest that Indonesian universities prioritised developing capacity to improve their international recognition more than the Australian partner. Consequently, the DDPs benefited the Indonesian universities through capacity development made possible by KT from the Australian DDP partners. KT processes occurred in DDP partnerships, particularly through curriculum collaboration, but they were more limited for the managerial area. Factors enabling the KT included both technology-aided and face-to-face communication, intention to acquire knowledge from the partners, capitalising on the unequal power relations to advance KT opportunities, and knowledge management system. The findings of this study suggest the importance of prioritising capacity development in DDP partnerships to enable KT, executing the KT stages to ensure institutionalisation of acquired knowledge into the university’s systems and policies, and maintaining financial sustainability of the DDPs to reach mutually beneficial outcomes between Australian and Indonesian universities.

E-government policies for broadband adoption : the case of the UK Government

Broadband technology has been introduced to the business community and the public as a fast and easy way of exploiting the Internet. The benefits of its use (fast reliable connections, always on) have been widely realised and broadband diffusion is one of the items at the top of the agenda for technology related polices of governments worldwide. Broadband diffusion and capacity development are central to debates in many countries concerning the role of the government in developing efficient broadband policies particularly in terms of the usage of public money. In this paper we examine the impact of government polices to broadband adoption in the case of the UK government, a country that is striving to have the most extensive and competitive broadband market in the G7 by 2005. We analyse specific institutional actions related to IT diffusion as pursued by the government with the view to identify best practices in government intervention. We believe that are our analysis is useful for broadband strategies in particular and technology in general when applied elsewhere.

Knowledge transfer through a transnational program partnership between Indonesian and Australian universities

As transnational programs are often advocated as a knowledge transfer opportunity between the partner universities, this case study investigated the knowledge transfer (KT) processes between Indonesian and Australian universities through an undergraduate transnational program partnership (TPP). An inter-organisational KT theoretical framework from the business sector was adapted and used to guide the study. The data were generated through semi-structured interviews with key university officers and document analysis from two partner universities. Based on the thematic analysis of the data, the findings demonstrated that the curriculum mapping process facilitated KT. However, different intentions of the partner universities in establishing the program led to declining interest to conduct more KT when expectations were not met. The Indonesian university’s existing knowledge, acquired from other sources through processes that were serendipitous and based on individual lecturers’ personal experience, meant that KT opportunities through the TPP were not always pursued despite written agreement to exchange knowledge with the Australian partner. While KT most evidently resulted in institutional capacity development for the Indonesian university’s school that managed the TPP, dissemination of knowledge to other units within the university was more challenging due to communication problems between the units. Hence, other universities seeking to conduct KT through TPPs need to understand each partner university's intention in establishing the partnerships, identify the institutions' needs before seeking knowledge input from the partner university and improve the communication between and within the universities for sustainable benefits.

Tripogon loliiformis elicits a rapid physiological and structural response to dehydration for desiccation tolerance

Resurrection plants can withstand extreme dehydration to an air-dry state and then recover upon receiving water. Tripogon loliiformis (F.Muell.) C.E.Hubb. is a largely uncharacterised native Australian desiccation-tolerant grass that resurrects from the desiccated state within 72 h. Using a combination of structural and physiological techniques the structural and physiological features that enable T. loliiformis to tolerate desiccation were investigated. These features include: - (i) a myriad of structural changes such as leaf folding, cell wall folding and vacuole fragmentation that mitigate desiccation stress; - (ii) potential role of sclerenchymatous tissue within leaf folding and radiation protection; - (iii) retention of ~70% chlorophyll in the desiccated state; - (iv) early response of photosynthesis to dehydration by 50% reduction and ceasing completely at 80 and 70% relative water content, respectively; - (v) a sharp increase in electrolyte leakage during dehydration, and; - (vi) confirmation of membrane integrity throughout desiccation and rehydration. Taken together, these results demonstrate that T. loliiformis implements a range of structural and physiological mechanisms that minimise mechanical, oxidative and irradiation stress. These results provide powerful insights into tolerance mechanisms for potential utilisation in the enhancement of stress-tolerance in crop plants.

The Capacity to Share : A Study of Cuba's International Cooperation in Educational Development

Since 1959, international cooperation has been a key feature of Cuba’s commitment to egalitarian social well-being. Aspects of this experience have been well documented , in general and with reference to specific initiatives across human development and occupational sectors. Others have been little examined, of which education is one. This book describes the internationalism of Cuban education policy as practised in Cuba and in other parts of the Global “South.”