Meta-evaluation to improve learning, evaluation capacity development and sustainability : Findings from a participatory evaluation project in Nepal

Recent studies suggest that meta-evaluation can be valuable in developing new approaches to evaluation, building evaluation capacities, and enhancing organizational learning. These new extensions of the concept of meta-evaluation are significant, given the growing emphasis on improving the quality and effectiveness of evaluation practices in the South Asian region. Following a review of the literature, this paper presents a case study of the use of concurrent meta-evaluation in the four-year project Assessing Communication for Social Change which developed and trialled a participatory impact assessment methodology in collaboration with a development communication Non-government organization (NGO) in Nepal. Key objectives of the meta-evaluation included to: continuously develop, adapt and improve the impact assessment methodology, Monitoring and Evaluation (M&E) systems and process and other project activities; identify impacts of the project; and build capacities in critical reflection and review. Our analysis indicates that this meta-evaluation was essential to understanding various constraints related to the organizational context that affected the success of the project and the development of improved M&E systems and capacities within the NGO. We identified several limitations of our meta-evaluation methods, which were balanced by the strengths of other methods. Our case study suggests that as well as assessing the quality, credibility and value of evaluation practices, meta-evaluations need to focus on important contextual issues that can have significant impacts on the outcomes of participatory evaluation projects. They include hierarchical organizational cultures, communication barriers, power/knowledge relations, and the time and resources available. Meta-evaluations also need to consider wider issues such as the sustainability of evaluation systems and approaches.

Forecasting success on large projects : developing reliable scales to predict multiple perspectives by multiple stakeholders over multiple time frames

Our aim is to develop a set of leading performance indicators to enable managers of large projects to forecast during project execution how various stakeholders will perceive success months or even years into the operation of the output. Large projects have many stakeholders who have different objectives for the project, its output, and the business objectives they will deliver. The output of a large project may have a lifetime that lasts for years, or even decades, and ultimate impacts that go beyond its immediate operation. How different stakeholders perceive success can change with time, and so the project manager needs leading performance indicators that go beyond the traditional triple constraint to forecast how key stakeholders will perceive success months or even years later. In this article, we develop a model for project success that identifies how project stakeholders might perceive success in the months and years following a project. We identify success or failure factors that will facilitate or mitigate against achievement of those success criteria, and a set of potential leading performance indicators that forecast how stakeholders will perceive success during the life of the project's output. We conducted a scale development study with 152 managers of large projects and identified two project success factor scales and seven stakeholder satisfaction scales that can be used by project managers to predict stakeholder satisfaction on projects and so may be used by the managers of large projects for the basis of project control.

Development of network-level project screening methods supporting the 4-Year pavement management plan in Texas

The Texas Department of Transportation (TxDOT) is concerned about the widening gap between pavement preservation needs and available funding. Thus, the TxDOT Austin District Pavement Engineer (DPE) has investigated methods to strategically allocate available pavement funding to potential projects that improve the overall performance of the District and Texas highway systems. The primary objective of the study presented in this paper is to develop a network-level project screening and ranking method that supports the Austin District 4-year pavement management plan development. The study developed candidate project selection and ranking algorithms that evaluated pavement conditions of each project candidate using data contained in the Pavement Management Information system (PMIS) database and incorporated insights from Austin District pavement experts; and implemented the developed method and supporting algorithm. This process previously required weeks to complete, but now requires about 10 minutes including data preparation and running the analysis algorithm, which enables the Austin DPE to devote more time and resources to conducting field visits, performing project-level evaluation and testing candidate projects. The case study results showed that the proposed method assisted the DPE in evaluating and prioritizing projects and allocating funds to the right projects at the right time.

A decision support system for sustainable urban development : the integrated land use and transportation indexing model

Broad, early definitions of sustainable development have caused confusion and hesitation among local authorities and planning professionals. This confusion has arisen because loosely defined principles of sustainable development have been employed when setting policies and planning projects, and when gauging the efficiencies of these policies in the light of designated sustainability goals. The question of how this theory-rhetoric-practice gap can be filled is the main focus of this chapter. It examines the triple bottom line approach–one of the sustainability accounting approaches widely employed by governmental organisations–and the applicability of this approach to sustainable urban development. The chapter introduces the ‘Integrated Land Use and Transportation Indexing Model’ that incorporates triple bottom line considerations with environmental impact assessment techniques via a geographic, information systemsbased decision support system. This model helps decision-makers in selecting policy options according to their economic, environmental and social impacts. Its main purpose is to provide valuable knowledge about the spatial dimensions of sustainable development, and to provide fine detail outputs on the possible impacts of urban development proposals on sustainability levels. In order to embrace sustainable urban development policy considerations, the model is sensitive to the relationship between urban form, travel patterns and socio-economic attributes. Finally, the model is useful in picturing the holistic state of urban settings in terms of their sustainability levels, and in assessing the degree of compatibility of selected scenarios with the desired sustainable urban future.

Developing a predictive contractor satisfaction model (CoSMo) for construction projects

Over the last few decades, construction project performance has been evaluated due to the increase of delays, cost overruns and quality failures. Growing numbers of disputes, inharmonious working environments, conflict, blame cultures, and mismatches of objectives among project teams have been found to be contributory factors to poor project performance. Performance measurement (PM) approaches have been developed to overcome these issues, however, the comprehensiveness of PM as an overall approach is still criticised in terms of the iron triangle; namely time, cost, and quality. PM has primarily focused on objective measures, however, continuous improvement requires the inclusion of subjective measures, particularly contractor satisfaction (Co-S). It is challenging to deal with the two different groups of large and small-medium contractor satisfaction as to date, Co-S has not been extensively defined, primarily in developing countries such as Malaysia. Therefore, a Co-S model is developed in this research which aims to fulfil the current needs in the construction industry by integrating performance measures to address large and small-medium contractor perceptions. The positivist paradigm used in the research was adhered to by reviewing relevant literature and evaluating expert discussions on the research topic. It yielded a basis for the contractor satisfaction model (CoSMo) development which consists of three elements: contractor satisfaction (Co-S) dimensions; contributory factors and characteristics (project and participant). Using valid questionnaire results from 136 contractors in Malaysia lead to the prediction of several key factors of contractor satisfaction and to an examination of the relationships between elements. The relationships were examined through a series of sequential statistical analyses, namely correlation, one-way analysis of variance (ANOVA), t-tests and multiple regression analysis (MRA). Forward and backward MRAs were used to develop Co-S mathematical models. Sixteen Co-S models were developed for both large and small-medium contractors. These determined that the large contractor Malaysian Co-S was most affected by the conciseness of project scope and quality of the project brief. Contrastingly, Co-S for small-medium contractors was strongly affected by the efficiency of risk control in a project. The results of the research provide empirical evidence in support of the notion that appropriate communication systems in projects negatively contributes to large Co-S with respect to cost and profitability. The uniqueness of several Co-S predictors was also identified through a series of analyses on small-medium contractors. These contractors appear to be less satisfied than large contractors when participants lack effectiveness in timely authoritative decision-making and communication between project team members. Interestingly, the empirical results show that effective project health and safety measures are influencing factors in satisfying both large and small-medium contractors. The perspectives of large and small-medium contractors in respect to the performance of the entire project development were derived from the Co-S models. These were statistically validated and refined before a new Co-S model was developed. Developing such a unique model has the potential to increase project value and benefit all project participants. It is important to improve participant collaboration as it leads to better project performance. This study may encourage key project participants; such as client, consultant, subcontractor and supplier; to increase their attention to contractor needs in the development of a project. Recommendations for future research include investigating other participants‟ perspectives on CoSMo and the impact of the implementation of CoSMo in a project, since this study is focused purely on the contractor perspective.

Islamic project financing implementation for sustainable infrastructure development in Indonesia

Islamic financing in Indonesia infrastructure projects development has not been optimally implemented. Therefore this paper serves as a catalyst to explore alternative financial scheme such as Islamic financing for infrastructure development. The purpose of this paper is to explore the enablers and barriers in implementing Islamic project financing for public infrastructure development. The findings are then culminated into enablers and barriers in the implementation of Islamic project financing. The two main enablers are the readily availability of huge fund that can be used to support infrastructure projects; and the acceptability of the concept of shariah-compliant financing. On the other hand, the barriers include: high cost of funding; lack of financial institution capability; lack of government policy and regulation; insufficient government support and commitment; conflict between infrastructure and Islamic finance business practices; profit oriented mindset; lack of understanding of Islamic project financing knowledge in infrastructure; and insufficient project preparation.

An exploratory review of whole life cycle costing for Malaysia property development

The demand of implementing Whole Life Cycle Costing (WLCC) towards property development has increased since the launch of Value Management Circular 3/2009 by the Economic Planning Unit Malaysia. The circular made compulsory for all public construction projects and programme including property development estimated more than RM 50 Million to conduct WLCC. However, a knowledge gap on WLCC approach still exists and become a barrier among the practitioner in Malaysia particularly in property development industry. The main focus of this paper is to discuss the WLCC approach and introduce the indicator that need to be considered in Malaysia property development.

Environmental rating tools and residential land development in Australia

Numerous environmental rating tools have developed around the world over the past decade or so, in an attempt to increase awareness of the impact buildings have on the environment. Whilst many of these tools can be applied across a variety of building types, the majority focus mainly on the commercial building sector. Only recently have some of the better known environmental rating tools become adaptable to the land development sector, where arguably the most visible environmental impacts are made. EnviroDevelopment is one such tool that enables rating of residential land development in Australia. This paper seeks to quantify the environmental benefits achieved by the environmental rating tool EnviroDevelopment, using data from its certified residential projects across Australia. This research will identify the environmental gains achieved in the residential land development sector that can be attributed to developers aspiring to gain certification under this rating tool.

Emerging from student to practitioner : a cumulative development tool for students

PURPOSE: This pilot project’s aim was to trial a tool and process for developing students’ ability to engage in self-assessment using reflection on their clinical experiences, including feedback from workplace learning, in order to aid them in linking theory to practice and develop strategies to improve performance. BACKGROUND: In nursing education, students can experience a mismatch in performance compared to theoretical learning, this is referred to as the ‘theory practice gap’ (Scully 2011, Chan Chan & Liu 2011). One specific contributing factor seems to be students’ inability to engage in meaningful reflection and self-correcting behaviours. A self-assessment strategy was implemented within a third year clinical unit to ameliorate this mismatch with encouraging results, as students developed self-direction in addressing learning needs. In this pilot project the above strategy was adapted for implementation between different clinical units, to create a whole of course approach to integrating workplace learning. METHOD: The methodology underpinning this project is a scaffolded, supported reflective practice process. Improved self-assessment skills is achieved by students reflecting on and engaging with feedback, then mapping this to learning outcomes to identify where performance can be improved. Evaluation of this project includes: collation of student feedback identifying successful strategies along with barriers encountered in implementation; feedback from students and teachers via above processes and tools; and comparison of the number of learning contracts issued in clinical nursing units with similar cohorts. RESULTS: Results will be complete by May 2012 and include analysis of the data collected via the above evaluation methods. Other outcomes will include the refined process and tool, plus resources that should improve cost effectiveness without reducing student support. CONCLUSION: Implementing these tools and processes over the entire student’s learning package, will assist them to demonstrate progressive development through the course. Students will have learnt to understand feedback and integrate these skills for life-long learning.

Implementing the Clean Development Mechanism in Fiji : opportunities and barriers in the energy sector

The Clean Development Mechanism (CDM) has been praised for its ingenuity in mobilising finance to implement sustainable development practices in non-industrialised countries (known as Non-Annex 1 parties under the Kyoto Protocol). During the first commitment period of the Kyoto Protocol (2008-2012), a large number of clean development mechanism projects have been registered with the CDM board. In addition to the large number of registered CDM projects, there are significant numbers of proposed projects stalled in implementation due to the cumbersome and lengthy CDM approval process. Despite this regulatory criticism it is recognised that the role performed by the CDM is essential for achieving a significant reduction in global green house gas emissions. This is because the CDM funds sustainable development in countries that lack capacity to do so on their own. It is anticipated that some form of CDM instrument will continue post the 2012 timeframe and that reform of the mechanism will be focused around making the mechanism’s approval and implementation processes faster and more efficient.

Agricultural biodiversity, farm level technical efficiency and conservation benefits : an empirical investigation

The issues involved in agricultural biodiversity are important and interesting areas for the application of economic theory. However, very little theoretical and empirical work has been undertaken to understand the benefits of conserving agricultural biodiversity. Accordingly, the main objectives of this PhD thesis are to: (1) Investigate farmers’ valuation of agricultural biodiversity; (2) Identify factors influencing farmers’ demand for agricultural biodiversity; (3) Examine farmers’ demand for biodiversity rich farming systems; (4) Investigate the relationship between agricultural biodiversity and farm level technical efficiency. This PhD thesis investigates these issues by using primary data in small-scale farms, along with secondary data from Sri Lanka. The overall findings of the thesis can be summarized as follows. Firstly, owing to educational and poverty issues of those being interviewed, some policy makers in developed countries question whether non-market valuation techniques such as Choice Experiment (CE) can be applied to developing countries such as Sri Lanka. The CE study in this thesis indicates that carefully designed and pre-tested nonmarket valuation techniques can be applied in developing countries with a high level of reliability. The CE findings support the priori assumption that small-scale farms and their multiple attributes contribute positively and significantly to the utility of farm families in Sri Lanka. Farmers have strong positive attitudes towards increasing agricultural biodiversity in rural areas. This suggests that these attitudes can be the basis on which appropriate policies can be introduced to improve agricultural biodiversity. Secondly, the thesis identifies the factors which influence farmers’ demand for agricultural biodiversity and farmers’ demands on biodiversity rich farming systems. As such they provide important tools for the implementation of policies designed to avoid the loss agricultural biodiversity which is shown to be a major impediment to agricultural growth and sustainable development in a number of developing countries. The results illustrate that certain key household, market and other characteristics (such as agricultural subsidies, percentage of investment of owned money and farm size) are the major determinants of demand for agricultural biodiversity on small-scale farms. The significant household characteristics that determine crop and livestock diversity include household member participation on the farm, off-farm income, shared labour, market price fluctuations and household wealth. Furthermore, it is shown that all the included market characteristics as well as agricultural subsidies are also important determinants of agricultural biodiversity. Thirdly, it is found that when the efficiency of agricultural production is measured in practice, the role of agricultural biodiversity has rarely been investigated in the literature. The results in the final section of the thesis show that crop diversity, livestock diversity and mix farming system are positively related to farm level technical efficiency. In addition to these variables education level, number of separate plots, agricultural extension service, credit access, membership of farm organization and land ownerships are significant and direct policy relevant variables in the inefficiency model. The results of the study therefore have important policy implications for conserving agricultural biodiversity in Sri Lanka.

Assessing eco-environmental performance of agricultural production in OECD countries : combination of soil surface, soil system and farm gate methods of nutrient auditing

Nitrogen balance is increasingly used as an indicator of the environmental performance of agricultural sector in national, international, and global contexts. There are three main methods of accounting the national nitrogen balance: farm gate, soil surface, and soil system. OECD (2008) recently reported the nitrogen and phosphorus balances for member countries for the 1985 - 2004 period using the soil surface method. The farm gate and soil system methods were also used in some international projects. Some studies have provided the comparison among these methods and the conclusion is mixed. The motivation of this present paper was to combine these three methods to provide a more detailed auditing of the nitrogen balance and flows for national agricultural production. In addition, the present paper also provided a new strategy of using reliable international and national data sources to calculate nitrogen balance using the farm gate method. The empirical study focused on the nitrogen balance of OECD countries for the period from 1985 to 2003. The N surplus sent to the total environment of OECD surged dramatically in early 1980s, gradually decreased during 1990s but exhibited an increasing trends in early 2000s. The overall N efficiency however fluctuated without a clear increasing trend. The eco-environmental ranking shows that Australia and Ireland were the worst while Korea and Greece were the best.

A prototype system dynamic model for assessing the sustainability of construction projects

There is a worldwide demand for an increasingly sustainable built environment. This has resulted in the need for a more accurate evaluation of the level of sustainability of construction projects. To do this it involves the development of better measurement and benchmarking methods. One approach is to use a theoretical model to assess construction projects in terms of their sustainable development value (SDV) and sustainable development ability (SDA) for implementation in the project life cycle, where SDA measures the contribution of a project to development sustainability and as a major criterion for assessing its feasibility. This paper develops an improved SDA prototype model that incorporates the effects of dynamical factors on project sustainability. This involves the introduction of two major factors concerning technological advancement and changes in people's perceptions. A case study is used to demonstrate the procedures involved in simulation and modeling, one outcome of which is to demonstrate the greater influence of technological advancement on project sustainability than changes in perception.

Downtime model development for construction equipment management

Downtime (DT) caused by non-availability of equipment and equipment breakdown has non-trivial impact on the performance of construction projects. Earlier research has often addressed this fact, but it has rarely explained the causes and consequences of DT – especially in the context of developing countries. This paper presents a DT model to address this issue. Using this model, the generic factors and processes related to DT are identified, and the impact of DT is quantified. By applying the model framework to nine road projects in Nepal, the impact of DT is explored in terms of its duration and cost. The research findings highlight how various factors and processes interact with each other to create DT, and mitigate or exacerbate its impact on project performance. It is suggested that construction companies need to adopt proactive equipment management and maintenance programs to minimize the impact of DT.

An expert system for the strategic planning of construction projects

This paper describes the instigation and development of an expert system to aid in the strategic planning of construction projects. The paper consists of four parts - the origin of the project, the development of the concepts needed for the proposed system, the building of the system itself, and assessment of its performance. The origin of the project is outlined starting with the Japanese commitment to 5th generation computing together with the increasing local reaction to theory based prescriptive research in the field. The subsequent development of activities via the Alvey Commission and the RICS in conjunction with Salford University are traced culminating in the proposal and execution of the first major expert system to be built for the UK construction industry, subsequently recognised as one of the most successful of the expert system projects commissioned under the Alvey programme