Investment decision-making under risk (reliability) and uncertainty for infrastructure asset management

Risks and uncertainties are inevitable in engineering projects and infrastructure investments. Decisions about investment in infrastructure such as for maintenance, rehabilitation and construction works can pose risks, and may generate significant impacts on social, cultural, environmental and other related issues. This report presents the results of a literature review of current practice in identifying, quantifying and managing risks and predicting impacts as part of the planning and assessment process for infrastructure investment proposals. In assessing proposals for investment in infrastructure, it is necessary to consider social, cultural and environmental risks and impacts to the overall community, as well as financial risks to the investor. The report defines and explains the concept of risk and uncertainty, and describes the three main methodology approaches to the analysis of risk and uncertainty in investment planning for infrastructure, viz examining a range of scenarios or options, sensitivity analysis, and a statistical probability approach, listed here in order of increasing merit and complexity. Forecasts of costs, benefits and community impacts of infrastructure are recognised as central aspects of developing and assessing investment proposals. Increasingly complex modelling techniques are being used for investment evaluation. The literature review identified forecasting errors as the major cause of risk. The report contains a summary of the broad nature of decision-making tools used by governments and other organisations in Australia, New Zealand, Europe and North America, and shows their overall approach to risk assessment in assessing public infrastructure proposals. While there are established techniques to quantify financial and economic risks, quantification is far less developed for political, social and environmental risks and impacts. The report contains a summary of the broad nature of decision-making tools used by governments and other organisations in Australia, New Zealand, Europe and North America, and shows their overall approach to risk assessment in assessing public infrastructure proposals. While there are established techniques to quantify financial and economic risks, quantification is far less developed for political, social and environmental risks and impacts. For risks that cannot be readily quantified, assessment techniques commonly include classification or rating systems for likelihood and consequence. The report outlines the system used by the Australian Defence Organisation and in the Australian Standard on risk management. After each risk is identified and quantified or rated, consideration can be given to reducing the risk, and managing any remaining risk as part of the scope of the project. The literature review identified use of risk mapping techniques by a North American chemical company and by the Australian Defence Organisation. This literature review has enabled a risk assessment strategy to be developed, and will underpin an examination of the feasibility of developing a risk assessment capability using a probability approach.

The public management issues of access to electronic gaming machines (EGMs) : a case study of policy in Victoria, its effects and requirements for evaluation

Wynne and Schaffer (2003) have highlighted both the strong growth of gambling activity in recent years, and the revenue streams this has generated for governments and communities. Gambling activities and the revenues derived from them have, unsurprisingly, therefore also been seen as a way in which to increase economic development in deprived areas (Jinkner-Lloyd, 1996). Consequently, according to Brown et al (2003), gambling is now a large taxation revenue earner for many western governments, at both federal and state levels, worldwide (for example UK, USA, Australia). In size and importance, the Australian gambling industry in particular has grown significantly over the last three decades, experiencing a fourfold increase in real gambling turnover. There are, however, also concerns expressed about gambling and Electronic Gaming in particular, as illustrated in economic, social and ethical terms in Oddo (1997). There are also spatial aspects to understanding these issues. Marshall’s (1998) study, for example, highlights that benefits from gambling are more likely to accrue at the macro as opposed to the local level, because of centralised tax gathering and spending of tax revenues, whilst localities may suffer from displacement of activities with higher multipliers than the institutions with EGMs that replace them. This also highlights a regional context of costs, where benefits accrue to the centre, but the costs accrue to the regions and localities, as simultaneously resources leave those communities through both the gambling activities themselves (in the form of revenue for the EGM owners), and the government (through taxes).

An evaluation framework for educational reform projects for teacher quality improvements in developing countries : a case study of Egyptian education reform

The role of the evaluation for Official Development Assistance (ODA) enterprises including educational development has become critical after increasing “aid fatigue” experienced by the international community in the 1990s. To date, however, monitoring and evaluating outcomes of the projects has been limited to the project life. Consequently these have been mainly through the international aid agencies. Furthermore, the monitoring and evaluation led by international aid agencies have paid little attention to aspects of the sustainability of technical cooperation in educational development. To sustain the impact of technical cooperation, the reinforcement of evaluation has drawn increasing attention in light of the emerging modalities in international development. Therefore this research was inspired to investigate alternative evaluation frameworks for an educational reform project for teacher quality improvement that may increase possibilities for long term sustainability. Importantly, the new modalities in international development and educational issues provide new options. In addition, the research reviewed theoretical and practical issues surrounding evaluation in general, and highlighted the evaluation of education reform projects. The research reported explored via case studies, the evaluation processes employed by the Egyptian education reform projects implemented by the Japan International Cooperation Agency (JICA) and the United Nations Children’s Fund (UNICEF). The case studies used three data sources (archival and relevant documents, a survey questionnaire and interviews) to illuminate the contextually-embedded evaluation processes. The research found that process evaluation is a potential alternative method since it is likely to be locally institutionalised, which may yield long-term sustainability of the projects.

The use of economic evaluation in CAM : an introductory framework

Background For CAM to feature prominently in health care decision-making there is a need to expand the evidence-base and to further incorporate economic evaluation into research priorities. In a world of scarce health care resources and an emphasis on efficiency and clinical efficacy, CAM, as indeed do all other treatments, requires rigorous evaluation to be considered in budget decision-making. Methods Economic evaluation provides the tools to measure the costs and health consequences of CAM interventions and thereby inform decision making. This article offers CAM researchers an introductory framework for understanding, undertaking and disseminating economic evaluation. The types of economic evaluation available for the study of CAM are discussed, and decision modelling is introduced as a method for economic evaluation with much potential for use in CAM. Two types of decision models are introduced, decision trees and Markov models, along with a worked example of how each method is used to examine costs and health consequences. This is followed by a discussion of how this information is used by decision makers. Conclusions Undoubtedly, economic evaluation methods form an important part of health care decision making. Without formal training it can seem a daunting task to consider economic evaluation, however, multidisciplinary teams provide an opportunity for health economists, CAM practitioners and other interested researchers, to work together to further develop the economic evaluation of CAM.

An evaluation framework for sustaining the impact of educational development

Notwithstanding significant efforts by international aid agencies, aid ineffectiveness became apparent in 1990s as the impact of continued development intervention did not endure the expected outcomes. Conventional monitoring and evaluation by those agencies is critiqued for focusing on measuring project outcomes and giving little attention to aspects of sustainability. As a result, devising a rigorous evaluation framework for educational development has been sought in light of recent paradigm shifts in international development. This paper reports on a case study of an Egyptian educational development project highlighting the importance of transforming the evaluation procedures to process evaluation so as to enhance project impact and longevity. This requires building evaluation capacity of the aid recipient country.

Evaluation of financial incentives as a quality improvement strategy in the public hospital context : clinician attitudes, design variables, and economic costs

In 2008, a three-year pilot ‘pay for performance’ (P4P) program, known as ‘Clinical Practice Improvement Payment’ (CPIP) was introduced into Queensland Health (QHealth). QHealth is a large public health sector provider of acute, community, and public health services in Queensland, Australia. The organisation has recently embarked on a significant reform agenda including a review of existing funding arrangements (Duckett et al., 2008). Partly in response to this reform agenda, a casemix funding model has been implemented to reconnect health care funding with outcomes. CPIP was conceptualised as a performance-based scheme that rewarded quality with financial incentives. This is the first time such a scheme has been implemented into the public health sector in Australia with a focus on rewarding quality, and it is unique in that it has a large state-wide focus and includes 15 Districts. CPIP initially targeted five acute and community clinical areas including Mental Health, Discharge Medication, Emergency Department, Chronic Obstructive Pulmonary Disease, and Stroke. The CPIP scheme was designed around key concepts including the identification of clinical indicators that met the set criteria of: high disease burden, a well defined single diagnostic group or intervention, significant variations in clinical outcomes and/or practices, a good evidence, and clinician control and support (Ward, Daniels, Walker & Duckett, 2007). This evaluative research targeted Phase One of implementation of the CPIP scheme from January 2008 to March 2009. A formative evaluation utilising a mixed methodology and complementarity analysis was undertaken. The research involved three research questions and aimed to determine the knowledge, understanding, and attitudes of clinicians; identify improvements to the design, administration, and monitoring of CPIP; and determine the financial and economic costs of the scheme. Three key studies were undertaken to ascertain responses to the key research questions. Firstly, a survey of clinicians was undertaken to examine levels of knowledge and understanding and their attitudes to the scheme. Secondly, the study sought to apply Statistical Process Control (SPC) to the process indicators to assess if this enhanced the scheme and a third study examined a simple economic cost analysis. The CPIP Survey of clinicians elicited 192 clinician respondents. Over 70% of these respondents were supportive of the continuation of the CPIP scheme. This finding was also supported by the results of a quantitative altitude survey that identified positive attitudes in 6 of the 7 domains-including impact, awareness and understanding and clinical relevance, all being scored positive across the combined respondent group. SPC as a trending tool may play an important role in the early identification of indicator weakness for the CPIP scheme. This evaluative research study supports a previously identified need in the literature for a phased introduction of Pay for Performance (P4P) type programs. It further highlights the value of undertaking a formal risk assessment of clinician, management, and systemic levels of literacy and competency with measurement and monitoring of quality prior to a phased implementation. This phasing can then be guided by a P4P Design Variable Matrix which provides a selection of program design options such as indicator target and payment mechanisms. It became evident that a clear process is required to standardise how clinical indicators evolve over time and direct movement towards more rigorous ‘pay for performance’ targets and the development of an optimal funding model. Use of this matrix will enable the scheme to mature and build the literacy and competency of clinicians and the organisation as implementation progresses. Furthermore, the research identified that CPIP created a spotlight on clinical indicators and incentive payments of over five million from a potential ten million was secured across the five clinical areas in the first 15 months of the scheme. This indicates that quality was rewarded in the new QHealth funding model, and despite issues being identified with the payment mechanism, funding was distributed. The economic model used identified a relative low cost of reporting (under $8,000) as opposed to funds secured of over $300,000 for mental health as an example. Movement to a full cost effectiveness study of CPIP is supported. Overall the introduction of the CPIP scheme into QHealth has been a positive and effective strategy for engaging clinicians in quality and has been the catalyst for the identification and monitoring of valuable clinical process indicators. This research has highlighted that clinicians are supportive of the scheme in general; however, there are some significant risks that include the functioning of the CPIP payment mechanism. Given clinician support for the use of a pay–for-performance methodology in QHealth, the CPIP scheme has the potential to be a powerful addition to a multi-faceted suite of quality improvement initiatives within QHealth.

Sing & Grow : the co-existence of evaluation research and clinical practice in an early intervention music therapy project

Sing & Grow is a short term early intervention music therapy program for at risk families. Sing & Grow uses music to strengthen parent-child relationships by increasing positive parent-child interactions, assisting parents to bond with their children, and extending the repertoire of parents’ skills in relating to their child through interactive . Both the Australian and New Zealand governments are looking for evidence based research to highlight the effectiveness of funded programs in early childhood. As a government funded program, independent evaluation is a requirement of the delivery of the service. This paper explains the process involved in setting up and managing this large scale evaluation from engaging the evaluators and designing the project, to the data gathering stage. It describes the various challenges encountered and concludes that a highly collaborative and communicative partnership bet en researchers and clinicians is essential to ensure data can be gathered with minimal disturbance to clinical music therapy practice.

Action inquiry into the use of standardized evaluation tools for music therapy : a real life journey within a parent-child community program

Sing & Grow is an early intervention music therapy project that provides community group music therapy programs to families with young children who encounter risk factors that may impact on parenting and optimal child develop variety of evaluation tools were devised and used over the first 3 years of the project. Upon the subsequent funding and expansion of the project at the end of this period, it was necessary to find, test and devise more rigorous, valid and reliable measures to withstand the scrutiny of researchers, and to combat the concerns and criticisms associated with the previous methods of data collection. An action inquiry project was therefore undertaken with two groups of project participants to trial the use of the Parenting Stress Index and Depression, Anxiety and Stress Scales, both recommended by leading psychologists. Key findings that will be discussed include the friction between the deficit-focussed nature of many psychometric tools and the strengths-based approach taken in service delivery, the level of difficulty in terms of literacy and comprehension for vulnerable respondents, and the lack of one tool with the ability to comprehensively measure all aspects of a broad scoping program. Keywords: music therapy, evaluation, PSI, DASS, action inquiry.

An economic evaluation of the Safe Motherhood programme in Guangxi, China

Maternal and infant mortality is a global health issue with a significant social and economic impact. Each year, over half a million women worldwide die due to complications related to pregnancy or childbirth, four million infants die in the first 28 days of life, and eight million infants die in the first year. Ninety-nine percent of maternal and infant deaths are in developing countries. Reducing maternal and infant mortality is among the key international development goals. In China, the national maternal mortality ratio and infant mortality rate were reduced greatly in the past two decades, yet a large discrepancy remains between urban and rural areas. To address this problem, a large-scale Safe Motherhood Programme was initiated in 2000. The programme was implemented in Guangxi in 2003. Interventions in the programme included both demand-side and supply side-interventions focusing on increasing health service use and improving birth outcomes. Little is known about the effects and economic outcomes of the Safe Motherhood Programme in Guangxi, although it has been implemented for seven years. The aim of this research is to estimate the effectiveness and cost-effectiveness of the interventions in the Safe Motherhood Programme in Guangxi, China. The objectives of this research include: 1. To evaluate whether the changes of health service use and birth outcomes are associated with the interventions in the Safe Motherhood Programme. 2. To estimate the cost-effectiveness of the interventions in the Safe Motherhood Programme and quantify the uncertainty surrounding the decision. 3. To assess the expected value of perfect information associated with both the whole decision and individual parameters, and interpret the findings to inform priority setting in further research and policy making in this area. A quasi-experimental study design was used in this research to assess the effectiveness of the programme in increasing health service use and improving birth outcomes. The study subjects were 51 intervention counties and 30 control counties. Data on the health service use, birth outcomes and socio-economic factors from 2001 to 2007 were collected from the programme database and statistical yearbooks. Based on the profile plots of the data, general linear mixed models were used to evaluate the effectiveness of the programme while controlling for the effects of baseline levels of the response variables, change of socio-economic factors over time and correlations among repeated measurements from the same county. Redundant multicollinear variables were deleted from the mixed model using the results of the multicollinearity diagnoses. For each response variable, the best covariance structure was selected from 15 alternatives according to the fit statistics including Akaike information criterion, Finite-population corrected Akaike information criterion, and Schwarz.s Bayesian information criterion. Residual diagnostics were used to validate the model assumptions. Statistical inferences were made to show the effect of the programme on health service use and birth outcomes. A decision analytic model was developed to evaluate the cost-effectiveness of the programme, quantify the decision uncertainty, and estimate the expected value of perfect information associated with the decision. The model was used to describe the transitions between health states for women and infants and reflect the change of both costs and health benefits associated with implementing the programme. Result gained from the mixed models and other relevant evidence identified were synthesised appropriately to inform the input parameters of the model. Incremental cost-effectiveness ratios of the programme were calculated for the two groups of intervention counties over time. Uncertainty surrounding the parameters was dealt with using probabilistic sensitivity analysis, and uncertainty relating to model assumptions was handled using scenario analysis. Finally the expected value of perfect information for both the whole model and individual parameters in the model were estimated to inform priority setting in further research in this area.The annual change rates of the antenatal care rate and the institutionalised delivery rate were improved significantly in the intervention counties after the programme was implemented. Significant improvements were also found in the annual change rates of the maternal mortality ratio, the infant mortality rate, the incidence rate of neonatal tetanus and the mortality rate of neonatal tetanus in the intervention counties after the implementation of the programme. The annual change rate of the neonatal mortality rate was also improved, although the improvement was only close to statistical significance. The influences of the socio-economic factors on the health service use indicators and birth outcomes were identified. The rural income per capita had a significant positive impact on the health service use indicators, and a significant negative impact on the birth outcomes. The number of beds in healthcare institutions per 1,000 population and the number of rural telephone subscribers per 1,000 were found to be positively significantly related to the institutionalised delivery rate. The length of highway per square kilometre negatively influenced the maternal mortality ratio. The percentage of employed persons in the primary industry had a significant negative impact on the institutionalised delivery rate, and a significant positive impact on the infant mortality rate and neonatal mortality rate. The incremental costs of implementing the programme over the existing practice were US $11.1 million from the societal perspective, and US $13.8 million from the perspective of the Ministry of Health. Overall, 28,711 life years were generated by the programme, producing an overall incremental cost-effectiveness ratio of US $386 from the societal perspective, and US $480 from the perspective of the Ministry of Health, both of which were below the threshold willingness-to-pay ratio of US $675. The expected net monetary benefit generated by the programme was US $8.3 million from the societal perspective, and US $5.5 million from the perspective of the Ministry of Health. The overall probability that the programme was cost-effective was 0.93 and 0.89 from the two perspectives, respectively. The incremental cost-effectiveness ratio of the programme was insensitive to the different estimates of the three parameters relating to the model assumptions. Further research could be conducted to reduce the uncertainty surrounding the decision, in which the upper limit of investment was US $0.6 million from the societal perspective, and US $1.3 million from the perspective of the Ministry of Health. It is also worthwhile to get a more precise estimate of the improvement of infant mortality rate. The population expected value of perfect information for individual parameters associated with this parameter was US $0.99 million from the societal perspective, and US $1.14 million from the perspective of the Ministry of Health. The findings from this study have shown that the interventions in the Safe Motherhood Programme were both effective and cost-effective in increasing health service use and improving birth outcomes in rural areas of Guangxi, China. Therefore, the programme represents a good public health investment and should be adopted and further expanded to an even broader area if possible. This research provides economic evidence to inform efficient decision making in improving maternal and infant health in developing countries.

A formative evaluation of the Stronger Smarter Learning Communities Project. 2011 report.

This report is a formative evaluation of the operations of the DEEWR funded Stronger Smarter Learning Community (SSLC) project from September 2009 to July 2011. It is undertaken by an independent team of researchers from Queensland University of Technology, the University of Newcastle and Harvard University. It reports on findings from: documentary analysis; qualitative case studies of SSLC Hub schools; descriptive, multivariate and multilevel analysis of survey data from school leaders and teachers from SSLC Hub and Affiliate schools and from a control group of non-SSLC schools; and multilevel analysis of school-level data on SSLC Hubs, Affiliates and ACARA like-schools. Key findings from this work are that: • SSLC school leaders and teachers are reporting progress in changing school ethos around issues of: recognition of Indigenous identity, Indigenous leadership, innovative approaches to staffing and school models, Indigenous community engagement and high expectations leadership; • Many Stronger Smarter messages are reportedly having better uptake in schools with high percentages of Indigenous students; • There are no major or consistent patterns of differences between SSLC and non-SSLC schools in teacher and school leader self-reports of curriculum and pedagogy practices; and • There is no evidence to date that SSLC Hubs and Affiliates have increased attendance or increased achievement gains compared to ACARA like-schools. Twenty-one months is relatively early in this school reform project. Hence the major focus of subsequent reports will be on the documentation of comparative longitudinal gains in achievement tests and improved attendance. The 2011 and 2012 research also will model the relationships between change in school ethos/climate, changed Indigenous community relations, improved curriculum/pedagogy, and gains in Indigenous student achievement, attendance and outcomes. The key challenge for SSLC and the Stronger Smarter approach will be whether it can systematically generate change and reform in curriculum and pedagogy practices that can be empirically linked to improved student outcomes.

Managing financial implications for sustainable highway infrastructure delivery

Highway construction works have significant bearings on all aspects of sustainability. As they typically involve huge capital funds, stakeholders tend to place all interests on the financial justifications of the project, especially when embedding sustainability principles and practices may demand significant initial investment. Increasing public awareness and government policies demand that infrastructure projects respond to environmental challenges and people start to realise the negative consequences of not to pursue sustainability. Stakeholders are now keen to identify sustainable alternatives and financial implications of including them on a whole lifecycle basis. Therefore tools that aid the evaluation of investment options, such as provision of environmentally sustainable features in roads and highways, are highly desirable. Life-cycle cost analysis (LCCA) is generally recognised as a valuable approach for investment decision making for construction works. However to date it has limited application because the current LCCA models tend to focus on economic issues alone and are not able to deal with sustainability factors. This paper reports a research on identifying sustainability related factors in highway construction projects, in quantitative and qualitative forms of a multi-criteria analysis. These factors are then incorporated into existing LCCA models to produce a new sustainability based LCCA model with cost elements specific to sustainability measures. This presents highway project stakeholders a practical tool to evaluate investment decisions and reach an optimum balance between financial viability and sustainability deliverables.

The development of a conceptual framework for a district 4-Year Pavement Management Plan, implementation of network-level project screening process

The Texas Department of Transportation (TxDOT) is concerned about the widening gap between preservation needs and available funding. Funding levels are not adequate to meet the preservation needs of the roadway network; therefore projects listed in the 4-Year Pavement Management Plan must be ranked to determine which projects should be funded now and which can be postponed until a later year. Currently, each district uses locally developed methods to prioritize projects. These ranking methods have relied on less formal qualitative assessments based on engineers’ subjective judgment. It is important for TxDOT to have a 4-Year Pavement Management Plan that uses a transparent, rational project ranking process. The objective of this study is to develop a conceptual framework that describes the development of the 4-Year Pavement Management Plan. It can be largely divided into three Steps; 1) Network-Level project screening process, 2) Project-Level project ranking process, and 3) Economic Analysis. A rational pavement management procedure and a project ranking method accepted by districts and the TxDOT administration will maximize efficiency in budget allocations and will potentially help improve pavement condition. As a part of the implementation of the 4-Year Pavement Management Plan, the Network-Level Project Screening (NLPS) tool including the candidate project identification algorithm and the preliminary project ranking matrix was developed. The NLPS has been used by the Austin District Pavement Engineer (DPE) to evaluate PMIS (Pavement Management Information System) data and to prepare a preliminary list of candidate projects for further evaluation.

Evaluation of the control of carbon emissions of construction facilities in Hong Kong

This paper investigates the policies and instruments adopted in Hong Kong to control the carbon emissions of construction facilities, including the whole building life cycle: production of material stage, construction stage, operation stage and demolition stage. This commences with a literature review comparing activities world-wide to those in Hong Kong to identify the main issues at stake, followed by a report on a series of local interviews to evaluate the present situation in Hong Kong, as well as future opportunities for local carbon mitigation. The interviewees included practitioners from engineering contracting firms, consulting firms, clients and energy provider, together with two university experts and a counsellor. A small case study is also provided of a building project in Hong Kong to illustrate some of the innovative design aspects being incorporated into buildings in Hong Kong as a result of the current emphasis on sustainability. The paper concludes with a summary of main findings and proposals for improvement in policy related to carbon mitigation and building sustainability in Hong Kong.