Dismantling the public sector bastion : evaluation capital works

Purpose: The determination of the most appropriate procurement system for a capital works project is a challenging task for public sector clients considering the array of assessment criteria that are considered and the procurement methods that are available. This is particularly pertinent to the Western Australian public sector where they have had a propensity to use traditional lump sum as the default procurement solution despite knowing that the selection of an inappropriate procurement method may lead to cost and time overruns, claims, and disputes’ on projects. This paper presents a six step procurement method evaluation approach that requires public sector agencies to consider in detail an array of options so as to obtain value for money. Design/methodology/approach: A procurement evaluation approach is developed and is examined using a focus group of 12 participants comprising of a public sector client, project team and key stakeholders. The focus group was used to examine the developed approach in the context of a real-life capital works project. Findings: The procurement method evaluation approach was deemed to be pragmatic and enabled decision-makers to re-evaluate outcomes from previous steps in the process. All focus group participants stated the six step process enabled a recommendation that was grounded in reflection and detailed evaluation. Practical implications: The developed procurement approach has enabled the public sector client evaluate the way in which they view procurement method selection and examine how they obtain ‘value for money’. Originality/value: The six step procurement approach makes use of quantitative and qualitative techniques and is reliant on discourse and reflection in making a procurement method recommendation. Consequently, the approach enables public sector clients to account for the complexities often associated with procurement selection.

Changing motorcycle rider safety attitudes and motives for risk taking : process evaluation of a rider training intervention

Risk-taking behaviour by motorcyclists has been shown to contribute to a substantial proportion of road crashes in Australia and abroad. Concern has been expressed that traditional motorcycle licence training programs do not sufficiently address such behaviour. Accordingly, the Three Steps to Safer Riding program was developed to address risk taking behaviour by riders as an adjunct to existing skills-based rider training. The program was designed to be delivered in a one hour classroom session at the start of training, with a 20 minute debrief to revise the key concepts at the end of training. This paper reports on the key training concepts, methodology and implementation of the pilot program with a major rider training organisation in Queensland and presents findings from a process evaluation. The Three Steps to Safer Riding intervention pilot was delivered to 518 learner riders over a three month period. Follow-up focus groups and one interview with intervention participants (N=18) five to eight months after completion of the program suggest that new riders (absolute novices) embraced and internalised many of the intervention concepts. However, some riders who had previous riding experience prior to training stated these issues were common sense, yet still expressed riding styles that were contrary to some of the key intervention messages. This is discussed in terms of raising awareness of risk issues for motorcyclists versus behaviour change. Additionally, interviews conducted with riding instructors are discussed regarding logistical challenges of implementation, training consistency, skills required to deliver the program, support for the program, and student engagement.

Adiabatic compression testing - part II : background and approach to estimating severity of test methodology

Adiabatic compression testing of components in gaseous oxygen is a test method that is utilized worldwide and is commonly required to qualify a component for ignition tolerance under its intended service. This testing is required by many industry standards organizations and government agencies; however, a thorough evaluation of the test parameters and test system influences on the thermal energy produced during the test has not yet been performed. This paper presents a background for adiabatic compression testing and discusses an approach to estimating potential differences in the thermal profiles produced by different test laboratories. A “Thermal Profile Test Fixture” (TPTF) is described that is capable of measuring and characterizing the thermal energy for a typical pressure shock by any test system. The test systems at Wendell Hull & Associates, Inc. (WHA) in the USA and at the BAM Federal Institute for Materials Research and Testing in Germany are compared in this manner and some of the data obtained is presented. The paper also introduces a new way of comparing the test method to idealized processes to perform system-by-system comparisons. Thus, the paper introduces an “Idealized Severity Index” (ISI) of the thermal energy to characterize a rapid pressure surge. From the TPTF data a “Test Severity Index” (TSI) can also be calculated so that the thermal energies developed by different test systems can be compared to each other and to the ISI for the equivalent isentropic process. Finally, a “Service Severity Index” (SSI) is introduced to characterizing the thermal energy of actual service conditions. This paper is the second in a series of publications planned on the subject of adiabatic compression testing.

“Filling out the forms was a nightmare” : project evaluation and the reflective practitioner in community theatre in contemporary Northern Ireland

Since the Good Friday Agreement of 1998, large sums have been invested in community theatre projects in Northern Ireland, in the interests of conflict transformation and peace building. While this injection of funds has resulted in an unprecedented level of applied theatre activity, opportunities to maximise learning from this activity are being missed. It is generally assumed that project evaluation is undertaken at least partly to assess the degree of success of projects against important social objectives, with a view to learning what works, what does not, and what might work in the future. However, three ethnographic case studies of organisations delivering applied theatre projects in Northern Ireland indicate that current processes used to evaluate such projects are both flawed and inadequate for this purpose. Practitioners report that the administrative work involved in applying for and justifying funding is onerous, burdensome, and occurs at the expense of artistic activity. This is a very real concern when the time and effort devoted to ‘filling out the forms’ does not ultimately result in useful evaluative information. There are strong disincentives for organisations to report honestly on their experiences of difficulties, or undesirable impacts of projects, and this problem is not transcended by the use of external evaluators. Current evaluation processes provide little opportunity to capture unexpected benefits of projects, and small but significant successes which occur in the context of over-ambitious objectives. Little or no attempt is made to assess long-term impacts of projects on communities. Finally, official evaluation mechanisms fail to capture the reflective practice and dialogic analysis of practitioners, which would richly inform future projects. The authors argue that there is a need for clearer lines of communication, and more opportunities for mutual learning, among stakeholders involved in community development. In particular, greater involvement of the higher education sector in partnership with government and non-government agencies could yield significant benefits in terms of optimizing learning from applied theatre project evaluations.

Dragline automation: experimental evaluation through productivity trial

Since 1993 we have been working on the automation of dragline excavators, the largest earthmoving machines that exist. Recently we completed a large-scale experimental program where the automation system was used for production purposes over a two week period and moved over 200,000 tonnes of overburden. This is a landmark achievement in the history of automated excavation. In this paper we briefly describe the robotic system and how it works cooperatively with the machine operator. We then describe our methodology for gauging machine performance, analyze results from the production trial and comment on the effectiveness of the system that we have created. © Springer-Verlag Berlin Heidelberg 2006.

Development and evaluation of hydrocarbon sorbent materials with the aid of chemometrics

Hydrocarbon spills on roads are a major safety concern for the driving public and can have severe cost impacts both on pavement maintenance and to the economy through disruption to services. The time taken to clean-up spills and re-open roads in a safe driving condition is an issue of increasing concern given traffic levels on major urban arterials. Thus, the primary aim of the research was to develop a sorbent material that facilitates rapid clean-up of road spills. The methodology involved extensive research into a range of materials (organic, inorganic and synthetic sorbents), comprehensive testing in the laboratory, scale-up and field, and product design (i.e. concept to prototype). The study also applied chemometrics to provide consistent, comparative methods of sorbent evaluation and performance. In addition, sorbent materials at every stage were compared against a commercial benchmark. For the first time, the impact of diesel on asphalt pavement has been quantified and assessed in a systematic way. Contrary to conventional thinking and anecdotal observations, the study determined that the action of diesel on asphalt was quite rapid (i.e. hours rather than weeks or months). This significant finding demonstrates the need to minimise the impact of hydrocarbon spills and the potential application of the sorbent option. To better understand the adsorption phenomenon, surface characterisation techniques were applied to selected sorbent materials (i.e. sand, organo-clay and cotton fibre). Brunauer Emmett Teller (BET) and thermal analysis indicated that the main adsorption mechanism for the sorbents occurred on the external surface of the material in the diffusion region (sand and organo-clay) and/or capillaries (cotton fibre). Using environmental scanning electron microscopy (ESEM), it was observed that adsorption by the interfibre capillaries contributed to the high uptake of hydrocarbons by the cotton fibre. Understanding the adsorption mechanism for these sorbents provided some guidance and scientific basis for the selection of materials. The study determined that non-woven cotton mats were ideal sorbent materials for clean-up of hydrocarbon spills. The prototype sorbent was found to perform significantly better than the commercial benchmark, displaying the following key properties: • superior hydrocarbon pick-up from the road pavement; • high hydrocarbon retention capacity under an applied load; • adequate field skid resistance post treatment; • functional and easy to use in the field (e.g. routine handling, transportation, application and recovery); • relatively inexpensive to produce due to the use of raw cotton fibre and simple production process; • environmentally friendly (e.g. renewable materials, non-toxic to environment and operators, and biodegradable); and • rapid response time (e.g. two minutes total clean-up time compared with thirty minutes for reference sorbents). The major outcomes of the research project include: a) development of a specifically designed sorbent material suitable for cleaning up hydrocarbon spills on roads; b) submission of patent application (serial number AU2005905850) for the prototype product; and c) preparation of Commercialisation Strategy to advance the sorbent product to the next phase (i.e. R&D to product commercialisation).

Evaluation of accessibility for knowledge-based cities

Purpose: The purpose of the paper is to develop a framework for evaluation of accessibility for knowledge based cities. ----- ----- Design/methodology/approach: This approach notifies common mistakes and problems in accessibility assessment for knowledge cities. ----- ----- Originality/value: Accessibility plays a key role in transport sustainability and recognizes the crucial links between transport and sustainable goals like air quality, environmental resource consumption & social equity. In knowledge cities, accessibility has significant effects on quality of life and social equity by improving the mobility of people and goods. Accessibility also influences patterns of growth and economic health by providing access to land. Accessibility is not only one of the components of knowledge cities but also affects other elements of knowledge cities directly or indirectly. ----- ----- Practical implications: The outcomes of the application will be helpful for developing particular methodologies for evaluating knowledge cities. On other words, this methodology attempts to develop an assessment procedure for examining accessibility of knowledge-based cities.

A comparative review of quantitative VFM methodology for PPP infrastructure project in Indonesia and Australia

The Government of Indonesia (GoI) increasingly relies on the private sector financing to build and operate infrastructures through public private partnership (PPP) schemes. However, PPP does not automatically provide the solution for the financing scheme due to value for money (VFM) issues. The procurement authority must show whether a PPP proposal is the optimal solution that provides best VFM outcome. The paper presents a literature review of comparing quantitative VFM methodology for PPP infrastructure project procurement in Indonesia and Australia. Public Sector Comparator (PSC) is used to assess the potential project VFM quantitatively in Australia. In Indonesia, the PSC has not been applied, where the PPP procurement authority tends to utilize a common project evaluation method that ignores the issues of risk. Unlike the conventional price bid evaluation, the PSC enables a financial comparison including costs/gains and risks. Since the construction of PSC is primarily on risk management approach, it can facilitate risk negotiation processes between the involved parties. The study indicates that the quantitative VFM methodology of PSC is potentially applicable in Indonesia for water supply sector. Various supporting regulations are available that emphasize the importance of VFM and risk management in infrastructure investment. However, the study also reveals a number of challenges that need to be anticipated, such as the need of a more comprehensive PPP policy at both central and local government level, a more specific legal instrument for bidding evaluation method and the issue of institutional capacity development in PPP Units at the local level.

Redland Youth Space development: evaluation report

This report provides an evaluation of the Capalaba Youth Space.The evaluation included elements of process and impact evaluation and used a participatory action research approach informed by engagement processes, focus groups, a community survey, interviews and secondary analysis of existing data.

Real-time evaluation of driver’s alertness on highways

Suburbanisation has been internationally a major phenomenon in the last decades. Suburb-to-suburb routes are nowadays the most widespread road journeys; and this resulted in an increment of distances travelled, particularly on faster suburban highways. The design of highways tends to over-simplify the driving task and this can result in decreased alertness. Driving behaviour is consequently impaired and drivers are then more likely to be involved in road crashes. This is particularly dangerous on highways where the speed limit is high. While effective countermeasures to this decrement in alertness do not currently exist, the development of in-vehicle sensors opens avenues for monitoring driving behaviour in real-time. The aim of this study is to evaluate in real-time the level of alertness of the driver through surrogate measures that can be collected from in-vehicle sensors. Slow EEG activity is used as a reference to evaluate driver's alertness. Data are collected in a driving simulator instrumented with an eye tracking system, a heart rate monitor and an electrodermal activity device (N=25 participants). Four different types of highways (driving scenario of 40 minutes each) are implemented through the variation of the road design (amount of curves and hills) and the roadside environment (amount of buildings and traffic). We show with Neural Networks that reduced alertness can be detected in real-time with an accuracy of 92% using lane positioning, steering wheel movement, head rotation, blink frequency, heart rate variability and skin conductance level. Such results show that it is possible to assess driver's alertness with surrogate measures. Such methodology could be used to warn drivers of their alertness level through the development of an in-vehicle device monitoring in real-time drivers' behaviour on highways, and therefore it could result in improved road safety.

Evaluation of financial incentives as a quality improvement strategy in the public hospital context : clinician attitudes, design variables, and economic costs

In 2008, a three-year pilot ‘pay for performance’ (P4P) program, known as ‘Clinical Practice Improvement Payment’ (CPIP) was introduced into Queensland Health (QHealth). QHealth is a large public health sector provider of acute, community, and public health services in Queensland, Australia. The organisation has recently embarked on a significant reform agenda including a review of existing funding arrangements (Duckett et al., 2008). Partly in response to this reform agenda, a casemix funding model has been implemented to reconnect health care funding with outcomes. CPIP was conceptualised as a performance-based scheme that rewarded quality with financial incentives. This is the first time such a scheme has been implemented into the public health sector in Australia with a focus on rewarding quality, and it is unique in that it has a large state-wide focus and includes 15 Districts. CPIP initially targeted five acute and community clinical areas including Mental Health, Discharge Medication, Emergency Department, Chronic Obstructive Pulmonary Disease, and Stroke. The CPIP scheme was designed around key concepts including the identification of clinical indicators that met the set criteria of: high disease burden, a well defined single diagnostic group or intervention, significant variations in clinical outcomes and/or practices, a good evidence, and clinician control and support (Ward, Daniels, Walker & Duckett, 2007). This evaluative research targeted Phase One of implementation of the CPIP scheme from January 2008 to March 2009. A formative evaluation utilising a mixed methodology and complementarity analysis was undertaken. The research involved three research questions and aimed to determine the knowledge, understanding, and attitudes of clinicians; identify improvements to the design, administration, and monitoring of CPIP; and determine the financial and economic costs of the scheme. Three key studies were undertaken to ascertain responses to the key research questions. Firstly, a survey of clinicians was undertaken to examine levels of knowledge and understanding and their attitudes to the scheme. Secondly, the study sought to apply Statistical Process Control (SPC) to the process indicators to assess if this enhanced the scheme and a third study examined a simple economic cost analysis. The CPIP Survey of clinicians elicited 192 clinician respondents. Over 70% of these respondents were supportive of the continuation of the CPIP scheme. This finding was also supported by the results of a quantitative altitude survey that identified positive attitudes in 6 of the 7 domains-including impact, awareness and understanding and clinical relevance, all being scored positive across the combined respondent group. SPC as a trending tool may play an important role in the early identification of indicator weakness for the CPIP scheme. This evaluative research study supports a previously identified need in the literature for a phased introduction of Pay for Performance (P4P) type programs. It further highlights the value of undertaking a formal risk assessment of clinician, management, and systemic levels of literacy and competency with measurement and monitoring of quality prior to a phased implementation. This phasing can then be guided by a P4P Design Variable Matrix which provides a selection of program design options such as indicator target and payment mechanisms. It became evident that a clear process is required to standardise how clinical indicators evolve over time and direct movement towards more rigorous ‘pay for performance’ targets and the development of an optimal funding model. Use of this matrix will enable the scheme to mature and build the literacy and competency of clinicians and the organisation as implementation progresses. Furthermore, the research identified that CPIP created a spotlight on clinical indicators and incentive payments of over five million from a potential ten million was secured across the five clinical areas in the first 15 months of the scheme. This indicates that quality was rewarded in the new QHealth funding model, and despite issues being identified with the payment mechanism, funding was distributed. The economic model used identified a relative low cost of reporting (under $8,000) as opposed to funds secured of over $300,000 for mental health as an example. Movement to a full cost effectiveness study of CPIP is supported. Overall the introduction of the CPIP scheme into QHealth has been a positive and effective strategy for engaging clinicians in quality and has been the catalyst for the identification and monitoring of valuable clinical process indicators. This research has highlighted that clinicians are supportive of the scheme in general; however, there are some significant risks that include the functioning of the CPIP payment mechanism. Given clinician support for the use of a pay–for-performance methodology in QHealth, the CPIP scheme has the potential to be a powerful addition to a multi-faceted suite of quality improvement initiatives within QHealth.

Development and evaluation of an integrated clinical learning model to inform continuing education for acute care nurses

Background Significant ongoing learning needs for nurses have occurred as a direct result of the continuous introduction of technological innovations and research developments in the healthcare environment. Despite an increased worldwide emphasis on the importance of continuing education, there continues to be an absence of empirical evidence of program and session effectiveness. Few studies determine whether continuing education enhances or develops practice and the relative cost benefits of health professionals’ participation in professional development. The implications for future clinical practice and associated educational approaches to meet the needs of an increasingly diverse multigenerational and multicultural workforce are also not well documented. There is minimal research confirming that continuing education programs contribute to improved patient outcomes, nurses’ earlier detection of patient deterioration or that standards of continuing competence are maintained. Crucially, evidence-based practice is demonstrated and international quality and safety benchmarks are adhered to. An integrated clinical learning model was developed to inform ongoing education for acute care nurses. Educational strategies included the use of integrated learning approaches, interactive teaching concepts and learner-centred pedagogies. A Respiratory Skills Update education (ReSKU) program was used as the content for the educational intervention to inform surgical nurses’ clinical practice in the area of respiratory assessment. The aim of the research was to evaluate the effectiveness of implementing the ReSKU program using teaching and learning strategies, in the context of organisational utility, on improving surgical nurses’ practice in the area of respiratory assessment. The education program aimed to facilitate better awareness, knowledge and understanding of respiratory dysfunction in the postoperative clinical environment. This research was guided by the work of Forneris (2004), who developed a theoretical framework to operationalise a critical thinking process incorporating the complexities of the clinical context. The framework used educational strategies that are learner-centred and participatory. These strategies aimed to engage the clinician in dynamic thinking processes in clinical practice situations guided by coaches and educators. Methods A quasi experimental pre test, post test non–equivalent control group design was used to evaluate the impact of the ReSKU program on the clinical practice of surgical nurses. The research tested the hypothesis that participation in the ReSKU program improves the reported beliefs and attitudes of surgical nurses, increases their knowledge and reported use of respiratory assessment skills. The study was conducted in a 400 bed regional referral public hospital, the central hub of three smaller hospitals, in a health district servicing the coastal and hinterland areas north of Brisbane. The sample included 90 nurses working in the three surgical wards eligible for inclusion in the study. The experimental group consisted of 36 surgical nurses who had chosen to attend the ReSKU program and consented to be part of the study intervention group. The comparison group included the 39 surgical nurses who elected not to attend the ReSKU program, but agreed to participate in the study. Findings One of the most notable findings was that nurses choosing not to participate were older, more experienced and less well educated. The data demonstrated that there was a barrier for training which impacted on educational strategies as this mature aged cohort was less likely to take up educational opportunities. The study demonstrated statistically significant differences between groups regarding reported use of respiratory skills, three months after ReSKU program attendance. Between group data analysis indicated that the intervention group’s reported beliefs and attitudes pertaining to subscale descriptors showed statistically significant differences in three of the six subscales following attendance at the ReSKU program. These subscales included influence on nursing care, educational preparation and clinical development. Findings suggest that the use of an integrated educational model underpinned by a robust theoretical framework is a strong factor in some perceptions of the ReSKU program relating to attitudes and behaviour. There were minimal differences in knowledge between groups across time. Conclusions This study was consistent with contemporary educational approaches using multi-modal, interactive teaching strategies and a robust overarching theoretical framework to support study concepts. The construct of critical thinking in the clinical context, combined with clinical reasoning and purposeful and collective reflection, was a powerful educational strategy to enhance competency and capability in clinicians.

Using participatory action research in a community-based initiative addressing complex mental health needs

Objective: This paper describes the first phase of a larger project that utilizes participatory action research to examine complex mental health needs across an extensive group of stakeholders in the community. Method: Within an objective qualitative analysis of focus group discussions the social ecological model is utilized to explore how integrative activities can be informed, planned and implemented across multiple elements and levels of a system. Seventy-one primary care workers, managers, policy-makers, consumers and carers from across the southern metropolitan and Gippsland regions of Victoria, Australia took part in seven focus groups. All groups responded to an identical set of focusing questions. Results: Participants produced an explanatory model describing the service system, as it relates to people with complex needs, across the levels of social ecological analysis. Qualitative themes analysis identified four priority areas to be addressed in order to improve the system's capacity for working with complexity. These included: (i) system fragmentation; (ii) integrative case management practices; (iii) community attitudes; and (iv) money and resources. Conclusions: The emergent themes provide clues as to how complexity is constructed and interpreted across the system of involved agencies and interest groups. The implications these findings have for the development and evaluation of this community capacity-building project were examined from the perspective of constructing interventions that address both top-down and bottom-up processes.

A methodology to assess the competitiveness of real estate developers in China

Real estate, or property development, is considered one of the pillar industries of the Chinese economy. As a result of the opening up of the economy as well as the "macro-control" policy of the Central Chinese Government to moderate the frenetic pace of growth of the economy, the real estate industry has faced fierce competition and ongoing change. Real estate firms in China must improve their competitiveness in order to maintain market share or even survive in this brutally competitive environment. This study developed a methodology to evaluate the competitiveness of real estate developers in the China and then used a case study to illustrate the effectiveness of the evaluation method. Four steps were taken to achieve this. The first step was to conduct a thorough literature review which included a review of the characteristics of real estate industry, theories about competitiveness and the competitive characteristics of real estate developers. Following this literature review, the competitive model was developed based on seven key competitive factors (the 'level 1') identified in the literature. They include: (1) financial competency; (2) market share; (3) management competency; (4) social responsibility; (5) organisational competency; (6) technological capabilities; and, (7) regional competitiveness. In the next step of research, the competitive evaluation criteria (the 'level 2') under each of competitive factors (the 'level 1') were evaluated. Additionally, there were identified a set of competitive attributes (the 'level 3') under each competitive criteria (the 'level 2'). These attributes were initially recognised during the literature review and then expanded upon through interviews with multidisciplinary experts and practitioners in various real estate-related industries. The final step in this research was to undertake a case study using the proposed evaluation method and attributes. Through the study of an actual real estate development company, the procedures and effectiveness of the evaluation method were illustrated and validated. Through the above steps, this research investigates and develops an analytical system for determining the corporate competitiveness of real estate developers in China. The analytical system is formulated to evaluate the "state of health" of the business from different competitive perspectives. The result of empirical study illustrates that a systematic and structured evaluation can effectively assist developers in identifying their strengths and highlighting potential problems. This is very important for the development of an overall corporate strategy and supporting key strategic decisions. This study also provides some insights, analysis and suggestions for improving the competitiveness of real estate developers in China from different perspectives, including: management competency, organisational competency, technological capabilities, financial competency, market share, social responsibility and regional competitiveness. In the case study, problems were found in each of these areas, and they appear to be common in the industry. To address these problems and improve the competitiveness and effectiveness of Chinese real estate developers, a variety of suggestions are proposed. The findings of this research provide an insight into the factors that influence competitiveness in the Chinese real estate industry while also assisting practitioners to formulate strategies to improve their competitiveness. References for studying the competitiveness of real estate developers in other countries are also provided.