Evaluation of a workplace alcohol and other drug policy

BACKGROUND The current impetus for developing alcohol and/or other drugs (AODs) workplace policies in Australia is to reduce workplace AOD impairment, improve safety, and prevent AOD-related injury in the workplace. For these policies to be effective, they need to be informed by scientific evidence. Evidence to inform the development and implementation of effective workplace AOD policies is currently lacking. There does not currently appear to be conclusive evidence for the effectiveness of workplace AOD policies in reducing impairment and preventing AOD-related injury. There is also no apparent evidence regarding which factors facilitate or impede the success of an AOD policy, or whether, for example, unsuccessful policy outcomes were due to poor policy or merely poor implementation of the policy. It was the aim of this research to undertake a process, impact, and outcome evaluation of a workplace AOD policy, and to contribute to the body of knowledge on the development and implementation of effective workplace AOD policies. METHODS The research setting was a state-based power-generating industry in Australia between May 2008 and May 2010. Participants for the process evaluation study were individuals who were integral to either the development or the implementation of the workplace AOD policy, or both of these processes (key informants), and comprised the majority of individuals who were involved in the process of developing and/or implementing the workplace AOD policy. The sample represented the two main groups of interest—management and union delegates/employee representatives—from all three of the participating organisations. For the impact and outcome evaluation studies, the population included all employees from the three participating organisations, and participants were all employees who consented to participate in the study and who completed both the pre-and post-policy implementation questionnaires. Qualitative methods in the form of interviews with key stakeholders were used to evaluate the process of developing and implementing the workplace AOD policy. In order to evaluate the impact of the policy with regard to the risk factors for workplace AOD impairment, and the outcome of the policy in terms of reducing workplace AOD impairment, quantitative methods in the form of a non-randomised single group pre- and post-test design were used. Changes from Time 1 (pre) to Time 2 (post) in the risk factors for workplace AOD impairment, and changes in the behaviour of interest—(self-reported) workplace AOD impairment—were measured. An integration of the findings from the process, impact, and outcome evaluation studies was undertaken using a combination of qualitative and quantitative methods. RESULTS For the process evaluation study Study respondents indicated that their policy was developed in the context of comparable industries across Australia developing workplace AOD policies, and that this was mainly out of concern for the deleterious health and safety impacts of workplace AOD impairment. Results from the process evaluation study also indicated that in developing and implementing the workplace AOD policy, there were mainly ‗winners', in terms of health and safety in the workplace. While there were some components of the development and implementation of the policy that were better done than others, and the process was expensive and took a long time, there were, overall, few unanticipated consequences to implementing the policy and it was reported to be thorough and of a high standard. Findings also indicated that overall the policy was developed and implemented according to best-practice in that: consultation during the policy development phase (with all the main stakeholders) was extensive; the policy was comprehensive; there was universal application of the policy to all employees; changes in the workplace (with regard to the policy) were gradual; and, the policy was publicised appropriately. Furthermore, study participants' responses indicated that the role of an independent external expert, who was trusted by all stakeholders, was integral to the success of the policy. For the impact and outcome evaluation studies Notwithstanding the limitations of pre- and post-test study designs with regard to attributing cause to the intervention, the findings from the impact evaluation study indicated that following policy implementation, statistically significant positive changes with regard to workplace AOD impairment were recorded for the following variables (risk factors for workplace AOD impairment): Knowledge; Attitudes; Perceived Behavioural Control; Perceptions of the Certainty of being punished for coming to work impaired by AODs; Perceptions of the Swiftness of punishment for coming to work impaired by AODs; and Direct and Indirect Experience with Punishment Avoidance for workplace AOD impairment. There were, however, no statistically significant positive changes following policy implementation for Behavioural Intentions, Subjective Norms, and Perceptions of the Severity of punishment for workplace AOD impairment. With regard to the outcome evaluation, there was a statistically significant reduction in self-reported workplace AOD impairment following the implementation of the policy. As with the impact evaluation, these findings need to be interpreted in light of the limitations of the study design in being able to attribute cause to the intervention alone. The findings from the outcome evaluation study also showed that while a positive change in self-reported workplace AOD impairment following implementation of the policy did not appear to be related to gender, age group, or employment type, it did appear to be related to levels of employee general alcohol use, cannabis use, site type, and employment role. Integration of the process, impact, and outcome evaluation studies There appeared to be qualitative support for the relationship between the process of developing and implementing the policy, and the impact of the policy in changing the risk factors for workplace AOD impairment. That is, overall the workplace AOD policy was developed and implemented well and, following its implementation, there were positive changes in the majority of measured risk factors for workplace AOD impairment. Quantitative findings lend further support for a relationship between the process and impact of the policy, in that there was a statistically significant association between employee perceived fidelity of the policy (related to the process of the policy) and positive changes in some risk factors for workplace AOD impairment (representing the impact of the policy). Findings also indicated support for the relationship between the impact of the policy in changing the risk factors for workplace AOD impairment and the outcome of the policy in reducing workplace AOD impairment: positive changes in the risk factors for workplace AOD impairment (impact) were related to positive changes in self reported workplace AOD impairment (representing the main goal and outcome of the policy). CONCLUSIONS The findings from the research indicate support for the conclusion that the policy was appropriately implemented and that it achieved its objectives and main goal. The Doctoral research findings also addressed a number of gaps in the literature on workplace AOD impairment, namely: the likely effectiveness of AOD policies for reducing AOD impairment in the workplace, which factors in the development and implementation of a workplace AOD policy are likely to facilitate or impede the effectiveness of the policy to reduce workplace AOD impairment, and which employee groups are less likely to respond well to policies of this type. The findings from this research not only represent an example of translational, applied research—through the evaluation of the study industry's policy—but also add to the body of knowledge on workplace AOD policies and provide policy-makers with evidence which may be useful in the development and implementation of effective workplace AOD policies. Importantly, the findings espouse the importance of scientific evidence in the development, implementation, and evaluation of workplace AOD policies.

A review of project evaluation methodologies to address net impacts and risks of toll road projects to the community

Project evaluation is a process of measuring costs, benefits, risks and uncertainties for the purpose of decision-making by estimating and assessing impacts of the project to the community. The effects of impacts of toll roads are similar but different from the general non-tolled roads. Project evaluation methodologies are extensively studied and applied to various transport infrastructure projects. However, there is no definitive methodology to evaluate toll roads. This review discusses the impacts of toll roads then reviews the limitations of existing project evaluation methodologies when evaluating toll road impacts. The review identified gaps of knowledge of toll evaluations. First, the treatment of toll in project evaluation, particularly in Cost-Benefit Analysis requires further study to explore the appropriate methodology. Secondly, the project evaluation methodology needs to place strong emphasis on empirically based risk and uncertainty assessment. Addressing the limitations of the existing project evaluation methodologies leads to improvements of the methodology in practical level as well as fills the gap of knowledge of project evaluation for toll roads with respect to net impacts to the community.

A review of project evaluation methodologies for toll road projects

Project evaluation is a process of measuring costs, benefits, risks and uncertainties for the purpose of decision-making by estimating and assessing impacts of the project to the community. The effects of impacts of toll roads are similar but different from the general non-tolled roads. Project evaluation methodologies are extensively studied and applied to various transport infrastructure projects. However, there is no definitive methodology to evaluate toll roads. This review discusses the impacts of toll roads then reviews the limitations of existing project evaluation methodologies when evaluating toll road impacts. The review identified gaps of knowledge of toll evaluations. First, the treatment of toll in project evaluation, particularly in Cost-Benefit Analysis requires further study to explore the appropriate methodology. Secondly, the project evaluation methodology needs to place strong emphasis on empirically based risk and uncertainty assessment. Addressing the limitations of the existing project evaluation methodologies leads to improvements of the methodology in practical level as well as fills the gap of knowledge of project evaluation for toll roads with respect to net impacts to the community.

Evaluation of tomato production systems at Bowen in mitigating the risk of fruit fly infestation: a systems approach for market access

Queensland fruit flies Bactrocera tryoni and B. neohumeralis are considered major quarantine pests of tomato, a major crop in the horticultural production district around Bowen, North Queensland, Australia. Preharvest and/or postharvest treatments are required to meet the market access requirements of both domestic and international trading partners. The suspension from use of dimethoate and fenthion, the two insecticides used for fruit fly control, has resulted in the loss of both pre and postharvest uses in fresh tomato. Research undertaken quantitatively at Bowen evaluated the effectiveness of pre-harvest production systems without specific fruit fly controls and postharvest mitigation measures in reducing the risk of fruit fly infestation in tomato. A district-wide trapping using cue-lure baited traps was undertaken to determine fruit fly seasonal patterns in relation to the cropping seasons. A total of 17,626 field-harvested and 11,755 pack-house tomatoes were sampled from ten farms over three cropping seasons (2006-2009). The fruit were incubated and examined for fruit fly infestation. No fruit fly infested fruit were recorded over the three seasons in either the field or the pack-house samples. Statistical analyses showed that upper infestation levels were extremely low (between 0.025 and 0.062%) at the 95% confidence level. The trap catches showed a seasonal pattern in fruit fly activity, with low numbers during the autumn and winter months, rising slightly in spring and peaking in summer. This seasonal pattern was similar over the four seasons. The main two species of fruit fly caught were B. tryoni and B. neohumeralis. Based on the results, it is clear that the risk of fruit fly infestation is extremely low under the current production systems in the Bowen region.

Structured systems methodology for evaluation of random interruptions in continuous process type manufacturing systems

A structured systems methodology was developed to analyse the problems of production interruptions occurring at random intervals in continuous process type manufacturing systems. At a macro level the methodology focuses on identifying suitable investment policies to reduce interruptions of a total manufacturing system that is a combination of several process plants. An interruption-tree-based simulation model was developed for macroanalysis. At a micro level the methodology focuses on finding the effects of alternative configurations of individual process plants on the overall system performance. A Markov simulation model was developed for microlevel analysis. The methodology was tested with an industry-specific application.

Structural Design under Seismic Risk Using Multiple Performance Objectives

Structural design is a decision-making process in which a wide spectrum of requirements, expectations, and concerns needs to be properly addressed. Engineering design criteria are considered together with societal and client preferences, and most of these design objectives are affected by the uncertainties surrounding a design. Therefore, realistic design frameworks must be able to handle multiple performance objectives and incorporate uncertainties from numerous sources into the process.

In this study, a multi-criteria based design framework for structural design under seismic risk is explored. The emphasis is on reliability-based performance objectives and their interaction with economic objectives. The framework has analysis, evaluation, and revision stages. In the probabilistic response analysis, seismic loading uncertainties as well as modeling uncertainties are incorporated. For evaluation, two approaches are suggested: one based on preference aggregation and the other based on socio-economics. Both implementations of the general framework are illustrated with simple but informative design examples to explore the basic features of the framework.

The first approach uses concepts similar to those found in multi-criteria decision theory, and directly combines reliability-based objectives with others. This approach is implemented in a single-stage design procedure. In the socio-economics based approach, a two-stage design procedure is recommended in which societal preferences are treated through reliability-based engineering performance measures, but emphasis is also given to economic objectives because these are especially important to the structural designer's client. A rational net asset value formulation including losses from uncertain future earthquakes is used to assess the economic performance of a design. A recently developed assembly-based vulnerability analysis is incorporated into the loss estimation.

The presented performance-based design framework allows investigation of various design issues and their impact on a structural design. It is a flexible one that readily allows incorporation of new methods and concepts in seismic hazard specification, structural analysis, and loss estimation.

A Delphi Process to Optimize Quality and Performance of Drug Evaluation in Neonates

Background: Neonatal trials remain difficult to conduct for several reasons: in particular the need for study sites to have an existing infrastructure in place, with trained investigators and validated quality procedures to ensure good clinical, laboratory practices and a respect for high ethical standards. The objective of this work was to identify the major criteria considered necessary for selecting neonatal intensive care units that are able to perform drug evaluations competently. Methodology and Main Findings: This Delphi process was conducted with an international multidisciplinary panel of 25 experts from 13 countries, selected to be part of two committees (a scientific committee and an expert committee), in order to validate criteria required to perform drug evaluation in neonates. Eighty six items were initially selected and classified under 7 headings: "NICUs description - Level of care'' (21), "Ability to perform drug trials: NICU organization and processes (15), "Research Experience'' (12), "Scientific competencies and area of expertise'' (8), "Quality Management'' (16), "Training and educational capacity'' (8) and "Public involvement'' (6). Sixty-one items were retained and headings were rearranged after the first round, 34 were selected after the second round. A third round was required to validate 13 additional items. The final set includes 47 items divided under 5 headings. Conclusion: A set of 47 relevant criteria will help to NICUs that want to implement, conduct or participate in drug trials within a neonatal network identify important issues to be aware of. Summary Points: 1) Neonatal trials remain difficult to conduct for several reasons: in particular the need for study sites to have an existing infrastructure in place, with trained investigators and validated quality procedures to ensure good clinical, laboratory practices and a respect for high ethical standards. 2) The present Delphi study was conducted with an international multidisciplinary panel of 25 experts from 13 countries and aims to identify the major criteria considered necessary for selecting neonatal intensive care units (NICUs) that are able to perform drug evaluations competently. 3) Of the 86 items initially selected and classified under 7 headings - "NICUs description - Level of care'' (21), "Ability to perform drug trials: NICU organization and processes (15), "Research Experience'' (12), "Scientific competencies and area of expertise'' (8), "Quality Management'' (16), "Training and educational capacity'' (8) and "Public involvement'' (6) - 47 items were selected following a three rounds Delphi process. 4) The present consensus will help NICUs to implement, conduct or participate in drug trials within a neonatal network.

Design, construction and evaluation of systems to predict risk in obstetrics.

We present a systematic, practical approach to developing risk prediction systems, suitable for use with large databases of medical information. An important part of this approach is a novel feature selection algorithm which uses the area under the receiver operating characteristic (ROC) curve to measure the expected discriminative power of different sets of predictor variables. We describe this algorithm and use it to select variables to predict risk of a specific adverse pregnancy outcome: failure to progress in labour. Neural network, logistic regression and hierarchical Bayesian risk prediction models are constructed, all of which achieve close to the limit of performance attainable on this prediction task. We show that better prediction performance requires more discriminative clinical information rather than improved modelling techniques. It is also shown that better diagnostic criteria in clinical records would greatly assist the development of systems to predict risk in pregnancy.