A review of evidence-based evaluation of measures for assessing social and emotional well-being in indigenous Australians

Addressing low levels of social and emotional well-being (SEWB) in Indigenous communities has been a national strategic priority for over 10 years and yet progress in assessing the impact of interventions has been slow. One of the key factors limiting the development of evidence-based practice has been the lack of well-validated instruments to assess SEWB and how it changes over time as a result of intervention. This article systematically reviews available measures, classifying them in terms of the evidence base that exists to support their use. It is concluded that there is an ongoing need to develop psychometrically sound, comprehensive, culturally appropriate measures to operationalise Indigenous SEWB at a population health, programme evaluation, and clinical level. It is suggested that seven pathways be followed to achieve this goal, including the need to recognise that the gold standard status for Indigenous measurement tools cannot be ascribed based on evidence-based assessment criteria alone.

Theorising about gender and computing interventions through an evaluation framework

Despite over 20 years of intervention programmes, the gender balance in the computing profession is not improving. It has been suggested that the problem with much of the research on gender and computing is that it is under-theorised. The contribution of this study is an evaluation framework, designed to evaluate gender and computing interventions, which will advance the theorisation of research through programme evaluation. This study was undertaken in three phases: The first involved theory building through an extensive review of the literature resulting in a conceptual framework for intervention programme evaluation. The second phase consisted of a multiple-case study of 14 major intervention programmes in Australia. Subsequent modifications to the conceptual framework resulted in the gender and computing intervention evaluation framework. The value of the framework was confirmed in phase three by intervention experts and showed that applying the framework will help programme champions to evaluate their programmes more thoroughly. The dissemination of sound evaluation results will then enable a deeper theorisation of the issues surrounding gender and computing interventions.

Peer mentoring in an accounting setting : a case study of mentor skill development

This paper provides the background to the adoption of a peer mentoring program including the preparation of mentors and the classroom role of mentors. The paper also includes a discussion of alternative models of peer mentoring and an examination of factors to be considered in adopting mentoring as a teaching and learning resource. Using a case study of third-year undergraduate accounting students who were employed to mentor fellow students in a second-year accounting unit at an Australian university, the paper examines peer mentoring as a classroom resource in the teaching of accounting concepts. The study includes an evaluation of the perceptions of generic skill development by peer mentors illustrating how mentoring can be a reciprocal process, in that the mentor's skills can be enhanced while fulfilling a supporting role to junior students. The programme evaluation also demonstrated that accounting educators could contribute to the development of students' generic skills outside the traditional classroom setting.

Distant relations : limits to relational contracting in domestic violence programmes

Purpose – The purpose of this paper is to assess the applicability of relational contract theory in situations where government departments contract with non-government welfare organisations to deliver human service programmes. Its limits are highlighted by an assessment of programmes for domestically violent men that epitomise “management of incomplete contracts” central to the theory.

Design/methodology/approach – The paper is based on an evaluation of contracted-out programmes for perpetrators of domestic violence in Australia that set out to compare and contrast distinct models of service delivery by documenting programme logic, service delivery effectiveness and effects on programme participants. It reflects on the difficulties of monitoring such programmes and considers the implications of this for contracting theory and for human service practice.

Findings – In contrast to critiques of contracting-out in a neo-liberal environment that emphasise how accountability and reporting requirements limit the autonomy of contracted agencies, this paper highlights considerable variation in how programmes were managed and delivered despite standardised service delivery contracts developed by the government department funding the programmes. This leads to a consideration of “incomplete contracts” where service delivery outcomes are hard to measure or there is limited knowledge of the contracted agencies by the contracting government department.

Originality/value – The paper highlights a situation in which the recommendations of relational contracting theory can exacerbate the difficulties of quality assurance rather than minimise them. It then argues a need for workforce development in the government departments and the contracted agencies, to enable a nuanced monitoring of the programmes' service delivery and promotion of quality assurance processes.

Major quantitative techniques for risk analysis in the construction industry : a comparative analysis

Risk analysis is one of the critical functions of the risk management process. It relies on a detailed understanding of risks and their possible implications. Construction projects, because of their large and complex nature, are plagued by a variety of risks which must be considered and responded to in order to ensure project success. This study conducts an extensive comparative analysis of major quantitative risk analysis techniques in the construction industry. The techniques discussed and comparatively analyzed in this report include: Programme Evaluation and Review Technique (PERT), Judgmental Risk Analysis Process (JRAP), Estimating Using Risk Analysis (ERA), Monte Carlo Simulation technique, Computer Aided Simulation for Project Appraisal and Review (CASPAR), Failure Modes and Effects Analysis technique (FMEA) and Advanced Programmatic Risk Analysis and Management model (APRAM). The findings highlight the fact that each risk analysis technique addresses risks in any or all of the following areas – schedule risks, budget risks or technical risks. Through comparative analysis, it has been revealed that a majority of risk analysis techniques focus on schedule or budget risks. Very little has been documented in terms of technical risk analysis techniques. In an era where clients are demanding and expecting higher quality projects and finishes, project managers must endeavor to invest time and resources to ensure that the few existing technical risk analysis techniques are developed and further refined, and that new technical risk analysis techniques are developed to suit the current construction industries requirements.

An evaluation of Sea Search as a citizen science programme in Marine Protected Areas

Citizen science involves collaboration between multi-sector agencies and the public to address a natural resource management issue. The Sea Search citizen science programme involves community groups in monitoring and collecting subtidal rocky reef and intertidal rocky shore data in Victorian Marine Protected Areas (MPAs), Australia. In this study we compared volunteer and scientifically collected data and the volunteer motivation for participation in the Sea Search programme. Intertidal rocky shore volunteer-collected data was found to be typically comparable to data collected by scientists for species richness and diversity measures. For subtidal monitoring there was also no significant difference for species richness recorded by scientists and volunteers. However, low statistical power suggest only large changes could be detected due to reduced data replication. Generally volunteers recorded lower species diversity for biological groups compared to scientists, albeit not significant. Species abundance measures for algae species were significantly different between volunteers and scientists. These results suggest difficulty in identification and abundance measurements by volunteers and the need for additional training requirements necessary for surveying algae assemblages. The subtidal monitoring results also highlight the difficulties of collecting data in exposed rocky reef habitats with weather conditions and volunteer diver availability constraining sampling effort. The prime motivation for volunteer participation in Sea Search was to assist with scientific research followed closely by wanting to work close to nature. This study revealed two important themes for volunteer engagement in Sea Search: 1) volunteer training and participation and, 2) usability of volunteer collected data for MPA managers. Volunteer-collected data through the Sea Search citizen science programme has the potential to provide useable data to assist in informed management practices of Victoria’s MPAs, but requires the support and commitment from all partners involved.

Evaluation framework underpinning the digital divas programme

In Australia, as elsewhere, women’s participation rates in Information Technology (IT) have been low. IT is the generic term used to refer to the many courses in the Computer Science, and Information Systems disciplines. While there have been a number of intervention programmes implemented aimed at encouraging women into IT and retaining them once there, few have included evaluations of the efficacy of the intervention. Thus little is known about the factors contributing to the success, or lack of success, of the interventions, or of the medium and longer term impacts for the participants. In this paper we briefly describe an intervention programme implemented with girls in the high school years. We present an evaluation framework providing a detailed overview of the aims and processes involved in the evaluation of the programme. Data for the evaluation were embedded within the data gathering methods associated with the research on the intervention itself.

The delivery of a telephone coaching programme to people with type 2 diabetes by practice nurses in Victoria, Australia : a qualitative evaluation

Aim.  To evaluate telephone coaching undertaken by practice nurses in a randomised controlled trial of self-management support for people with type 2 diabetes.

Background.  Qualitative evaluation of the processes that take place in randomised controlled trials has the advantage of providing information on those variables that contribute to the success or failure of the randomised controlled trial. This additional information can be used to improve or modify chronic disease management programme designs.

Methods.  Grounded theory was used to analyse transcriptions of telephone coaching sessions between practice nurses and patient participants in the randomised controlled trial.

Findings.   Analysis of transcriptions found that patient participants had complex multiple medical conditions to manage, as well as maintaining their daily lives. Two approaches to working with this complexity by practice nurses emerged. We characterised one as ‘treat to target’ and the other as ‘personalised care’. While each approach shapes identities available to patients within the relationship with the practice nurse, the impact or effectiveness of these approaches on outcomes has yet to be reported.

Conclusions.  Telephone coaching takes place in complex social contexts as well as complex medical conditions. People with type 2 diabetes must manage their diabetes care and their care of other conditions within their social contexts. This means a constant negotiation of priorities.

Relevance to clinical practice.  Awareness of how health professional support for patients’ self-management becomes a relationship and element of the negotiated identity of patients is important in adapting clinical guideline-based protocols to achieving targets in the management of chronic illness.

A longitudinal evaluation of a project-based learning initiative in an engineering undergraduate programme

Project-based learning (PBL) is a well-known student-centred methodology for engineering design education. The methodology claims to offer a number of educational benefits. This paper evaluates the student perceptions of the initial and second offering of a first-year design unit at Griffith University in Australia. It builds on an earlier evaluation conducted after the initial offering of the unit. It considers the implementation of the recommended changes. Evaluations of the two offerings reveal that students (in both the initial and second offering) generally enjoyed the experience, but that the second offering was found to be a significantly more enjoyable learning experience. Students in the second offering also reported a significantly better understanding of what they needed to do for the design projects and where to find the requisite information. The oral presentation aspect of the initial and second offerings received the lowest satisfaction rating. The inclusion (and delivery) of the computer-aided drawing component of the unit is seen as a positive aspect by some students, but many others comment on it negatively. The best aspects of the PBL unit and those aspects needing further improvement were similar to the findings of other investigations documented in the literature.

The 6-PACK programme to decrease falls and fall-related injuries in acute hospitals : protocol for an economic evaluation alongside a cluster randomised controlled trial

Background Falls are a common hospital occurrence complicating the care of patients. From an economic perspective, the impact of in-hospital falls and related injuries is substantial. However, few studies have examined the economic implications of falls prevention interventions in an acute care setting. The 6-PACK programme is a targeted nurse delivered falls prevention programme designed specifically for acute hospital wards. It includes a risk assessment tool and six simple strategies that nurses apply to patients classified as high-risk by the tool.
Objective To examine the incremental cost-effectiveness of the 6-PACK programme for the prevention of falls and fall-related injuries, compared with usual care practice, from an acute hospital perspective.
Methods and design The 6-PACK project is a multicentre cluster randomised controlled trial (RCT) that includes 24 acute medical and surgical wards from six hospitals in Australia to investigate the efficacy of the 6-PACK programme. This economic evaluation will be conducted alongside the 6-PACK cluster RCT. Outcome and hospitalisation cost data will be prospectively collected on approximately 16 000 patients admitted to the participating wards during the 12-month trial period. The results of the economic evaluation will be expressed as ‘cost or saving per fall prevented’ and ‘cost or saving per fall-related injury prevented’ calculated from differences in mean costs and effects in the intervention and control groups, to generate an incremental cost-effectiveness ratio (ICER).
Discussion This economic evaluation will provide an opportunity to explore the cost-effectiveness of a targeted nurse delivered falls prevention programme for reducing in-hospital falls and fall-related injuries. This protocol provides a detailed statement of a planned economic evaluation conducted alongside a cluster RCT to investigate the efficacy of the 6-PACK programme to prevent falls and fall-related injuries.

Evaluation of the medium-term outcomes and impact of the Rowan Nicks Scholarship Programme

Rowan Nicks was a cardiothoracic surgeon in Sydney. He endowed the Rowan Nicks Scholarship Programme of the Royal Australasian College of Surgeons, which was initiated in 1991 to provide opportunities for clinicians from developing countries so that they return to their countries as leaders and teachers. This paper's objective was to evaluate the outcomes and impact of the scholarship on individuals and their communities.

Teaching general practitioners and doctors-in-training to discuss advance care planning : evaluation of a brief multimodality education programme

To develop and evaluate an interactive advance care planning (ACP) educational programme for general practitioners and doctors-in-training.

Moving beyond a ‘bums-on-seats’ analysis of progress towards widening participation: reflections on the context, design and evaluation of an Australian government-funded mentoring programme

In 2010 the Australian Government established the Higher Education Participation and Partnerships programme – a funding agenda to promote programmes that respond to the underrepresentation in higher education of people from what is often denoted low socioeconomic status (SES) backgrounds. Many government-funded programmes and projects have since emerged that respond to the problem of low SES underrepresentation, based on partnerships between higher education providers and other organisations. The arguments made in this paper draw on one such project: a mentoring programme implemented from 2011 to 2013 that targeted the aspirations of Year 9 regional secondary students. We discuss data and documentation that provide insights into the conception and design of the mentoring programme, and the strategies used to evaluate it, in order to discuss how funding and policy contexts influences the possible solutions that might be implemented in response to the underrepresentation in higher education of people from low SES backgrounds.