Purposeful program theory: Effective use of theories of change and logic models [Book Review]

The purpose of this book by two Australian authors is to: introduce the audience to the full complement of contextual elements found within program theory; offer practical suggestions to engage with theories of change, theories of action and logic models; and provide substantial evidence for this approach through scholarly literature, practice case studies together with the authors' combined experience of 60 years.

Boosting the effects of a curriculum based injury prevention program through a school connectedness intervention : final report to NRMA-ACT Road Safety Trust

Injury is the leading cause of death among young people (AIHW, 2008). A primary contributing factor to injury among adolescents is risk taking behaviour, including road related risks such as risky bicycle and motorcycle use and riding with dangerous or drink-drivers. Injury rates increase dramatically throughout adolescence, at the same time as adolescents are becoming more involved in risk taking behaviour. Also throughout this period, adolescents‟ connectedness to school is decreasing (Monahan, Oesterle & Hawkins, 2010; Whitlock, 2004). School connectedness refers to „the extent to which students feel personally accepted, respected, included, and supported by others in the school‟ (Goodenow, 1993, p. 80), and has been repeatedly shown to be a critical protective factor in adolescent development. For example, school connectedness has been shown to be associated with decreased risk taking behaviour, including violence and alcohol and other drug use (e.g., Resnick et al., 1997), as well as with decreased transport risk taking and vehicle related injuries (Chapman et al., accepted April 2011). This project involved the pilot evaluation of a school connectedness intervention (a professional development program for teachers) to reduce adolescent risk taking behaviour and injury. This intervention has been developed for use as a component of the Skills for Preventing Injury in Youth (SPIY) curriculum based injury prevention program for early adolescents. The objectives of this research were to: 1. Implement a trial School Connectedness intervention (professional development program for teachers) in ACT high schools, and evaluate using comparison high schools. 2. Determine whether the School Connectedness program impacts on adolescent risk taking behaviour and associated injuries (particularly transport risks and injuries). 3. Evaluate the process effectiveness of the School Connectedness program.

Evaluation of the potential impact of the global economic crisis on HIV epidemics in Southeast Asia

Economic conditions around the world are likely to deteriorate in the short to medium term. The potential impact of this crisis on the spread of HIV is not clear. Government revenues and aid flows from international donors may face constraints, possibly leading to reductions in funding for HIV programs. Economic conditions (leading to increases in unemployment, for example) may also have an indirect impact on HIV epidemics by affecting the behaviour of individual people. Some behavioural changes may influence the rate of HIV transmission. This report presents findings from a study that investigates the potential impact of the economic crisis on HIV epidemics through the use of mathematical modelling. The potential epidemiological impacts of changes in the economy are explored for two distinctly characterised HIV epidemics: (i) a well-defined, established, and generalised HIV epidemic (specifically Cambodia, where incidence is declining); (ii) an HIV epidemic in its early expansion phase (specifically Papua New Guinea, where incidence has not yet peaked). Country-specific data are used for both settings and the models calibrated to accurately reflect the unique HIV epidemics in each population in terms of both incidence and prevalence. Models calibrated to describe the past and present epidemics are then used to forecast epidemic trajectories over the next few years under assumptions that behavioural or program conditions may change due to economic conditions. It should be noted that there are very limited solid data on how HIV/AIDS program funds may decrease or how social determinants related to HIV risk may change due to the economic crisis. Potential changes in key relevant factors were explored, along with sensitivity ranges around these assumptions, based on extensive discussions with in-country and international experts and stakeholders. As with all mathematical models, assumptions should be reviewed critically and results interpreted cautiously.

Youth off the Streets "Our Place, Walgett Youth and Young Families Project" Project Evaluation.

The Project: • YOTS is a major youth specific agency established in 1991. It is a non-denominational, non-discriminatory and not-for-profit organisation, providing a wide range of services and offering a full continuum of care. It seeks to build on the strengths and positive aspects of marginalised young people and communities. • The 'Our Place, Walgett Youth and Young Families Project' further develops an existing YOTS capacity to provide services to Aboriginal young people. • The project adopted an action-research and community development model in which YOTS worked in partnership with the Youth Sub-committee of the Walgett Interagency. • Specific goals/objectives of the program were to: Coordinate youth and young family activities in partnership with local services and the community to build self-esteem, pride, resilience, motivation and skills; Contribute to the prevention and reduction of homelessness, unstable and unsafe housing and disruptive mobility (Walgett/Redfern) in youth and young families; Increase and improve collaborative engagement between youth and family focused services; and, research, adapt and implement Australian and international best-practice homelessness prevention/reduction initiatives to contribute to new models of practice relevant to rural and regional areas. • The project centred around an out-reach model that focused on providing a safe space with relevant structured activities coordinated by YOTS youth and family workers. Through community and service provider consultation, it was proposed that local services could coordinate strategies and activities and run them, where possible, from the centre, providing ease of access in a safe and supportive context. • Specific activities included: Implementing regular meetings with the stakeholders and community representatives; Developing a Terms of Reference for YOTS presence in the Walgett community; Undertaking a community consultation prior to finalising program activities; Implementing a range of recreational activities (sports, music, arts and crafts) early on in the activity; Implementing young family support initiatives; implementing a volunteering program, including volunteer support to young families through intergenerational volunteering; running a series of Culture and Healing Camps in partnership with local Elders and other services; Running a series of Music Camps; Providing alternative education support and referrals in partnership with local schools; Researching, identifying and adapting other best-practice models.

Designing for collaborative learning spaces : an academic staff development pilot program

To prepare for the delivery of new Bachelor of Science units in collaborative learning spaces, academic and professional staff at Queensland University of Technology piloted an academic development program over the period of a semester. The program was informed by Rogers’ theory of innovation and diffusion (2003) and structured according to Wilson’s framework for faculty development (2007). Through a series of workshops and group mentoring activities, the program modelled inquiry-based learning in a collaborative learning space, and the participants designed and practiced the delivery of teaching activities. This paper provides a preliminary evaluation of the effectiveness of the pilot based on survey responses from participants, notes from the development team who coordinated the program and audience feedback from the final showcase session. The design and structure of the program is discussed as well as possible future directions.

Development of the OnTrack Diabetes program

Background: Type 2 diabetes affects an estimated 347 million people worldwide and often leads to serious complications including blindness, kidney disease, and limb amputation. Comorbid dysphoria is common and is an independent risk factor for poor glycaemic control. Professional support for diabetes self-management and dysphoria has limited availability and involves high costs, especially after regular hours, and in rural and remote areas. Web-based cognitive behavior therapy offers highly accessible, acceptable, and cost-effective support for people with diabetes. This paper describes the development of OnTrack Diabetes, a self-guided, Web-based program to promote improved physical and emotional self-management in people with Type 2 diabetes. Objective: The objective of the study is to describe the development of the OnTrack Diabetes program, which is a self-guided, Web-based program aimed to promote euthymia and improved disease self-management in people with Type 2 diabetes. Methods: Semistructured interviews with 12 general practitioners and 13 patients with Type 2 diabetes identified enablers of and barriers to effective diabetes self-management, requirements for additional support, and potential program elements. Existing resources and research data informed the development of content, and consultants from relevant disciplines provided feedback on draft segments and reviewed the program before release. Using a self-guided delivery format contained costs, in addition to adapting program features and modules from an existing OnTrack program. Results: A separate paper describes the protocol for a randomized controlled trial to provide this required evaluation. Conclusions: Development of the OnTrack Diabetes program demonstrates strategies that help ensure that a program is acceptable to users. The next stages involve testing users’ experiences and examining the program’s effectiveness and cost-effectiveness in randomized controlled trials.

Best practice in peer-led curriculum content : informing an interactive program to improve passenger safety among high school seniors

Despite ongoing improvements in behaviour change strategies, licensing models and road law enforcement measures young drivers remain significantly over-represented in fatal and non-fatal road related crashes. This paper focuses on the safety of those approaching driving age and identifies both high priority road safety messages and relevant peer-led strategies to guide the development school programs. It summarises the review in a program logic model built around the messages and identified curriculum elements, as they may be best operationalised within the licensing and school contexts in Victoria. This paper summarises a review of common deliberate risk-taking and non-deliberate unsafe driving behaviours among novice drivers, highlighting risks associated with speeding, driving while fatigued, driving while impaired and carrying passengers. Common beliefs of young people that predict risky driving were reviewed, particularly with consideration of those beliefs that can be operationalised in a behaviour change school program. Key components of adolescent risk behaviour change programs were also reviewed, which identified a number of strategies for incorporation in a school based behaviour change program, including: a well-structured theoretical design and delivery, thoughtfully considered peer-selected processes, adequate training and supervision of peer facilitators, a process for monitoring and sustainability, and interactive delivery and participant discussions. The research base is then summarised in a program logic model with further discussion about the quality of the current state of knowledge of evaluation of behaviour change programs and the need for considerable development in program evaluation.

Patient perspectives of using a web based intervention program for patients with type 2 diabetes and acute coronary syndrome

Introduction Research highlights patients with dual diagnoses of type 2 diabetes and acute coronary syndrome (ACS) have higher readmission rates and poorer health outcomes than patients with singular chronic conditions. Despite this, there is a lack of education programs targeted for this dual diagnosis population to improve self-management and decrease negative health outcomes. There is evidence to suggest that internet based interventions may improve health outcomes for patients with singular chronic conditions, however there is a need to develop an evidence base for ACS patients with comorbid diabetes. There is a growing awareness of the importance of a participatory model in developing effective online interventions. That is, internet interventions are more effective if end users’ perceptions of the intervention are incorporated in their final development prior to testing in large scale trials. Objectives This study investigated patients’ perspectives of the web-based intervention designed to promote self-management of the dual conditions in order to refine the intervention prior to clinical trial evaluation. Methods An interpretive approach with thematic analysis was used to obtain deeper understanding regarding participants’ experience when using web-application interventions for patients with ACS and type 2 diabetes. Semi-structured interviews were undertaken on a purposive sample of 30 patients meeting strict inclusion and exclusion criteria to obtain their perspectives on the program. Results Preliminary results indicate patients with dual diagnoses express more complex needs than those with a singular condition. Participants express a positive experience with the proposed internet intervention and emerging themes include that the web page is seen as easy to use and comforting as a support, in that patients know they are not alone. Further results will be reported as they become available. Conclusion The results indicate potential for patient acceptability of the newly developed internet intervention for patients with ACS and comorbid diabetes. Incorporation of patient perspectives into the final development of the intervention is likely to maximise successful outcomes of any future trials that utilise this intervention. Future quantitative evaluation of the effectiveness of the intervention is being planned.

Illuminative evaluation as a method applied to Australian Government policy borrowing and implementation in higher education

Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.

A randomized controlled trial of a program to reduce adolescent road-related medically-treated injuries

Background Injury is the leading cause of adolescent death and injury around the road is a common source of adolescent injuries. Skills for Preventing Injury in Youth (SPIY) is a comprehensive program developed in Australia for early adolescents (term-long curriculum, including looking out for friends, first-aid training coupled with teacher school-connectedness professional development). Jessors’ Protection-Risk Framework guided the program approach focusing on building protective relationships. Method A randomized controlled trial with 35 schools was undertaken. Students completed surveys at baseline, six-months post-intervention and twelve-months post intervention. There were 1686 students (56% female) who completed the twelve-month survey, including the Extended-Adolescent Injury Checklist whereby students self-report on medically-treated injuries over the previous three-months (only road-related items are reported in this study; cycling, motorcycle riding, pedestrian, and riding as a passenger). Randomly selected SPIY classes also participated in focus groups and reported on perceptions of SPIY and injury risk behavior. Results As a check of randomization baseline differences of the variables were examined, with no significant differences between intervention and control groups. At the 12-month follow-up, there were fewer medically-treated injuries among the intervention students compared with the control group, particularly associated with being a passenger. The process evaluation revealed students perceived change in injury risk and risk behaviors. Conclusions While data analyses are continuing, the results indicate that the program seeking to encourage adolescents to look out for their friends, build connections to school and provide first aid skills training goes some way to reducing self-reported medically-treated injuries around the road.

Improving wound management for residents in Residential Aged Care Facilities: National dissemination and implementation of the evidence based Champions for Skin Integrity Program

Overview The incidence of skin tears, pressure injuries and chronic wounds increases with age [1-4] and therefore is a serious issue for staff and residents in Residential Aged Care Facilities (RACFs). A pilot project funded in Round 2 of the Encouraging Best Practice in Residential Aged Care (EBPRAC) program by the then Australian Government Department of Health and Ageing found that a substantial proportion of residents in aged care facilities experienced pressure injuries, skin tears or chronic wounds. It also found the implementation of the evidence based Champions for Skin Integrity (CSI) model of wound care was successful in significantly decreasing the prevalence and severity of wounds in residents, improving staff skills and knowledge of evidence based wound management, increasing staff confidence with wound management, increasing implementation of evidence based wound management and prevention strategies, and increasing staff awareness of their roles in evidence based wound care at all levels [5]. Importantly, during the project, the project team developed a resource kit on evidence based wound management. Two critical recommendations resulting from the project were that: - The CSI model or a similar strategic approach should be implemented in RACFs to facilitate the uptake of evidence based wound management and prevention - The resource kit on evidence based wound management should be made available to all Residential Aged Care Facilities and interested parties A proposal to disseminate or rollout the CSI model of wound care to all RACFs across Australia was submitted to the department in 2012. The department approved funding from the Aged Care Services Improvement Healthy Ageing Grant (ACSIHAG) at the same time as the Round 3 of the Encouraging Better Practice in Aged Care (EBPAC) program. The dissemination involved two crucial elements: 1. The updating, refining and distribution of a Champions for Skin Integrity Resource Kit, more commonly known as a CSI Resource Kit and 2. The presentation of intensive one day Promoting Healthy Skin “Train the Trainer” workshops in all capital cities and major regional towns across Australia Due to demand, the department agreed to fund a second round of workshops focussing on regional centres and the completion date was extended to accommodate the workshops. Later, the department also decided to host a departmental website for a number of clinical domains, including wound management, so that staff from the residential aged care sector had easy access to a central repository of helpful clinical resource material that could be used for improving the health and wellbeing of their older adults, consumers and carers. CSI Resource Kit Upgrade and Distribution: At the start of the project, a full evidence review was carried out on the material produced during the EBPRAC-CSI Stage 1 project and the relevant evidence based changes were made to the documentation. At the same time participants in the EBPRAC-CSI Stage 1 project were interviewed for advice on how to improve the resource material. Following this the documentation, included in the kit, was sent to independent experts for peer review. When this process was finalised, a learning designer and QUT’s Visual Communications Services were engaged to completely refine and update the design of the resources, and combined resource kit with the goal of keeping the overall size of the kit suitable for bookshelf mounting and the cost at reasonable levels. Both goals were achieved in that the kit is about the same size as a 25 mm A4 binder and costs between $19.00 and $28.00 per kit depending on the size of the print run. The dissemination of the updated CSI resource kit was an outstanding success. Demand for the kits was so great that a second print run of 2,000 kits was arranged on top of the initial print run of 4,000 kits. All RACFs across Australia were issued with a kit, some 2,740 in total. Since the initial distribution another 1,100 requests for kits has been fulfilled as well as 1,619 kits being distributed to participants at the Promoting Healthy Skin workshops. As the project was winding up a final request email was sent to all workshop participants asking if they required additional kits or resources to distribute the remaining kits and resources. This has resulted in requests for 200 additional kits and resources. Feedback from the residential aged care sector and other clinical providers who have interest in wound care has been very positive regarding the utility of the kit, (see Appendix 4). Promoting Healthy Skin Workshops The workshops also exceeded the project team’s initial objective. Our goal of providing workshop training for staff from one in four facilities and 450 participants was exceeded, with overwhelming demand for workshop places resulting in the need to provide a second round of workshops across Australia. At the completion of the second round, 37 workshops had been given, with 1286 participants, representing 835 facilities. A number of strategies were used to promote the workshops ranging from invitations included in the kit, to postcard mail-outs, broadcast emailing to all facilities and aged care networks and to articles and paid advertising in aged care journals. The most effective method, by far, was directly phoning the facilities. This enabled the caller to contact the relevant staff member and enlist their support for the workshop. As this is a labour intensive exercise, it was only used where numbers needed bolstering, with one venue rising from 3 registrants before the calls to 53 registrants after. The workshops were aimed at staff who had the interest and the capability of implementing evidence-based wound management within their facility or organisation. This targeting was successful in that a large proportion (68%) of participants were Registered Nurses, Nurse Managers, Educators or Consultants. Twenty percent were Endorsed Enrolled Nurses with the remaining 12% being made up of Personal Care Workers or Allied Health Professionals. To facilitate long term sustainability, the workshop employed train-the-trainer strategies. Feedback from the EBPRAC-CSI Stage 1 interviews was used in the development of workshop content. In addition, feedback from the workshop conducted at the end of the EBPRAC-CSI Stage 1 project suggested that change management and leadership training should be included in the workshops. The program was trialled in the first workshop conducted in Brisbane and then rolled out across Australia. Participants were asked to complete pre and post workshop surveys at the beginning and end of the workshop to determine how knowledge and confidence improved over the day. Results from the pre and post surveys showed significant improvements in the level of confidence in attendees’ ability to implement evidence based wound management. The results also indicated a significant increase in the level of confidence in ability to implement change within their facility or organisation. This is an important indication that the inclusion of change management/leadership training with clinical instruction can increase staff capacity and confidence in translating evidence into practice. To encourage the transfer of the evidence based content of the workshop into practice, participants were asked to prepare an Action Plan to be followed by a simple one page progress report three months after the workshop. These reports ranged from simple (e.g. skin moisturising to prevent skin tears), to complex implementation plans for introducing the CSI model across the whole organisation. Outcomes described in the project reports included decreased prevalence of skin tears, pressure injuries and chronic wounds, along with increased staff and resident knowledge and resident comfort. As stated above, some organisations prepared large, complex plans to roll out the CSI model across their organisation. These plans included a review of the organisation’s wound care system, policies and procedures, the creation of new processes, the education of staff and clients, uploading education and resource material onto internal electronic platforms and setting up formal review and evaluation processes. The CSI Resources have been enthusiastically sought and incorporated into multiple health care settings, including aged care, acute care, Medicare Local intranets (e.g. Map of Medicine e-pathways), primary health care, community and home care organisations, education providers and New Zealand aged and community health providers. Recommendations: Recommendations for RACFs, aged care and health service providers and government  Skin integrity and the evidence-practice gap in this area should be recognised as a major health issue for health service providers for older adults, with wounds experienced by up to 50% of residents in aged care settings (Edwards et al. 2010). Implementation of evidence based wound care through the Champions for Skin Integrity model in this and the pilot project has demonstrated the prevalence of wounds, wound healing times and wound infections can be halved.  A national program and Centre for Evidence Based Wound Management should be established to: - expand the reach of the model to other aged care facilities and health service providers for older adults - sustain the uptake of models such as the Champions for Skin Integrity (CSI) model - ensure current resources, expertise and training are available for consumers and health care professionals to promote skin integrity for all older adults  Evidence based resources for the CSI program and similar projects should be reviewed and updated every 3 – 4 years as per NH&MRC recommendations  Leadership and change management training is fundamental to increasing staff capacity, at all levels, to promote within-organisation dissemination of skills and knowledge gained from projects providing evidence based training Recommendations for future national dissemination projects  A formal program of opportunities for small groups of like projects to share information and resources, coordinate activities and synergise education programs interactively would benefit future national dissemination projects - Future workshop programs could explore an incentive program to optimise attendance and reduce ‘no shows’ - Future projects should build in the capacity and funding for increased follow-up with workshop attendees, to explore the reasons behind those who are unable to translate workshop learnings into the workplace and identify factors to address these barriers.

Where to after Drink Rite: An evaluation of opportunities for police and community promotion of an anti-drink driving message in the licensed venue setting. Report to the National Drug Strategy Law Enforcement Funding Committee

Estimating the prevalence of drink driving is a difficult task. Self‐reported drink driving indicates that drink driving is far more common than official statistics suggest. In order to promote a responsible attitude towards alcohol consumption and drink driving within the Queensland community, the Queensland Police Service, Queensland Health and Queensland Transport developed the ‘Drink Rite’ program (Queensland Police Service information sheet, 2009). However, the feasibility of the program is now in doubt as the National Health and Medical Research Council’s guidelines for alcohol consumption changed in 2009 to state “For healthy men and women, drinking no more than four standard drinks on a single occasion reduces the risk of alcohol‐related injury arising from that occasion” (NHMRC Publication, 2009, p. 51). As such, adhering to the NHMRC guidelines places restrictions on how the existing Drink Rite program can be operated (i.e. by reducing the number of standard drinks provided to participants from eight to four). It is arguable that a reduction in the number of alcoholic drinks provided to participants in the program will result in a large reduction in observed BAC readings. This, in turn, will lead to a potential loss of message content when discussing the variation in the effects of alcohol.

Differential segmentation responses to an alcohol social marketing program

Objective This study seeks establish whether meaningful subgroups exist within a 14-16 year old adolescent population and if these segments respond differently to the Game On: Know Alcohol (GOKA) intervention, a school-based alcohol social marketing program. Methodology This study is part of a larger cluster randomized controlled evaluation of the Game On: Know Alcohol (GOKA) program implemented in 14 schools in 2013/2014. TwoStep cluster analysis was conducted to segment 2114 high school adolescents (14-16 years old) on the basis of 22 demographic, behavioral and psychographic variables. Program effects on knowledge, attitudes, behavioral intentions, social norms, expectancies and refusal self-efficacy of identified segments was subsequently examined. Results Three segments were identified: (1) Abstainers (2) Bingers (3) Moderate Drinkers. Program effects varied significantly across segments. The strongest positive change effects post participation were observed for the Bingers, while mixed effects were evident for Moderate Drinkers and Abstainers. Conclusions These findings provide preliminary empirical evidence supporting application of social marketing segmentation in alcohol education programs. Development of targeted programs that meet the unique needs of each of the three identified segments is indicated to extend the social marketing footprint in alcohol education.