Sing & Grow : the co-existence of evaluation research and clinical practice in an early intervention music therapy project

Sing & Grow is a short term early intervention music therapy program for at risk families. Sing & Grow uses music to strengthen parent-child relationships by increasing positive parent-child interactions, assisting parents to bond with their children, and extending the repertoire of parents’ skills in relating to their child through interactive . Both the Australian and New Zealand governments are looking for evidence based research to highlight the effectiveness of funded programs in early childhood. As a government funded program, independent evaluation is a requirement of the delivery of the service. This paper explains the process involved in setting up and managing this large scale evaluation from engaging the evaluators and designing the project, to the data gathering stage. It describes the various challenges encountered and concludes that a highly collaborative and communicative partnership bet en researchers and clinicians is essential to ensure data can be gathered with minimal disturbance to clinical music therapy practice.

"Listening, hearing and drawing things together to work on solutions" : NAPCAN Play a Part, Queensland, 2010-2013, Evaluation-Research Report

Responding to the idea of child friendly communities, Play a Part is an innovative program advancing preventative strategies for children and young people to minimise exposure to abuse and neglect. The program was developed ensuing an increase in notifications of suspected child abuse and neglect in 2007. Now completing the second phase, the program is a community engagement strategy that aims to prevent child abuse. Play a Part is described as “a whole of community approach to creating child friendly communities” (NAPCAN, 2012). The Play a Part program was piloted between 2007 and 2010 in five southeast Queensland communities, and is currently operating in parts of Logan City region and the Redlands region. To assess the merit of the second phase of the program the Children and Youth Research Centre at Queensland University of Technology was contracted to undertake an evaluation-research at the beginning of 2013.

Program Evaluation Research Task (PERT) : summary report.

"September 1958."

Victoria's “Day in Prison Program” : an evaluation and critique

The use of “Day in Prison” programs to deter young adult offenders is a concept which originated in the United States and was replicated in Australia during the late 1970s. After almost a decade of uncertainty this model of ‘crime prevention’ re-emerged in Victoria with the introduction of a pilot “Day in Prison” program. This article traces the development and operation of the Victorian experience and provides evaluation research findings which conclude that coercive, intimidatory and degrading aversion techniques should not be utilised by the criminal justice system for the purposes of individual deterrence.

Evaluation of the skills for preventing injury in youth program : impact on health risk behaviors, injury and school connectedness

Injury is the leading cause of death among young people, and involvement in health risk behaviors, such as alcohol use and transport-related risks, is related to increased risk for injury. Effective health promotion programs for adolescents focus on multiple levels, including relationships with peers and parents, student knowledge, behavior and attitudes, and school-level factors such as school connectedness. This study describes the pilot evaluation of a comprehensive, multi-level injury prevention program for 13-14 year old adolescents, targeting change in injury associated with transport and alcohol risks. The program, called Skills for Preventing Injury in Youth (SPIY), incorporates two primary elements: an 8-week, teacher delivered attitude and behavior change curriculum with peer protection and first aid messages; and professional development for program teachers focusing on strategies to increase students’ connectedness to school. Five Australian high schools were recruited for the pilot evaluation research, with three being assigned to receive intervention components and two assigned as curriculum-as-usual controls. In the intervention schools, 118 Year 8 students participated in surveys at baseline, with 105 completing surveys at follow up, six months following the intervention. In the control schools, 196 Year 8 students completed surveys at baseline and 207 at follow up. Survey measures included self-reported injury, risk taking behavior and school connectedness. Results showed that students in the control schools were significantly more likely to report riding bikes without helmets, riding with dangerous drivers, having driven cars on the road, and using alcohol six months after the program, while the intervention group showed no such increase in these behaviors. Additionally, students in the control schools were significantly more likely to report having had pedestrian-related injuries at follow up than they were at the baseline measurement, while intervention school students showed no change. There was also a trend observed in terms of a decrease in bicycle related injuries among intervention school students, compared with a slight increasing trend in bicycle injuries among control students. Overall, scores on the school connectedness scale decreased significantly from baseline to follow up for both intervention and control students, however measurement limitations may have impacted on results relating to students’ connectedness. Overall, the SPIY program has shown promising results in regards to prevention of students’ health risk behavior and injuries. Evidence suggests that the curriculum component was important; however there was limited evidence to suggest that teacher training in school connectedness strategies contributed to these promising results. While school connectedness may be an important factor to target in risk and injury prevention programs, programs may need to incorporate whole-of-school strategies or target a broader range of teachers than were selected for the current research.

Queensland Emergency Medicine Research Foundation: Special report

The development of any new profession is dependent on the development of a special body of knowledge which is the domain of the profession and key to this is the conduct of research. In 2007, as part of the settlement of an Enterprise Bargaining Agreement and following sustained lobbying by Emergency Physicians, the Queensland Government agreed to establish an Emergency Medicine Research Fund to foster the development of research activities in Emergency Medicine in Queensland. That fund is now managed by the Queensland Emergency Medicine Research Foundation. The aims of this article are to describe the strategic approaches taken by the Foundation and its first three years of experience, to describe the application of research funds and to foreshadow an evaluative framework for determining the strategic value of this community investment. The Foundation has developed a range of personnel and project support funding programs and competition for funding has increased. Ongoing evaluation will seek to determine the effectiveness of this funding strategy on improving the effectiveness of research performance and the clinical and organisational outcomes that may derive from that initiative.

Engaging paramedic students in research: A case report

The use of volunteer undergraduate students to support simulated training for peers is common in Paramedic Science. However, there are limited examples of engaging paramedic student-volunteers in research as compared to that reported in cognate disciplines such as Medicine and Nursing. This case report shares our experience with engaging a penultimate year paramedic student in evaluation research. This information we hope will start the dialogue on the epistemology and pedagogies for effective engagement of undergraduate paramedic students as future researchers.

Illuminative evaluation as a method applied to Australian Government policy borrowing and implementation in higher education

Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.

Tropical peach breeding and evaluation

Breeding new peach and nectarine cultivars that are adapted to tropical climates and produce fruit of high quality.

Hardwood Drying Length Evaluation

Evaluating the length of time required to dry hardwood timber using vacuum drying compared to conventional drying facilities and technology.

Managing patient deterioration: evaluation of what works.

Background: Rapid Response Systems (RRS) have been implemented nationally and internationally to improve patient safety in hospital. However, to date the majority of the RRS research evidence has focused on measuring the effectiveness of the intervention on patient outcomes. To evaluate RRS it has been recommended that a multimodal approach is required to address the broad range of process and outcome measures required to determine the effectiveness of the RRS concept. Aim: The aim of this paper is to evaluate the official RRS programme theoretical assumptions regarding how the programme is meant to work against actual practice in order to determine what works. Methods: The research design was a multiple case study approach of four wards in two hospitals in Northern Ireland. It followed the principles of realist evaluation research which allowed empirical data to be gathered to test and refine RRS programme theory [1]. This approach used a variety of mixed methods to test the programme theories including individual and focus group interviews with a purposive sample of 75 nurses and doctors, observation of ward practices and documentary analysis. The findings from the case studies were analysed and compared within and across cases to identify what works for whom and in what circumstances. Results: The RRS programme theories were critically evaluated and compared with study findings to develop a mid-range theory to explain what works, for whom in what circumstances. The findings of what works suggests that clinical experience, established working relationships, flexible implementation of protocols, ongoing experiential learning, empowerment and pre-emptive management are key to the success of RRS implementation.  Conclusion:These findings highlight the combination of factors that can improve the implementation of RRS and in light of this evidence several recommendations are made to provide policymakers with guidance and direction for their success and sustainability.References: 1.Pawson R and Tilley N. (1997) Realistic Evaluation. Sage Publications; LondonType of submission: Concurrent session Source of funding: Sandra Ryan Fellowship funded by the School of Nursing & Midwifery, Queen’s University of Belfast

Realistic evaluation of EWS and ALERT: factors enabling and constraining implementation.

Realistic Evaluation of EWS and ALERT: factors enabling and constraining implementation Background The implementation of EWS and ALERT in practice is essential to the success of Rapid Response Systems but is dependent upon nurses utilising EWS protocols and applying ALERT best practice guidelines. To date there is limited evidence on the effectiveness of EWS or ALERT as research has primarily focused on measuring patient outcomes (cardiac arrests, ICU admissions) following the implementation of a Rapid Response Team. Complex interventions in healthcare aimed at changing service delivery and related behaviour of health professionals require a different research approach to evaluate the evidence. To understand how and why EWS and ALERT work, or might not work, research needs to consider the social, cultural and organisational influences that will impact on successful implementation in practice. This requires a research approach that considers both the processes and outcomes of complex interventions, such as EWS and ALERT, implemented in practice. Realistic Evaluation is such an approach and was used to explain the factors that enable and constrain the implementation of EWS and ALERT in practice [1]. Aim The aim of this study was to evaluate factors that enabled and constrained the implementation and service delivery of early warnings systems (EWS) and ALERT in practice in order to provide direction for enabling their success and sustainability. Methods The research design was a multiple case study approach of four wards in two hospitals in Northern Ireland. It followed the principles of realist evaluation research which allowed empirical data to be gathered to test and refine RRS programme theory. This approach used a variety of mixed methods to test the programme theories including individual and focus group interviews, observation and documentary analysis in a two stage process. A purposive sample of 75 key informants participated in individual and focus group interviews. Observation and documentary analysis of EWS compliance data and ALERT training records provided further evidence to support or refute the interview findings. Data was analysed using NVIVO8 to categorise interview findings and SPSS for ALERT documentary data. These findings were further synthesised by undertaking a within and cross case comparison to explain the factors enabling and constraining EWS and ALERT. Results A cross case analysis highlighted similarities, differences and factors enabling or constraining successful implementation across the case study sites. Findings showed that personal (confidence; clinical judgement; personality), social (ward leadership; communication), organisational (workload and staffing issues; pressure from managers to complete EWS audit and targets), educational (constraints on training; no clinical educator on ward) and cultural (routine task delegated) influences impact on EWS and acute care training outcomes. There were also differences noted between medical and surgical wards across both case sites. Conclusions Realist Evaluation allows refinement and development of the RRS programme theory to explain the realities of practice. These refined RRS programme theories are capable of informing the planning of future service provision and provide direction for enabling their success and sustainability. References: 1. McGaughey J, Blackwood B, O’Halloran P, Trinder T. J. & Porter S. (2010) A realistic evaluation of Track and Trigger systems and acute care training for early recognition and management of deteriorating ward–based patients. Journal of Advanced Nursing 66 (4), 923-932. Type of submission: Concurrent session Source of funding: Sandra Ryan Fellowship funded by the School of Nursing & Midwifery, Queen’s University of Belfast

A realistic evaluation of Early Warning Systems and acute care training in Northern Ireland: findings and recommendations

Statement of purpose The purpose of this concurrent session is to present the main findings and recommendations from a five year study evaluating the implementation of Early Warning Systems (EWS) and the Acute Life-threatening Events: Recognition and Treatment (ALERT) course in Northern Ireland. The presentation will provide delegates with an understanding of those factors that enable and constrain successful implementation of EWS and ALERT in practice in order to provide an impetus for change. Methods The research design was a multiple case study approach of four wards in two hospitals in Northern Ireland. It followed the principles of realist evaluation research which allowed empirical data to be gathered to test and refine RRS programme theory [1]. The stages included identifying the programme theories underpinning EWS and ALERT, generating hypotheses, gathering empirical evidence and refining the programme theories. This approach used a variety of mixed methods including individual and focus group interviews, observation and documentary analysis of EWS compliance data and ALERT training records. A within and across case comparison facilitated the development of mid-range theories from the research evidence. Results The official RRS theories developed from the realist synthesis were critically evaluated and compared with the study findings to develop a mid-range theory to explain what works, for whom in what circumstances. The findings of what works suggests that clinical experience, established working relationships, flexible implementation of protocols, ongoing experiential learning, empowerment and pre-emptive management are key to the success of EWS and ALERT implementation. Each concept is presented as ‘context, mechanism and outcome configurations’ to provide an understanding of how the context impacts on individual reasoning or behaviour to produce certain outcomes. Conclusion These findings highlight the combination of factors that can improve the implementation and sustainability of EWS and ALERT and in light of this evidence several recommendations are made to provide policymakers with guidance and direction for future policy development. References: 1. Pawson R and Tilley N. (1997) Realistic Evaluation. Sage Publications; London Type of submission: Concurrent session Source of funding: Sandra Ryan Fellowship funded by the School of Nursing & Midwifery, Queen’s University of Belfast

The development, assessment, and implementation of an evaluation of the EMPOWER Program at shelters for abused women /

The development, assessment, and implementation of a program evaluation instrument was carried out to evaluate the impact and efficacy of the EMPOWER Program. This intervention was created to educate residents at a shelter for abused women with an anticipated outcome of prevention. Participants included the staff and residents at 2 shelters in Southern Ontario. Client pre, post and follow-up measures were obtained and analyzed statistically and using keyword content analysis. A single staff measure was obtained and summarized using keyword content analysis. Qualitative results were suggestive of important change in participants. All women in the post and follow-up measures believed their participation in the EMPOWER Program provided them with the knowledge, skills, and confidence to avoid abusive relationships in the fliture. This transformational impact was repeatedly expressed in both resident and staff feedback. Limitations of this research, as well as suggestions for future study were discussed.