An evaluation approach for a new paradigm - health care integration

This paper explores an approach to the implementation and evaluation of integrated health service delivery. It identifies the key issues involved in integration evaluation, provides a framework for assessment and identifies areas for the development of new tools and measures. A proactive role for evaluators in responding to health service reform is advocated.

An evaluation of tobacco brief intervention training in three Indigenous health care settings in north Queensland

Objective: To evaluate the pilot phase of a tobacco brief intervention program in three Indigenous health care settings in rural and remote north Queensland. Methods: A combination of in-depth interviews with health staff and managers and focus groups with health staff and consumers. Results: The tobacco brief intervention initiative resulted in changes in clinical practice among health care workers in all three sites. Although health workers had reported routinely raising the issue of smoking in a variety of settings prior to the intervention, the training provided them with an additional opportunity to become more aware of new approaches to smoking cessation. Indigenous health workers in particular reported that their own attempts to give up smoking following the training had given them confidence and empathy in offering smoking cessation advice. However, the study found no evidence that anybody had actually given up smoking at six months following the intervention. Integration of brief intervention into routine clinical practice was constrained by organisational, interpersonal and other factors in the broader socio-environmental context. Conclusions/implications: While modest health gains may be possible through brief intervention, the potential effectiveness in Indigenous settings will be limited in the absence of broader strategies aimed at tackling community-identified health priorities such as alcohol misuse, violence, employment and education. Tobacco and other forms of lifestyle brief. intervention need to be part of multi-level health strategies. Training in tobacco brief intervention should address both the Indigenous context and the needs of Indigenous health care workers.

An evaluation of a family psychoeducation program in community mental health

The aim of the research project was to identify the efficacy of the family psychoeducation program as a strategy for reducing the hospital admissions of young people. It also aimed to determine if the family psychoeducation program had an impact on the experience of caregiving and knowledge and satisfaction of services provided by the mental health service. A retrospective chart audit compared readmission history of 27 clients whose families attended a psychoeducation program with readmission history of a matched group of young people whose families did not attend the program. A telephone survey was conducted for both groups of families to investigate knowledge and understanding of services and burden of care. The results indicated that family participation in a brief multiple family psychoeducation program did not reduce the number or duration of admissions of the young people. There was no impact on the level of care for families who attended the psychoeducation program, however, this group showed some evidence of increased knowledge and understanding of services as compared to the control group.

Teamwork in Health and Rehabilitation Sciences: Evaluation of a multiprofessional workshop

This paper outlines a multiprofessional education workshop piloted and subsequently conducted with a cohort of 81 graduate entry students of occupational therapy, physiotherapy, speech pathology and audiology. The rationale for, and format of, the workshop is outlined, followed by comparisons between students' knowledge about teamwork prior to and after the four-hour workshop. The workshop was based on a real case scenario of a child with Developmental Coordination Disorder (DCD). Students completed pre- and post-workshop questionnaires about their knowledge of DCD, teamwork and the roles of various professionals and parents; and a post-workshop questionnaire about their views regarding the utility of the workshop, its strengths, and learning outcomes. The evaluation indicated that the workshop was overwhelmingly successful from the students' perspective in: (1) enhancing their understanding about DCD and its multifaceted impact on school age children; (2) developing a deeper appreciation of the importance of teamwork itself; (3) refining their understanding of their own profession's role and (4) developing an appreciation of the role of other professions and parents in working with children with complex needs, and their families. Limitations of this study and directions for future research are discussed.

Evaluation of the Western Australian Department of Health telehealth project

In 1999, the Department of Health in Western Australia began a telehealth project, which finished in 2004. The 75 videoconferencing sites funded by the project were part of a total state-wide videoconference network of 104 sites. During the period from January 2002 to December 2003, a total of 3266 consultations, case reviews and patient education sessions took place. Clinical use grew to 30% of all telehealth activity. Educational use was approximately 40% (1416 sessions) and management use was about 30% (1031 sessions). The average overhead cost per telehealth session across all regions and usage types was $A192. Meaningful comparisons of the results of the present study with other public health providers were difficult, because many of the available Websites on telehealth were out of date. Despite the successful use of telehealth to deliver clinical services in Western Australia, sustaining the effort in the post-project phase will present significant challenges.

Development and evaluation of a youth mental health community awareness campaign - The Compass Strategy

Background: Early detection and treatment of mental disorders in adolescents and young adults can lead to better health outcomes. Mental health literacy is a key to early recognition and help seeking. Whilst a number of population health initiatives have attempted to improve mental health literacy, none to date have specifically targeted young people nor have they applied the rigorous standards of population health models now accepted as best practice in other health areas. This paper describes the outcomes from the application of a health promotion model to the development, implementation and evaluation of a community awareness campaign designed to improve mental health literacy and early help seeking amongst young people. Method: The Compass Strategy was implemented in the western metropolitan Melbourne and Barwon regions of Victoria, Australia. The Precede-Proceed Model guided the population assessment, campaign strategy development and evaluation. The campaign included the use of multimedia, a website, and an information telephone service. Multiple levels of evaluation were conducted. This included a cross-sectional telephone survey of mental health literacy undertaken before and after 14 months of the campaign using a quasi-experimental design. Randomly selected independent samples of 600 young people aged 12 - 25 years from the experimental region and another 600 from a comparison region were interviewed at each time point. A series of binary logistic regression analyses were used to measure the association between a range of campaign outcome variables and the predictor variables of region and time. Results: The program was judged to have an impact on the following variables, as indicated by significant region-by-time interaction effects ( p < 0.05): awareness of mental health campaigns, self-identified depression, help for depression sought in the previous year, correct estimate of prevalence of mental health problems, increased awareness of suicide risk, and a reduction in perceived barriers to help seeking. These effects may be underestimated because media distribution error resulted in a small amount of print material leaking into the comparison region. Conclusion: We believe this is the first study to apply the rigorous standards of a health promotion model including the use of a control region to a mental health population intervention. The program achieved many of its aims despite the relatively short duration and moderate intensity of the campaign.

Evaluation of NHMRC funded research completed in 1992, 1997 and 2003: gains in knowledge, health and wealth

Objective: To report on strategies for, and outcomes of, evaluation of knowledge (publications), health and wealth (commercial) gains from medical research funded by the Australian Government through the National Health and Medical Research Council (NHMRC). Design and methods: End-of-grant reports submitted by researchers within 6 months of completion of NHMRC funded project grants which terminated in 2003 were used to capture self-reported publication number, health and wealth gains. Self-reported gains were also examined in retrospective surveys of grants completed in 1992 and 1997 and awards primarily supporting people (“people awards”) held between 1992 and 2002. Results: The response rate for the 1992 sample was too low for meaningful analysis. The mean number of publications per grant in the basic biomedical, clinical and health services research areas was very similar in 1997 and 2003. The publication output for population health was somewhat higher in the 2003 than in the 1997 analysis. For grants completed in 1997, 24% (31/131) affected clinical practice; 14% (18/131) public health practice; 9% (12/131) health policy; and 41% (54/131) had commercial potential with 20% (26/131) resulting in patents. Most respondents (89%) agreed that NHMRC people awards improved their career prospects. Interpretation is limited by the relatively low response rates (50% or less). Conclusions: A mechanism has been developed for ongoing assessment of NHMRC funded research. This process will improve accountability to the community and to government, and refine current funding mechanisms to most efficiently deliver health and economic returns for Australia.

Development of the student placement evaluation form: A tool for assessing student fieldwork performance

Evaluation of students undertaking fieldwork education placements is a critical process in the health professions. As training programs and practice evolve, systems for assessing students need to be reviewed and updated constantly. In 1995, staff of the occupational therapy training program at the University of Queensland, Australia decided to develop a new tool for assessing student fieldwork performance. Using an action research methodology, a team developed the Student Placement Evaluation Form, a flexible and comprehensive criterion-referenced evaluation tool. The present paper examines action research as an appropriate methodology for considering real-life organisational problems in a systematic and participatory manner. The action research cycles undertaken, including preliminary information gathering, tool development, trial stages and current use of the tool, are detailed in the report. Current and future development of the tool is also described.