Considerations in the design of a mixed-method cluster evaluation of a community programme for 'at-risk' young people

This article discusses the design of a comprehensive evaluation of a community development programme for young people 'at-risk' of self-harming behaviour. It outlines considerations in the design of the evaluation and focuses on the complexities and difficulties associated with the evaluation of a community development programme. The challenge was to fulfil the needs of the funding body for a broad, outcome-focused evaluation while remaining close enough to the programme to accurately represent its activities and potential effects at a community level. Specifically, the strengths and limitations of a mixed-method evaluation plan are discussed with recommendations for future evaluation practice.

Development and evaluation of a youth mental health community awareness campaign - The Compass Strategy

Background: Early detection and treatment of mental disorders in adolescents and young adults can lead to better health outcomes. Mental health literacy is a key to early recognition and help seeking. Whilst a number of population health initiatives have attempted to improve mental health literacy, none to date have specifically targeted young people nor have they applied the rigorous standards of population health models now accepted as best practice in other health areas. This paper describes the outcomes from the application of a health promotion model to the development, implementation and evaluation of a community awareness campaign designed to improve mental health literacy and early help seeking amongst young people. Method: The Compass Strategy was implemented in the western metropolitan Melbourne and Barwon regions of Victoria, Australia. The Precede-Proceed Model guided the population assessment, campaign strategy development and evaluation. The campaign included the use of multimedia, a website, and an information telephone service. Multiple levels of evaluation were conducted. This included a cross-sectional telephone survey of mental health literacy undertaken before and after 14 months of the campaign using a quasi-experimental design. Randomly selected independent samples of 600 young people aged 12 - 25 years from the experimental region and another 600 from a comparison region were interviewed at each time point. A series of binary logistic regression analyses were used to measure the association between a range of campaign outcome variables and the predictor variables of region and time. Results: The program was judged to have an impact on the following variables, as indicated by significant region-by-time interaction effects ( p < 0.05): awareness of mental health campaigns, self-identified depression, help for depression sought in the previous year, correct estimate of prevalence of mental health problems, increased awareness of suicide risk, and a reduction in perceived barriers to help seeking. These effects may be underestimated because media distribution error resulted in a small amount of print material leaking into the comparison region. Conclusion: We believe this is the first study to apply the rigorous standards of a health promotion model including the use of a control region to a mental health population intervention. The program achieved many of its aims despite the relatively short duration and moderate intensity of the campaign.

Clinical evaluation of the WOMAC 3.0 OA Index in numeric rating scale format using a computerized touch screen version

Background: The Western Ontario and McMaster Universities (WOMAC) Osteoarthritis Index is a previously described self-administered questionnaire covering three domains: pain, stiffness and function. It has been validated in patients with osteoarthritis (OA) of the hip or knee in a paper-based format. Aim: To validate the WOMAC 3.0 using a numerical rating scale in a computerized touch screen format allowing immediate evaluation of the questionnaire. In the computed version cartoons, written and audio instruments were included in order facilitate application. Methods: Fifty patients, demographically balanced, with radiographically proven primary hip or knee OA completed the classical paper and the new computerized WOMAC version. Subjects were randomized either to paper format or computerized format first to balance possible order effects, Results: The intra-class correlation coefficients for pain, stiffness and function values were 0.915, 0.745 and 0.940, respectively. The Spearman correlation coefficients for pain, stiffness and function were 0.88, 0.77 and 0.87, respectively. Conclusion: These data indicate that the computerized WOMAC OA index 3.0 is comparable to the paper WOMAC in all three dimensions. The computerized version would allow physicians to get an immediate result and if present a direct comparison with a previous exam. (C) 2002 OsteoArthritis Research Society International. Published by Elsevier Science Ltd. All rights reserved.

Development and evaluation of the Clinician Suicide Risk Assessment Checklist

This paper describes the development and evaluation of a new instrument – the Clinician Suicide Risk Assessment Checklist (CSRAC). The instrument assesses the clinician’s competency in three areas: clinical interviewing, assessment of specific suicide risk factors, and formulating a management plan. A draft checklist was constructed by integrating information from 1) literature review 2) expert clinician focus group and 3) consultation with experts. It was utilised in a simulated clinical scenario with clinician trainees and a trained actor in order to test for inter-rater agreement. Agreement was calculated and the checklist was re-drafted with the aim of maximising agreement. A second phase of simulated clinical scenarios was then conducted and inter-rater agreement was calculated for the revised checklist. In the first phase of the study, 18 of 35 items had inadequate inter-rater agreement (60%>), while in the second phase, using the revised version, only 3 of 39 items failed to achieve adequate inter-rater agreement. Further evidence of reliability and validity are required. Continued development of the CSRAC will be necessary before it can be utilised to assess the effectiveness of risk assessment training programs.

Skin cancer medicine in primary care: towards an agenda for quality health outcomes

The number of skin cancer clinics functioning within Australia's primary care environment is increasing rapidly, and significant concerns have been raised about the type and quality of work done by some doctors in some clinics. Mainstream general practice is threatened by perceived fragmentation, and specialist practice in dermatology and plastic surgery is threatened by encroachment into their domains of practice. We propose an agenda of training, standards, accreditation, audit and research to ensure that skin cancer clinics provide optimal health outcomes for patients.