Students' knowledge about tobacco one year after receiving the PAL smoking prevention program

A review of literature revealed that the control of cigarette smoking could do more to improve health than any other single action in the field of preventive medicine. In Ontario, since 1989, both Public Health Units and Boards ofEducations have been mandated to provide educational studies related to tobacco use prevention. Even given this fact, there has been an increase in smoking behaviQurs at an earlier age and in females in particular. Smoking prevention progralns must use the most effective means to assist students to obtain the knowledge and skills required to remain or becom'e nonsmokers. In the Niagara Region, PAL smoking prevention programs are offered in some, but not all, schools. As a form of program evaluation, this research sought to determine if students who had PAL could answer correctly a greater number of smoking-related questions than students who did not have this program. Findings reported that students who had PAL in Grade 6 were able to correctly answer more knowledge-based questions (at a statistically significant level), could provide ways to refuse cigarettes at a greater rate, and were able to provide more reasons for remaining nonsmokers. Students who had smoking prevention programming re,ported smoking behavio'urs at a lower rate than those who did not receive this type of program.

Evaluation in Sport for Development: A Case Study of the Gansbaai Project, Football Foundation of South Africa, From A Critical Perspective

The purpose of this case study was to determine the effectiveness of sport for development (SDP) evaluation within one program in Gansbaai, South Africa through critical, independent participant inclusive program evaluation. Qualitative research was conducted on the Football Foundation of South Africa (FFSA), where semi-structured interview data were collected from administrators and participants, as were data from direct participant observations and organizational documents. Data analysis followed, according to Kvale and Brinkman’s (2008) methodology. FFSA goals were found, as were themes of social impact (i.e., regarding coach-player relationships, trust, and coaching impact on social integration). A further theme related to evaluation components and procedures. Further themes included life skill development, competition within programming, participants’ home life and social integration. Findings contribute to the SDP literature relating to program evaluation research and to FFSA administrators by providing an understanding of SDP program shortcomings, limitations, and suggested improvements.

Quantitative evaluation in health promotion : evaluation with confidence

Over the past 10 years or so, confidence intervals have become increasingly recognised in program evaluation and quantitative health measurement generally as the preferred way of reporting the accuracy of statistical estimates. Statisticians have found that the more traditional ways of reporting results - using P-values and hypothesis tests - are often very difficult to interpret and can be misleading. This is particularly the case when sample sizes are small and results are 'negative' (ie P>0.05); in these cases, a confidence interval can communicate much more information about the sample and, by inference, about the population. Despite this trend among statisticians and health promotion evaluators towards the use of confidence intervals, it is surprisingly difficult to find succinct and reasonably simple methods to actually compute a confidence interval. This is particularly the case for proportions or percentages. Much of the data which are analysed in health promotion are binary or categorical, rather than the quantities and continuous variables often found in laboratories or other branches of science, so there is a need for health promotion evaluators to be able to present confidence intervals for percentages or proportions. However, the most popular statistical analysis computer package among health promotion professionals, SPSS does not have a routine to compute a simple confidence interval for a proportion! To address this shortcoming, I present in this paper some fairly simple strategies for computing confidence intervals for population percentages, both manually and using the right computer software.

Use of the program explication method to explore the benefits of a service for homeless and marginalized young people

Maintaining the alignment between the dynamic development of health and social services and the rapidly advancing scientific evaluation literature is a central challenge facing service administrators. We describe “program explication,” a consulting method designed to assist services to identify and review implicit program logic assumptions against the evaluation literature. Program explication initially facilitates agency staff to identify and document service components and activities considered critical for improving client outcomes. Program assumptions regarding the relationship between service activities and client outcomes are then examined against available scientific evidence. We demonstrate the application of this method using an example of its use in reviewing a service for homeless young people operating in Melbourne, Australia, known as the Young People's Health Service (YPHS). The YPHS involved 21 activities organized within 4 components. The intended benefits of each of the activities were coherently articulated and logically consistent. Our literature search revealed moderate to strong evidence for around 1 quarter of the activities. The program explication method proved feasible for describing and appraising the YPHS service assumptions, thereby enhancing service evaluability.

Expanding children's food experiences : the impact of a school-based kitchen garden program

Objective: Evaluate achievement of the Stephanie Alexander Kitchen Garden Program in increasing child appreciation of diverse, healthy foods.

Design: Comparative 2-year study.

Setting: Six program and 6 comparison primary schools in rural and metropolitan Victoria, Australia, matched for socioeconomic status and size.

Participants: A total of 764 children in grades 3 to 6 (8–12 years of age) and 562 parents recruited. Retention rates at follow-up included 85% children and 75% parents.

Intervention: Each week of the school year, children spent 45 to 60 minutes in a garden class and 90 minutes in a kitchen class.

Phenomenon of interest: Program impact on children’s willingness to try new foods, capacity to describe foods, and healthy eating.

Analysis: Qualitative data analyzed using inductive thematic analysis. Quantitative data analyzed using random-effects linear regressions adjusted for school clustering.

Results: Child and parent qualitative and quantitative measures (if never tried before, odds ratio 2.0; confidence interval, 1.06–3.58) showed increases in children’s reported willingness to try new foods. No differences in articulation of food descriptions (program vs comparison groups). Qualitative evidence showed that the program extended its influence to healthy eating, but this was not reflected in the quantitative evidence.

Conclusions and Implications: Findings indicate program success in achieving its primary objective, meriting further program research.

Adaptation and validation of the Japanese version of the health education impact questionnaire (heiQ-J) for the evaluation of self-management education interventions

Aim:  

Measurement issues in the evaluation of chronic disease self-management programs

Objectives: 

Scaling up diabetes prevention in Victoria, Australia: policy development, implementation and evaluation

Objective
The Australian lifestyle intervention program Life! is only the second reported, large-scale diabetes prevention program. This paper describes the genesis and the successful establishment of Life! and its key outcomes for participants and implementation.

Research
Design and Methods Life!, a behavior change intervention, comprises six group sessions over eight months. The Victorian Department of Health funded Diabetes Australia-Victoria to implement the program. Experience of the Greater Green Triangle diabetes prevention implementation trial was used for intervention design, workforce development, training and infrastructure. Clinical and anthropometric data from participants, used for program evaluation, was recorded on a central database.

Results
Life! has a state-wide workforce of 302 trained facilitators within 137 organizations. 29,000 Victorians showed interest in Life! and 15,000 individuals have been referred to the program. In total, 8,412 participants commenced a Life! program between October 2007 and June 2011. 37% of the original participants completed the eight month program. Participants completing sessions one to five lost an average of 1·4 kg weight (p<0·001) and waist circumference of 2·5cm (p<0.001). Those completing six sessions lost an average of 2·4 kg weight (p<0·001) and waist circumference of 3·8 cm (p<0·001). The weight loss of 2.4 kg represents 2.7% of participants’ starting body weight.

Conclusion
The impact of Life! is attributable to applying available evidence for the systems design of the intervention, and collaboration between policy makers, implementers and evaluators using the principles of continuous quality improvement to support successful, large scale recruitment and implementation.

Unpacking black boxes: Mechanisms and theory building in evaluation

There is growing interest in the concept of ‘‘mechanism’’ across many areas of the social sciences. In the field of program and policy evaluation, a number of scholars have also emphasized the importance of causal mechanisms for explaining how and why programs work. However, there appears to be some ambiguity about the meaning and uses of mechanism-based thinking in both the social science and evaluation literature. In this article we attempt to clarify what is meant by mechanisms in the context of program evaluation by identifying three main characteristics of mechanisms and outlining a possible typology of mechanisms. A number of theoretical and practical implications for evaluators are also discussed, along with some precautions to consider when investigating mechanisms that might plausibly account for program outcomes.

Assessing and intervening on OSH programmes : effectiveness evaluation of the Wellworks-2 intervention in 15 manufacturing worksites

To develop a transparent and broadly applicable method for assessing occupational safety and health (OSH) programmes or management systems; (2) to assess OSH programmes in a sample of manufacturing worksites; and (3) to determine whether a management focused occupational health intervention results in greater improvement in OSH programmes compared to minimal intervention controls. OSH programmes were assessed using an adaptation of the US Occupational Safety & Health Administration's 1995 Program Evaluation Profile. Scores were generated from 91 binary indicator variables grouped under four "Essential Elements". Essential Element scores were weighted to contribute to an overall programme score on a 100 point scale. Seventeen large manufacturing worksites were assessed at baseline; 15 sites completed the 16 month intervention and follow up assessments. There was considerable variation in Essential Element scores across sites at baseline as judged by our instrument, particularly in "management commitment and employee participation" and "workplace analysis". Most sites scored highly on "hazard prevention and control" and "training and education". For overall OSH programme scores, most sites scored in the 60-80% range at baseline, with four sites scoring below 60%, suggesting weak programmes. Intervention sites showed greater improvements than controls in the four programme elements and in overall programme scores, with significantly greater improvements in "management commitment and employee participation". The OSH programme assessment method used is broadly applicable to manufacturing work settings, and baseline profiles suggest needs for improvement in OSH programmes in most such worksites. Despite a small sample size, results showed that sustained management focused intervention can result in improvement in these OSH programme measures.

A review of program evaluations in an Australian independent school: participants’ perspectives

This paper reports on ways in which one Australian independent school seeks to develop and sustain best practice and academic integrity in its programs through a system of ongoing program evaluation, involving a systematic, cyclical appraisal of the school’s suite of six faculties. A number of different evaluation methods have been and continue to be used, each developed to best suit the particular program under evaluation. In order to gain an understanding of the effectiveness of this process, we conducted a study into participants’ perceptions of the strengths and weaknesses of the four program evaluations undertaken between 2009 and 2011. Drawing on documentary analysis of the evaluation reports and analysis of questionnaire data from the study participants, a number of findings were generated. These findings are provided and discussed, together with suggestions about ways in which the conceptualisation and conduct of school program evaluations might be enhanced.

Linking secondary school physical education with community sport and recreation for girls: a process evaluation

The purpose of this study was to undertake a process evaluation to examine the reach, adoption and implementation of a school-community linked physical activity (PA) program for girls aged 12 - 15 years (School Years 7 - 9) using the RE-AIM framework.

Coproduction and Casula's Pacifica Program: Initial research findings

This research takes Casula Powerhouse Art Centre’s Pacifica program as a case study to investigate the ways in which museum and galleries are involved in coproduction with culturally diverse communities. Coproduction is defined here as:Museum and gallery practice conducted jointly with communities or other external partiesThe benefits of coproduction are that it leads to more effective and efficient public services (including arts and cultural services) while also building the skills and capacity of the community. However coproduction is not easy, particularly because it requires public service providers and communities to work in ‘equal and reciprocal’ relationships.As an organisation with strong and strategic alliances to its governing body (Liverpool City Council), Casula brings a strong capacity for coproduction. Internally it has support and commitment to coproduction from across the organisation. The staff at Casula bring exceptional relational skills. The organisation’s capacity to coproduce draws heavily on their skills as cultural brokers and experience in community cultural development practice. The communities Casula works with bring strong cultural knowledge and practice, along with a desire to maintain and preserve these community resources. Casula’s coproduction work also meets external political needs for public services to deliver increased public value as well as a greater diversity in the profile of arts audiences.The key challenge for Casula Powerhouse’s coproduction work is the extent to which it aims for joint delivery of public services through ‘equal and reciprocal’ relationships with the community, or uses coproduction as a tool for community engagement and audience development. Advocates of coproduction in the public sector argue for its value as a means of delivering more effective and efficient public services while at the same time building the skills and capacity of local communities. A critical element of coproduction according to these writers and scholars is the development and delivery of public services through ‘equal and reciprocal’ relationships between providers and users.The value of coproduction for Casula Powerhouse and the Pacifica program is its use as a means of community engagement and audience development. Coproduction is a feature of the components of Pacifica that enable the participation of the community and provide entry points for audiences to engage with contemporary art. Evidence of this approach to coproduction can be seen in the dual ‘stakeholder’ and ‘audience’ role that the community have within the Pacifica program. The community is therefore both a contributor to Pacifica and a beneficiary of this work. The benefits Casula Powerhouse receives from the community’s involvement in Pacifica are greater public value of its work and stronger engagement with communities and audiences.Although coproduction may not be the focus of all aspects of Pacifica, the involvement of Pacific Islander communities in the program results in exhibitions and public programs that are not typical contemporary art gallery offerings. Pacifica is further evidence of Casula Powerhouse’s innovative and entrepreneurial approach to gallery practice. The use of coproduction also ensures Pacifica offers an authentic and distinctive gallery experience.