The development of the extended adolescent injury checklist (E-AIC) : a measure for injury prevention program evaluation

The Extended Adolescent Injury Checklist (E-AIC), a self-report measure of injury based on the model of the Adolescent Injury Checklist (AIC), was developed for use in the evaluation of school-based interventions. The three stages of this development involved focus groups with adolescents and consultations with medical staff, pilot testing of the revised AIC in a high school context, and use of the finalised checklist in pre- and post-questionnaires to examine its utility. Results revealed that responses to the final version of the E-AIC were meaningful and remained consistent over time. The E-AIC appears to be a promising measure of adolescent injury that is simple, time-efficient and appropriate for use in the evaluation of school-based injury prevention programs.

The Mirror Room Project: A Critical Ethnographic Program Evaluation of a Teaching Personal and Social Responsibility Based Youth Development Program

Amongst a host of other benefits, proper physical education has the possibility to create a safe place where responsibility can be transferred from the teacher/facilitator, to the student. This is especially true with an underserved population. This critical program evaluation of the program CHARM was done for the purpose of program improvement. This research was a place for participants to share their experiences of the program. The participants were 5 underserved youth, 5 undergraduate students, 3 teachers and 1 graduate student. Observations, interviews, and document analysis were used to gather data. Data was analyzed using a first level read-through, and two second-level analyses. Summaries were written, and cross-case analyses were completed. The main finding of the research was the development of a Handbook, which is a guide to running the program. Secondary findings include issues of program structure, goal setting, meaningful relationships, roles, SNAP, and an outlier in the data.

Sources of selection bias in evaluating social programs: An interpretation of conventional measures and evidence on the effectiveness of matching as a program evaluation method

This paper decomposes the conventional measure of selection bias in observational studies into three components. The first two components are due to differences in the distributions of characteristics between participant and nonparticipant (comparison) group members: the first arises from differences in the supports, and the second from differences in densities over the region of common support. The third component arises from selection bias precisely defined. Using data from a recent social experiment, we find that the component due to selection bias, precisely defined, is smaller than the first two components. However, selection bias still represents a substantial fraction of the experimental impact estimate. The empirical performance of matching methods of program evaluation is also examined. We find that matching based on the propensity score eliminates some but not all of the measured selection bias, with the remaining bias still a substantial fraction of the estimated impact. We find that the support of the distribution of propensity scores for the comparison group is typically only a small portion of the support for the participant group. For values outside the common support, it is impossible to reliably estimate the effect of program participation using matching methods. If the impact of participation depends on the propensity score, as we find in our data, the failure of the common support condition severely limits matching compared with random assignment as an evaluation estimator.

A Process and Outcome Evaluation of the Under the Limit (UTL) Therapeutic Drink Driving Program for Recidivist and High Range Offenders

Drink driving is a major public health issue and this report examines the experiences of convicted offenders who participated in an established drink driving rehabilitation program Under the Limit (UTL). Course completers were surveyed at least three months after they had finished the 11-week UTL course. The aim of this study was to examine whether the UTL program reduced the level of alcohol consumption either directly as a result of participation in the UTL drink driving program or through increased use of community alcohol program by participants. The research involved a self-report outcome evaluation to determine whether the self-reported levels of alcohol use after the course had changed from the initial alcohol use reported by offenders. The findings are based on the responses of 30 drink-driving offenders who had completed the UTL program (response rate: 20%). While a process evaluation was proposed in the initial application, the low response rate meant that this follow up research was not feasible. The response rate was low for two reasons, it was difficult to: recruit participants who consented to follow up, and subsequently locate and survey those who had consented to involvement.

Impact of definitions of loss to follow-up (LTFU) in antiretroviral therapy program evaluation: variation in the definition can have an appreciable impact on estimated proportions of LTFU

OBJECTIVE To examine the impact of different definitions of loss to follow-up (LTFU) on estimates of program outcomes in cohort studies of patients on antiretroviral therapy (ART). STUDY DESIGN AND SETTING We examined the impact of different definitions of LTFU using data from the International Epidemiological Databases to Evaluate AIDS-Southern Africa. The reference approach, Definition A, was compared with five alternative scenarios that differed in eligibility for analysis and the date assigned to the LTFU outcome. Kaplan-Meier estimates of LTFU were calculated up to 2 years after starting ART. RESULTS Estimated cumulative LTFU were 14% and 22% at 12 and 24 months, respectively, using the reference approach. Differences in the proportion LTFU were reported in the alternative scenarios with 12-month estimates of LTFU varying by up to 39% compared with Definition A. Differences were largest when the date assigned to the LTFU outcome was 6 months after the date of last contact and when the site-specific definition of LTFU was used. CONCLUSION Variation in the definitions of LTFU within cohort analyses can have an appreciable impact on estimated proportions of LTFU over 2 years of follow-up. Use of a standardized definition of LTFU is needed to accurately measure program effectiveness and comparability between programs.

Program Evaluation of an After-School Socialization Group Program

Use of evidence-based practice is becoming more expected and necessary for mental health practitioners. This allows for proof, accountability, and rigorous standards to be upheld, facilitates healthcare reimbursement, and provides a wider range of services to more clients. Appropriate monitoring and outcome measurement is essential to determine the effectiveness of a given intervention. An organization providing group social skills interventions for children 7-18 years was analyzed to facilitate the best plan for evaluating treatment effectiveness. Measurable goals and objectives consistent with the organizations mission and values were developed. Appropriate social skills measurement tools were identified. Strengths and weaknesses of each measure were compared, and existing literature was reviewed to ensure cohesion between this evaluation and current standards in literature. Parent report, self report, and teacher report on Piers-Harris II, BASC-2, and Skills Improvement System Rating Scales were determined to be the most relevant measures of social skills development. A timeline for administration and plan for how to implement measurement and use data was suggested as well as considerations for future research.

Barriers and facilitators of a designated driver program

The general aim of designated driver programs is to reduce the level of drink driving by encouraging potential drivers to travel with a driver who has abstained from (or at least limited) consuming alcohol. Designated driver programs are quite widespread around the world, however a limited number have been rigorously evaluated. This paper reports the qualitative results from an evaluation of a designated driver program known as ‘Skipper’, in a provincial city in Queensland. Focus groups were conducted with 108 individuals from the intervention area. These focus groups aimed to assess the barriers and facilitators to the programs’ effectiveness by obtaining information about the patrons’ views on various aspects of the program, as well as designated driver and travelling after drinking more generally. A brief questionnaire was also given to participants in order to present responses in terms of the participants’ characteristics. Results suggest general support for the designated driver concept and the ‘Skipper’ program specifically. Facilitating factors reported by participants included the media coverage highlighting the risks associated with drink driving and the social acceptability of choosing not to drink. However, there was also some suggestion that the impact of the program was mainly to encourage those who already engage in designated driver behaviour to continue doing so, rather than encouraging the uptake of the behaviour among potential new users. Some of the suggested barriers to this kind of behaviour change include: social pressure to drink; alcohol dependency; and a failure to plan ahead.

"Listening, hearing and drawing things together to work on solutions" : NAPCAN Play a Part, Queensland, 2010-2013, Evaluation-Research Report

Responding to the idea of child friendly communities, Play a Part is an innovative program advancing preventative strategies for children and young people to minimise exposure to abuse and neglect. The program was developed ensuing an increase in notifications of suspected child abuse and neglect in 2007. Now completing the second phase, the program is a community engagement strategy that aims to prevent child abuse. Play a Part is described as “a whole of community approach to creating child friendly communities” (NAPCAN, 2012). The Play a Part program was piloted between 2007 and 2010 in five southeast Queensland communities, and is currently operating in parts of Logan City region and the Redlands region. To assess the merit of the second phase of the program the Children and Youth Research Centre at Queensland University of Technology was contracted to undertake an evaluation-research at the beginning of 2013.

The Brazilian Zero Hunger social program implications in achieving sustainable development

This article aims to analyse the progress taken by Brazil towards the accomplishment of a sustainable development, mainly highlighting the success of the Zero Hunger social programme in achieving the elimination of starvation. As one of the essential pillars to the sustainable development, the State’s social performance demands actions directed to the elimination of extreme poverty and hunger, establishing a basis for equitable growth. It is necessary to clarify the juridical and constitutional framework of Brazil, aiming to emphasise the Zero Hunger importance in achieving the Brazilian Republic’s goals and reaching an international pattern on sustainable development. Also, it will be stressed that the international order influences Brazilian domestic law and is one of the main aspects for the Zero Hunger program development.

Methodological convergence of program evaluation designs

Resumen tomado de la publicación