Learning What Works: Demonstrating Practice Effectiveness with Children and Families Through Retrospective Investigation

Intensive family preservation services (IFPS), designed to stabilize at-risk families and avert out-of-home care, have been the focus of many randomized, experimental studies. Employing a retrospective “clinical data-mining” (CDM) methodology (Epstein, 2001), this study makes use of available information extracted from client records in one IFPS agency over the course of two years. The primary goal of this descriptive and associational study was to gain a clearer understanding of IFPS service delivery and effectiveness. Interventions provided to families are delineated and assessed for their impact on improved family functioning, their impact on the reduction of family violence, as well as placement prevention. Findings confirm the use of a wide range of services consistent with IFPS program theory. Because the study employs a quasi-experimental, retrospective use of available information, clinical outcomes described cannot be causally attributed to interventions employed as with randomized controlled trials. With regard to service outcomes, findings suggest that family education, empowerment services and advocacy are most influential in placement prevention and in ameliorating unmanageable behaviors in children as well as the incidence of family violence.

Looking Inside The Black Box

Intensive family preservation services (IFPS), designed to stabilize at-risk families and avert out-of-home care, have been the focus of many randomized, experimental studies. The emphasis on "gold-standard" evaluation of IFPS has resulted in fewer "black box" studies that describe actual IFPS service patterns and the fidelity with which they adhere to IFPS program theory. Intervention research is important to the advancement of programs designed to protect the safety of children, improve family functioning, as well as prevent out-of-home placement. Employing a retrospective “clinical data-mining” (CDM) methodology, this exploratory study of Families First, an IFPS program, makes use of available information extracted from client records to describe interventions and service patterns provided over a two year period. This study uncovers actual IFPS service patterns, demonstrates IFPS program fidelity, as well as reveals the usefulness of CDM as a social work research methodology. These findings are particularly valuable for program planning and treatment, policy development and evidence-based practice research.

Diffusion and implementation of a tobacco prevention program: A replication study

Tobacco use is a major health hazard, and the onset of tobacco use occurs almost entirely in the teenage years. For this reason, schools are an ideal site for tobacco prevention programs. Although studies have shown that effective school-based tobacco prevention programs exist, all too frequently these programs are not used. In order for effective programs to achieve their potential impact, strategies for speeding the diffusion of these programs to school districts and seeing that, once adopted, programs are implemented as they are intended, must be developed and tested.^ This study (SC2) set out to replicate the findings of an earlier quasi-experimental study (The Smart Choices Diffusion Study, or SC1) in which strategies based on diffusion theory and social learning theory were found to be effective in encouraging adoption and implementation of an effective tobacco prevention program in schools. To increase awareness and encourage adoption, intervention strategies in both studies utilized opinion leaders, messages highlighting positive aspects of the program, and modeling of benefits and effective use through videotape and newsletters. To encourage accurate implementation of the curriculum, teacher training for the two studies utilized videotaped modeling and practice of activities by teachers. SC2 subjects were 38 school districts that make up one of Texas' 20 education service regions. These districts had served as the comparison group in SC1, and findings for the SC1 comparison and intervention groups were utilized as historic controls.^ SC2 achieved a 76.3% adoption rate and found that an average of 84% of the curriculum was taught with an 82% fidelity to methods utilized by the curriculum. These rates and rates for implementation of dissemination strategies were equal to or greater than corresponding rates for SC1. The proportion of teachers implementing the curriculum in SC2 was found to be equal to SC1's video-trained districts but lower than the SC1 workshop-trained group.^ SC2's findings corroborate and support the findings from the earlier study, and increase our confidence in its findings. Taken together, the findings from SC2 and SC1 point to the effectiveness of their theory-based intervention strategies in encouraging adoption and accurate implementation of the tobacco prevention curriculum. ^

Advancing the theory of collective empowerment: A qualitative study

In order to fully describe the construct of empowerment and to determine possible measures for this construct in racially and ethnically diverse neighborhoods, a qualitative study based on Grounded Theory was conducted at both the individual and collective levels. Participants for the study included 49 grassroots experts on community empowerment who were interviewed through semi-structured interviews and focus groups. The researcher also conducted field observations as part of the research protocol.^ The results of the study identified benchmarks of individual and collective empowerment and hundreds of possible markers of collective empowerment applicable in diverse communities. Results also indicated that community involvement is essential in the selection and implementation of proper measures. Additional findings were that the construct of empowerment involves specific principles of empowering relationships and particular motivational factors. All of these findings lead to a two dimensional model of empowerment based on the concepts of relationships among members of a collective body and the collective body's desire for socio-political change.^ These results suggest that the design, implementation, and evaluation of programs that foster empowerment must be based on collaborative ventures between the population being served and program staff because of the interactive, synergistic nature of the construct. In addition, empowering programs should embrace specific principles and processes of individual and collective empowerment in order to maximize their effectiveness and efficiency. And finally, the results suggest that collaboratively choosing markers to measure the processes and outcomes of empowerment in the main systems and populations living in today's multifaceted communities is a useful mechanism to determine change. ^

Implementation analysis of the Texas Vendor Drug Program

An agency is accountable to a legislative body in the implementation of public policy. It has a responsibility to ensure that the implementation of that policy is consistent with its statutory objectives.^ The analysis of the effectiveness of implementation of the Vendor Drug Program proceeded in the following manner. The federal and state roles and statutes pursuant to the formulation of the Vendor Drug Program were reviewed to determine statutory intent and formal provisions. The translation of these into programmatic details was examined focusing on the factors impacting the implementation process. Lastly, the six conditions outlined by Mazmanian and Sabatier as criteria for effective implementation, were applied to the implementation of the Vendor Drug Program to determine if the implementation was effective in relation to consistency with statutory objectives.^ The implementation of the statutes clearly met four of the six conditions for effective implementation: (1) clear and consistent objectives; (2) a valid causal theory; (3) structured the process to maximize agency and target compliance with the objectives; and (4) had continued support of constituency groups and sovereigns.^ The implementation was basically consistent with the statutory objectives, although the determination of vendor reimbursement has had and continues to have problems. ^

The Texas driver responsibility program: A preliminary analysis of the impact on impaired driving and trauma system funding

Objective. In 2003, the State of Texas instituted the Driver Responsibility Program (TDRP), a program consisting of a driving infraction point system coupled with a series of graded fines and annual surcharges for specific traffic violations such as driving while intoxicated (DWI). Approximately half of the revenues generated are earmarked to be disbursed to the state's trauma system to cover uncompensated trauma care costs. This study examined initial program implementation, the impact of trauma system funding, and initial impact on impaired driving knowledge, attitudes and behaviors. A model for targeted media campaigns to improve the program's deterrence effects was developed. ^ Methods. Data from two independent driver survey samples (conducted in 1999 and 2005), department of public safety records, state health department data and a state auditor's report were used to evaluate the program's initial implementation, impact and outcome with respect to drivers' impaired driving knowledge, attitudes and behavior (based on constructs of social cognitive theory) and hospital uncompensated trauma care funding. Survey results were used to develop a regression model of high risk drivers who should be targeted to improve program outcome with respect to deterring impaired driving. ^ Results. Low driver compliance with fee payment (28%) and program implementation problems were associated with lower surcharge revenues in the first two years ($59.5 million versus $525 million predicted). Program revenue distribution to trauma hospitals was associated with a 16% increase in designated trauma centers. Survey data demonstrated that only 28% of drivers are aware of the TDRP and that there has been no initial impact on impaired driving behavior. Logistical regression modeling suggested that target media campaigns highlighting the likelihood of DWI detection by law enforcement and the increased surcharges associated with the TDRP are required to deter impaired driving. ^ Conclusions. Although the TDRP raised nearly $60 million in surcharge revenue for the Texas trauma system over the first two years, this study did not find evidence of a change in impaired driving knowledge, attitudes or behaviors from 1999 to 2005. Further research is required to measure whether the program is associated with decreased alcohol-related traffic fatalities. ^

Process evaluation of intervention program for prevention of early sexual activity and alcohol use among Hispanic teenagers

Hispanic teens are a high-risk population for initiation of early sexual activity and alcohol use which in turn has numerous social and health consequences. One strategy to address prevention of these behaviors is to implement a capacity building intervention that promotes parent child communication, encompasses their cultural values and community participation. This study describes the process evaluation of a pilot intervention program amongst Hispanic teens and their families living along the Texas-Mexico border. “Girls Lets Talk” is a small group intervention with 10-14 year old teens and their female adult family members that involves education regarding effects of alcohol use and sexual activity as well as activities for monitoring and refusal skills to prevent risky behaviors. Two waves of the program each consisting of at least seven mother daughter dyads were conducted. During the designing process, community advisory board meetings and focus groups were held to review course materials and ensure they were appropriate to the Mexican American culture. Parent and adolescent surveys were administered at the beginning and end of the intervention to assess for psychosocial outcome variables. All sessions received high mean satisfactory scores (mean of 4.00 or better on a five point scale) for both adult and adolescent participants. Qualitative feedback was obtained via debriefing sessions to evaluate experience as well as alter recruitment strategies. A Wilcoxon Sign Rank analysis of the pre and post intervention surveys was done that showed significant changes in some outcome variables such as intentions and confidence for monitoring behaviors for adults and beliefs regarding sexual activity. “Girls Lets Talk” is a promising example of how a process evaluation plan can help develop a theory based health promotion program using the community based participatory research approach. The intervention may also be effective in altering intentions and enhancing self-efficacy among parents and teens in order to decrease risky behaviors such as early sexual activity and alcohol use.^

Development and evaluation of a nutrition education program designed to increase fruit and vegetable consumption in girls, ages 9--12

The purpose of this study is to evaluate the theory-based Eat 5 nutrition badge. It is designed to increase fruit and vegetable (F&V) intake in 4th-6th grade junior Girl Scouts. Twenty-two troops were recruited and randomized by grade level (4th, 5th, 6th, or mixed) into either the intervention or control conditions. The leaders in the intervention condition received a brief training and the materials and conducted the program with their troops during four meetings. The Girl Scouts in the intervention condition completed 1-day Food Frequency Questionnaires and Nutrition Questionnaires both before and after completing the Eat 5 badge, and a third measurement of F&V intake three months after the posttest. Girl Scouts in the control condition were only evaluated at the three time periods.^ The primary hypotheses were that the Girl Scouts in the intervention condition would increase their daily intake of fruits and vegetables at both the posttest and three months later, compared to the Girl Scouts in the control condition. Other study questions investigated the impact of the Eat 5 program on intervening variables such as knowledge, self-efficacy, barriers, norms, F&V preference, and F&V selection and preparation skills.^ A nested ANOVA, with troop as the unit of analysis nested within condition, was used to assess the effects of the program. Pretest F&V intake and grade level were used as covariates. Pretest mean F&V intake for the total sample of 210 girls was 2.50 servings per day; 3.0 for the intervention group (n = 101). Significant increases in F&V intake (to 3.4 servings per day), knowledge, and fruit and vegetable preference were found for the intervention condition troops compared to the troops in the control condition. Three months later, the mean F&V intake had returned to pretest levels.^ This study indicates that social groups such as Girl Scouts can provide a channel for nutrition education. Long term effects were not sustained by the intervention; a possible cause was the lack of change in self-efficacy. Therefore, additional interventions are recommended such as booster lessons to maintain increased F&V intake by Girl Scouts. ^

THE POTENTIAL EFFECTS OF A COMMUNITY DEVELOPMENT PROGRAM ON DEMOGRAPHIC VARIABLES IN HIMALAYAN INDIAN VILLAGES

A community development program operating in the mountains of North India was studied to assess its potential effects on mortality, fertility and migration patterns in the community which it served. The development program operated in Jaunpur Block, Tehri-Garhwal District, Uttar Pradesh State. Two comparable villages in the district were studied. The development program had been working in one for two years, and the other was completely untouched by the program.^ Since not enough time had elapsed since the beginning of the development program's work for any effects on demographic patterns to be visable in Jaunpur Block, this study looked to attitudes of village residents as indicators of future demographic trends. Existing demographic patterns and their interrelationship with socio-religious customs were examined in the test village. A questionnaire was then administered to ascertain attitudinal differences between the residents of the test village and the control village.^ The primary work of the community development program was to train women as village health workers. The results of the attitudinal comparison of the residents of the two villages showed a marked difference in attitudes relating to the position of women in society. The data showed a higher esteem for women in the test village than in the control village, and it is argued that this difference may be attributable to the work of the development program.^ Predicting future demographic trends in Jaunpur Block on the basis of the observed difference in villagers' attitudes toward the status of women is speculatory. Jaunpur Block appears to be in the demographic stage of pre-transition, maintaining relatively high rates of both mortality and fertility. Based on demographic transition theory the next significant change in demographic patterns in Jaunpur is predicted to be a decline in mortality, and an increase in the status of women is unrelated to this prediction.^ The community development program which was studied terminated unexpectedly during the time of this study. A case study of the program's final months is presented, and speculation on the future course of demographic trends in Jaunpur Block is related to the possible alternatives for future development in the area. ^

Using intervention mapping to develop a community garden program at the women's home

This culminating experience was a practice based intervention conducted by an organization, utilizing an intervention mapping approach for the program planning. It took place summer 2010 through spring 2011 and included incorporating a community garden into the Gusto wellness program at The Women's Home. This organization offers long-term residential care, and therapeutic services. Literature relating to community gardens and nutrition behavior change was reviewed. Short-term objectives included: 1) Conducting a needs assessment using focus groups, 2) Designing gardening program components based on intervention mapping guidelines, 3) Constructing a garden bed at Midtown Community Garden for use of The Women's Home, 4) Planning and implementing gardening education, and 5) Assessing feasibility of the garden program. The target population included 24 residents living at the residential dormitory of The Women's Home at the time of this project. The major variables are intervention mapping constructs including: 1) Needs assessment, 2) Preparing matrices of change objectives, 3) Selecting theory-informed intervention methods and practical strategies, 4) Producing program components and materials, 5) Planning program adoption, implementation, and sustainability, and 6) Planning for evaluation. The specific focus was lack of access to fresh fruits and vegetables (FV) for this population. Focus group responses revealed interest in community garden participation. Matrices of change were developed for lack of FV access based on performance objectives for behavioral and environmental factors and related determinants and theory. Methods and strategies were developed to implement a community garden and encourage participation. Program components included initiating a garden club, networking activities, creating gardening curriculum, and participating at Midtown Community Garden. Adoption and implementation performance objectives were outlined, and many were carried out. Evaluation questions were designed and outcomes of the garden project were discussed. ^ Outcomes of the project included exposure of garden topics and activities for The Women's Home residents, focus group responses revealing an interest in gardening among this population, gardening program components designed based on intervention mapping steps, and a constructed garden bed that was used for planting vegetables and flowers through fall 2010. Limited resources and budget along with a lack of a residential coordinator at The Women's Home were the main limiting factors for this project. Future garden projects can be developed using the intervention mapping process.^

Effects of a health advocacy program to promote physical activity and nutrition behaviors among adolescents: An evaluation of the Young Leaders for Healthy Change program

Background. This study was designed to evaluate the effects of the Young Leaders for Healthy Change program, an internet-delivered program in the school setting that emphasized health advocacy skills-development, on nutrition and physical activity behaviors among older adolescents (13–18 years). The program consisted of online curricular modules, training modules, social media, peer and parental support, and a community service project. Module content was developed based on Social Cognitive Theory and known determinants of behavior for older adolescents. ^ Methods. Of the 283 students who participated in the fall 2011 YL program, 38 students participated in at least ten of the 12 weeks and were eligible for this study. This study used a single group-only pretest/posttest evaluation design. Participants were 68% female, 58% white/Caucasian, 74% 10th or 11th graders, and 89% mostly A and/or B students. The primary behavioral outcomes for this analysis were participation in 60-minutes of physical activity per day, 20-minutes of vigorous- or moderate- intensity physical activity (MVPA) participation per day, television and computer time, fruit and vegetable (FV) intake, sugar-sweetened beverage intake, and consumption of breakfast, home-cooked meals, and fast food. Other outcomes included knowledge, beliefs, and attitudes related to healthy eating, physical activity, and advocacy skills. ^ Findings. Among the 38 participants, no significant changes in any variables were observed. However, among those who did not previously meet behavioral goals there was an 89% increase in students who participated in more than 20 minutes of MVPA per day and a 58% increase in students who ate home-cooked meals 5–7 days per week. The majority of participants met program goals related to knowledge, beliefs, and attitudes prior to the start of the program. Participants reported either maintaining or improving to the goal at posttest for all items except FV intake knowledge, taste and affordability of healthy foods, interest in teaching others about being healthy, and ease of finding ways to advocate in the community. ^ Conclusions. The results of this evaluation indicated that promoting healthy behaviors requires different strategies than maintaining healthy behaviors among high school students. In the school setting, programs need to target the promotion and maintenance of health behaviors to engage all students who participate in the program as part of a class or club activity. Tailoring the program using screening and modifying strategies to meet the needs of all students may increase the potential reach of the program. The Transtheoretical Model may provide information on how to develop a tailored program. Additional research on how to utilize the constructs of TTM effectively among high school students needs to be conducted. Further evaluation studies should employ a more expansive evaluation to assess the long-term effectiveness of health advocacy programming.^

Development and evaluation of a video-based modeling strategy for the dissemination phase of a school health tobacco -use prevention program

Even the best school health education programs will be unsuccessful if they are not disseminated effectively in a manner that encourages classroom adoption and implementation. This study involved two components: (1) the development of a videotape intervention to be used in the dissemination phase of a 4-year, NCI-funded diffusion study and (2) the evaluation of that videotape intervention strategy in comparison with a print (information transfer) strategy. Conceptualization has been guided by Social Learning Theory, Diffusion Theory, and communication theory. Additionally, the PRECEDE Framework has been used. Seventh and 8th grade classroom teachers from Spring Branch Independent School District in west Houston participated in the evaluation of the videotape and print interventions using a 57-item preadoption survey instrument developed by the UT Center for Health Promotion Research and Development. Two-way ANOVA was used to study individual score differences for five outcome variables: Total Scale Score (comprised of 57 predisposing, enabling, and reinforcing items), Adoption Characteristics Subscale, Attitude Toward Innovation Subscale, Receptivity Toward Innovation, and Reinforcement Subscale. The aim of the study is to compare the effect upon score differences of video and print interventions alone and in combination. Seventy-three 7th and 8th grade classroom teachers completed the study providing baseline and post-intervention measures on factors related to the adoption and implementation of tobacco-use prevention programs. Two-way ANOVA, in relation to the study questions, found significant scoring differences for those exposed to the videotape intervention alone for both the Attitude Toward Innovation Subscale and the Receptivity to Adopt Subscale. No significant results were found to suggest that print alone influences favorable scoring differences between baseline and post-intervention testing. One interaction effect was found suggesting video and print combined are more effective for influencing favorable scoring differences for the Reinforcement for the Adoption Subscale.^ This research is unique in that it represents a newly emerging field in health promotion communications research with implications for Social Learning Theory, Diffusion Theory, and communication science that are applicable to the development of improved school health interventions. ^