Formative Evaluation of the "Time Twisters": A Comic Book Based Obesity Prevention Curriculum for Elementary School Students

Introduction: Obesity is a public health problem, particularly in Hispanic children. Alternative media channels may offer the potential to motivate children to engage in health promoting behaviors. A comic book, “Time Twisters”, was developed to impact screen time use, physical activity, and dietary behavior for elementary school children and evaluated for acceptability and feasibility prior to implementation in a multi-component physical activity intervention. [See PDF for complete abstract]

Development and formative evaluation of an obesity prevention intervention for the pediatric primary care setting: Helping HAND (Healthy Activity and Nutrition Directions)

As the obesity epidemic continues to increase, the pediatric primary care office setting remains a relatively unexplored arena to offer obesity prevention interventions for children. The increased risk for adult obesity among 10 to 14 year-old children who are overweight, suggests obesity prevention programs should be introduced just before this age or early in this age period. Research is also accumulating on the importance of targeting parents along with children, since parents are in charge of the home environment for children. Therefore, the aim of this project was to develop an obesity prevention program called Helping HAND (Healthy Activity and Nutrition Directions) based on Social Cognitive Theory and authoritative parenting techniques for the pediatric primary care setting and conduct one-on-one interviews with parents as the initial formative evaluation of the intervention material for the obesity prevention intervention. A secondary aim of the project was to determine the feasibility of identifying appropriate subjects for the intervention, and conducting qualitative evaluations of the materials through recruitment through pediatric primary care settings. ^

A formative evaluation of the American Diabetes Association's Choosing Health and Sensible Exercise childhood obesity intervention

Background. The increasing prevalence of overweight among youth in the United States, and the parallel rise in related medical comorbidities has led to a growing need for efficient weight-management interventions. Purpose. The aim of this study was to evaluate the effects of the Choosing Health and Sensible Exercise (C.H.A.S.E.) childhood obesity prevention program on Body Mass Index (BMI), physical activity and dietary behaviors. Methods. This study utilized de-identified data collected during the fall 2006 session of the C.H.A.S.E. program. A total of 65 students at Woodview Elementary School and Deepwater Elementary School participated in this intervention. The C.H.A.S.E. program is a 10-week obesity prevention program that focuses on nutrition and physical activity education. Collection of height and weight data, and a health behavior survey was conducted during the first and last week of the intervention. Paired t-tests were used to determine statistically significant differences between pre- and post-intervention measurements. One-way analysis of variance was used to adjust for potential confounders, such as gender, age, BMI category ("normal weight", "at risk overweight", or "overweight"), and self-reported weight loss goals. Data were analyzed using STATA, v. 9.2. Results. A significant decrease in mean BMI (p< 0.05) was found after the 10-week intervention. While the results were statistically significant for the group as a whole, changes in BMI were not significant when stratified by age, sex, or ethnicity. The mean overall scores for the behavior survey did not change significantly pre- and post-intervention; however, significant differences were found in the dietary intention scale, indicating that students were more likely to intend to make healthier food choices (p<0.05). No statistically significant decreases in BMI were found when stratified for baseline BMI-for-age percentiles or baseline weight loss efforts (p>0.05). Conclusion. The results of this evaluation provide information that will be useful in planning and implementing an effective childhood obesity intervention in the future. Changes in the self-reported dietary intentions and BMI show that the C.H.A.S.E. program is capable of modifying food choice selection and decreasing BMI. Results from the behavior questionnaire indicate that students in the intervention program were making changes in a positive direction. Future implementation of the C.H.A.S.E. program, as well as other childhood obesity interventions, may want to consider incorporating additional strategies to increase knowledge and other behavioral constructs associated with decreased BMI. In addition, obesity prevention programs may want to increase parental involvement and increase the dose or intensity of the intervention. ^

Assessing fun in serious games: Formative evaluation of games from two diabetes prevention interventions

Purpose. To determine the usability of two video games to prevent type 2 diabetes and obesity among youth through analysis of data collected during alpha-testing. ^ Subjects. Ten children aged 9 to 12 were selected for three 2-hour alpha testing sessions.^ Methods. "Escape from Diab" and "Nanoswarm" were designed to change dietary and physical inactivity behaviors, based on a theoretical framework of mediating variables obtained from social cognitive theory, self-determination theory, elaboration likelihood model, and behavioral inoculation theory. Thirteen mini-games developed by the software company were divided into 3 groups based on completion date. Children tested 4-5 mini-games in each of three sessions. Observed game play was followed by a scripted interview. Results from observation forms and interview transcripts were tabulated and coded to determine usability. Suggestions for game modifications were delivered to the software design firm, and a follow-up table reports rationale for inclusion or exclusion of such modifications.^ Results. Participants were 50% frequent video game players and 20% non game-players. Most (60%) were female. The mean grade (indicating likeability as a subset of usability) across all games given by children was significantly greater than a neutral grade of 80% (89%, p < 0.01), indicating a positive likeability score. The games on average also received positive ratings for fun, helpfulness of instructions and length compared to neutral values (midpoint on likert scales) (all p < 0.01). Observation notes indicated that participants paid attention to the instructions, did not appear to have much difficulty with the games, and were "not frustrated", "not bored", "very engaged", "not fidgety" and "very calm" (all p < 0.01). The primary issues noted in observations and interviews were unclear instructions and unclear purpose of some games. Player suggestions primarily involved ways to make on screen cues more visible or noticeable, instructions more clear, and games more elaborate or difficult.^ Conclusions. The present study highlights the importance of alpha testing video game components for usability prior to completion to enhance usability and likeability. Results indicate that creating clear instructions, making peripheral screen cues more eye-catching or noticeable, and vigorously stating the purpose of the game to improve understandability are important elements. However, future interventions will each present unique materials and user-interfaces and should therefore also be thoroughly alpha-tested. ^

Formative evaluation of a parental involvement program for obesity prevention with low-income elementary school children

This study addresses the responses to a postcard campaign with health messages targeting the parents of children in a sample of low-income elementary schools and assesses the feasibility and areas of possible improvements in such a project. The campaign was implemented in Spring 2009 with 4 th grade students (n=1070) in fifteen economically disadvantaged elementary schools in Travis County, Texas. Postcards were sent home with children, and parents filled out a feedback card that the children returned to school. Response data, in the form of self-administered feedback cards (n=2665) and one-on-one teacher interviews (n=8), were qualitatively analyzed using NVivo 8 software. Postcard reception and points of improvement were then identified from the significant themes that emerged including health, cessation or reduction of unhealthy behaviors, motivation, family, and the comprehension of abstract health concepts. ^ Responses to the postcard campaign were almost completely positive, with less than 1% of responses reporting some sort of dislike, and many parents reported a modification of their behavior. However, possible improvements that could be made to the campaign are: increased focus of the postcards on the parents as the target population, increased information about serving size, greater emphasis on the link between obesity and health, alteration of certain skin tones used in the graphical depiction of people on the cards, and smaller but more frequent incentives to return the feedback cards for the students. The program appears to be an effective method of communicating health messages to the parents of 4th grade children.^

The Comprehensive Assessment of Leadership for Learning: A Next Generation Formative Evaluation and Feedback System

Recent developments in federal policy have prompted the creation of state evaluation frameworks for principals and teachers that hold educators accountable for effective practices and student outcomes. These changes have created a demand for formative evaluation instruments that reflect current accountability pressures and can be used by schools to focus school improvement and leadership development efforts. The Comprehensive Assessment of Leadership for Learning (CALL) is a next generation, 360-degree on-line assessment and feedback system that reflect best practices in feedback design. Some unique characteristics of CALL include a focus on: leadership distributed throughout the school rather than as carried out by an individual leader; assessment of leadership tasks rather than perceptions of leadership practice; a focus on larger complex systems of middle and high school; and transparency of assessment design. This paper describes research contributing to the design and validation of the CALL survey instrument.

Evaluation and Model of a Clinical Librarian Program

Clinical medical librarianship is entering its second decade, but little evaluative data has accrued in the literature. Variations from the original programs and novel new approaches have insured the survival of the program so far. The clinical librarian (CL) forms a vital link between the library and the health care professional, operating as an important information transfer agent. However, to further insure the survival of these vital programs, hard evaluative evidence is needed. The University of Texas Medical Branch (UTMB) at Galveston began a CL Program in 1978/79. An extensive three-year pre/post evaluation study was conducted using a specifically developed evaluation model, which, if adopted by others, will provide the needed comparative data. Both a pilot study, or formative evaluation, and a summative evaluation were conducted. The results of this evaluation confirmed many of the conclusions reported by other CL programs. Eight hypotheses were proposed at the beginning of this study. Data were collected and used to support acceptance or rejection of the null hypotheses, and conclusions were drawn according to the results. Implications relevant to the study conclusions and future trends in medical librarianship are also discussed in the closing chapter.

An evaluation of the personal response system program for the elderly in Harris County, Texas

The Personal Response System Program at Huffington Center on Aging, Baylor College of Medicine, provides emergency call systems for elderly people living independently in Houston, Texas. The goal of the project was to complete a formative evaluation of the Personal Response System Program. The specific aims of the evaluation were three-fold. One aim was to evaluate participant health status and level of disability. The second aim was to develop a health care cost estimation strategy. Finally, a preliminary cost-effectiveness analysis was completed to evaluate the program's impact on health care costs and health status among the elderly target population. ^ The evaluation was a longitudinal, randomized experimental design. After the screening of 120 volunteers for eligibility, clients were asked to complete a written questionnaire and a monthly health service contact diary. Volunteers were contacted by telephone interviewers to collect health status information from 100 eligible clients (83%) on three occasions during the six months of follow-up. ^ Initially, volunteers were randomized to two experimental groups. The two groups were found to be comparable at the beginning of the study. No significant differences were detected related to health status, level of disability, or history of physician visits at baseline. However, the group with the Personal Response System (PRS) device had more adverse health events, higher IADL scores, more frequent use of walkers, lower average health status scores, and fewer community volunteers hours than the usual care comparison group. ^ The health care costs were estimated based on an algorithm adapted from the American Medical Association guidelines. Average total health care costs for the group with the PRS device ($912) were greater than the usual care group ($464). However, median health care values for the PRS group ($263) were similar to the usual care comparison group ($234). The preliminary findings indicated that the use of the PRS device was not associated with health care cost savings. ^ In the preliminary cost-effectiveness analysis, use of the personal response system was found to be associated with increased mental health status among elderly clients. The cost-effectiveness evaluation indicated that the associated cost for six months was $710 per unit increase in mental component score when the $150 PRS subscription was included. However, clients with the PRS device were found to have a greater decline in physical health status during the six-month follow-up. The beneficial effect on mental health status was found to be in contrast to negative findings associated with changes in physical health status. The implications for future research relate to the need to identify risk factors among geriatric populations to better target groups that would most likely benefit from PRS Program enrollment. ^

Team: Bee Me. A formulative evaluation of a negative eating attitudes and behaviors primary prevention curriculum targeted at 5 th grade elementary school girls

Objective. To evaluate a school-based intervention aimed at the primary prevention of negative eating attitudes and behaviors among preadolescent girls, and to revise curriculum lessons based on quantitative and qualitative findings. ^ Intervention Design. A formative evaluation was conducted on four Team: Bee Me curriculum lessons at a Houston elementary school. Evaluation focused on program satisfaction and short-term effect on knowledge and eating attitudes and behaviors. ^ Results. Sixteen girls participated in the five-day project. Statistically significant improvements in overall knowledge were observed (p<0.05), however only modest changes were observed in eating attitudes and behaviors. Program satisfaction was high among student participants and the teacher who implemented it. Insight for future modifications to this program and for similar interventions was provided by the students and teacher. ^ Conclusions. This program led to positive trends in outcome variables; however longer and more intensive testing of this program is needed to better evaluate its effectiveness.^

The use of social media to communicate child health information to low-income parents: A formative study

The Internet, and specifically web 2.0 social media applications, offers an innovative method for communicating child health information to low-income parents. The main objective of this study was to use qualitative data to determine the value of using social media to reach low-income parents with child health information. A qualitative formative evaluation employing focus groups was used to determine the value of using social media for dissemination of child health information. Inclusion criteria included: (1) a parent with a child that attends a school in a designated Central Texas school district; and (2) English-speaking. The students who attend these schools are generally economically disadvantaged and are predominately Hispanic. The classic analysis strategy was used for data analysis. Focus group participants (n=19) were female (95%); White (53%), Hispanic (42%) or African American (5%); and received government assistance (63%). Most had access to the Internet (74%) and were likely to have low health literacy (53%). The most preferred source of child health information was the family pediatrician or general practitioner. Many participants were familiar with social media applications and had profiles on popular social networking sites, but used them infrequently. Objections to social media sites as sources of child health information included lack of credibility and parent time. Social media has excellent potential for reaching low-income parents when used as part of a multi-channel communication campaign. Further research should focus on the most effective type and format of messages that can promote behavior change in this population, such as story-telling. ^

Adherence to HIV /AIDS therapies among low -literacy populations: The ALP project

Adherence to HIV/AIDS therapies has been an important health problem since the early 1980s when AZT was first prescribed as a therapy for HIV/AIDS. It became particularly important between 1995 and 1997 with the advent of protease inhibitors (Chesney, Ickovics, Hecht, Sikipa, & Rabkin J., 1999) and became even more significant as persons with HIV/AIDS began to develop resistance to medications. Low-literacy populations have poorer health (Brez & Taylor, 1997) and higher AIDS rates (Simon, Hu, Diaz, & Kerndt, 1995), than their higher literacy counterparts due to delayed treatment (Baker, Parker, Williams, Clark, & Nurss, 1997), shame of literacy skills (Parikh, 1996), and poor access to care (Williams, et al., 1995). Poorer health and higher AIDS rates can also be attributed to poor patient-provider relationships (Crespo-Fierro, 1997; Eldred, Wu, Chaisson, & Moore, 1998) to a poorer understanding of medical protocols (Murphy, 1997), and inadequate patient education (Ungvarski, 1997; Davis, Michielutte, Askov, Williams, & Weiss, 1998, Doak, Doak, & Root, 1996). ^ The ALP intervention was developed for HIV positive low-literacy populations of African American women in Houston, Texas. The intervention was based on a needs assessment, using the PRECEDE model, an innovative process referred to as Intervention Mapping, and validated using formative evaluation methods with 54 individuals. The needs assessment resulted in a list of behavioral, environmental, predisposing, enabling, and reinforcing determinants of adherence. The Intervention Mapping framework was used to refine these determinants and develop a list of objectives describing what must be learned or changed to for the target population to adhere to HIV/AIDS therapies. Methods and strategies, were developed using theoretical constructs from the Health Belief Model (Rosenstock, 1974) and Social Cognitive Theory (Bandura, 1986). These theories, empirical evidence, and information from the target population indicated that perceived susceptibility, perceived severity, outcome expectations, and self-efficacy were important and changeable determinants of adherence to HIV/AIDS therapies for this population. ^ These components were brought together in the form of a theory-based color cartoon book and 10-minute cassette tape. The book was developed for people with 2.9 years of U.S. education as measured with the Flesch-Kincaid Grade Level method and the script was recorded onto a cassette tape to make it suitable for populations with even lower-literacy skills. A formative evaluation was conducted to ensure that the content and structure were accurate, clear, realistic, readable, appropriate, and likely to be used as intended. ^