Formative Evaluation of the "Time Twisters": A Comic Book Based Obesity Prevention Curriculum for Elementary School Students

Introduction: Obesity is a public health problem, particularly in Hispanic children. Alternative media channels may offer the potential to motivate children to engage in health promoting behaviors. A comic book, “Time Twisters”, was developed to impact screen time use, physical activity, and dietary behavior for elementary school children and evaluated for acceptability and feasibility prior to implementation in a multi-component physical activity intervention. [See PDF for complete abstract]

Development and formative evaluation of an obesity prevention intervention for the pediatric primary care setting: Helping HAND (Healthy Activity and Nutrition Directions)

As the obesity epidemic continues to increase, the pediatric primary care office setting remains a relatively unexplored arena to offer obesity prevention interventions for children. The increased risk for adult obesity among 10 to 14 year-old children who are overweight, suggests obesity prevention programs should be introduced just before this age or early in this age period. Research is also accumulating on the importance of targeting parents along with children, since parents are in charge of the home environment for children. Therefore, the aim of this project was to develop an obesity prevention program called Helping HAND (Healthy Activity and Nutrition Directions) based on Social Cognitive Theory and authoritative parenting techniques for the pediatric primary care setting and conduct one-on-one interviews with parents as the initial formative evaluation of the intervention material for the obesity prevention intervention. A secondary aim of the project was to determine the feasibility of identifying appropriate subjects for the intervention, and conducting qualitative evaluations of the materials through recruitment through pediatric primary care settings. ^

A formative evaluation of the American Diabetes Association's Choosing Health and Sensible Exercise childhood obesity intervention

Background. The increasing prevalence of overweight among youth in the United States, and the parallel rise in related medical comorbidities has led to a growing need for efficient weight-management interventions. Purpose. The aim of this study was to evaluate the effects of the Choosing Health and Sensible Exercise (C.H.A.S.E.) childhood obesity prevention program on Body Mass Index (BMI), physical activity and dietary behaviors. Methods. This study utilized de-identified data collected during the fall 2006 session of the C.H.A.S.E. program. A total of 65 students at Woodview Elementary School and Deepwater Elementary School participated in this intervention. The C.H.A.S.E. program is a 10-week obesity prevention program that focuses on nutrition and physical activity education. Collection of height and weight data, and a health behavior survey was conducted during the first and last week of the intervention. Paired t-tests were used to determine statistically significant differences between pre- and post-intervention measurements. One-way analysis of variance was used to adjust for potential confounders, such as gender, age, BMI category ("normal weight", "at risk overweight", or "overweight"), and self-reported weight loss goals. Data were analyzed using STATA, v. 9.2. Results. A significant decrease in mean BMI (p< 0.05) was found after the 10-week intervention. While the results were statistically significant for the group as a whole, changes in BMI were not significant when stratified by age, sex, or ethnicity. The mean overall scores for the behavior survey did not change significantly pre- and post-intervention; however, significant differences were found in the dietary intention scale, indicating that students were more likely to intend to make healthier food choices (p<0.05). No statistically significant decreases in BMI were found when stratified for baseline BMI-for-age percentiles or baseline weight loss efforts (p>0.05). Conclusion. The results of this evaluation provide information that will be useful in planning and implementing an effective childhood obesity intervention in the future. Changes in the self-reported dietary intentions and BMI show that the C.H.A.S.E. program is capable of modifying food choice selection and decreasing BMI. Results from the behavior questionnaire indicate that students in the intervention program were making changes in a positive direction. Future implementation of the C.H.A.S.E. program, as well as other childhood obesity interventions, may want to consider incorporating additional strategies to increase knowledge and other behavioral constructs associated with decreased BMI. In addition, obesity prevention programs may want to increase parental involvement and increase the dose or intensity of the intervention. ^

Assessing fun in serious games: Formative evaluation of games from two diabetes prevention interventions

Purpose. To determine the usability of two video games to prevent type 2 diabetes and obesity among youth through analysis of data collected during alpha-testing. ^ Subjects. Ten children aged 9 to 12 were selected for three 2-hour alpha testing sessions.^ Methods. "Escape from Diab" and "Nanoswarm" were designed to change dietary and physical inactivity behaviors, based on a theoretical framework of mediating variables obtained from social cognitive theory, self-determination theory, elaboration likelihood model, and behavioral inoculation theory. Thirteen mini-games developed by the software company were divided into 3 groups based on completion date. Children tested 4-5 mini-games in each of three sessions. Observed game play was followed by a scripted interview. Results from observation forms and interview transcripts were tabulated and coded to determine usability. Suggestions for game modifications were delivered to the software design firm, and a follow-up table reports rationale for inclusion or exclusion of such modifications.^ Results. Participants were 50% frequent video game players and 20% non game-players. Most (60%) were female. The mean grade (indicating likeability as a subset of usability) across all games given by children was significantly greater than a neutral grade of 80% (89%, p < 0.01), indicating a positive likeability score. The games on average also received positive ratings for fun, helpfulness of instructions and length compared to neutral values (midpoint on likert scales) (all p < 0.01). Observation notes indicated that participants paid attention to the instructions, did not appear to have much difficulty with the games, and were "not frustrated", "not bored", "very engaged", "not fidgety" and "very calm" (all p < 0.01). The primary issues noted in observations and interviews were unclear instructions and unclear purpose of some games. Player suggestions primarily involved ways to make on screen cues more visible or noticeable, instructions more clear, and games more elaborate or difficult.^ Conclusions. The present study highlights the importance of alpha testing video game components for usability prior to completion to enhance usability and likeability. Results indicate that creating clear instructions, making peripheral screen cues more eye-catching or noticeable, and vigorously stating the purpose of the game to improve understandability are important elements. However, future interventions will each present unique materials and user-interfaces and should therefore also be thoroughly alpha-tested. ^

Formative evaluation of a parental involvement program for obesity prevention with low-income elementary school children

This study addresses the responses to a postcard campaign with health messages targeting the parents of children in a sample of low-income elementary schools and assesses the feasibility and areas of possible improvements in such a project. The campaign was implemented in Spring 2009 with 4 th grade students (n=1070) in fifteen economically disadvantaged elementary schools in Travis County, Texas. Postcards were sent home with children, and parents filled out a feedback card that the children returned to school. Response data, in the form of self-administered feedback cards (n=2665) and one-on-one teacher interviews (n=8), were qualitatively analyzed using NVivo 8 software. Postcard reception and points of improvement were then identified from the significant themes that emerged including health, cessation or reduction of unhealthy behaviors, motivation, family, and the comprehension of abstract health concepts. ^ Responses to the postcard campaign were almost completely positive, with less than 1% of responses reporting some sort of dislike, and many parents reported a modification of their behavior. However, possible improvements that could be made to the campaign are: increased focus of the postcards on the parents as the target population, increased information about serving size, greater emphasis on the link between obesity and health, alteration of certain skin tones used in the graphical depiction of people on the cards, and smaller but more frequent incentives to return the feedback cards for the students. The program appears to be an effective method of communicating health messages to the parents of 4th grade children.^

The Comprehensive Assessment of Leadership for Learning: A Next Generation Formative Evaluation and Feedback System

Recent developments in federal policy have prompted the creation of state evaluation frameworks for principals and teachers that hold educators accountable for effective practices and student outcomes. These changes have created a demand for formative evaluation instruments that reflect current accountability pressures and can be used by schools to focus school improvement and leadership development efforts. The Comprehensive Assessment of Leadership for Learning (CALL) is a next generation, 360-degree on-line assessment and feedback system that reflect best practices in feedback design. Some unique characteristics of CALL include a focus on: leadership distributed throughout the school rather than as carried out by an individual leader; assessment of leadership tasks rather than perceptions of leadership practice; a focus on larger complex systems of middle and high school; and transparency of assessment design. This paper describes research contributing to the design and validation of the CALL survey instrument.

Evaluation and Model of a Clinical Librarian Program

Clinical medical librarianship is entering its second decade, but little evaluative data has accrued in the literature. Variations from the original programs and novel new approaches have insured the survival of the program so far. The clinical librarian (CL) forms a vital link between the library and the health care professional, operating as an important information transfer agent. However, to further insure the survival of these vital programs, hard evaluative evidence is needed. The University of Texas Medical Branch (UTMB) at Galveston began a CL Program in 1978/79. An extensive three-year pre/post evaluation study was conducted using a specifically developed evaluation model, which, if adopted by others, will provide the needed comparative data. Both a pilot study, or formative evaluation, and a summative evaluation were conducted. The results of this evaluation confirmed many of the conclusions reported by other CL programs. Eight hypotheses were proposed at the beginning of this study. Data were collected and used to support acceptance or rejection of the null hypotheses, and conclusions were drawn according to the results. Implications relevant to the study conclusions and future trends in medical librarianship are also discussed in the closing chapter.

Implementation of an Anthropomorphic Phantom for the Evaluation of Proton Therapy Treatment Procedures

With an increasing number of institutions offering proton therapy, the number of multi-institutional clinical trials involving proton therapy will also increase in the coming years. The Radiological Physics Center monitors sites involved in clinical trials through the use of site visits and remote auditing with thermoluminescent dosimeters (TLD) and mailable anthropomorphic phantoms. Currently, there are no heterogeneous phantoms that have been commissioned to evaluate proton therapy. It was hypothesized that an anthropomorphic pelvis phantom can be designed to audit treatment procedures (patient simulation, treatment planning and treatment delivery) at proton facilities to confirm agreement between the measured dose and calculated dose within 5%/3mm with a reproducibility of 3%. A pelvis phantom originally designed for use with photon treatments was retrofitted for use in proton therapy. The relative stopping power (SP) of each phantom material was measured. Hounsfield Units (HU) for each phantom material were measured with a CT scanner and compared to the relative stopping power calibration curve. The tissue equivalency for each material was calculated. Two proton treatment plans were created; one which did not correct for material SP differences (Plan 1) and one plan which did correct for SP differences (Plan 2). Film and TLD were loaded into the phantom and the phantom was irradiated 3 times per plan. The measured values were compared to the HU-SP calibration curve and it was found that the stopping powers for the materials could be underestimated by 5-10%. Plan 1 passed the criteria for the TLD and film margins with reproducibility under 3% between the 3 trials. Plan 2 failed because the right-left film dose profile average displacement was -9.0 mm on the left side and 6.0 mm on the right side. Plan 2 was intended to improve the agreements and instead introduced large displacements along the path of the beam. Plan 2 more closely represented the actual phantom composition with corrected stopping powers and should have shown an agreement between the measured and calculated dose within 5%/3mm. The hypothesis was rejected and the pelvis phantom was found to be not suitable to evaluate proton therapy treatment procedures.

An evaluation of the personal response system program for the elderly in Harris County, Texas

The Personal Response System Program at Huffington Center on Aging, Baylor College of Medicine, provides emergency call systems for elderly people living independently in Houston, Texas. The goal of the project was to complete a formative evaluation of the Personal Response System Program. The specific aims of the evaluation were three-fold. One aim was to evaluate participant health status and level of disability. The second aim was to develop a health care cost estimation strategy. Finally, a preliminary cost-effectiveness analysis was completed to evaluate the program's impact on health care costs and health status among the elderly target population. ^ The evaluation was a longitudinal, randomized experimental design. After the screening of 120 volunteers for eligibility, clients were asked to complete a written questionnaire and a monthly health service contact diary. Volunteers were contacted by telephone interviewers to collect health status information from 100 eligible clients (83%) on three occasions during the six months of follow-up. ^ Initially, volunteers were randomized to two experimental groups. The two groups were found to be comparable at the beginning of the study. No significant differences were detected related to health status, level of disability, or history of physician visits at baseline. However, the group with the Personal Response System (PRS) device had more adverse health events, higher IADL scores, more frequent use of walkers, lower average health status scores, and fewer community volunteers hours than the usual care comparison group. ^ The health care costs were estimated based on an algorithm adapted from the American Medical Association guidelines. Average total health care costs for the group with the PRS device ($912) were greater than the usual care group ($464). However, median health care values for the PRS group ($263) were similar to the usual care comparison group ($234). The preliminary findings indicated that the use of the PRS device was not associated with health care cost savings. ^ In the preliminary cost-effectiveness analysis, use of the personal response system was found to be associated with increased mental health status among elderly clients. The cost-effectiveness evaluation indicated that the associated cost for six months was $710 per unit increase in mental component score when the $150 PRS subscription was included. However, clients with the PRS device were found to have a greater decline in physical health status during the six-month follow-up. The beneficial effect on mental health status was found to be in contrast to negative findings associated with changes in physical health status. The implications for future research relate to the need to identify risk factors among geriatric populations to better target groups that would most likely benefit from PRS Program enrollment. ^

Federally regulated institutional review boards: The aspects of a "model" IRB---Is the current system in need of a tune up or complete overhaul?

Institutional Review Boards (IRBs) are the primary gatekeepers for the protection of ethical standards of federally regulated research on human subjects in this country. This paper focuses on what general, broad measures that may be instituted or enhanced to exemplify a "model IRB". This is done by examining the current regulatory standards of federally regulated IRBs, not private or commercial boards, and how many of those standards have been found either inadequate or not generally understood or followed. The analysis includes suggestions on how to bring about changes in order to make the IRB process more efficient, less subject to litigation, and create standardized educational protocols for members. The paper also considers how to include better oversight for multi-center research, increased centralization of IRBs, utilization of Data Safety Monitoring Boards when necessary, payment for research protocol review, voluntary accreditation, and the institution of evaluation/quality assurance programs. ^ This is a policy study utilizing secondary analysis of publicly available data. Therefore, the research for this paper focuses on scholarly medical/legal journals, web information from the Department of Health and Human Services, Federal Drug Administration, and the Office of the Inspector General, Accreditation Programs, law review articles, and current regulations applicable to the relevant portions of the paper. ^ Two issues are found to be consistently cited by the literature as major concerns. One is a need for basic, standardized educational requirements across all IRBs and its members, and secondly, much stricter and more informed management of continuing research. There is no federally regulated formal education system currently in place for IRB members, except for certain NIH-based trials. Also, IRBs are not keeping up with research once a study has begun, and although regulated to do so, it does not appear to be a great priority. This is the area most in danger of increased litigation. Other issues such as voluntary accreditation and outcomes evaluation are slowing gaining steam as the processes are becoming more available and more sought after, such as JCAHO accrediting of hospitals. ^ Adopting the principles discussed in this paper should promote better use of a local IRBs time, money, and expertise for protecting the vulnerable population in their care. Without further improvements to the system, there is concern that private and commercial IRBs will attempt to create a monopoly on much of the clinical research in the future as they are not as heavily regulated and can therefore offer companies quicker and more convenient reviews. IRBs need to consider the advantages of charging for their unique and important services as a cost of doing business. More importantly, there must be a minimum standard of education for all IRB members in the area of the ethical standards of human research and a greater emphasis placed on the follow-up of ongoing research as this is the most critical time for study participants and may soon lead to the largest area for litigation. Additionally, there should be a centralized IRB for multi-site trials or a study website with important information affecting the trial in real time. There needs to be development of standards and metrics to assess the performance of the IRBs for quality assurance and outcome evaluations. The boards should not be content to run the business of human subjects' research without determining how well that function is actually being carried out. It is important that federally regulated IRBs provide excellence in human research and promote those values most important to the public at large.^

Team: Bee Me. A formulative evaluation of a negative eating attitudes and behaviors primary prevention curriculum targeted at 5 th grade elementary school girls

Objective. To evaluate a school-based intervention aimed at the primary prevention of negative eating attitudes and behaviors among preadolescent girls, and to revise curriculum lessons based on quantitative and qualitative findings. ^ Intervention Design. A formative evaluation was conducted on four Team: Bee Me curriculum lessons at a Houston elementary school. Evaluation focused on program satisfaction and short-term effect on knowledge and eating attitudes and behaviors. ^ Results. Sixteen girls participated in the five-day project. Statistically significant improvements in overall knowledge were observed (p<0.05), however only modest changes were observed in eating attitudes and behaviors. Program satisfaction was high among student participants and the teacher who implemented it. Insight for future modifications to this program and for similar interventions was provided by the students and teacher. ^ Conclusions. This program led to positive trends in outcome variables; however longer and more intensive testing of this program is needed to better evaluate its effectiveness.^

The use of social media to communicate child health information to low-income parents: A formative study

The Internet, and specifically web 2.0 social media applications, offers an innovative method for communicating child health information to low-income parents. The main objective of this study was to use qualitative data to determine the value of using social media to reach low-income parents with child health information. A qualitative formative evaluation employing focus groups was used to determine the value of using social media for dissemination of child health information. Inclusion criteria included: (1) a parent with a child that attends a school in a designated Central Texas school district; and (2) English-speaking. The students who attend these schools are generally economically disadvantaged and are predominately Hispanic. The classic analysis strategy was used for data analysis. Focus group participants (n=19) were female (95%); White (53%), Hispanic (42%) or African American (5%); and received government assistance (63%). Most had access to the Internet (74%) and were likely to have low health literacy (53%). The most preferred source of child health information was the family pediatrician or general practitioner. Many participants were familiar with social media applications and had profiles on popular social networking sites, but used them infrequently. Objections to social media sites as sources of child health information included lack of credibility and parent time. Social media has excellent potential for reaching low-income parents when used as part of a multi-channel communication campaign. Further research should focus on the most effective type and format of messages that can promote behavior change in this population, such as story-telling. ^

Perceptions of institutional support of community -based participatory research within prevention research centers

The Institute of Medicine (IOM) report on the future of health care states that the focus on health needs to shift to the management and prevention of chronic illnesses and that academic health centers (AHCs) should play an active role in this process through community partnerships (IOM, 2002). Grant funding from the National Institutes of Health and the creation of the Centers for Disease Control and Prevention (CDC) Prevention Research Centers (PRC) across the county represent a transition toward more proactively seeking out community partnerships to better design and disseminate health promotion programs (Green, 2001). ^ The focus of the PRCs is to conduct rigorous, community-based, prevention research, to seek outcomes applicable to public health programs and policies. The PRCs work is to create and foster partnerships among public health and community organizations, to address health promotion and disease prevention issues (CDC, 2003). ^ The W.K. Kellogg Foundation defines CBPR as "a collaborative approach to research that equitably involves all partners in the research process and recognizes the unique strengths that each brings. CBPR begins with a research topic of importance to the community with the aim of combining knowledge and action for social change to improve community health." ^ In 1995, CDC asked the IOM to review the PRC program to examine the extent to which the program is providing the public health community with strategies to address public health problems in disease prevention and health promotion (IOM, 1997). No comprehensive evaluation n of the individual PRCs had ever been done (IOM, 1997). ^ The CDC was interested in understanding how it could better support the PRC program through improved management and oversight to influence the program's success. The CDC only represents one of the entities that influence the success of a PRC. Another key entity to consider is the support of and influence of the Schools of Public Health in which the PRCs reside. Using evaluation criteria similar to those that were developed by the IOM, this study examined how aspects of structural capacity of the Schools of Public Health in which the PRCs reside are perceived to influence PRC community-based research activities. ^