Program evaluation for a summer science internship: Short and long term benefits as reported by interns

Undergraduate research programs have been used as a tool to attract and retain student interest in science careers. This study evaluates the short and long-term benefits of a Summer Science Internship (SSI) at the University of Texas Health Science Center at Houston– School of Public Health – in Brownsville, Texas, by analyzing survey data from alumni. Questions assessing short-term program impact were aimed at three main topics, student: satisfaction with program, self-efficacy for science after completing the program, and perceived benefits. Long-term program impact was assessed by looking at student school attendance and college majors along with perceived links between SSI and future college plans. Students reported high program satisfaction, a significant increase in science self-efficacy and high perceived benefits. At the time data were collected for the study, one-hundred percent of alumni were enrolled in school (high school or college). The majority of students indicated they were interested in completing a science major/career, heavily influenced by their participation in the program.^

A summative evaluation of a HIV/AIDS Early Childhood Care, Education and Development Teacher Training Workshop in Mongu, Zambia

Objective. To conduct a summative evaluation of an Early Childhood Care, Education and Development (ECCED) Teacher Training Workshop in Mongu, Zambia by assessing changes in knowledge, attitudes and intent to use the information. ^ Study design. A matched cohort survey design was used with additional qualitative data collected by structured observation of workshop sessions, daily facilitator and participant debriefs and participant interviews. ^ Results. Matching pre and post tests were completed by 27 individuals in addition to daily debriefs, structured workshop observation and participant interviews with 22% of the group. The participant population was predominantly female individuals aged 15-44 years old that had completed high school and additional post-secondary training, been teaching children aged 0 – 8 years for 2-5 years in the Western Province and received other HIV/AIDS and ECCED education. Pre-tests indicated a strong understanding of ECCED principles and misconceptions regarding HIV transmission, prevention and the disease's impact on early childhood development. The workshop was found to significantly increase the participants' knowledge of topics covered by the curriculum (paired t-test, N=27, p = 0.004, 95% CI 1.8, 8.6). Participants began with a more limited understanding of HIV/AIDS than ECCED, but the mean gain was much greater at 7.4 +/- 12.3 points. Significantly more participants believed at post-test that HIV/AIDS education should increase for future educators. The 77.8% of participants that increased their knowledge scores at post-test expressed significantly less fear of having a child with HIV/AIDS in the classroom (Independent Samples t-test, N= 27, p = 0.011). Overall participant fear decreased 15.5%. 92.6% and 88.9% of participants planned at post-test to respectively use and share the taught information in their daily professional lives and reported on innovative strategies to communicate with the community. ^ Conclusions. Teacher training workshops can significantly increase HIV/AIDS awareness and promote positive attitudes in educators working with children affected by HIV/AIDS. Using participant suggested teaching techniques such as poems and songs and translating the materials to the local language could assist future facilitators to both culturally and professionally relate to the workshop audience as well as increase participant capacity to share the information with the local community. ^

Using technology to teach technology: design and evaluation of bilingual online physician education about electronic medical records.

The "EMR Tutorial" is designed to be a bilingual online physician education environment about electronic medical records. After iterative assessment and redesign, the tutorial was tested in two groups: U.S. physicians and Mexican medical students. Split-plot ANOVA revealed significantly different pre-test scores in the two groups, significant cognitive gains for the two groups overall, and no significant difference in the gains made by the two groups. Users rated the module positively on a satisfaction questionnaire.

Design and evaluation of the parent education component of a comprehensive violence prevention project in middle schools

Low parental monitoring is related to youth risk behaviors such as delinquency and aggression. The purpose of this dissertation was to describe the development and evaluation of a parent education intervention to increase parental monitoring in Hispanic parents of middle school children.^ The first study described the process of intervention mapping as used to develop Padres Trabajando por la Paz, a newsletter intervention for parents. Using theory, empirical literature, and information from the target population, performance objectives and determinants for monitoring were defined. Learning objectives were specified and a staged social-cognitive approach was used to develop methods and strategies delivered through newsletters.^ The second study examined the outcomes of a randomized trial of the newsletter intervention. Outcome measures consisted of a general measure of monitoring, parent and child reports of monitoring behaviors targeted by the intervention, and psychosocial determinants of monitoring (self-efficacy, norms, outcome expectancies, knowledge, and beliefs). Seventy-seven parents completed the randomized trial, half of which received four newsletters over an eight-week period. Results revealed a significant interaction effect for baseline and treatment for parent's reports of norms for monitoring (p =.009). Parents in the experimental condition who scored low at baseline reported increased norms for monitoring at follow-up. A significant interaction effect for child reports of parental monitoring behaviors (p =.04) reflected an small increase across baseline levels in the experimental condition and decreases for the control condition at higher baseline scores. Both groups of parents reported increased levels of monitoring at follow-up. No other outcome measures varied significantly by condition.^ The third study examined the relationship between the psychosocial determinants of parental monitoring and parental monitoring behaviors in the study population. Weak evidence for a relationship between outcome expectancies and parental monitoring behaviors suggests further research in the area utilizing stronger empirical models such as longitudinal design and structural equation modeling.^ The low-cost, minimal newsletter intervention showed promise for changing norms among Hispanic parents for parental monitoring. In light of the importance of parental monitoring as a protective factor for youth health risk behaviors, more research needs to be done to develop and evaluate interventions to increase parental monitoring. ^

An evaluation of the impact of Inner City Development's Cooperative Home School: An educational alternative program to increase the quality and level of education in San Antonio's West Side community

The research study was intended to evaluate the effectiveness of Inner City Development's (I.C.D.) Cooperative Home School, an educational alternative program to the Title I public schools of San Antonio's West Side community. The study investigated students', parents' and tutors' perception of parental involvement and educational resources. The study also investigated each student's academic achievement. ^ The study found that students progressed toward expected math proficiency at a faster rate than they did in reading proficiency. However, because the target population size was small and a comparison group was not used, the results of this study are only suggestive. This research also indicated that study subjects believed students' quality and level of education increased substantially since program exposure. Study subjects mainly attributed the students' strides in academic performance to the increased amount of individualized attention students received in the small twelve-student class size. Study subjects were more satisfied with the home school's educational resources than those of the Title I public schools. Study subjects also perceived that parental involvement both at home and at school increased since enrollment in the home school program because: (1) there were more opportunities for involvement in the home school; and (2) parents felt closer to the tutors than the teachers in public school. ^ This evaluation also suggested improvements to program operations. With the help of additional volunteers, I.C.D. program operators could improve collection and organization of academic records. Furthermore, as suggested by program participants, science could be added to the curriculum. Lastly, a formal tutor orientation could be implemented to familiarize and train tutors on classroom management procedures. ^

The School Sealant Program: An evaluation of the oral health education component

Dental caries is a common preventable childhood disease leading to severe physical, mental and economic repercussions for children and their families if left untreated. A needs assessment in Harris County reported that 45.9% of second graders had untreated dental caries. In order to address this growing problem, the School Sealant Program (SSP), a primary preventive initiative, was launched by the Houston Department of Health and Human Services (HDHHS) to provide oral health education, and underutilized dental preventive services to second grade children from participating Local School Districts (LSDs). ^ To determine the effectiveness and efficiency of the SSP, a program evaluation was conducted by the HDHHS between September 2007 and June 2008 for the Oral Health Education (OHE) component of the SSP. The objective of the evaluation was to assess short term changes in oral health knowledge of the participants and determine if these changes, if any, were due to the OHE sessions. An 8-item multiple choice pre/post test was developed for this purpose and administered to the participants before and immediately after the OHE sessions. ^ The present project analyzed pre and post test data of 1,088 second graders from 22 participating schools. Changes in overall and topic-specific knowledge of the program participants before and after the OHE sessions were analyzed using the Wilcoxon's signed rank test. ^ Results. The overall knowledge assessment showed a statistically significant (p <0.001) increase in the dental health knowledge of the participants after the oral health education sessions. Participants in the higher scoring category (7-8 correct responses) increased from 9.5% at baseline to 60.8% after the education sessions. Overall knowledge increased in all school regions with the highest knowledge gains seen in the Central and South regions. Males and females had similar knowledge gains. Significant knowledge differences were also found for each of the topic specific categories (functions of teeth, healthy diet, healthy habits, dental sealants; p<0.001) indicating an increase in topic specific knowledge of the participants post-health education sessions. ^ Conclusions. The OHE sessions were successful in increasing the short term oral health knowledge of the participants. ^

An evaluation of the Texas "A Woman's Right to Know" booklet: A state mandated patient education document for induced abortion

Many patient educational documents are written at a grade level higher than the level at which most individuals can read. This discrepancy can lead to treatment noncompliance and negative health outcomes. Therefore, it is important that patients receive readable health information. The Texas "A Woman's Right to Know" booklet is a state mandated informational document provided to women seeking abortion services. Given the significance of the abortion procedure, it is imperative that women considering having an abortion receive accurate and readable health materials. However, no published studies were found that evaluated the readability of the "A Woman's Right to Know" booklet. Therefore, the purpose of this study was to assess the readability of the "A Woman's Right to Know" booklet. To assess the readability, the Flesch-Kincaid readability test was used to evaluate the reading grade level of the entire "A Woman's Right to Know" booklet and each of the 7 sections of the booklet. The results showed that the readability of the entire booklet as well as each section of the booklet was written below the 8th grade reading level. Although the booklet was written below the estimated United States reading level (8th grade), the reading level of this booklet may still be too high for people in Texas who read below the 8th grade level. Based on these results, it is recommended that health care professionals involved in the distribution and explanation of the "A Woman's Right to Know" booklet provide their patients with both written and verbal medical information. The patients should be allowed to ask questions about the abortion procedure so that they can make the most informed choice.^

Development and evaluation of a nutrition education program designed to increase fruit and vegetable consumption in girls, ages 9--12

The purpose of this study is to evaluate the theory-based Eat 5 nutrition badge. It is designed to increase fruit and vegetable (F&V) intake in 4th-6th grade junior Girl Scouts. Twenty-two troops were recruited and randomized by grade level (4th, 5th, 6th, or mixed) into either the intervention or control conditions. The leaders in the intervention condition received a brief training and the materials and conducted the program with their troops during four meetings. The Girl Scouts in the intervention condition completed 1-day Food Frequency Questionnaires and Nutrition Questionnaires both before and after completing the Eat 5 badge, and a third measurement of F&V intake three months after the posttest. Girl Scouts in the control condition were only evaluated at the three time periods.^ The primary hypotheses were that the Girl Scouts in the intervention condition would increase their daily intake of fruits and vegetables at both the posttest and three months later, compared to the Girl Scouts in the control condition. Other study questions investigated the impact of the Eat 5 program on intervening variables such as knowledge, self-efficacy, barriers, norms, F&V preference, and F&V selection and preparation skills.^ A nested ANOVA, with troop as the unit of analysis nested within condition, was used to assess the effects of the program. Pretest F&V intake and grade level were used as covariates. Pretest mean F&V intake for the total sample of 210 girls was 2.50 servings per day; 3.0 for the intervention group (n = 101). Significant increases in F&V intake (to 3.4 servings per day), knowledge, and fruit and vegetable preference were found for the intervention condition troops compared to the troops in the control condition. Three months later, the mean F&V intake had returned to pretest levels.^ This study indicates that social groups such as Girl Scouts can provide a channel for nutrition education. Long term effects were not sustained by the intervention; a possible cause was the lack of change in self-efficacy. Therefore, additional interventions are recommended such as booster lessons to maintain increased F&V intake by Girl Scouts. ^

A practice-based culminating experience: Writing a HRSA grant application with Texas Organ Sharing Alliance for public education efforts to increase solid organ donation

In November 2010, nearly 110,000 people in the United States were waiting for organs for transplantation. Despite the fact that the organ donor registration rate has doubled in the last year, Texas has the lowest registration rate in the nation. Due to the need for improved registration rates in Texas, this practice-based culminating experience was to write an application for federal funding for the central Texas organ procurement organization, Texas Organ Sharing Alliance. The culminating experience has two levels of significance for public health – (1) to engage in an activity to promote organ donation registration, and (2) to provide professional experience in grant writing. ^ The process began with a literature review. The review was to identify successful intervention activities in motivating organ donation registration that could be used in intervention design for the grant application. Conclusions derived from the literature review included (1) the need to specifically encourage family discussions, (2) religious and community leaders can be leveraged to facilitate organ donation conversations in families, (3) communication content must be culturally sensitive and (4) ethnic disparities in transplantation must be acknowledged and discussed.^ Post the literature review; the experience followed a five step process of developing the grant application. The steps included securing permission to proceed, assembling a project team, creation of a project plan and timeline, writing each element of the grant application including the design of proposed intervention activities, and completion of the federal grant application. ^ After the grant application was written, an evaluation of the grant writing process was conducted. Opportunities for improvement were identified. The first opportunity was the need for better timeline management to allow for review of the application by an independent party, iterative development of the budget proposal, and development of collaborative partnerships. Another improvement opportunity was the management of conflict regarding the design of the intervention that stemmed from marketing versus evidence-based approaches. The most important improvement opportunity was the need to develop a more exhaustive evaluation plan.^ Eight supplementary files are attached to appendices: Feasibility Discussion in Appendix 1, Grant Guidance and Workshop Notes in Appendix 2, Presentation to Texas Organ Sharing Alliance in Appendix 3, Team Recruitment Presentation in Appendix 5, Grant Project Narrative in Appendix 7, Federal Application Form in Appendix 8, and Budget Workbook with Budget Narrative in Appendix 9.^