Evaluation and Model of a Clinical Librarian Program

Clinical medical librarianship is entering its second decade, but little evaluative data has accrued in the literature. Variations from the original programs and novel new approaches have insured the survival of the program so far. The clinical librarian (CL) forms a vital link between the library and the health care professional, operating as an important information transfer agent. However, to further insure the survival of these vital programs, hard evaluative evidence is needed. The University of Texas Medical Branch (UTMB) at Galveston began a CL Program in 1978/79. An extensive three-year pre/post evaluation study was conducted using a specifically developed evaluation model, which, if adopted by others, will provide the needed comparative data. Both a pilot study, or formative evaluation, and a summative evaluation were conducted. The results of this evaluation confirmed many of the conclusions reported by other CL programs. Eight hypotheses were proposed at the beginning of this study. Data were collected and used to support acceptance or rejection of the null hypotheses, and conclusions were drawn according to the results. Implications relevant to the study conclusions and future trends in medical librarianship are also discussed in the closing chapter.

A Multi- Faceted, Intensive Family Preservation Program Evaluation

This evaluation of a county intensive family preservation services (ifps) program makes several important methodological contributions to assessing post-treatment placement patterns of ifps clients. It is the first published ifps evaluation that utilizes an interval-level, overall measure of restrictiveness of placement, and one of the few that has followed placement patterns for a full two-years after treatment. The study is also a good example of complementing placement data with measures of family health and stability, and with qualitative feedback from former ifps clients. Finally, this study demonstrates the potential for doing methodologically sound evaluations of local ifps programs.

Evaluation of a date rape prevention program for new students in a university setting

The purpose of this study was to design, implement and evaluate the effectiveness of a date rape prevention program among new students at Rice University. Six-hundred and fifteen new students were randomly assigned to one of eight residential colleges or dormitories. The distribution of students to each of the dormitories was carried out in accordance with a stratified random sampling procedure. The study population was divided into strata based on ethnicity, gender, geographical region, and academic major. The number of students randomly assigned to each of the eight dormitories was approximately 75. After this procedure was completed, each of the colleges was randomly selected to either the intervention or control group. A randomized pretest and posttest control group design was used to assess changes in attitudes, self-efficacy, and behavior with regard to date rape. All participants were given an anonymous pretest and posttest measuring attitudes, self-efficacy, and behavior immediately prior to and following the intervention. The intervention group attended the play Scruples, designed to promote date rape prevention, after which they were immediately posttested. After this initial posttest the intervention group also participated in an interactive group role-playing activity led by trained peer instructors. The control group was pretested and subject to the placebo intervention of a multiculturalism play and was posttested immediately afterwards. Later in the week this group saw the Scruples play only. Both control and intervention groups were sent a two month follow-up survey questionnaire, to measure any changes in attitudes, self-efficacy, and behavior over time. As hypothesized students who saw the play Scruples showed a change in attitudes immediately posttest but no difference in self-efficacy or behavior. The two month follow-up survey showed no change in attitudes, self-efficacy, or behavior. There was a difference at pretest in males and females attitudes, with males showing significantly more rape tolerant attitudes than females. Thus, the proposed research findings will provide a better understanding of the attitudes that perpetuate date rape, and will inform strategies for prevention programs. ^

A process evaluation of the childhood lead poisoning prevention program in Houston, Texas, 1992--1995

A process evaluation of the Houston Childhood Lead Poisoning Prevention Program, 1992-1995, was conducted. The Program's goal is to reduce lead poisoning prevalence. The study proposed to determine to what extent the Program was implemented as planned by measuring how well Program services were actually: (1) received by the intended target population; (2) delivered to children with elevated blood lead levels; (3) delivered in compliance with the Centers for Disease Control and Prevention and Program guidelines and timetables; and (4) able to reduce lead poisoning prevalence among those rescreened. Utilizing a program monitoring design, the Program's pre-collected computer records were reviewed. The study sample consisted of 820 children whose blood lead levels were above 15 micrograms per deciLiter, representing approximately 2.9% of the 28,406 screened over this period. Three blood lead levels from each participant were examined: the initial elevated result; the confirmatory result; and the next rescreen result, after the elevated confirmatory level. Results showed that the Program screened approximately 18% (28,406 of 161,569) of Houston's children under age 6 years for lead poisoning. Based on Chi-square tests of significance, results also showed that lead-poisoned participants were more likely to be younger than 3 years, male and Hispanic, compared to those not lead poisoned. The age, gender and ethnic differences observed were statistically significant (p =.01, p =.00, p =.00). Four of the six Program services: medical evaluations, rescreening, environmental inspections and confirmation, had satisfactory delivery completion rates of 71%-98%. Delivery timetable compliance rates for three of the six services examined: outreach contacts, home visits and environmental inspections were below 32%. However, dangerously elevated blood lead levels fell and lead poisoning prevalence dropped from 3.3% at initial screening to 1.2% among those rescreened, after intervention. From a public health perspective, reductions in lead poisoning prevalence are very meaningful. Based on these findings, the following are recommendations for future research: (1) integrate Program database files by utilizing a computer database management program; (2) target services at Hispanic male children under age 3 years living in the highest risk neighborhoods; (3) increase resources to: improve tracking and documentation of service delivery and provide more non-medical case management and environmental services; and (4) share the evaluation methodology/findings with the Centers for Disease Control and Prevention administrators; the implications may be relevant to other program managers conducting such assessments. ^

An evaluation of a community-based program for homeless adults with mental disorders and comorbid alcohol drug and mental disorders

Limited research has been conducted evaluating programs that are designed to improve the outcomes of homeless adults with mental disorders and comorbid alcohol, drug and mental disorders. This study conducted such an evaluation in a community-based day treatment setting with clients of the Harris County Mental Health and Mental Retardation Authority's Bristow Clinic. The study population included all clients who received treatment at the clinic for a minimum of six months between January 1, 1995 and August 31, 1996. An electronic database was used to identify clients and to track their program involvement. A profile was developed of the study participants and their level of program involvement included an examination of the amount of time spent in clinical, social and other interventions, the type of interventions encountered and the number of interventions encountered. Results were analyzed to determine whether social, demographic and mental history affected levels of program involvement and the effects of the levels of program involvement on housing status and psychiatric functioning status.^ A total of 101 clients met the inclusion criteria. Of the 101 clients, 96 had a mental disorder, and five had comorbidity. Due to the limited numbers of participants with comorbidity, only those with mental disorders were included in the analysis. The study found the Bristow Clinic population to be primarily single, Black, male, between the ages of 31 and 40 years, and with a gross family income of less than $4,000. There were more persons residing on the streets at entry and at six months following treatment than in any other residential setting. The most prevalent psychiatric diagnoses were depressive disorders and schizophrenia. The Global Assessment of Functioning (GAF) scale which was used to determine the degree of psychiatric functioning revealed a modal GAF score of 31--40 at entry and following six months in treatment. The study found that the majority of clients spent less than 17 hours in treatment, had less than 51 encounters and had clinical, social, and other encounters. In regard to social and demographic factors and levels of program involvement, there were statistically significant associations between gender and ethnicity and the types of interventions encountered as well as the number of interventions encountered. There was also a statistically significant difference between the amount of time spent in clinical interventions and gender. Relative to outcomes measured, the study found female gender to be the only background variable that was significantly associated with improved housing status and the female gender and previous MHMRA involvement to be statistically associated with improvement in GAF score. The total time in other (not clinical or social) interventions and the total number of encounters with other interventions were also significantly associated with improvement in housing outcome. The analysis of previous services and levels of program involvement revealed significant associations between time spent in social and clinical interventions and previous hospitalizations and previous MHMRA involvement.^ Major limitations of this study include the small sample size which may have resulted in very little power to detect differences and the lack of generalizability of findings due to site locations used in the study. Despite these limitations, the study makes an important contribution to the literature by documenting the levels of program involvement and the social and demographic factors necessary to produce outcomes of improved housing status and psychiatric functioning status. ^

An evaluation of the personal response system program for the elderly in Harris County, Texas

The Personal Response System Program at Huffington Center on Aging, Baylor College of Medicine, provides emergency call systems for elderly people living independently in Houston, Texas. The goal of the project was to complete a formative evaluation of the Personal Response System Program. The specific aims of the evaluation were three-fold. One aim was to evaluate participant health status and level of disability. The second aim was to develop a health care cost estimation strategy. Finally, a preliminary cost-effectiveness analysis was completed to evaluate the program's impact on health care costs and health status among the elderly target population. ^ The evaluation was a longitudinal, randomized experimental design. After the screening of 120 volunteers for eligibility, clients were asked to complete a written questionnaire and a monthly health service contact diary. Volunteers were contacted by telephone interviewers to collect health status information from 100 eligible clients (83%) on three occasions during the six months of follow-up. ^ Initially, volunteers were randomized to two experimental groups. The two groups were found to be comparable at the beginning of the study. No significant differences were detected related to health status, level of disability, or history of physician visits at baseline. However, the group with the Personal Response System (PRS) device had more adverse health events, higher IADL scores, more frequent use of walkers, lower average health status scores, and fewer community volunteers hours than the usual care comparison group. ^ The health care costs were estimated based on an algorithm adapted from the American Medical Association guidelines. Average total health care costs for the group with the PRS device ($912) were greater than the usual care group ($464). However, median health care values for the PRS group ($263) were similar to the usual care comparison group ($234). The preliminary findings indicated that the use of the PRS device was not associated with health care cost savings. ^ In the preliminary cost-effectiveness analysis, use of the personal response system was found to be associated with increased mental health status among elderly clients. The cost-effectiveness evaluation indicated that the associated cost for six months was $710 per unit increase in mental component score when the $150 PRS subscription was included. However, clients with the PRS device were found to have a greater decline in physical health status during the six-month follow-up. The beneficial effect on mental health status was found to be in contrast to negative findings associated with changes in physical health status. The implications for future research relate to the need to identify risk factors among geriatric populations to better target groups that would most likely benefit from PRS Program enrollment. ^

Development and evaluation of a cervical cancer screening program for Chinese women

Despite of the proven efficacy of the Pap test, Asian populations still have low Pap screening compliance. The purpose of this dissertation was to investigate factors that influencing women's decision to obtain a Pap test, and to describe the development and evaluation of a cervical cancer educational program promoting the Pap screening behavior among women in Taiwan. ^ The first study examined factors associated with Pap screening compliance. Psychometric properties of measurement instruments were also assessed. The scale reliabilities were as the follows: Cronbach alpha 0.70 for knowledge scale, 0.88 for pros scale, 0.68 for cons scale, and 0.72 for perceived norms scale. Results from multiple logistic regression analysis, after adjusted for marital status, showed women who compliant to Pap screening guidelines had significantly higher knowledge, higher perceived benefits (pros), lower perceived barriers (cons), and higher perceived norms to receive a Pap test. ^ The second study described the development of a program called “Love yourself before you take care of your family”, designed to increase Pap screening behavior among women in Taiwan. The development of this program was guided by Intervention Mapping (IM), an innovative process of intervention design. The program used methods such as information transmission, modeling, persuasion, and facilitation. Strategies included direct mail campaigns, role model stories with women's testimonials, and phone intervention. ^ The third study examined the effectiveness of a randomized trial of the carefully-designed intervention (N = 424). Participants were female family members of inpatients admitted to one of the major teaching hospitals in Taiwan during August and September 1999. Women in the intervention group reported a higher rate of receiving a Pap test than women in the control group (50% versus 32%) after a three-month intervention (p = 0.002). Women in the intervention group showed increased knowledge (p = .016), perceived pros (p = 0.008), and susceptibility (p = .011) between baseline and follow-up. They also showed higher perceived pros of Pap tests than women in control group at follow-up (p = .031). This result suggested that program development based on theories and evidences could maximize the intervention impact for a specific target population. ^

An evaluation of the impact of Inner City Development's Cooperative Home School: An educational alternative program to increase the quality and level of education in San Antonio's West Side community

The research study was intended to evaluate the effectiveness of Inner City Development's (I.C.D.) Cooperative Home School, an educational alternative program to the Title I public schools of San Antonio's West Side community. The study investigated students', parents' and tutors' perception of parental involvement and educational resources. The study also investigated each student's academic achievement. ^ The study found that students progressed toward expected math proficiency at a faster rate than they did in reading proficiency. However, because the target population size was small and a comparison group was not used, the results of this study are only suggestive. This research also indicated that study subjects believed students' quality and level of education increased substantially since program exposure. Study subjects mainly attributed the students' strides in academic performance to the increased amount of individualized attention students received in the small twelve-student class size. Study subjects were more satisfied with the home school's educational resources than those of the Title I public schools. Study subjects also perceived that parental involvement both at home and at school increased since enrollment in the home school program because: (1) there were more opportunities for involvement in the home school; and (2) parents felt closer to the tutors than the teachers in public school. ^ This evaluation also suggested improvements to program operations. With the help of additional volunteers, I.C.D. program operators could improve collection and organization of academic records. Furthermore, as suggested by program participants, science could be added to the curriculum. Lastly, a formal tutor orientation could be implemented to familiarize and train tutors on classroom management procedures. ^

Evaluation of the city of Houston HIV/STD prevention program in Houston Independent School District (HISD) high schools

This cross-sectional study was undertaken to evaluate the impact in terms of HIV/STD knowledge and sexual behavior that the City of Houston HIV/STD prevention program in HISD high schools has had on students who have participated in it by comparing them with their peers who have not, based on self reports. The study further evaluated the program cost-effectiveness for averting future HIV infections by computing Cost-Utility Ratios based on reported sexual behavior. ^ Mixed results were obtained, indicating a statistically significant difference in knowledge with the intervention group having scored higher (p-value 0.001) but not for any of the behaviors assessed. The knowledge score outcome's overall p-value after adjusting for each stratifying variable (age, grade, gender and ethnicity) was statistically significant. The Odds Ratio of intervention group participants aged 15 years or more scoring 70% or higher was 1.86 times; that of intervention group female participants was 2.29 times; and that of intervention group Black/African American participants was 2.47 times relative to their comparison group counterparts. The knowledge score results remained statistically significant in the logistic regression model, which controlled for age, grade level, gender and ethnicity. The Odds Ratio in this case was 1.74. ^ Three scenarios based on the difference in the risk of HIV infection between the intervention and comparison group were used for computation of Cost-Utility Ratios: Base, worst and best-case scenario. The best-case scenario yielded cost-effective results for male participants and cost-saving results for female participants when using ethnicity-adjusted HIV prevalence. The scenario remained cost-effective for female participants when using the unadjusted HIV prevalence. ^ The challenge to the program is to devise approaches that can enhance benefits for male participants. If it is a threshold problem implying that male participants require more intensive programs for behavioral change, then programs should first be piloted among boys before being implemented across the board. If it is a reflection of gender differences, then we might have to go back to the drawing board and engage boys in focus group discussions that will help formulate more effective programs. Gender-blind approaches currently in vogue do not seem to be working. ^

Evaluation of the effectiveness of the Fresh and Healthy Program

Background. Diets high in fat and calories are promoted by the toxic food environment in which high fat, high calorie foods are readily accessible, thus contributing to high rates of overweight and obesity.^ Hypothesis. Changing the food environment to make low-fat, low-calorie foods readily identifiable and accessible while simultaneously offering incentives for choosing those foods will result in increased consumption of targeted foods, thus decreasing caloric and fat intake and ultimately decreasing obesity rates.^ Objective. To conduct an outcome evaluation study on the effectiveness of The Fresh & Healthy Program, a health promotion project designed to promote healthy eating among The Methodist Hospital employees by labeling and promoting low calorie, low fat items in the hospital cafeteria. ^ Program. By promoting healthy eating, this program seeks to address unhealthy dietary behaviors, one of the most widely known and influential behavioral causes of obesity. Food items that are included in the program meet nutritional criteria for calories and fat and are labeled with a special logo. Program participants receive incentives for purchasing Fresh & Healthy items. The program was designed and implemented by a team of registered dietitians, two health education specialists, and retail foodservice managers at The Methodist Hospital in the Texas Medical Center in Houston and has been in existence since April 2006.^ Methods. The evaluation uses a non-randomized, one-group, time series design to evaluate the effect of the program on sales of targeted food items.^ Key words. point-of-purchase, menu labeling, environmental obesity interventions, food pricing interventions ^

The School Sealant Program: An evaluation of the oral health education component

Dental caries is a common preventable childhood disease leading to severe physical, mental and economic repercussions for children and their families if left untreated. A needs assessment in Harris County reported that 45.9% of second graders had untreated dental caries. In order to address this growing problem, the School Sealant Program (SSP), a primary preventive initiative, was launched by the Houston Department of Health and Human Services (HDHHS) to provide oral health education, and underutilized dental preventive services to second grade children from participating Local School Districts (LSDs). ^ To determine the effectiveness and efficiency of the SSP, a program evaluation was conducted by the HDHHS between September 2007 and June 2008 for the Oral Health Education (OHE) component of the SSP. The objective of the evaluation was to assess short term changes in oral health knowledge of the participants and determine if these changes, if any, were due to the OHE sessions. An 8-item multiple choice pre/post test was developed for this purpose and administered to the participants before and immediately after the OHE sessions. ^ The present project analyzed pre and post test data of 1,088 second graders from 22 participating schools. Changes in overall and topic-specific knowledge of the program participants before and after the OHE sessions were analyzed using the Wilcoxon's signed rank test. ^ Results. The overall knowledge assessment showed a statistically significant (p <0.001) increase in the dental health knowledge of the participants after the oral health education sessions. Participants in the higher scoring category (7-8 correct responses) increased from 9.5% at baseline to 60.8% after the education sessions. Overall knowledge increased in all school regions with the highest knowledge gains seen in the Central and South regions. Males and females had similar knowledge gains. Significant knowledge differences were also found for each of the topic specific categories (functions of teeth, healthy diet, healthy habits, dental sealants; p<0.001) indicating an increase in topic specific knowledge of the participants post-health education sessions. ^ Conclusions. The OHE sessions were successful in increasing the short term oral health knowledge of the participants. ^

The coordinated approach to child health in underserved population (CATCH UP) curriculum: A quantitative evaluation of an innovative preschool obesity prevention program

Obesity has been on the rise in the United States over the last 30 years for all populations, including preschoolers. The purpose of the project was to develop an observation tool to measure physical activity levels in preschool children and use the tool in a pilot test of the CATCH UP curriculum at two Head Start Centers in Houston. Pretest and posttest interobserver agreements were all above 0.60 for physical activity level and physical activity type. Preschoolers spent the majority of their time in light physical activity (75.33% pretest, 87.77% posttest), and spent little time in moderate to vigorous physical activity (MVPA) (24.67% pretest, 12.23% posttest). Percent time spent in MVPA decreased significantly pretest to posttest from (F=5.738, p=0.043). While the pilot testing of the CATCH UP curriculum did not show an increase in MVPA, the SOFIT-P tool did show promising results as being a new method for collecting physical activity level data for preschoolers. Once the new tool has undergone more reliability and validity testing, it could allow for a more convenient method of collecting physical activity levels for preschoolers. ^

Formative evaluation of a parental involvement program for obesity prevention with low-income elementary school children

This study addresses the responses to a postcard campaign with health messages targeting the parents of children in a sample of low-income elementary schools and assesses the feasibility and areas of possible improvements in such a project. The campaign was implemented in Spring 2009 with 4 th grade students (n=1070) in fifteen economically disadvantaged elementary schools in Travis County, Texas. Postcards were sent home with children, and parents filled out a feedback card that the children returned to school. Response data, in the form of self-administered feedback cards (n=2665) and one-on-one teacher interviews (n=8), were qualitatively analyzed using NVivo 8 software. Postcard reception and points of improvement were then identified from the significant themes that emerged including health, cessation or reduction of unhealthy behaviors, motivation, family, and the comprehension of abstract health concepts. ^ Responses to the postcard campaign were almost completely positive, with less than 1% of responses reporting some sort of dislike, and many parents reported a modification of their behavior. However, possible improvements that could be made to the campaign are: increased focus of the postcards on the parents as the target population, increased information about serving size, greater emphasis on the link between obesity and health, alteration of certain skin tones used in the graphical depiction of people on the cards, and smaller but more frequent incentives to return the feedback cards for the students. The program appears to be an effective method of communicating health messages to the parents of 4th grade children.^

Process evaluation of intervention program for prevention of early sexual activity and alcohol use among Hispanic teenagers

Hispanic teens are a high-risk population for initiation of early sexual activity and alcohol use which in turn has numerous social and health consequences. One strategy to address prevention of these behaviors is to implement a capacity building intervention that promotes parent child communication, encompasses their cultural values and community participation. This study describes the process evaluation of a pilot intervention program amongst Hispanic teens and their families living along the Texas-Mexico border. “Girls Lets Talk” is a small group intervention with 10-14 year old teens and their female adult family members that involves education regarding effects of alcohol use and sexual activity as well as activities for monitoring and refusal skills to prevent risky behaviors. Two waves of the program each consisting of at least seven mother daughter dyads were conducted. During the designing process, community advisory board meetings and focus groups were held to review course materials and ensure they were appropriate to the Mexican American culture. Parent and adolescent surveys were administered at the beginning and end of the intervention to assess for psychosocial outcome variables. All sessions received high mean satisfactory scores (mean of 4.00 or better on a five point scale) for both adult and adolescent participants. Qualitative feedback was obtained via debriefing sessions to evaluate experience as well as alter recruitment strategies. A Wilcoxon Sign Rank analysis of the pre and post intervention surveys was done that showed significant changes in some outcome variables such as intentions and confidence for monitoring behaviors for adults and beliefs regarding sexual activity. “Girls Lets Talk” is a promising example of how a process evaluation plan can help develop a theory based health promotion program using the community based participatory research approach. The intervention may also be effective in altering intentions and enhancing self-efficacy among parents and teens in order to decrease risky behaviors such as early sexual activity and alcohol use.^

Development and evaluation of a nutrition education program designed to increase fruit and vegetable consumption in girls, ages 9--12

The purpose of this study is to evaluate the theory-based Eat 5 nutrition badge. It is designed to increase fruit and vegetable (F&V) intake in 4th-6th grade junior Girl Scouts. Twenty-two troops were recruited and randomized by grade level (4th, 5th, 6th, or mixed) into either the intervention or control conditions. The leaders in the intervention condition received a brief training and the materials and conducted the program with their troops during four meetings. The Girl Scouts in the intervention condition completed 1-day Food Frequency Questionnaires and Nutrition Questionnaires both before and after completing the Eat 5 badge, and a third measurement of F&V intake three months after the posttest. Girl Scouts in the control condition were only evaluated at the three time periods.^ The primary hypotheses were that the Girl Scouts in the intervention condition would increase their daily intake of fruits and vegetables at both the posttest and three months later, compared to the Girl Scouts in the control condition. Other study questions investigated the impact of the Eat 5 program on intervening variables such as knowledge, self-efficacy, barriers, norms, F&V preference, and F&V selection and preparation skills.^ A nested ANOVA, with troop as the unit of analysis nested within condition, was used to assess the effects of the program. Pretest F&V intake and grade level were used as covariates. Pretest mean F&V intake for the total sample of 210 girls was 2.50 servings per day; 3.0 for the intervention group (n = 101). Significant increases in F&V intake (to 3.4 servings per day), knowledge, and fruit and vegetable preference were found for the intervention condition troops compared to the troops in the control condition. Three months later, the mean F&V intake had returned to pretest levels.^ This study indicates that social groups such as Girl Scouts can provide a channel for nutrition education. Long term effects were not sustained by the intervention; a possible cause was the lack of change in self-efficacy. Therefore, additional interventions are recommended such as booster lessons to maintain increased F&V intake by Girl Scouts. ^