UTILIZATION OF EVALUATION RESULTS

The issue over utilization of evaluation results has been receiving increasing attention lately. However, a continuing problem has been the lack of knowledge of the specific determinants of the utilization of evaluation results, the extent that these results are used, and the absence in the literature of empirical investigations of those determinants. The purposes of the current study were to examine the patterns of utilization of the recommendations made to an organization following an extensive evaluation, and to quantitatively examine some determinants of that utilization. During the course of the ten-month study, a number of important points regarding evaluation studies, and the utilization of their results were identified. Primary among these was the fact that a definition of utilization that requires adoption of recommendations is limited. Tremendous amounts of organizational activity may accrue from the results of an evaluation regardless of whether the results are adopted. In addition, none of the characteristics of the recommendations that were examined proved to be significant predictors of the extent of their utilization other than whether adoption was considered an authority decision or the results of staff concensus. ^

Use of empowerment evaluation to determine a sustainability plan for a community-based preconception care program

Women With IMPACT (WWI) is a community-based preconception care educational intervention. WWI is being implemented by the Impacting Maternal and Prenatal Care Together (IMPACT) Collaborative and targets zip codes in Harris County, Texas at high risk for infant mortality, low birthweight, and preterm birth. WWI started March 2012 and continues through August 2013. Three workshop series are planned. This study was conducted with participants and facilitators from the first workshop series. This study aimed to 1) evaluate the WWI program using empowerment evaluation, 2) engage all WWI stakeholders in an empowerment evaluation so the method could be adopted as a participatory evaluation process for future IMPACT activities, and 3) develop recommendations for sustainability of the WWI intervention, based on empowerment evaluation findings and results from the pre/post program evaluation completed by WWI participants. Study participants included WWI participants and facilitators and IMPACT Collaborative Steering Committee members. WWI participants were female, 18-35 year-old, non-pregnant residents of zip codes at high risk of adverse birth outcomes. All other study participants were 18 years or older. A two-phased empowerment evaluation (EE) was utilized in this study. Sessions 1-4 were conducted independently of one another – 3 with participants at different sites and one with the facilitators. The fifth session included WWI participant and facilitator representatives, and IMPACT Steering Committee members. Session 5 built upon the work of the other sessions. Observation notes were recorded during each session. Thematic content analysis was conducted on all EE tables and observation notes. Mission statements drafted by each group focused on improvement of physical and mental health through behavior change and empowerment of all participants. The top 5 overall program components were: physical activity, nutrition, self-worth, in-class communication, and stress. Goals for program improvement were set by EE participants for each of these components. Through thematic content analysis of the tables and observation notes, social support emerged as an important theme of the program among all participant groups. Change to a healthy lifestyle emerged as an important theme in terms of program improvement. Two-phased EE provided an opportunity for all program stakeholders to provide feedback regarding important program components and provide suggestions for program improvement. EE, thematic content analysis, pre/post evaluation results, and inherent program knowledge were triangulated to make recommendations to sustain the program once the initial funding ends. ^

Use of tabletop exercises for disaster preparedness training

Background. In public health preparedness, disaster preparedness refers to the strategic planning of responses to all types of disasters. Preparation and training for disaster response can be conducted using different teaching modalities, ranging from discussion-based programs such as seminars, drills and tabletop exercises to more complex operation-based programs such as functional exercises and full-scale exercises. Each method of instruction has its advantages and disadvantages. Tabletop exercises are facilitated discussions designed to evaluate programs, policies, and procedures; they are usually conducted in a classroom, often with tabletop props (e.g. models, maps or diagrams). ^ Objective. The overall goal of this project was to determine whether tabletop exercises are effective teaching modalities for disaster preparedness, with an emphasis on intentional chemical exposure. ^ Method. The target audience for the exercise was the Medical Reserve Brigade of the Texas State Guard, a group of volunteer healthcare providers and first responders who prepare for response to local disasters. A new tabletop exercise was designed to provide information on the complex, interrelated organizations within the national disaster preparedness program that this group would interact with in the event of a local disaster. This educational intervention consisted of a four hour multipart program that included a pretest of knowledge, lecture series, an interactive group discussion using a mock disaster scenario, a posttest of knowledge, and a course evaluation. ^ Results. Approximately 40 volunteers attended the intervention session; roughly half (n=21) had previously participated in a full scale drill. There was an 11% improvement in fund of knowledge between the pre- and post-test scores (p=0.002). Overall, the tabletop exercise was well received by those with and without prior training, with no significant differences found between these two groups in terms of relevance and appropriateness of content. However, the separate components of the tabletop exercise were variably effective, as gauged by written text comments on the questionnaire. ^ Conclusions. Tabletop exercises can be a useful training modality in disaster preparedness, as evidenced by improvement in knowledge and qualitative feedback on its value. Future offerings could incorporate recordings of participant responses during the drill, so that better feedback can be provided to them. Additional research should be conducted, using the same or similar design, in different populations that are stakeholders in disaster preparedness, so that the generalizability of these findings can be determined.^

Developing a Framework for Child Welfare Supervision

The roles of the child welfare supervisor in guiding practice and in retaining child welfare workers are well established in the literature. In this article, we discuss a framework for child welfare supervision that was developed and implemented in the state of Iowa with support from the Children’s Bureau through a five-year grant to improve recruitment and retention in public child welfare. The framework supports family centered practice through a parallel process of supervision reflecting these guiding principles: strength-based, competency-based, culturally competent, reflective, individualized to workers’ learning styles and stages of development, and aimed at enhancing worker skill, autonomy, teamwork, and commitment to the organization. We present key elements of the framework, an overview of implementation, and evaluation results regarding knowledge gain, use of skills, and rates of worker retention.