An examination of attitude toward change as a predictor of policy/program implementation

Policy/program implementation, e.g., the process of fulfilling policy/program directives, is fundamentally tied to change. Implementation studies have examined the process, identifying many critical organizational variables although individuals perform the activities.^ Many of the studies are predicated on the rational, goal oriented model of organizations and examine implementation, presenting only the goal-oriented view. Organizational change and its resistance, however, are not fully explained by the rational model of organizations. There are other schools of thought providing different views of organizations from which explanation may emerge. Bolman and Deal (1984, 1991a, 1994) provide a different perspective for examining organizations Bolman and Deal argue organizations should be viewed through four different frames or lenses. Framing and reframing organizational action captures the complexity of action and provides better understanding of organizational processes. Understanding of implementation of policies/programs also will benefit from the use of the four-frame approach.^ The goal of this research is to provide a better understanding of the implementation process by examining individual attitudes toward change, the dependent variable of this research, and studying the relationship between the dependent variable and frame. The research was conducted in two phases. In Phase One, a survey was sent to 306 school administrators and teachers in magnet programs in Dade County, Florida. The survey instrument was composed of 55 questions including six from Bolman and Deal's Leadership Orientation Survey (1988) and 38 questions about organizational change. In Phase Two, more in-depth analysis of four school was conducted, to further explore the relationship between frame and attitude toward change.^ The results revealed that frame was a factor in explaining differences in personal Attitude Toward Change and Comfort Level with Change. Individuals using the symbolic frame had more positive attitudes toward change and were also more comfortable with change. The results of Phase Two of the research partially supported this finding in that the most fully implemented program was the product of an administrator who had chosen the symbolic frame. ^

Truancy reduction: Extension of a county program

The primary purpose of this research was to examine the effect of the Truancy Intervention Program (TIP) on attendance patterns of elementary school students. Longitudinal archival data were used from Miami-Dade County Public School system's data system, ISIS. Data included the students' school information from fifth through ninth grade for attendance, academic grades, referral information, and referral consequences. The sample for this study was drawn from students at TIP-participating M-DCPS elementary schools in Miami-Dade County. Data collected spanned five years for each participant from the fifth grade to the ninth grade. To examine the effect of TIP on attendance, participation in middle school TIP was compared with non-TIP participation. In addition to immediate effects on attendance, the durability of the effects of TIP was studied through an analysis of attendance at the ninth grade level. A secondary purpose was to examine the relation of TIP participation to Grade Point Average (GPA). ^ The data were analyzed using 2 (group) x 3 (grade) Repeated Measures Analysis of Variance (ANOVA) on yearly attendance (number of absences), and grade point average for each year. The interaction between group and grade was significant. Post hoc tests indicated that absences were not significantly different in the two programs in seventh, eighth or ninth grade. Students enrolled in a middle school with TIP showed a significantly higher number of absences in ninth grade than for seventh or eighth grade. There were no differences by grade level for students enrolled in non-TIP middle schools. GPA analysis indicated that students enrolled in a non-TIP middle school had a significantly higher GPA across seventh, eighth, and ninth grades when compared to students enrolled at a TIP middle school. ^ An examination of attendance disciplinary referrals and consequences further revealed that the referral rates for students enrolled at a TIP middle school were higher at the seventh, eighth, and ninth grade level, then for students enrolled at a non-TIP middle school. This pattern was not readily apparent at non-TIP middle schools. Limitations of the research were noted and further research regarding program implementation (process evaluation) was suggested. ^

Factors influencing the implementation of substance use prevention programs in elementary schools

Prevention scientists have called for more research on the factors affecting the implementation of substance use prevention programs. Given the lack of literature in this area, coupled with evidence that children as early as elementary school engage in substance use, the purpose of this study was to identify the factors that influence the implementation of substance use prevention programs in elementary schools. This study involved a mixed methods approach comprised of a survey and in-person interviews. Sixty-five guidance counselors and teachers completed the survey, and 9 guidance counselors who completed the survey were interviewed individually. Correlation analyses and hierarchical multiple regression were conducted. Quantitative findings revealed ease of implementation most frequently influenced program implementation, followed by beliefs about the program's effectiveness. Qualitative findings showed curriculum modification as an important theme, as well as difficulty of program implementation. The in-person interviews also shed light on three interrelated themes influencing program implementation – The Wheel, time, and scheduling. Results indicate the majority of program providers modified the curriculum in some way. Implications for research, policy, and practice are discussed, and areas for future research are suggested.^

Factors Influencing the Implementation of Substance Use Prevention Programs in Elementary Schools

Prevention scientists have called for more research on the factors affecting the implementation of substance use prevention programs. Given the lack of literature in this area, coupled with evidence that children as early as elementary school engage in substance use, the purpose of this study was to identify the factors that influence the implementation of substance use prevention programs in elementary schools. This study involved a mixed methods approach comprised of a survey and in-person interviews. Sixty-five guidance counselors and teachers completed the survey, and 9 guidance counselors who completed the survey were interviewed individually. Correlation analyses and hierarchical multiple regression were conducted. Quantitative findings revealed ease of implementation most frequently influenced program implementation, followed by beliefs about the program’s effectiveness. Qualitative findings showed curriculum modification as an important theme, as well as difficulty of program implementation. The in-person interviews also shed light on three interrelated themes influencing program implementation – The Wheel, time, and scheduling. Results indicate the majority of program providers modified the curriculum in some way. Implications for research, policy, and practice are discussed, and areas for future research are suggested.

An analysis of teachers' perceptions of informal science professional development

Science professional development, which is fundamental to science education improvement, has been described as being weak and fragmentary. The purpose of this study was to investigate teachers' perceptions of informal science professional development to gain an in-depth understanding of the essence of the phenomenon and related science-teaching dispositions. Based on the frameworks of phenomenology, constructivism, and adult learning theory, the focus was on understanding how the phenomenon was experienced within the context of teachers' everyday world. ^ Data were collected from eight middle-school teachers purposefully selected because they had participated in informal programs during Project TRIPS (Teaching Revitalized Through Informal Programs in Science), a collaboration between the Miami-Dade school district, government agencies (including NASA), and non-profit organizations (including Audubon of Florida). In addition, the teachers experienced hands-on labs offered through universities (including the University of Arizona), field sites, and other agencies. ^ The study employed Seidman's (1991) three-interview series to collect the data. Several methods were used to enhance the credibility of the research, including using triangulation of the data. The interviews were transcribed, color-coded and organized into six themes that emerged from the data. The themes included: (a) internalized content knowledge, (b) correlated hands-on activities, (c) enhanced science-teaching disposition, (d) networking/camaraderie, (e) change of context, and (f) acknowledgment as professionals. The teachers identified supportive elements and constraints related to each theme. ^ The results indicated that informal programs offering experiential learning opportunities strengthened understanding of content knowledge. Teachers implemented hands-on activities that were explicitly correlated to their curriculum. Programs that were conducted in a relaxed context enhanced teachers' science-teaching dispositions. However, a lack of financial and administrative support, perceived safety risks, insufficient reflection time, and unclear itineraries impeded program implementation. The results illustrated how informal educators can use this cohesive model as they develop programs that address the supports and constraints to teachers' science instruction needs. This, in turn, can aid teachers as they strive to provide effective science instruction to students; notions embedded in reforms. Ultimately, this can affect how learners develop the ability to make informed science decisions that impact the quality of life on a global scale. ^

The early development and pilot -testing of a case management intervention with victims of stalking: An evaluation study

This study evaluated the early development and pilot-testing of Project IMPACT, a case management intervention for victims of stalking. The Design and Development framework (Rothman & Thomas, 1994) was used as a guide for program development and evaluation. Nine research questions examined the processes and outcomes associated with program implementation. ^ The sample included all 36 clients who participated in Project IMPACT between February of 2000 and June of 2001, as well as the victim advocates who provided them with services. Quantitative and qualitative data were drawn from client case files, participant observation field notes and interview transcriptions. Quantitative data were entered into three databases where: (1) clients were the units of analysis (n = 36), (2) services were the units of analysis (n = 1146), and (3) goals were the units of analysis (n = 149). These data were analyzed using descriptive statistics, Pearson's Chi-square, Spearman's Rho, Phi, Cramer's V, Wilcoxon's Matched Pairs Signed-Ranked Test and McNemar's Test Statistic. Qualitative data were reduced via open, axial and selective coding methods. Grounded theory and case study frameworks were utilized to analyze these data. ^ Results showed that most clients noted an improved sense of well-being and safety, although residual symptoms of trauma remained for numerous individuals. Stalkers appeared to respond to criminal and civil justice-based interventions by reducing violent and threatening behaviors; however, covert behaviors continued. The study produced findings that provided preliminary support for the use of several intervention components including support services, psycho-education, safety planning, and boundary spanning. The psycho-education and safety planning in particular seemed to help clients cognitively reframe their perceptions of the stalking experience and gain a sense of increased safety and well-being. A 65% level of satisfactory goal achievement was observed overall, although goals involving justice-based organizations were associated with lower achievement. High service usage was related to low-income clients and those lacking in social support. Numerous inconsistencies in program implementation were found to be associated with the skills and experiences of victim advocates. Thus, recommendations were made to further refine, develop and evaluate the intervention. ^

Performance accountability in the federal-state transportation program: Factors affecting successful implementation

Governmental accountability is the requirement of government entities to be accountable to the citizenry in order to justify the raising and expenditure of public resources. The concept of service efforts and accomplishments measurement for government programs was introduced by the Governmental Accounting Standards Board (GASB) in Service Efforts and Accomplishments Reporting: Its Time Has Come (1990). This research tested the feasibility of implementing the concept for the Federal-aid highway construction program and identified factors affecting implementation with a case study of the District of Columbia. Changes in condition and performance ratings for specific highway segments in 15 projects, before and after construction expenditures, were evaluated using data provided by the Federal Highway Administration. The results of the evaluation indicated difficulty in drawing conclusions on the state program performance, as a whole. The state program reflects problems within the Federally administered program that severely limit implementation of outcome-oriented performance measurement. Major problems identified with data acquisition are: data reliability, availability, compatibility and consistency among states. Other significant factors affecting implementation are institutional barriers and political barriers. Institutional issues in the Federal Highway Administration include the lack of integration of the fiscal project specific database with the Highway Performance Monitoring System database. The Federal Highway Administration has the ability to resolve both of the data problems, however interviews with key Federal informants indicate this will not occur without external directives and changes to the Federal “stewardship” approach to program administration. ^ The findings indicate many issues must be resolved for successful implementation of outcome-oriented performance measures in the Federal-aid construction program. The issues are organizational and political in nature, however in the current environment resolution is possible. Additional research is desirable and would be useful in overcoming the obstacles to successful implementation. ^

Pilot Implementation of a Heat Illness Prevention Program in the Southeastern US

Objective: To evaluate the ease of application of a heat illness prevention program (HIPP). Design: A mixed-method research design was used: questionnaire and semi-structured interview. Setting: Eleven South Florida high schools in August (mean ambient temperature=84.0°F, mean relative humidity=69.5%) participated in the HIPP. Participants: Certified Athletic Trainers (AT) (n=11; age=22.2+1.2yr; 63.6% female, 36.4% male; 63.6%) implemented the HIPP with their football athletes which included a pre-screening tool, the Heat Illness Index Score- Risk Assessment. Data Collection and Analysis: Participants completed a 17-item questionnaire, 4 of which provided space for open-ended responses. Additionally, semi-structured interviews were voice recorded, and separately transcribed. Results: Three participants (27.7%) were unable to implement the HIPP with any of their athletes. Of the 7 participants (63.6%) who implemented the HIPP to greater than 50% of their athletes, a majority reported that the HIPP was difficult (54.5%) or exceedingly difficult (18.2%) to implement. Lack of appropriate instrumentation (81.8%, n=9/11), lack of coaching staff/administrative support (54.5%, n=6/11), insufficient support staff (54.5%, n=6/11), too many athletes (45.5%, n=5/11), and financial restrictions (36.4%, n=4/11) deterred complete implementation of the HIPP. Conclusions: Because AT in the high school setting often lack the resources, time, and coaches’ support to identify risk factors, predisposing athletes to exertional heat Illnesses (EHI) researchers should develop and validate a suitable screening tool. Further, ATs charged with the health care of high school athletes should seek out prevention programs and screening tools to identify high-risk athletes and monitor athletes throughout exercise in extreme environments.

Understanding the dynamics of information technology implementation: The case of clinical information systems

The ultimate intent of this dissertation was to broaden and strengthen our understanding of IT implementation by emphasizing research efforts on the dynamic nature of the implementation process. More specifically, efforts were directed toward opening the "black box" and providing the story that explains how and why contextual conditions and implementation tactics interact to produce project outcomes. In pursuit of this objective, the dissertation was aimed at theory building and adopted a case study methodology combining qualitative and quantitative evidence. Precisely, it examined the implementation process, use and consequences of three clinical information systems at Jackson Memorial Hospital, a large tertiary care teaching hospital.^ As a preliminary step toward the development of a more realistic model of system implementation, the study proposes a new set of research propositions reflecting the dynamic nature of the implementation process.^ Findings clearly reveal that successful implementation projects are likely to be those where key actors envision end goals, anticipate challenges ahead, and recognize the presence of and seize opportunities. It was also found that IT implementation is characterized by the systems theory of equifinality, that is, there are likely several equally effective ways to achieve a given end goal. The selection of a particular implementation strategy appears to be a rational process where actions and decisions are largely influenced by the degree to which key actors recognize the mediating role of each tactic and are motivated to action. The nature of the implementation process is also characterized by the concept of "duality of structure," that is, context and actions mutually influence each other. Another key finding suggests that there is no underlying program that regulates the process of change and moves it form one given point toward a subsequent and already prefigured end. For this reason, the implementation process cannot be thought of as a series of activities performed in a sequential manner such as conceived in stage models. Finally, it was found that IT implementation is punctuated by a certain indeterminacy. Results suggest that only when substantial efforts are focused on what to look for and think about, it is less likely that unfavorable and undesirable consequences will occur. ^

Development and implementation of a tool to assess the micro/macro elements in an organization

This study examines the congruency of planning between organizational structure and process, through an evaluation and planning model known as the Micro/Macro Dynamic Planning Grid. The model compares day-to-day planning within an organization to planning imposed by organizational administration and accrediting agencies. A survey instrument was developed to assess the micro and macro sociological analysis elements utilized by an organization.^ The Micro/Macro Dynamic Planning Grid consists of four quadrants. Each quadrant contains characteristics that reflect the interaction between the micro and macro elements of planning, objectives and goals within an organization. The Over Macro/Over Micro, Quadrant 1, contains attributes that reflect a tremendous amount of action and ongoing adjustments, typical of an organization undergoing significant changes in either leadership, program and/or structure. Over Macro/Under Micro, Quadrant 2, reflects planning characteristics found in large, bureaucratic systems with little regard given to the workings of their component parts. Under Macro/Under Micro, Quadrant 3, reflects the uncooperative, uncoordinated organization, one that contains a multiplicity of viewpoints, language, objectives and goals. Under Macro/Under Micro, Quadrant 4 represents the worst case scenario for any organization. The attributes of this quadrant are very reactive, chaotic, non-productive and redundant.^ There were three phases to the study: development of the initial instrument, pilot testing the initial instrument and item revision, and administration and assessment of the refined instrument. The survey instrument was found to be valid and reliable for the purposes and audiences herein described.^ In order to expand the applicability of the instrument to other organizational settings, the survey was administered to three professional colleges within a university.^ The first three specific research questions collectively answered, in the affirmative, the basic research question: Can the Micro/Macro Dynamic Planning Grid be applied to an organization through an organizational development tool? The first specific question: Can an instrument be constructed that applies the Micro/Macro Dynamic Planning Grid? The second specific research question: Is the constructed instrument valid and reliable? The third specific research question: Does an instrument that applies the Micro/Macro Dynamic Planning Grid assess congruency of micro and macro planning, goals and objectives within an organization? The fourth specific research question: What are the differences in the responses based on roles and responsibilities within an organization? involved statistical analysis of the response data and comparisons obtained with the demographic data. (Abstract shortened by UMI.) ^

Profile of planning: A study of a three-year project on the implementation of collaborative library media programs

The implementation of collaborative planning and teaching models in ten flexibly scheduled elementary and middle school library media centers was studied to determine which factors facilitated the collaborative planning process and to learn what occurs when library media specialists (LMSs) and classroom teachers (CTs) plan together. In this qualitative study, 61 principals, CTs, and LMSs were interviewed on a range of topics including the principal's role, school climate, the value of team planning, the importance of information literacy instruction, and the ideal learning environment. Other data sources were observations, videotapes of planning sessions, and documents. This three-year school reform effort was funded by the Library Power Project to improve library programs, to encourage collaborative planning, and to increase curricular integration of information literacy skills instruction. ^ The findings included a description of typical planning sessions and the identification of several major factors which impacted the success of collaborative planning: the individuals involved, school climate, time for planning, the organization of the school, the facility and collection, and training. Of these factors, the characteristics and actions of the people involved were most critical to the implementation of the innovation. The LMS was the pivotal player and, in the views of CTs, principals, and LMSs themselves, must be knowledgeable about curriculum, the library collection, and instructional design and delivery; must be open and welcoming to CTs and use good interpersonal skills; and must be committed to information literacy instruction and willing to act as a change agent. The support of the principal was vital; in schools with successful programs, the principal served as an advocate for collaborative planning and information literacy instruction, provided financial support for the library program including clerical staff, and arranged for LMSs and CTs to have time during the school day to plan together. ^ CTs involved in positive planning partnerships with LMSs were flexible, were open to change, used a variety of instructional materials, expected students to be actively involved in their own learning, and were willing to team teach with LMSs. Most CTs planning with LMSs made lesson plans in advance and preferred to plan with others. Also, most CTs in this study planned with grade level or departmental groups, which expedited the delivery of information literacy instruction and the effective use of planning time. ^ Implications of the findings of this research project were discussed for individual schools, for school districts, and for colleges and universities training LMSs, CTs, and administrators. Suggestions for additional research were also included. ^

The effect of dual -language instruction on academic achievement as measured by the SAT: An evaluation of the "International Studies Program" of Miami -Dade County Public Schools

This study investigated the influence that receiving instruction in two languages, English and Spanish, had on the performance of students enrolled in the International Studies Program (delayed partial immersion model) of Miami Dade County Public Schools on a standardized test in English, the Stanford Achievement Test, eighth edition, for three of its sections, Reading Comprehension, Mathematics Computations, and Mathematics Applications.^ The performance of the selected IS program/Spanish section cohort of students (N = 55) on the SAT Reading Comprehension, Mathematics Computation, and Mathematics Application along four consecutive years was contrasted with that of a control group of comparable students selected within the same feeder pattern where the IS program is implemented (N = 21). The performance of the group was also compared to the cross-sectional achievement patterns of the school's corresponding feeder pattern, region, and district.^ The research model for the study was a variation of the "causal-comparative" or "ex post facto design" sometimes referred to as "prospective". After data were collected from MDCPS, t-tests were performed to compare IS-Spanish students SAT performance for grades 3 to 6 for years 1994 to 1997 to control group, feeder pattern, region and district norms for each year for Reading Comprehension, Mathematics Computation, and Mathematics Applications. Repeated measures ANOVA and Tukey's tests were calculated to compare the mean percentiles of the groups under study and the possible interactions of the different variables. All tests were performed at the 5% significance level.^ From the analyses of the tests it was deduced that the IS group performed significantly better than the control group for all the three measures along the four years. The IS group mean percentiles on the three measures were also significantly higher than those of the feeder pattern, region, and district. The null hypotheses were rejected and it was concluded that receiving instruction in two languages did not negatively affect the performance of IS program students on tests taken in English. It was also concluded that the particular design the IS program enhances the general performance of participant students on Standardized tests.^ The quantitative analyses were coupled with interviews from teachers and administrators of the IS program to gain additional insight about different aspects of the implementation of the program at each particular school. ^

Program evaluation primer: A review of three evaluations

Program evaluation—a “tool used to assess the implementation and outcomes of a program, to increase a program’s efficiency and impact over time, and to demonstrate accountability” (MacDonald et. al, 2001, p. 1)—is an essential process to program assessment and improvement. This paper overviews three published program evaluations and considers important aspects of program evaluation more broadly.

Florida's Voluntary Prekindergarten (VPK) Program's Influence on Preschool Teachers' Beliefs and Practices

Three veteran preschool teachers told their stories about navigating the implementation of VPK. Even with parents' misconceptions about VPK's goals, teachers were able to meet its standards by continuing to embrace a play-based, hands-on philosophy that also emphasized children's social and emotional needs.

Development and Application of a Multi-dimensional Strategy for Measuring Fidelity of Implementation of a Districtwide Intervention

This symposium describes a multi-dimensional strategy to examine fidelity of implementation in an authentic school district context. An existing large-district peer mentoring program provides an example. The presentation will address development of a logic model to articulate a theory of change; collaborative creation of a data set aligned with essential concepts and research questions; identification of independent, dependent, and covariate variables; issues related to use of big data that include conditioning and transformation of data prior to analysis; operationalization of a strategy to capture fidelity of implementation data from all stakeholders; and ways in which fidelity indicators might be used.