Development and Application of a Multi-dimensional Strategy for Measuring Fidelity of Implementation of a Districtwide Intervention

This symposium describes a multi-dimensional strategy to examine fidelity of implementation in an authentic school district context. An existing large-district peer mentoring program provides an example. The presentation will address development of a logic model to articulate a theory of change; collaborative creation of a data set aligned with essential concepts and research questions; identification of independent, dependent, and covariate variables; issues related to use of big data that include conditioning and transformation of data prior to analysis; operationalization of a strategy to capture fidelity of implementation data from all stakeholders; and ways in which fidelity indicators might be used.

The eye of the beholder: The impact of a district implementation plan of state mandates as perceived by the implementers

The purpose of this inquiry was to investigate the impact of a large, urban school district's experience in implementing a mandated school improvement plan and to examine how that plan was perceived, interpreted, and executed by those charged with the task. The research addressed the following questions: First, by whom was the district implementation plan designed, and what factors were considered in its construction? Second, what impact did the district implementation plan have on those charged with its implementation? Third, what impact did the district plan have on the teaching and learning practices of a particular school? Fourth, what aspects of the implemention plan were perceived as most and least helpful by school personnel in achieving stated goals? Last, what were the intended and unintended consequences of an externally mandated and directed plan for improving student achievement? The implementation process was measured against Fullan's model as expounded upon in The Meaning of Educational Change (1982) and The New Meaning of Educational Change (1990). The Banya implementation model (1993), because it added a dimension not adequately addressed by Fullan, was also considered.^ A case study was used as the methodological framework of this qualitative study. Sources of data used in this inquiry included document analysis, participant observations in situ, follow-up interviews, the "long" interview, and triangulation. The study was conducted over a twelve-month period. Findings were obtained from the content analysis of interview transcripts of multiple participants. Results were described and interpreted using the Fullan and Banya models as the descriptive framework. A cross-case comparison of the multiple perspectives of the same phenomena by various participants was constructed.^ The study concluded that the school district's implementation plan to improve student achievement was closely aligned to Fullan's model, although not intentionally. The research also showed that where there was common understanding at all levels of the organization as to the expectations for teachers, level of support to be provided, and availability of resources, successful implementation occured. The areas where successful implementation did not occur were those where the complexity of the changes were underestimated and processes for dealing with unintended consequences were not considered or adequately addressed. The unique perspectives of the various participants, from the superintendent to the classroom teacher, are described. Finally, recommendations for enhancement of implementation are offered and possible topics for further research studies are postulated. ^

A qualitative case study of teacher perceptions and concerns during the implementation of literacy folders in a select elementary school in Broward County, Florida

Classroom teachers are often required to implement new procedures or practices in response to local or federal education mandates. Attempts to implement innovations, often do not take into account the personal side of change; the perceptions, concerns and needs of those required to implement the innovation. One innovation that was required by the School Board of Broward County, Florida for all elementary classroom teachers was the implementation of Literacy Folders. ^ This study attempted to address the personal side of change by identifying teacher concerns during the implementation of Literacy Folders in a select elementary school in Broward County Florida. The Concerns Based Adoption model (CBAM) for change was used as the conceptual framework for this qualitative case study. ^ Sources of data for this study included participant interviews, observations and analysis of documents. Informal conversations with the participants and unscheduled classroom visits were also sources of data. Seven classroom teachers were interviewed using a predesigned interview guide developed based on the CBAM of change, specifically the Stages of Concern Dimension. Participant responses were coded into two categories, (a) recollections of past perceptions, and (b) present perceptions regarding the innovation. ^ Data analysis resulted in the emergence of one major theme and two subordinate themes. The themes were related to time and purpose of the innovation. The researcher also discovered that the participants exhibited responses typically representative of the CBAM for individuals who are in the process of adjusting to a new innovation. ^ Recommendations based on participant concerns are made for improving the implementation of the innovation. Recommendations for alternatives to the innovation and suggestions regarding areas for further research in the field of educational change are also made. ^

Factors influencing the implementation of substance use prevention programs in elementary schools

Prevention scientists have called for more research on the factors affecting the implementation of substance use prevention programs. Given the lack of literature in this area, coupled with evidence that children as early as elementary school engage in substance use, the purpose of this study was to identify the factors that influence the implementation of substance use prevention programs in elementary schools. This study involved a mixed methods approach comprised of a survey and in-person interviews. Sixty-five guidance counselors and teachers completed the survey, and 9 guidance counselors who completed the survey were interviewed individually. Correlation analyses and hierarchical multiple regression were conducted. Quantitative findings revealed ease of implementation most frequently influenced program implementation, followed by beliefs about the program's effectiveness. Qualitative findings showed curriculum modification as an important theme, as well as difficulty of program implementation. The in-person interviews also shed light on three interrelated themes influencing program implementation – The Wheel, time, and scheduling. Results indicate the majority of program providers modified the curriculum in some way. Implications for research, policy, and practice are discussed, and areas for future research are suggested.^

The relationship between faculty perceptions and implementation of elements of transactional distance theory and online web-based course completion rates

The purpose of this study was to explore the relationship between faculty perceptions, selected demographics, implementation of elements of transactional distance theory and online web-based course completion rates. This theory posits that the high transactional distance of online courses makes it difficult for students to complete these courses successfully; too often this is associated with low completion rates. Faculty members play an indispensable role in course design, whether online or face-to-face. They also influence course delivery format from design through implementation and ultimately to how students will experience the course. This study used transactional distance theory as the conceptual framework to examine the relationship between teaching and learning strategies used by faculty members to help students complete online courses. Faculty members' sex, number of years teaching online at the college, and their online course completion rates were considered. A researcher-developed survey was used to collect data from 348 faculty members who teach online at two prominent colleges in the southeastern part of United States. An exploratory factor analysis resulted in six factors related to transactional distance theory. The factors accounted for slightly over 65% of the variance of transactional distance scores as measured by the survey instrument. Results provided support for Moore's (1993) theory of transactional distance. Female faculty members scored higher in all the factors of transactional distance theory when compared to men. Faculty number of years teaching online at the college level correlated significantly with all the elements of transactional distance theory. Regression analysis was used to determine that two of the factors, instructor interface and instructor-learner interaction, accounted for 12% of the variance in student online course completion rates. In conclusion, of the six factors found, the two with the highest percentage scores were instructor interface and instructor-learner interaction. This finding, while in alignment with the literature concerning the dialogue element of transactional distance theory, brings a special interest to the importance of instructor interface as a factor. Surprisingly, based on the reviewed literature on transactional distance theory, faculty perceptions concerning learner-learner interaction was not an important factor and there was no learner-content interaction factor.

A study of the implementation of junior college *evaluation system in Taiwan

Since the establishment of the evaluation system in 1975, the junior colleges in the Republic of China (Taiwan), have gone through six formal evaluations. We know that evaluation in schooling, like quality control in businesses, should be a systematic, formal, and a continual process. It can doubtless serve as a strategy to refine the quality of education. The purpose of this research is to explore the current practice of junior college evaluation in Taiwan. This provides insight into the development of and quality of the current evaluation system. Moreover, this study also identified the source of problems with the current evaluation system and provided suggestion for improvements.^ In order to attain the above purposes, this research was undertaken in both theoretical and practical ways. First, theoretically, on the basis of a literature review, the theories of educational evaluation and, according to the course and principles of development, a view of the current practice in Taiwan. Secondly, in practice, by means of questionnaires, an analysis of the views of evaluation committeemen, junior college presidents, and administrators were obtained on evaluation models, methods, contents, organization, functions, criteria, grades reports, and others with suggestions for improvement. The summary of findings concludes that most evaluators and evaluatees think the purpose of evaluation can help the colleges explore their difficulties and problems. In addition, it was found that there is significant difference between the two groups regarding the evaluation methods, contents, organization, functions, criteria, grades reports and others, while analyzing these objective data forms the basis for an improved method of evaluation for Junior Colleges in Taiwan. ^

The design, implementation and production process of creating a viable series of Latin shows in Hollywood, Florida

The purpose of this research was to design and implement a Series of Latin Shows to be featured at the Satine Restaurant located in The Diplomat Hotel in Hollywood, Florida. Three shows were created: "Electro Tango," "Bossa Nova Jazz," and "Piel Canela Night" to help generate interest for not only the Satine Restaurant but also for the surrounding area. The artistic concept included big bands, costumes, dancers and a DJ. A production book was created and included the most important aspects of the individual shows such as budgets, costumes, and ground plans, to assure the success of each event. Careful analysis was done for the demographic area and a marketing plan was designed and implemented. The research and practical application of similar shows in the industry determined that the production of these particular shows, although costly, have a qualifiable chance to succeed in this venue.

Factors Influencing the Implementation of Substance Use Prevention Programs in Elementary Schools

Prevention scientists have called for more research on the factors affecting the implementation of substance use prevention programs. Given the lack of literature in this area, coupled with evidence that children as early as elementary school engage in substance use, the purpose of this study was to identify the factors that influence the implementation of substance use prevention programs in elementary schools. This study involved a mixed methods approach comprised of a survey and in-person interviews. Sixty-five guidance counselors and teachers completed the survey, and 9 guidance counselors who completed the survey were interviewed individually. Correlation analyses and hierarchical multiple regression were conducted. Quantitative findings revealed ease of implementation most frequently influenced program implementation, followed by beliefs about the program’s effectiveness. Qualitative findings showed curriculum modification as an important theme, as well as difficulty of program implementation. The in-person interviews also shed light on three interrelated themes influencing program implementation – The Wheel, time, and scheduling. Results indicate the majority of program providers modified the curriculum in some way. Implications for research, policy, and practice are discussed, and areas for future research are suggested.

The Relationship between Faculty Perceptions and Implementation of Elements of Transactional Distance Theory and Online Web-Based Course Completion Rates

The purpose of this study was to explore the relationship between faculty perceptions, selected demographics, implementation of elements of transactional distance theory and online web-based course completion rates. This theory posits that the high transactional distance of online courses makes it difficult for students to complete these courses successfully; too often this is associated with low completion rates. Faculty members play an indispensable role in course design, whether online or face-to-face. They also influence course delivery format from design through implementation and ultimately to how students will experience the course. This study used transactional distance theory as the conceptual framework to examine the relationship between teaching and learning strategies used by faculty members to help students complete online courses. Faculty members’ sex, number of years teaching online at the college, and their online course completion rates were considered. A researcher-developed survey was used to collect data from 348 faculty members who teach online at two prominent colleges in the southeastern part of United States. An exploratory factor analysis resulted in six factors related to transactional distance theory. The factors accounted for slightly over 65% of the variance of transactional distance scores as measured by the survey instrument. Results provided support for Moore’s (1993) theory of transactional distance. Female faculty members scored higher in all the factors of transactional distance theory when compared to men. Faculty number of years teaching online at the college level correlated significantly with all the elements of transactional distance theory. Regression analysis was used to determine that two of the factors, instructor interface and instructor-learner interaction, accounted for 12% of the variance in student online course completion rates. In conclusion, of the six factors found, the two with the highest percentage scores were instructor interface and instructor-learner interaction. This finding, while in alignment with the literature concerning the dialogue element of transactional distance theory, brings a special interest to the importance of instructor interface as a factor. Surprisingly, based on the reviewed literature on transactional distance theory, faculty perceptions concerning learner-learner interaction was not an important factor and there was no learner-content interaction factor.

Assessing the effectiveness of five-year mechanical engineering technology programs of junior colleges in Taiwan, R.O.C.: An application of the CIPP evaluation model

The purpose of this study was to evaluate the mechanical engineering technology curriculum effectiveness at the junior college in Taiwan by using the CIPP evaluation model. The study concerned the areas of the curriculum, curriculum materials, individualized instruction, support services, teaching effectiveness, student achievement, and job performance. A descriptive survey method was used with questionnaires for data collection from faculty, students, graduates, and employers.^ All categories of respondents tended to agree that the curriculum provides appropriate occupational knowledge and skills. Students, graduates, and faculty tended to be satisfied with the curriculum; faculty tended to be satisfied with student achievement; graduates tended to be satisfied with their job preparation; and employers were most satisfied with graduates' job performance.^ Conclusions were drawn in the context, input, process, and product of the CIPP model. In Context area: Students were dissatisfied with curriculum flexibility in students characteristics. Graduates were dissatisfied with curriculum design for student's adaptability in new economic and industrial conditions; practicum flexibility in student characteristics; and course overlap. Both students and graduates were dissatisfied with practicum credit hours. Both faculty and students were dissatisfied with the number of required courses.^ In Input area: Students, faculty, and graduates perceived audiovisuals and manipulative aids positively. Faculty and students perceive CAI implementation positively. Students perceived textbooks negatively.^ In Process area: Faculty, students, and graduates perceived all support service negatively. Faculty tended to perceive the ratios of graduates who enter advanced study and related occupation, and who passed the professional skills certification, negatively. Students tended to perceive teaching effectiveness in terms of instructional strategies, the quality of instruction, overall suitability, and receivable, negatively. Graduates also tended to identify the instructional strategies as a negative perception. Faculty and students perceived curriculum objectives and practicum negatively. Both faculty and students felt that instructors should be more interested in making the courses a useful learning experience.^ In Product area: Employers were satisfied with graduates' academic preparation and job performance, adaptability, punctuality, and their ability to communicate, cooperate, and meet organization needs. Graduates were weak in terms of equipment familiarity and supervisory ability.^ In sum, the curriculum of the five-year mechanical engineering technology programs of junior college in Taiwan has served adequately up to this time in preparing a work force to enter industry. It is now time to look toward the future and adapt the curriculum and instruction for the future needs of this high-tech society. ^

Perceptions of beginning teachers in Florida regarding their preparation as it relates to the Florida Essential Generic Competencies

The purpose of the study is to investigate how beginning teachers in the state of Florida perceive their preparation to demonstrate the 27 Florida Essential Generic Competencies.^ The basic research question of this study was: How do beginning teachers perceive their level of preparation regarding their implementation of the Florida Essential Generic Competencies? This study identified and categorized the perceived degree of preparation for each of the competencies. Also, elementary, middle, and high school beginning teachers were compared to find significant differences and similarities in their perception of their preparation. A comparison was also done for graduates from in-state versus out-of-state and private versus public institutions.^ A survey developed in collaboration with the Department of Education, Florida State University, members of the Professional Orientation Program (POP) Coordinators, and the Project Director of Program Review in the College of Education at the University of South Florida, was sent to 5,076 beginning teachers. A total of 1,995 returned the survey in February of 1993. The Multivariate Analysis of Variance (MANOVA) procedure was used (Alpha =.05). Statistical analysis of the data involved a comparison of the different groups of beginning teachers by school level and kind of graduating institutions. The dependent variables analyzed were the responses to all items representing the generic competencies.^ The study identified and categorized the degree of preparation for each competency. The competencies receiving the lowest ratings for degree of preparation were: integrate computers in instruction; manage situations involving child abuse and/or neglect; severe emotional stress; alcohol and drug abuse.^ The Wilkes lambda and the Hotellings multivariate tests of significance were used to examine the differences among the groups. The competency items were further analyzed by a univariate F test. Results indicated that: (1) significant differences were found in nine competency items in which elementary teachers felt better prepared than middle and high school beginning teachers, (2) graduates from a Florida teacher education program felt they were better prepared in demonstrating the competencies than those from out-of-state schools, and (3) no significant difference was found in the perceptions of those who graduated from public versus private institutions.^ Based on the findings of this study, the following recommendations are made: (1) Florida's institutions responsible for teacher preparation programs need to focus on those competencies receiving the lowest ratings, (2) Districts should provide an orientation program for out-of-state beginning teachers, and (3) The survey instrument should be used annually to evaluate teacher education programs. ^

A history of the presidency of National Taiwan University: The relationship between the university and an evolving democratic nation

Higher education always plays an important role in the development of a nation. Taiwan is no exception. Graduates of the National Taiwan University have occupied most of the important positions in this country and today many devote themselves to the development of Taiwan since the central government of the Republic of China (ROC) withdrew from Mainland China and re-located to Taiwan in the winter of 1949. The higher education system in Taiwan, including university and junior colleges, received special attention from the government except from 1945 to 1949 during the transitional period; the time of the early restoration year and the central government's retreating period from Mainland China.^ The five presidents of National Taiwan University who served from 1949 to 1993, Fu Szu-nien, Chien Seu-liang, Yen Chen-Hsing, Yu Chao-chung, and Sun Chen, are the subject of this research. All of the presidents were appointed by the government which established a direct connection between the government and the university leadership. The purpose of this study is to understand how each president balanced politically assigned roles and expectations with personal visionary academic responsiveness to the principles which define the university.^ Each president and his tenure were analyzed using historical research, a developed leadership model, an integration of role theory, Locke's leadership model, Wiles and Bondi curriculum leader tasks, and Burn's leadership style. Results of analyses of documents showed that all presidents of the National Taiwan University were highly respected due to their academic background, personal characteristics, and contribution to the university as a leader. Meanwhile, implementation and achievement of the presidents led to the conclusion that appointed university presidents had significant relationships with government policy. Their leadership style was affected strongly by their personal traits and knowledge and the social and political climate of the time. ^

A qualitative analysis of the lived experience of nursing staff developers in the process of changing to a new educational model

The purpose of this study was to document and critically analyze the lived experience of selected nursing staff developers in the process of moving toward a new model for hospital nursing education. Eleven respondents were drawn from a nation-wide population of about two hundred individuals involved in nursing staff development. These subjects were responsible for the implementation of the Performance Based Development System (PBDS) in their institutions.^ A purposive, criterion-based sampling technique was used with respondents being selected according to size of hospital, primary responsibility for orchestration of the change, influence over budgetary factors and managerial responsibility for PBDS. Data were gathered by the researcher through both in-person and telephone interviews. A semi-structured interview guide, designed by the researcher was used, and respondents were encouraged to amplify on their recollections as desired. Audiotapes were transcribed and resulting computer files were analyzed using the program "Martin". Answers to interview questions were compiled and reported across cases. The data was then reviewed a second time and interpreted for emerging themes and patterns.^ Two types of verification were used in the study. Internal verification was done through interview transcript review and feedback by respondents. External verification was done through review and feedback on data analysis by readers who were experienced in management of staff development departments.^ All respondents were female, so Gilligan's concept of the "ethic of care" was examined as a decision making strategy. Three levels of caring which influenced decision making were found. They were caring: (a) for the organization, (b) for the employee, and (c) for the patient. The four existentials of the lived experience, relationality, corporeality, temporality and spatiality were also examined to reveal the everydayness of making change. ^

An examination of attitude toward change as a predictor of policy/program implementation

Policy/program implementation, e.g., the process of fulfilling policy/program directives, is fundamentally tied to change. Implementation studies have examined the process, identifying many critical organizational variables although individuals perform the activities.^ Many of the studies are predicated on the rational, goal oriented model of organizations and examine implementation, presenting only the goal-oriented view. Organizational change and its resistance, however, are not fully explained by the rational model of organizations. There are other schools of thought providing different views of organizations from which explanation may emerge. Bolman and Deal (1984, 1991a, 1994) provide a different perspective for examining organizations Bolman and Deal argue organizations should be viewed through four different frames or lenses. Framing and reframing organizational action captures the complexity of action and provides better understanding of organizational processes. Understanding of implementation of policies/programs also will benefit from the use of the four-frame approach.^ The goal of this research is to provide a better understanding of the implementation process by examining individual attitudes toward change, the dependent variable of this research, and studying the relationship between the dependent variable and frame. The research was conducted in two phases. In Phase One, a survey was sent to 306 school administrators and teachers in magnet programs in Dade County, Florida. The survey instrument was composed of 55 questions including six from Bolman and Deal's Leadership Orientation Survey (1988) and 38 questions about organizational change. In Phase Two, more in-depth analysis of four school was conducted, to further explore the relationship between frame and attitude toward change.^ The results revealed that frame was a factor in explaining differences in personal Attitude Toward Change and Comfort Level with Change. Individuals using the symbolic frame had more positive attitudes toward change and were also more comfortable with change. The results of Phase Two of the research partially supported this finding in that the most fully implemented program was the product of an administrator who had chosen the symbolic frame. ^

Facing a paradigm shift in professional credentialing: Preparedness of registered dietitians for Professional Development 2001

In the year 2001, the Commission on Dietetic Registration (CDR) will begin a new process of recertifying Registered Dietitians (RD) using a self-directed lifelong learning portfolio model. The model, entitled Professional Development 2001 (PD 2001), is designed to increase competency through targeted learning. This portfolio consists of five steps: reflection, learning needs assessment, formulation of a learning plan, maintenance of a learning log, and evaluation of the learning plan. By targeting learning, PD 2001 is predicted to foster more up-to-date practitioners than the current method that requires only a quantity of continuing education hours. This is the first major change in the credentialing system since 1975. The success or failure of the new system will impact the future of approximately 60,000 practitioners. The purpose of this study was to determine the readiness of RDs to change to the new system. Since the model is dependent on setting goals and developing learning plans, this study examined the methods dietitians use to determine their five-year goals and direction in practice. It also determined RD's attitudes towards PD 2001 and identified some of the factors that influenced their beliefs. A dual methodological design using focus groups and questionnaires was utilized. Sixteen focus groups were held during state dietetic association meetings. Demographic data was collected on the 132 registered dietitians who participated in the focus groups using a self-administered questionnaire. The audiotaped sessions were transcribed into 643 pages of text and analyzed using Non-numerical Unstructured Data - Indexing Searching and Theorizing (NUD*IST version 4). Thirty-four of the 132 participants (26%) had formal five-year goals. Fifty-four participants (41%) performed annual self-assessments. In general, dietitians did not currently have professional goals nor conduct self-assessments and they claimed they did not have the skills or confidence to perform these tasks. Major barriers to successful implementation of PD 2001 are uncertainty, misinterpretation, and misinformation about the process and purpose, which in turn contribute to negative impressions. Renewed vigor to provide a positive, accurate message along with presenting goal-setting strategies will be necessary for better acceptance of this professional development process. ^