Development and implementation of a tool to assess the micro/macro elements in an organization

This study examines the congruency of planning between organizational structure and process, through an evaluation and planning model known as the Micro/Macro Dynamic Planning Grid. The model compares day-to-day planning within an organization to planning imposed by organizational administration and accrediting agencies. A survey instrument was developed to assess the micro and macro sociological analysis elements utilized by an organization.^ The Micro/Macro Dynamic Planning Grid consists of four quadrants. Each quadrant contains characteristics that reflect the interaction between the micro and macro elements of planning, objectives and goals within an organization. The Over Macro/Over Micro, Quadrant 1, contains attributes that reflect a tremendous amount of action and ongoing adjustments, typical of an organization undergoing significant changes in either leadership, program and/or structure. Over Macro/Under Micro, Quadrant 2, reflects planning characteristics found in large, bureaucratic systems with little regard given to the workings of their component parts. Under Macro/Under Micro, Quadrant 3, reflects the uncooperative, uncoordinated organization, one that contains a multiplicity of viewpoints, language, objectives and goals. Under Macro/Under Micro, Quadrant 4 represents the worst case scenario for any organization. The attributes of this quadrant are very reactive, chaotic, non-productive and redundant.^ There were three phases to the study: development of the initial instrument, pilot testing the initial instrument and item revision, and administration and assessment of the refined instrument. The survey instrument was found to be valid and reliable for the purposes and audiences herein described.^ In order to expand the applicability of the instrument to other organizational settings, the survey was administered to three professional colleges within a university.^ The first three specific research questions collectively answered, in the affirmative, the basic research question: Can the Micro/Macro Dynamic Planning Grid be applied to an organization through an organizational development tool? The first specific question: Can an instrument be constructed that applies the Micro/Macro Dynamic Planning Grid? The second specific research question: Is the constructed instrument valid and reliable? The third specific research question: Does an instrument that applies the Micro/Macro Dynamic Planning Grid assess congruency of micro and macro planning, goals and objectives within an organization? The fourth specific research question: What are the differences in the responses based on roles and responsibilities within an organization? involved statistical analysis of the response data and comparisons obtained with the demographic data. (Abstract shortened by UMI.) ^

Design and evaluation of a consulting system for database design

Database design is a difficult problem for non-expert designers. It is desirable to assist such designers during the problem solving process by means of a knowledge based (KB) system. A number of prototype KB systems have been proposed, however there are many shortcomings. Few have incorporated sufficient expertise in modeling relationships, particularly higher order relationships. There has been no empirical study that experimentally tested the effectiveness of any of these KB tools. Problem solving behavior of non-experts, whom the systems were intended to assist, has not been one of the bases for system design. In this project a consulting system for conceptual database design that addresses the above short comings was developed and empirically validated.^ The system incorporates (a) findings on why non-experts commit errors and (b) heuristics for modeling relationships. Two approaches to knowledge base implementation--system restrictiveness and decisional guidance--were used and compared in this project. The Restrictive approach is proscriptive and limits the designer's choices at various design phases by forcing him/her to follow a specific design path. The Guidance system approach which is less restrictive, provides context specific, informative and suggestive guidance throughout the design process. The main objectives of the study are to evaluate (1) whether the knowledge-based system is more effective than a system without the knowledge-base and (2) which knowledge implementation--restrictive or guidance--strategy is more effective. To evaluate the effectiveness of the knowledge base itself, the two systems were compared with a system that does not incorporate the expertise (Control).^ The experimental procedure involved the student subjects solving a task without using the system (pre-treatment task) and another task using one of the three systems (experimental task). The experimental task scores of those subjects who performed satisfactorily in the pre-treatment task were analyzed. Results are (1) The knowledge based approach to database design support lead to more accurate solutions than the control system; (2) No significant difference between the two KB approaches; (3) Guidance approach led to best performance; and (4) The subjects perceived the Restrictive system easier to use than the Guidance system. ^

Performance accountability in the federal-state transportation program: Factors affecting successful implementation

Governmental accountability is the requirement of government entities to be accountable to the citizenry in order to justify the raising and expenditure of public resources. The concept of service efforts and accomplishments measurement for government programs was introduced by the Governmental Accounting Standards Board (GASB) in Service Efforts and Accomplishments Reporting: Its Time Has Come (1990). This research tested the feasibility of implementing the concept for the Federal-aid highway construction program and identified factors affecting implementation with a case study of the District of Columbia. Changes in condition and performance ratings for specific highway segments in 15 projects, before and after construction expenditures, were evaluated using data provided by the Federal Highway Administration. The results of the evaluation indicated difficulty in drawing conclusions on the state program performance, as a whole. The state program reflects problems within the Federally administered program that severely limit implementation of outcome-oriented performance measurement. Major problems identified with data acquisition are: data reliability, availability, compatibility and consistency among states. Other significant factors affecting implementation are institutional barriers and political barriers. Institutional issues in the Federal Highway Administration include the lack of integration of the fiscal project specific database with the Highway Performance Monitoring System database. The Federal Highway Administration has the ability to resolve both of the data problems, however interviews with key Federal informants indicate this will not occur without external directives and changes to the Federal “stewardship” approach to program administration. ^ The findings indicate many issues must be resolved for successful implementation of outcome-oriented performance measures in the Federal-aid construction program. The issues are organizational and political in nature, however in the current environment resolution is possible. Additional research is desirable and would be useful in overcoming the obstacles to successful implementation. ^

Parent involvement on school advisory councils: A process evaluation at the middle school level

Increasing parental involvement was made an important goal for all Florida schools in educational reform legislation in the 1990's. A forum for this input was established and became known as the School Advisory Council (SAC). To demonstrate the importance of process and inclusion, a south Florida school district and its local teacher's union agreed on the following five goals for SACs: (a) to foster an environment of professional collaboration among all stakeholders, (b) to assist in the preparation and evaluation of the school improvement plan, (c) to address all state and district goals, (d) to serve as the avenue for authentic and representative input from all stakeholders, and (e) to ensure the continued existence of the consensus-building process on all issues related to the school's instructional program. ^ The purpose of this study was to determine to what extent and in what ways the parent members of one south Florida middle school's SAC achieved the five district goals during its first three years of implementation. The primary participants were 16 parents who served as members of the SAC, while 16 non-parent members provided perspective on parent involvement as “outside sources.” Being qualitative by design, factors such as school climate, leadership styles, and the quality of parental input were described from data collected from four sources: parent interviews, a questionnaire of non-parents, researcher observations, and relevant documents. A cross-case analysis of all data informed a process evaluation that described the similarities and differences of intended and observed outcomes of parent involvement from each source using Stake's descriptive matrix model. A formative evaluation of the process compared the observed outcomes with standards set for successful SACs, such as the district's five goals. ^ The findings indicated that parents elected to the SACs did not meet the intended goals set by the state and district. The school leadership did not foster an environment of professional collaboration and authentic decision-making for parents and other stakeholders. The overall process did not include consensus-building, and there was little if any input by parents on school improvement and other important issues relating to the instructional program. Only two parents gave the SAC a successful rating for involving parents in the decision-making process. Although compliance was met in many of the procedural transactions of the SAC, the reactions of parents to their perceived role and influence often reflected feelings of powerlessness and frustration with a process that many thought lacked meaningfulness and productivity. Two conclusions made from this study are as follows: (a) that the role of the principal in the collaborative process is pivotal, and (b) that the normative-re-educative approach to change would be most appropriate for SACs. ^

Team Primacy Concept-based employee evaluation and job performance

This qualitative case study explored how employees learn from Team Primacy Concept (TPC)-based employee evaluation and how they apply the knowledge in their job performance. Kolb's experiential learning model (1974) served as a conceptual framework for the study to reveal the process of how employees learn from TPC evaluation, namely, how they experience, reflect, conceptualize and act on performance feedback. TPC based evaluation is a form of multirater evaluation that consists of three components: self-feedback, supervisor's feedback, and peer feedback. The distinctive characteristic of TPC based evaluation is the team evaluation component during which the employee's professional performance is discussed by one's peers in a face-to-face team setting, while other forms of multirater evaluation are usually conducted in a confidential and anonymous manner.^ Case study formed the methodological framework. The case was the Southeastern Virginia (SEVA) region of the Institute for Family Centered Services, and the participants were eight employees of the SEVA region. Findings showed that the evaluation process was anxiety producing for employees, especially the process of peer evaluation in a team setting. Preparation was found to be an important phase of TPC evaluation. Overall, the positive feedback delivered in a team setting made team members feel acknowledged. The study participants felt that honesty in providing feedback and openness to hearing challenges were significant prerequisites to the TPC evaluation process. Further, in the planning phase, employees strove to develop goals for themselves that were meaningful. Also, the catalyst for feedback implementation appeared to stem from one's accountability to self and to the client or community. Generally, the participants identified a number of performance improvement goals that they attained during their employment with IFCS, which were supported by their developmental plans.^ In conclusion, the study identified the process by which employees learned from TPC-based employee evaluation and the ways in which they used the knowledge to improve their job performance. Specifically, the study examined how participants felt and what they thought about TPC-based feedback, in what ways they reflected and made meaning of the feedback, and how they used the feedback to improve their job performance.^

The effect of dual -language instruction on academic achievement as measured by the SAT: An evaluation of the "International Studies Program" of Miami -Dade County Public Schools

This study investigated the influence that receiving instruction in two languages, English and Spanish, had on the performance of students enrolled in the International Studies Program (delayed partial immersion model) of Miami Dade County Public Schools on a standardized test in English, the Stanford Achievement Test, eighth edition, for three of its sections, Reading Comprehension, Mathematics Computations, and Mathematics Applications.^ The performance of the selected IS program/Spanish section cohort of students (N = 55) on the SAT Reading Comprehension, Mathematics Computation, and Mathematics Application along four consecutive years was contrasted with that of a control group of comparable students selected within the same feeder pattern where the IS program is implemented (N = 21). The performance of the group was also compared to the cross-sectional achievement patterns of the school's corresponding feeder pattern, region, and district.^ The research model for the study was a variation of the "causal-comparative" or "ex post facto design" sometimes referred to as "prospective". After data were collected from MDCPS, t-tests were performed to compare IS-Spanish students SAT performance for grades 3 to 6 for years 1994 to 1997 to control group, feeder pattern, region and district norms for each year for Reading Comprehension, Mathematics Computation, and Mathematics Applications. Repeated measures ANOVA and Tukey's tests were calculated to compare the mean percentiles of the groups under study and the possible interactions of the different variables. All tests were performed at the 5% significance level.^ From the analyses of the tests it was deduced that the IS group performed significantly better than the control group for all the three measures along the four years. The IS group mean percentiles on the three measures were also significantly higher than those of the feeder pattern, region, and district. The null hypotheses were rejected and it was concluded that receiving instruction in two languages did not negatively affect the performance of IS program students on tests taken in English. It was also concluded that the particular design the IS program enhances the general performance of participant students on Standardized tests.^ The quantitative analyses were coupled with interviews from teachers and administrators of the IS program to gain additional insight about different aspects of the implementation of the program at each particular school. ^

The early development and pilot -testing of a case management intervention with victims of stalking: An evaluation study

This study evaluated the early development and pilot-testing of Project IMPACT, a case management intervention for victims of stalking. The Design and Development framework (Rothman & Thomas, 1994) was used as a guide for program development and evaluation. Nine research questions examined the processes and outcomes associated with program implementation. ^ The sample included all 36 clients who participated in Project IMPACT between February of 2000 and June of 2001, as well as the victim advocates who provided them with services. Quantitative and qualitative data were drawn from client case files, participant observation field notes and interview transcriptions. Quantitative data were entered into three databases where: (1) clients were the units of analysis (n = 36), (2) services were the units of analysis (n = 1146), and (3) goals were the units of analysis (n = 149). These data were analyzed using descriptive statistics, Pearson's Chi-square, Spearman's Rho, Phi, Cramer's V, Wilcoxon's Matched Pairs Signed-Ranked Test and McNemar's Test Statistic. Qualitative data were reduced via open, axial and selective coding methods. Grounded theory and case study frameworks were utilized to analyze these data. ^ Results showed that most clients noted an improved sense of well-being and safety, although residual symptoms of trauma remained for numerous individuals. Stalkers appeared to respond to criminal and civil justice-based interventions by reducing violent and threatening behaviors; however, covert behaviors continued. The study produced findings that provided preliminary support for the use of several intervention components including support services, psycho-education, safety planning, and boundary spanning. The psycho-education and safety planning in particular seemed to help clients cognitively reframe their perceptions of the stalking experience and gain a sense of increased safety and well-being. A 65% level of satisfactory goal achievement was observed overall, although goals involving justice-based organizations were associated with lower achievement. High service usage was related to low-income clients and those lacking in social support. Numerous inconsistencies in program implementation were found to be associated with the skills and experiences of victim advocates. Thus, recommendations were made to further refine, develop and evaluate the intervention. ^

Moving toward change: Institutionalizing reform through implementation of the Learning Assistant model and Open Source Tutorials

Florida International University has undergone a reform in the introductory physics classes by focusing on the laboratory component of these classes. We present results from the secondary implementation of two research-based instructional strategies: the implementation of the Learning Assistant model as developed by the University of Colorado at Boulder and the Open Source Tutorial curriculum developed at the University of Maryland, College Park. We examine the results of the Force Concept Inventory (FCI) for introductory students over five years (n=872) and find that the mean raw gain of students in transformed lab sections was 0.243, while the mean raw gain of the traditional labs was 0.159, with a Cohen’s d effect size of 0.59. Average raw gains on the FCI were 0.243 for Hispanic students and 0.213 for women in the transformed labs, indicating that these reforms are not widening the gaps between underrepresented student groups and majority groups. Our results illustrate how research-based instructional strategies can be successfully implemented in a physics department with minimal department engagement and in a sustainable manner.

Parent involvement on school advisory councils: a process evaluation at the middle school level

Increasing parental involvement was made an important goal for all Florida schools in educational reform legislation in the 1990's. A forum for this input was established and became known as the School Advisory Council (SAC). To demonstrate the importance of process and inclusion, a south Florida school district and its local teacher's union agreed on the following five goals for SACs: (a) to foster an environment of professional collaboration among all stakeholders, (b) to assist in the preparation and evaluation of the school improvement plan, (c) to address all state and district goals, (d) to serve as the avenue for authentic and representative input from all stakeholders, and (e) to ensure the continued existence of the consensus-building process on all issues related to the school's instructional program. The purpose of this study was to determine to what extent and in what ways the parent members of one south Florida middle school's SAC achieved the five district goals during its first three years of implementation. The primary participants were 16 parents who served as members of the SAC, while 16 non-parent members provided perspective on parent involvement as "outside sources." Being qualitative by design, factors such as school climate, leadership styles, and the quality of parental input were described from data collected from four sources: parent interviews, a questionnaire of non-parents, researcher observations, and relevant documents. A cross-case analysis of all data informed a process evaluation that described the similarities and differences of intended and observed outcomes of parent involvement from each source using Stake's descriptive matrix model. A formative evaluation of the process compared the observed outcomes with standards set for successful SACs, such as the district's five goals. The findings indicated that parents elected to the SACs did not meet the intended goals set by the state and district. The school leadership did not foster an environment of professional collaboration and authentic decision-making for parents and other stakeholders. The overall process did not include consensus-building, and there was little if any input by parents on school improvement and other important issues relating to the instructional program. Only two parents gave the SAC a successful rating for involving parents in the decision-making process. Although compliance was met in many of the procedural transactions of the SAC, the reactions of parents to their perceived role and influence often reflected feelings of powerlessness and frustration with a process that many thought lacked meaningfulness and productivity. Two conclusions made from this study are as follows: (a) that the role of the principal in the collaborative process is pivotal, and (b) that the normative-re-educative approach to change would be most appropriate for SACs.

Design and evaluation of a consulting system for database design

Database design is a difficult problem for non-expert designers. It is desirable to assist such designers during the problem solving process by means of a knowledge based (KB) system. Although a number of prototype KB systems have been proposed, there are many shortcomings. Firstly, few have incorporated sufficient expertise in modeling relationships, particularly higher order relationships. Secondly, there does not seem to be any published empirical study that experimentally tested the effectiveness of any of these KB tools. Thirdly, problem solving behavior of non-experts, whom the systems were intended to assist, has not been one of the bases for system design. In this project, a consulting system, called CODA, for conceptual database design that addresses the above short comings was developed and empirically validated. More specifically, the CODA system incorporates (a) findings on why non-experts commit errors and (b) heuristics for modeling relationships. Two approaches to knowledge base implementation were used and compared in this project, namely system restrictiveness and decisional guidance (Silver 1990). The Restrictive system uses a proscriptive approach and limits the designer's choices at various design phases by forcing him/her to follow a specific design path. The Guidance system approach, which is less restrictive, involves providing context specific, informative and suggestive guidance throughout the design process. Both the approaches would prevent erroneous design decisions. The main objectives of the study are to evaluate (1) whether the knowledge-based system is more effective than the system without a knowledge-base and (2) which approach to knowledge implementation - whether Restrictive or Guidance - is more effective. To evaluate the effectiveness of the knowledge base itself, the systems were compared with a system that does not incorporate the expertise (Control). An experimental procedure using student subjects was used to test the effectiveness of the systems. The subjects solved a task without using the system (pre-treatment task) and another task using one of the three systems, viz. Control, Guidance or Restrictive (experimental task). Analysis of experimental task scores of those subjects who performed satisfactorily in the pre-treatment task revealed that the knowledge based approach to database design support lead to more accurate solutions than the control system. Among the two KB approaches, Guidance approach was found to lead to better performance when compared to the Control system. It was found that the subjects perceived the Restrictive system easier to use than the Guidance system.

Baseline analysis of the existing capacities and needs for capacity building for Marine Strategy Framework Directive implementation in the Celtic Seas sub-region.

This report evaluates the existing situation in the Celtic Seas sub-region and determines the current state of preparedness for transboundary management of marine ecosystems and MSFD implementation. Recommendations for capacity building are provided through the analysis of the existing conflicts and potential synergies between relevant policies, institutions and information resources for MSFD implementation across the region. This report strives to empower stakeholders through the provision of a sound baseline with accurate and up-to-date information on the current status of MSFD implementation, potential opportunities and suggested approaches for building capacities in their region and across the Celtic Seas. It is evident that there are a number of national marine planning processes currently underway and at different stages throughout the United Kingdom and the pre-planning context for MSP in Ireland. On a similar note, this evaluation of MSFD implementation progress to-date in the United Kingdom, Ireland and France highlights that each Member State has implemented the legal and procedural requirements of preparatory steps in differing manners and using different time scales. This variance across the sub-region has the potential to impact the achievement of GES by 2020 across the Celtic Seas.

Collimation of a D-D Neutron Generator for Clinical Implementation of Neutron Stimulated Emission Computed Tomography: a Monte Carlo Study

This work is an investigation into collimator designs for a deuterium-deuterium (DD) neutron generator for an inexpensive and compact neutron imaging system that can be implemented in a hospital. The envisioned application is for a spectroscopic imaging technique called neutron stimulated emission computed tomography (NSECT).

Previous NSECT studies have been performed using a Van-de-Graaff accelerator at the Triangle Universities Nuclear Laboratory (TUNL) in Duke University. This facility has provided invaluable research into the development of NSECT. To transition the current imaging method into a clinically feasible system, there is a need for a high-intensity fast neutron source that can produce collimated beams. The DD neutron generator from Adelphi Technologies Inc. is being explored as a possible candidate to provide the uncollimated neutrons. This DD generator is a compact source that produces 2.5 MeV fast neutrons with intensities of 1012 n/s (4π). The neutron energy is sufficient to excite most isotopes of interest in the body with the exception of carbon and oxygen. However, a special collimator is needed to collimate the 4π neutron emission into a narrow beam. This work describes the development and evaluation of a series of collimator designs to collimate the DD generator for narrow beams suitable for NSECT imaging.

A neutron collimator made of high-density polyethylene (HDPE) and lead was modeled and simulated using the GEANT4 toolkit. The collimator was designed as a 52 x 52 x 52 cm3 HDPE block coupled with 1 cm lead shielding. Non-tapering (cylindrical) and tapering (conical) opening designs were modeled into the collimator to permit passage of neutrons. The shape, size, and geometry of the aperture were varied to assess the effects on the collimated neutron beam. Parameters varied were: inlet diameter (1-5 cm), outlet diameter (1-5 cm), aperture diameter (0.5-1.5 cm), and aperture placement (13-39 cm). For each combination of collimator parameters, the spatial and energy distributions of neutrons and gammas were tracked and analyzed to determine three performance parameters: neutron beam-width, primary neutron flux, and the output quality. To evaluate these parameters, the simulated neutron beams are then regenerated for a NSECT breast scan. Scan involved a realistic breast lesion implanted into an anthropomorphic female phantom.

This work indicates potential for collimating and shielding a DD neutron generator for use in a clinical NSECT system. The proposed collimator designs produced a well-collimated neutron beam that can be used for NSECT breast imaging. The aperture diameter showed a strong correlation to the beam-width, where the collimated neutron beam-width was about 10% larger than the physical aperture diameter. In addition, a collimator opening consisting of a tapering inlet and cylindrical outlet allowed greater neutron throughput when compared to a simple cylindrical opening. The tapering inlet design can allow additional neutron throughput when the neck is placed farther from the source. On the other hand, the tapering designs also decrease output quality (i.e. increase in stray neutrons outside the primary collimated beam). All collimators are cataloged in measures of beam-width, neutron flux, and output quality. For a particular NSECT application, an optimal choice should be based on the collimator specifications listed in this work.

Program evaluation primer: A review of three evaluations

Program evaluation—a “tool used to assess the implementation and outcomes of a program, to increase a program’s efficiency and impact over time, and to demonstrate accountability” (MacDonald et. al, 2001, p. 1)—is an essential process to program assessment and improvement. This paper overviews three published program evaluations and considers important aspects of program evaluation more broadly.

Development and Application of a Multi-dimensional Strategy for Measuring Fidelity of Implementation of a Districtwide Intervention

This symposium describes a multi-dimensional strategy to examine fidelity of implementation in an authentic school district context. An existing large-district peer mentoring program provides an example. The presentation will address development of a logic model to articulate a theory of change; collaborative creation of a data set aligned with essential concepts and research questions; identification of independent, dependent, and covariate variables; issues related to use of big data that include conditioning and transformation of data prior to analysis; operationalization of a strategy to capture fidelity of implementation data from all stakeholders; and ways in which fidelity indicators might be used.